Smartphones have pretty much eliminated the need for point-and-shoot pocket cameras. For a while now, they have even been threatening to replace professional rigs. As early as 2012, Time Magazine featured a cover shot snapped by an iPhone 4S, and recently a Sports Illustrated cover photo was shot with a Motorola Z and Billboard Magazine’s cover was taken using an iPhone 7 plus. As smartphones cameras improve, the market has come to expect DSLR-like image and video quality as well as DSLR-like features. But, until now, DSLRs have still been irreplaceable in certain cases.
Photo shot with the ASUS Zenfone 3 Zoom (Credit: CEVA)
The Limited Potential of a Hardware Image Signal Processor
One of the most prominent of these cases is shooting in low-light conditions. A major reason for this is the limited capabilities of the common application processors (AP). In addition to many other modules, these APs are equipped with a hardware Image Signal Processor (ISP). The image processing is not the main focus of the AP, but rather is one of many other features that the AP must support. Therefore, they all offer decent quality cameras, but not state-of-the-art photographic capabilities. These solutions are limited in their ability to provide DSLR-like image and video quality such as low light, HDR and AAA (Auto exposure, Auto white balance and Auto focus), while retaining a long battery life. They also lack the flexibility to cope with the evolving changes and demand for higher quality, as they are built on a fixed hardware pipeline. This also eliminates the option of on-the-fly updates for fixes and efficient real-time technical support.
To offer better image quality and differentiate themselves from the competition, OEMs need to offer something more. Some companies tackle this issue by manufacturing their own chipset. For example, Xiaomi developed its own AP, with a built-in dual ISP. But there is a faster and much lower-risk solution that addresses all the above issues, and does not entail developing an entire chipset. The solution is to embed a flexible and power digital signal processor (DSP) with an efficient vector processing unit (VPU) in the existing pipeline.
Enabling Superior Image and Video Quality with a DSP
Adding a DSP to improve the image processing of the ISP opens the door to many features. It can offer WRGB to RGB conversion, distortion correction, video stabilization and other ISP enhancements. It also enables many features that can significantly improve quality of both still images and video footage, like low-light enhancements, HDR and super resolution. Using a DSP, all this is achieved with minimal power so that the battery can last for all the other smartphone tasks. The DSP solution also maximizes flexibility and keeps the options open to address future needs.
The DSP can be added to the ISP in three different configurations. It can be placed inside the camera module as a companion chip to perform pre-ISP processing. This enables support of non-standard Bayer patterns, dual sensor cameras, and a wide field of view lens (like 180 or 360 degrees). A second option is to add the DSP as a companion chip for post-ISP processing. The third setup places the DSP within the ISP.
Putting It All Together in an SoC
This potential has been put to use in Rockchip’s new vision coprocessor, the RK1608, which just launched during MWC 2017. The RK1608 utilizes the CEVA-XM4 DSP core to perform frame-based processing, rather than line-based as in ISP. Because it is specifically designed for image processing, it can perform much faster than a standard AP. It includes features like multi frame de-noise, zig-zag HDR for preview and capture, HDR processing of multi-exposed image, depth map generation, super resolution and pseudo optical zoom. The vision coprocessor’s function is not dependent on the ISP, so it can be adapted to any platform.
One phone already available on the market that uses the RK1608 for enhanced photography, is the ASUS ZenFone 3 Zoom smartphone, launched earlier this year at CES 2017. If there was ever a phone that truly threatened DSLRs, this is it. This device was designed to be a camera just as much as it was designed to be a phone. The aperture, pixel size and sensor are among the largest in the industry at f/1.7, 1.4µm and 1/2.55”, respectively. It uses new SuperPixel camera technology for clearer images at night or in low-light environments and offers 2.5-times greater light sensitivity than the Apple iPhone 7 Plus.
Side by side comparison of ASUS ZenFone 3 Zoom (left) and iPhone 7 Plus (right) (Credit: Rockchip)
Sampled cropped areas ASUS ZenFone 3 Zoom (left) and iPhone 7 Plus (right) (Credit: Rockchip)
To demonstrate the image quality, London-based urban photographer Nicholas Goodden, created a series of photos exclusively shot with the ASUS Zenphone 3 Zoom, which you can view here. The photos were printed and exhibited at the W Hotel in Barcelona during MWC 2017. As the photographer states on his site, those impressive images were acquired in only seven days, including printing for the exhibit.
Photo shot with the ASUS Zenfone 3 Zoom (Credit: Nicholas Goodden)
But that’s a professional photographer, right? We wanted to see the results in controlled conditions. So, at our booth at MWC, we set up a Macbeth Colorchecker in low-light conditions (5 LUX) to test the performance of different smartphones (while using automatic shooting mode). None of the phones came close to the image quality of the Zenfone 3 Zoom, including camera modules that are still in development and haven’t been released yet! Check out the images below to see for yourself.
CEVA’s low light setup at MWC (Credit: CEVA)
These are the results, clearly showing the superiority of the Zenfone 3 Zoom. Provides better color accuracy, more details with less noise at the fraction of the power consumption (Credit: CEVA)
Learn more
As you can see, the results speak for themselves. To find out how you can harness our low-power, high-performance imaging and vision platforms for your photography solution, click here.
By Liran Bar
Director of Product Marketing, Imaging & Vision, CEVA.
