This blog post was originally published at Visidon’s website. It is reprinted here with the permission of Visidon.
As discussed in previous blog articles, taking a photo means that your camera is capturing data, which is used to create a digital image. Every graphic you see on a display is an image file. Image files come in a variety of formats, and each is optimized for a specific use. The choice for the image file depends on the required level of quality and the level of post-processing needed.
As one of the main technology providers for embedded imaging solutions, we are considering the choice based on our experience. The formats presented here are discovered to be the most optimal ones for mobile imaging solutions with requirements for fast computing, low power, and low memory use.
Let´s go through some of the most common formats in image file size order and how they affect to the end result.
It´s RAW
A camera RAW image file contains minimally processed data from the image sensor and is usually uncompressed. RAW images offer the highest image quality and they are easier to manipulate when it comes to exposure, contrast, and white balance. Therefore, they are preferred by professional photographers, who want to have full control over quality during the post-processing. RAW images cannot be projected as such to displays.
RAW format defines the raw data from an image sensor, which is usually Bayer pattern. Instead of each pixel having R, G, and B values (Red, Green and Blue), each pixel in a 2×2 grid is assigned to one color. The missing colors, needed to make a full RGB image, are calculated, i.e. interpolated, from neighboring pixels. The used interpolation methods vary and are often compromises between processing time and image quality.
It is also good to know, that every camera manufacturer has their own RAW file format. When it comes to mobile phones, most Android phones supporting RAW image format are shooting in DNG, which is a more universal format developed by Adobe. Apple has developed their own ProRAW format.
Bayer pattern with each pixel in a 2×2 grid assigned to one color
Good old RGB
RGB is a color model that you are able to see on any digital display. In this model, all the colors can be presented on the display by three primary colors: red, green, and blue. Each of these can be recorded in a computer in 1 byte (8 bit) and therefore a pixel in a monitor has 3 bytes of information – one byte each for red, green, and blue. Every pixel is presented with 3 different light sources and the RGB image tells the intensity of the light in each source.
This format might take unnecessary memory space during image processing when for example only gray image (luminance) is needed.
Original picture and the primary colors of RGB model
YUV
To reduce processing times and memory requirements, images can be formatted to YUV format. YUV resembles the RGB color model but using YUV we can dramatically decrease the file size very easily. Human eye is more sensitive to light and brightness than to different colors and YUV representation of color utilizes this. In this format the brightness (luminance) and color (chrominance) are separated between Y and UV images. Y is a gray-scale image and UV represents the coordinates in an UV chart.
There are many subformats of YUV, and a common one (especially on Android devices) is YUV420. In this format every pixel has a grey value, but the same color is used for 4 pixels. This results in a tiny loss in color accuracy but is often acceptable and satisfactory for human eye.
In RGB model we have to use 3 bytes (24 bits) to record a color. In YUV model a new color is not needed for every pixel and therefore only half of the memory space is required compared to a RGB file. Occasionally only gray image is needed, and YUV model allows us to process just the Y part instead of all R, G, and B values. Both RGB and YUV have their strengths and weaknesses, so in case you wonder which one is better, the answer depends on the application.
An image with its Y´, U and V components
Image file size comparison between the uncompressed RAW, YUV, and RGB
Image processing can involve copying and accessing the data multiple times. Less data means less time and energy used per image and therefore YUV is preferred in some embedded imaging applications. RGB image files might use unnecessary big amount of data, and even 10bit RAW format would be less data intensive if pre-processing, like white balance, is needed.
| RAW 16bit | YUV 8bit | RGB 8bit | ||
| 100 x 100 pixel image | 20 kB | 15 kB | 30 kB |
JPEG compression
After processing it is preferred to compress the images to JPEG images which are much smaller in size and need less storage space. For displaying, JPEG needs to be decompressed to RGB again. The compressed images are result images so for image processing, the above formats are used.
The amount of JPEG compression is typically measured as a percentage of the quality level. An image at 100% quality has (almost) no loss, and 1% quality is a very low-quality image. In general, quality levels of 90% or higher are considered high quality, 80%-90% is medium quality, and 70%-80% is low quality. Anything below 70% is typically a very low-quality image.
JPEG compression example
About Visidon:
Visidon specializes in the development of fast and energy efficient image and video processing technologies. Our software products include for example automatic image quality enhancement, computational imaging, and face recognition.
Our technologies utilize the latest microprocessor architectures and machine learning algorithms especially for embedded industries. Our solutions can be found in over 1 billion mobile phones.
