This blog post was originally published at Visidon’s website. It is reprinted here with the permission of Visidon.
In the era of digital photography and social media, capturing and sharing images and videos has become an integral part of our lives. However, sometimes the captured content may suffer from face distortion, resulting in unrealistic or unflattering facial proportions. Face distortion can occur due to various factors, such as camera angles, lens distortion, or wide-angle lenses.
Fortunately, advancements in image processing, computer vision, and AI have paved the way for effective face distortion correction techniques, allowing us to enhance photographic accuracy and aesthetics. In this article, we will explore the concept of face distortion correction, its significance, and popular methods employed in the field.
Understanding Face Distortion
Face distortion refers to the alteration of facial features’ shape or proportions in an image or video compared to their real-life appearance. It can manifest as elongated or compressed facial structures, distorted facial symmetry, or skewed perspectives. Face distortion often occurs when capturing close-up shots or using wide-angle lenses, as these can introduce perspective distortion, leading to unnatural facial shapes.
In short, whenever the original image is in widescreen format it needs to be displayed as a 2D image with a smaller field of view (FOV) and this always requires some form of projection. However, each projection method has its own set of problems, such as significant changes in aspect ratios or distortions when examining the images at small or large scales. All image projection types can be useful, depending on the application. Perspective projection is useful to preserve straight lines in an image, while only stereographic projection saves shape of small objects. Hence, stereographic projection can be used as a tool to transform a distorted face image into a corrected version.
Face distortion correction is crucial for preserving the integrity of facial features and ensuring a faithful representation of individuals in photographs and videos. By rectifying distortions, the captured content can depict subjects as they appear in reality, improving the accuracy of facial recognition systems and forensic investigations. Face distortion correction also plays a significant role in enhancing the aesthetic appeal of photographs. By restoring natural facial proportions, images can achieve a more pleasing and balanced composition, making them visually appealing to viewers.
It is important to note, that due to the inherent differences between images captured with a wide-angle lens and the visual perception of the human eye, the process of correcting distortion in wide-angle lens photography presents unique challenges that differ from those encountered when correcting distortion in images taken with a standard lens. This means that correcting distortion in wide-angle lens photography involves tackling complex and non-uniform distortions and some unnatural artifacts may remain in the image.
Perspective projection
Stereographic projection
Visidon Output
Popular Face Distortion Correction Techniques
Geometric correction methods focus on rectifying the spatial distortion in images. These techniques involve identifying key facial landmarks, such as eyes, nose, and mouth, and applying geometric transformations to align them with a standardized reference. Methods like affine transformations, thin-plate spline, or Delaunay triangulation can be employed to achieve accurate alignment and proportional correction.
Mesh warping techniques utilize a mesh grid overlaid on facial regions to manipulate and adjust the facial geometry. By selectively moving grid points, facial features can be reshaped and realigned to their original proportions. This technique allows localized corrections, enabling precise adjustments to specific facial regions while preserving the overall structure.
Deep learning approaches: With the advent of deep learning, neural networks have been employed for face distortion correction. Convolutional Neural Networks (CNNs) can be trained to learn the mapping between distorted and undistorted face images. By leveraging large-scale datasets, these models can effectively correct various types of face distortion.
Face and body segmentation identifies and separates the facial and bodily regions from the background or surrounding objects within each frame of a video. This segmentation enables targeted distortion correction specifically on the face and body, while preserving the overall temporal coherence and natural flow of the footage.
Differences between face distortion correction for still images vs. videos
There can be differences between methods used for face distortion correction in single images and videos. While the underlying principles and techniques may be similar, there are certain considerations specific to each domain that may affect the approach used.
In the case of single images, the correction process typically involves analyzing the facial features and applying transformations to restore the natural proportions. Various computer vision techniques can be employed, such as detecting facial landmarks, estimating the pose and orientation of the face, and applying geometric transformations like affine or perspective transformations. These methods aim to correct the distortion present in the image by manipulating the facial geometry.
Video face distortion correction introduces additional challenges due to the temporal aspect. The correction must be performed consistently across frames to maintain visual continuity. In addition, real-time processing requires light weight algorithms. Methods often involve tracking facial landmarks across consecutive frames to establish correspondences and ensure smooth correction. Techniques such as optical flow, feature tracking, or more advanced approaches like facial tracking using deep learning models may be used. Additionally, temporal smoothing or filtering can be applied to reduce flickering or other artifacts that may occur during the correction process.
While the underlying techniques may overlap, the main difference lies in how the distortion correction is applied consistently over time in videos, considering the temporal coherence and potential artifacts that can arise due to frame-by-frame processing.
Visidon solution for Face Distortion Correction for images and videos in real-time
Visidon has developed a real-time face distortion correction for both images and videos, and it is combining AI-based face and body segmentation with continuously updating locally adaptive mesh warping. It keeps background straight lines naturally straight while applying natural looking stereographic projection to facial regions to avoid unwanted stretching of perspective projection especially caused by wide angle lens corrections. Advanced algorithm utilized adaptive combination of different projection models, seamless both in space and time, and provides overall visually pleasant images and videos.
