This blog post was originally published at TechNexion’s website. It is reprinted here with the permission of TechNexion.
3D scanning has gained immense popularity in the last decade. It is used in AR (Augmented Reality), VR (Virtual Reality), gaming simulation, automotive, manufacturing, and more. From scanning a motorcycle to a human being and machines, the use cases are endless.
To achieve 360-degree object scanning, technologists have traditionally relied on a combination of 60 to 120 digital SLR cameras from Canon or Nikon. The scanning unit will also include other equipment – such as lighting, for example. As you can imagine, this is a very expensive solution and has not seen widespread adoption.
But embedded vision can change this. A path-breaking technology called RGBD cameras can help you achieve what an expensive set of DSLR cameras can do – at a much lower cost.
In this article, we explore:
- What is 3D photogrammetry, and what are its commercial applications?
- How does embedded vision become a game-changer in 3D photogrammetry?
- What are RGBD cameras, and how do they perfectly suit your 3D scanning system?
What is 3D Photogrammetry?
3D photogrammetry is an imaging technique used to create 3D images of an object. It is used for various processes such as digital remodeling, 3D printing, 3D product demonstrations, etc. Photogrammetry involves taking images of the subject from all angles to reproduce it using a 3D modeling software like Solidworks.
Both static and handheld 3D scanners are in use for photogrammetry. The former involves placing the subject in the middle of a cabinet with cameras on all sides. The number of cameras depends on the level of detail needed and the nature of the subject. The latter uses a portable handheld scanning machine a human operator handles to capture a 360-degree surround view of the subject.
For example, imagine you need to create a 3D demonstration of a car or a motorbike. In the case of a static 3D scanning setup, the vehicle will have to be brought to where the cabinet is, whereas a handheld device can be used to scan the object in any location since it is portable. Though static 3D scanning setups tend to offer higher accuracy, portable 3D scanners are getting increasingly advanced and are almost as good as static ones.
How does Embedded Vision Become a Game-Changer in 3D Photogrammetry?
VLS3-AR0522-CB
Embedded vision has changed the face of imaging in various applications. Photogrammetry is one of them, especially in terms of reducing cost. While an SLR-based 3D scanner costs tens of thousands of dollars, a Raspberry Pi-based embedded multi-camera solution for 3D photogrammetry would cost $10K.
This cost advantage comes with the ability to achieve similar quality levels as that of an SLR-based solution. Embedded camera technology has become more advanced, with many features matching that of consumer-end digital cameras. For example, modern embedded cameras can offer a resolution of more than 20MP, which is more than enough for new-age commercial and industrial applications, including 3D photogrammetry.
Another distinctive feature of embedded cameras helpful for 3D scanning is the ability to capture depth information of the scene. Camera modules can now measure the depth of any object with high accuracy using technologies like stereo and time of flight.
Some of the other features that make embedded cameras a suitable solution for 3D photogrammetry are:
- Multi-camera synchronization: Both software and hardware-based methods can be used in embedded cameras for synchronization.
- Multiple camera parameters, such as exposure time, contrast, gain, sharpness, etc., can be adjusted as the application requires.
- IR (InfraRed) sensitivity: In 3D photogrammetry, the vision system creates a 3D model by projecting infrared dots onto the subject and uses the camera to figure out where these dots are. Many camera modules of today come with excellent IR sensitivity and can be comfortably used for IR-enabled 3D scanning.
What is an RGBD Camera?
Ordinary embedded color cameras are called RGB cameras since they read image data using a combination of red, green, and blue colors. The Bayer pattern uses the RGGB pixel format with the sensor capturing data with 25% red, 50% green, and 25% blue colors. Each pixel’s pixel values are combined to obtain the final image in an RGB camera.
On the other hand, a depth camera is used to capture depth information alone – which helps measure the distance to or between objects. This is done using any depth technologies such as stereo, time of flight, and structured light.
Now, what is an RGBD camera?
It is nothing but the combination of an RGB and depth camera in a single system. This essentially means that, instead of using two separate cameras for collecting color data and measuring depth, you have both in one camera system. This offers the advantage of a single camera serving multiple purposes, reducing selection and integration time.
RGBD cameras also come with the ability to identify the type of object in a scene simultaneously while measuring depth. This is helpful in systems where safety is of utmost concern. For example, consider an autonomous tractor. During its navigation, it must stop much quicker if it finds a human obstacle than any other object for safety reasons. This requires the depth camera in the tractor to be able to identify the nature of the objects in addition to measuring depth.
RGBD Cameras: a Perfect Solution for 3D Photogrammetry in Commercial Applications
A key advantage when using embedded cameras is the ability to integrate them with ARM-based platforms such as Raspberry Pi, NVIDIA Jetson, TI Jacinto TDA4VM, and NXP i.MX8, etc. For instance, TechNexion’s cameras come with out-of-the-box integration with all these processing platforms – making integration easy and fast for product developers.
Now, let’s consider the pros of having RGB and depth cameras in one system.
We already touched upon the advantages of using RGBD cameras for 3D scanning, such as cost, ease of integration, and convenience of having an all-in-one solution. Specifically for 3D photogrammetry, RGBD cameras can help ensure even the tiniest details are captured.
For example, an ordinary camera might struggle to create digital 3D models of shiny objects and surfaces. By combining depth and color data, this challenge can be overcome.
But the biggest advantage is that you can create colored 3D models of any object you like – whether it’s a human, a motorbike, or a machine. This cannot be done using a camera with a depth or RGB sensor alone.
Applications Where RGBD 3D Scanning can be Used
3D Model of a Motorbike
The automotive industry extensively uses 3D photogrammetry to give a real feel of their vehicles in ads and virtual product experiences.
Here are a few more applications where 3D photogrammetry has been picking up lately:
- Human digital modeling: For creating 3D human body models for research and analysis purposes.
- Augmented/Virtual reality: For recreating objects in a room or the entire space in 3D.
- Gaming: To offer real game-like experiences, gaming companies use sensors and cameras to create real-time and moving 3D models of gamers. An example is a running race simulation where the gamers run on something like a treadmill or an elevated platform while 3D cameras capture the motion in real time and display it on the screen within the game.
Selecting an RGBD Camera for Your 3D Scanning System
Whether you are building a static 3D scanning system or a portable one, you need the ability to get going fast. Purchasing digital cameras and integrating them into your system can be time-consuming (apart from being super expensive), which might prevent you from focusing on the other core aspects of product development.
This is where embedded camera solutions can be your best friend. With the help of a camera solutions expert like TechNexion, you can reduce your costs by up to 80% and prototyping time by 50%.
Next, let us talk specifically about a few key things you must remember while selecting an RGBD camera for your 3D scanning system.
As mentioned earlier, 3D scanning using RGBD cameras is done with the help of IR dots projected onto the subject. Since most 3D scanning systems are indoors, IR light of wavelength 850nm is typically the source. This helps the camera accurately capture depth information without any interference from sunlight (under bright sunlight, a wavelength of 940nm is a better choice). Hence, an RGBD camera with high QE (Quantum Efficiency) at 850nm is recommended.
Further, keep a check on the following features while selecting an RGBD camera:
- Strobe function – This offers the ability to switch on and off lights for better control of the scene and power optimization.
- Depth accuracy – Should be ideally in the range of a few millimeters.
- Range – Choose this depending on the application. Some might have a working distance of 1 to 2 meters, while others go beyond 5.
- Ability to integrate with other sensors like LiDAR or IMU.
- Ability to operate in both 850nm and 940nm (940nm is needed only in the case of outdoor applications).
- Flexibility in terms of platform/processor compatibility.
TechNexion: Your Go-to Imaging Partner
TechNexion has been striving to make a difference in the embedded vision space for a few years now. We rely on the two decades of experience we have in the embedded systems space to keep coming up with next-gen products that support the imaging needs of our customers – not just for today but for the future, too. Check out our embedded vision cameras page to learn more about our camera solutions.
