By Carlton Heard
Product Engineer – Vision Hardware and Software
National Instruments
With the increase in performance and decrease in cost, smart cameras have become increasingly more accessible over the past decade. Given this trend, how do you determine which smart camera best meets your needs or decide if a smart camera is appropriate for your application? Learn about the top 8 considerations you don't want to forget when selecting a smart camera, including application examples and alternative solutions.
Smart cameras have been around for many years, but advances in processor technologies have made smart cameras much more accessible and popular within the past decade, especially in applications such as machine vision and surveillance. However, when the term smart camera is mentioned, a wide variety of ideas still come to mind, because there is no widespread agreement on the technical definition of just what a smart camera is.
It is generally agreed that the basics of a smart camera includes not only the image sensor, but also some type of processing chip. This can be a CPU, DSP, FPGA, or other type of processing device. However, today even an off-the-shelf point & shoot digital camera has some type of built-in image processing, for example to make the image look more desirable, remove red eye effects, conduct facial recognition, and/or other types of image processing.
So, if the inclusion of a processor along with an image sensor isn’t the defining attribute of a smart camera, what makes a smart camera “smart”? They key lies in the output. Unlike most cameras, the primary output of a smart camera is not an image but a decision or some other type of information (Figure 1). Since the image processing or machine vision algorithm is done directly on the smart camera, the image doesn’t need to be passed onto a PC or another device. Instead, the result of the processing can be passed directly to an operator or another device in the system.
Figure 1. Smart cameras differ from normal cameras in that the primary output is not an image, but a decision or some other kind of information. This difference results in a potential reduction in system size, cost, and complexity.
For example, a smart camera may be selected for use in an in-line inspection system for a manufacturing line. The output of the smart camera could be a pass/fail report over a network to a database, a digital signal triggering a sorting system, or a serial command to a PLC. A smart camera is a decision maker. Still, if you conduct an internet search for a smart camera you will receive a large variety of results with very different features and appearance options. How do you first decide if a smart camera is best suited for a specific application and then choose between the sea of options available? Let’s review eight considerations to keep in mind when evaluating smart cameras.
1. Processor
As previously mentioned, the growing popularity of smart cameras can mainly be attributed to the increase in processor performance over the past decade. A 1 MHz smart camera 15 years ago would have been four times the size and cost of a >1 GHz smart camera of today. Up until a few years ago, smart cameras did not have the necessary capacity for processing and interpreting images and were instead reduced to simple tasks such as reading bar codes. Today, many may be surprised when they realize that the processing performance on some smart cameras rivals what can be done on PC-based systems. Smart cameras come with a range of available processors including DSPs, PowerPC-class, and Atom-class processors. There are also options with a mixed offering including a CPU with a DSP co-processor for certain algorithms.
2. Size
One benefit to using a smart camera is that multiple components of a vision system are integrated into a single package, resulting in a small size and the potential to save a lot of space. Space savings can be very important for machine vision and embedded applications where the space for a particular inspection cell may be fixed, but where more and more inspection or control steps are being added to the same space. Smart cameras also benefit from Moore’s Law, with today's available cameras being smaller than 55mm x 50mm and weighing less than 60 grams.
3. Image Sensor
Of course, a smart camera is still a camera and must acquire images. Both CMOS and CCD sensors can be found in smart cameras, with resolutions of up to 5MP and available in both color and monochrome variants. Smart cameras aren’t just limited to area scan. Line scan smart cameras are available with frequencies over 10 kHz. While smart cameras do not cover the full range of options as normal cameras, the ever increasing availability of smart cameras still spans a wide diversity of options, including some of the most popular sensors.
4. Operating System
As with any embedded system, the operating system is a variable between models of smart cameras. Real-time variants provide the software stability and deterministic operation desired for many embedded applications, such as machine vision, surveillance or robotics. However, there are other options such as Linux- and Windows-based models that exist for those who need to run Linux or Windows-only software (such as a database access program) along with the vision processing or to simply ensure software familiarity.
5. Software
Many times software is the key differentiation between smart cameras because most manufacturers will offer a range of smart cameras instead of a single model. This means that there can be a lot of overlap based on features such as the image sensor, processor performance, and size. Of course, the main differentiation is in the way that it’s programmed (Figure 2). Most smart cameras include relatively simple-to-use software so an advanced knowledge of programming skills is not required to use them, but it’s important to keep flexibility and scalability in mind.
Figure 2. Most smart cameras come with relatively intuitive programming software, reducing the need for an advanced knowledge of programming skills. The program should not only be easy-to-use, but should also be flexible enough to scale to different hardware targets without a complete rewrite.
The investment to learn a new piece of software and write an application should be somewhat long-term in nature This means that the software should scale with application requirements and future projects. Sensor and processor technologies are advancing rapidly, so the best-case scenario takes place when the smart camera model and software are well integrated but not exclusive to each other. As a result, if you change smart camera models to a new version or need to move to a different system hardware or operating system platform, a complete rewrite of the application or IP should not be required.
6. Distributed Versus Standalone
This aspect is one of the key differentiators between when to use a smart camera versus another type of a vision system. By a strict definition, the processing unit and image sensor are integrated into one device (which includes a host of benefits mentioned previously), but there are other times when multiple cameras are needed for the vision application. Synchronizing the outputs of multiple processing units can be more difficult than making sure multiple cameras are connected to the same processing unit. Also, this can lead to an increase in system costs.
The ability to upgrade hardware can also be a challenge with smart cameras since proprietary hardware is often used and the components are tightly coupled together. To retrieve extra memory or change the image sensor, the entire smart camera may need to be replaced with a different unit. Upgradability and distributed imaging are often reasons to use a PC-based system or other type of vision system.
7. Ruggedness
The level of ruggedness required is dependent upon the environment in which the smart camera is to be deployed; many applications take place in fairly harsh environments. Whether a manufacturing line or the outdoors, environmental specifications must always be considered. The ingress protection (IP) rating can help with this decision (Figure 3). IP ratings are usually two-number ratings in which each number represents a certain protection level:
- Protection from solid material such as dust (on a scale between 0 and 6)
- Protection from liquids (on a scale between 0 and 8)
Figure 3. Smart cameras such as National Instruments' NI 1772 have high IP ratings for protection against dust, water and other environmental exposures. Higher IP ratings are beneficial for applications in harsh environments such as outdoor monitoring and industrial vision inspection.
The higher the number, the more protected the smart camera is from that specific ingress. For example, a camera with an IP rating of 40 can protect against solid objects over 1mm such as wires and operators fingers, but offers no protection against dust or water. A smart camera with an IP rating of 67 offers total protection against dust and can be submerged in water up to 1m deep. An example of where a higher IP rating can be beneficial is in food inspection, where many times the cleaning process for the line includes washing everything on the line (including the camera).
Another aspect of ruggedness is the temperature rating. To reduce the chance of failure, smart cameras are usually fanless, which increases heat dissipation complexity for the manufacturer, especially with higher performance smart cameras containing Atom-class processors and above that require more power. Wider range temperature ratings make it possible for the smart camera to operate in a broader range of environments, which can be very beneficial in applications such as traffic monitoring or other types of surveillance where the smart camera is located outside.
8. Integration
Those who have completed a vision application know that vision is often part of a much larger system. Since the primary output of a smart camera is a decision, result, or some other information beyond an image, most smart cameras have built-in I/O to communicate or control other devices in the system (Figure 4). With industrial automation, the smart camera may need to control actuators to sort products, communicate inspection results to a robot controller, programmable logic controller (PLC), or programmable automation controller (PAC), save images and data to network servers, or communicate inspection parameters and results to a local or remote user interface.
Figure 4. Since the primary output of a smart camera is a decision or some type of signal other than an image, many smart cameras include integrated I/O and communication ports. These ports can include digital I/O, serial, Ethernet, and USB buses as well as display ports for user interfaces.
With USB and display ports, smart cameras can completely replace PC-vision systems where an operator interface is required, since everything is integrated in a single device. Often, for scientific imaging applications, vision must integrate with motion stages, data acquisition systems, microscopes, specialized optics, and advanced triggering. As a result, today many smart cameras include I/O such as industrial digital inputs and outputs, encoder inputs for image synchronization, displays, and communication ports.
Models are also available with built-in lighting or with light controllers to control illumination directly from the smart camera, reducing the need for external illumination or light controllers and cabling. More and more industrial communication protocols are also being supported natively in smart cameras with DeviceNet, EthernetIP, serial, etc. to effectively communicate to other devices. It is critical to think about the big picture to understand how the smart camera will best integrate into the overall system.
With the capabilities of today’s smart cameras, the adoption of these devices continues to grow and newer application areas are emerging which could help feed that growth, such as the use of vision in robotics applications. Robotic cells are usually space-confined, with a consequent need to reduce the amount of cables as much as possible. Smart cameras can be mounted on the end effector of a robot or in a position to conduct guidance or inspection.
The smart camera can essentially become the system master, performing image acquisition and outputting coordinates to the robot with in-line quality inspection or carrying out visual servoing, which gives continuous position feedback to the robot to align or track parts and locations. Smart cameras are also moving into the 3D vision space by providing solutions with multiple image sensors integrated into stereoscope or laser triangulation packages (Figure 5). The potential cost savings, ease of integration, and increasing performance makes smart cameras a cutting-edge option for many vision applications.
Figure 5. Smart cameras are now moving into emerging application areas such as 3D vision. 3D smart cameras incorporate the image sensor, laser, and processing into one device to report information about objects' height, shape and volume.
