By Alexandra Dopplinger
Global Segment Marketing Manager, Freescale Semiconductor
and Brian Dipert
Editor-In-Chief
Embedded Vision Alliance
Senior Analyst
BDTI
This article was originally published by Electronic Products Magazine. It is reprinted here with the permission of the original publisher.
If you order a pair of shoes (or a dozen) from Zappos (the largest online footwear retailer in America), a robot will play a key role in ensuring that the right shoes arrive at your door when promised (www.youtube.com/watch?v=Fdd6sQ8Cbe0). Several years ago, Zappos became a pioneering adopter of warehouse material-handling technology from robot manufacturer Kiva Systems (www.youtube.com/watch?v=lWsMdN7HMuA). Now, hundreds of small orange Kiva robots scurry around Zappos’ warehouses in a complex choreography, carrying inventory shelves containing every type of shoe to the human operators who pack them into shipping boxes (see Fig. 1).
Fig. 1: Kiva Systems’ robots make warehouse management more efficient.
Multiple robots could not rapidly crisscross the warehouse floor without a robust control and collision avoidance system. Kiva robots navigate using scanners that read stickers on the warehouse floor and barcodes on the bases of the mobile inventory shelves. They communicate their position and receive real-time dispatch instructions from servers using Wi-Fi. The robots’ applications processor handles motor control, sensor inputs, and automation and safety algorithms to move warehouse inventory with exceptional speed, ease, and precision.
Now that Amazon has purchased Kiva Systems, your next Kindle eReader order might also be carted around the warehouse by Kiva robots. Conversely, you may be surprised to learn that your Kindle, smartphone, tablet computer, and laptop PC are likely still assembled by human beings at large electronics manufacturers such as Foxconn, Inventec, FIC, Compal, and Pegatron. But it shouldn’t be too long before humanoid robots like FRIDA from the ABB Group (www.abb.com) begin doing the most tedious electronics assembly tasks, completely reliably, 24 hours a day without rest, food, or emotional distress. This transition will boost productivity and free human workers for other roles, such as maintaining the robots.
Many humanoid manufacturing robots mimic the upper torso of a factory worker, complete with shoulders, arms, elbows, wrists, fingers, thumbs, and advanced machine vision to guide the numerous precise motions needed for manufacturing miniscule electronics components. The embedded vision systems in manufacturing robots have a camera connected to a high-performance image processor for object recognition and a real-time control system for precise component placement. Such complex machine vision algorithms often run on the AltiVec SIMD (single-instruction multiple-data) vector-processing engine within Freescale processors such as MPC7448 and (soon) QorIQ Advanced Multiprocessing (AMP) series processor family (www.freescale.com/webapp/sps/site/overview.jsp?code=DRPPCALTVC). AltiVec is a floating-point and integer SIMD instruction set designed and owned by Apple, IBM, and Freescale Semiconductor.
If robots like FRIDA give you pause because of their humanoid behavior, here’s a heart-warming story about a robot that became an extension of an ill child in Texas. When high school student Lyndon Baty became too ill to attend school in Knox City, TX, his parents turned to a robot called VGo (http://today.msnbc.msn.com/id/26184891/vp/41641714 – 41641714). VGo is a telepresence robot that allows users to virtually attend events from a distance (www.freescale.com/files/industrial/doc/brochure/VGORBTBR.pdf). Although Lyndon’s autoimmune system made it too dangerous to be around other students at his school, his telepresence robot allowed him to continue to visually and verbally interact with schoolmates and teachers in spite of not being physically present at the school (see Fig. 2). Virtual attendance is superior to video conferencing because you can move the robot to various rooms and events just as you would move your own body as if you were actually present.
Fig. 2: VGo enables you to virtually attend events even if you can’t be there in person.
VGo robots don’t visually interpret their environments. Instead, users remotely control them, seeing through the robots’ video cameras and steering them with a computer mouse, similar to the way you’d drive a remote control toy car. Infrared sensors on each robot help dock it at the battery charger (and warn if it gets too close to tall objects), pressure sensors detect collisions, and accelerometers determine if the robot has fallen over. In the near future, telepresence robot cameras will connect to real-time machine vision processors and complex sensors to navigate autonomously between meeting rooms and battery chargers. This will save time and allow the robots to navigate through areas inside buildings that lack Wi-Fi, cellular, or GPS coverage.
Significant advances in robotic vision are enabled by more efficient image processors and new vision algorithms developed from many years of scientific research, such as the DARPA Mind’s Eye project (www.darpa.mil/Our_Work/I2O/Programs/Minds_Eye.aspx). Such efforts will allow automated systems to recognize objects and execute actions never seen before, deduce what might be around hidden corners, identify faces in human conversations, and predict what might happen next in various situations.
As the technology’s cost and power consumption both continue to decrease, service robots in homes will include embedded vision. Early-generation service robots such as iRobot’s Roomba vacuum and Scooba floor washer use pattern-based cleaning programs that reroute the robot in response to collisions detected by a pressure sensor. Elementary infrared “cliff” detectors prevent the robot from driving down stairs, and allow it to be contained in a defined space. Augmenting service robots with embedded vision will allow them to analyze a room to determine more efficient cleaning routes based on room size and shape, and location of contents.
More generally, embedded vision technology will enable a wide range of electronic products, including autonomous robots of all shapes, sizes, prices, and functions, that are more intelligent and responsive, and thus more valuable. It can add helpful features to existing products. And it can provide significant new markets for hardware, software, and semiconductor manufacturers.
The Embedded Vision Alliance, a worldwide organization of technology developers and providers, is working to empower engineers to transform this potential into reality in a rapid and efficient manner. The mission of the Alliance is to provide engineers with practical education, information, and insights to help them incorporate embedded vision capabilities into products. The Alliance’s website (www.embedded-vision.com) is freely accessible to all and includes articles, videos, a daily news portal, and a discussion forum staffed by a diversity of technology experts. Registered website users can receive the Embedded Vision Alliance’s e-mail newsletter, Embedded Vision Insights (www.embeddedvisioninsights.com), and have access to the Embedded Vision Academy (www.embeddedvisionacademy.com), containing numerous training videos, technical papers, and file downloads, intended to enable those new to the embedded vision application space to rapidly ramp up their expertise.
A few examples of the website’s content include:
- The BDTI Quick-Start OpenCV Kit, a VMware image that includes OpenCV and all required tools pre-installed, configured, and built, thereby making it easy to quickly get it running. OpenCV (Open Source Computer Vision) is a library of programming functions for real-time computer vision (www.embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImage).
- The BDTI OpenCV Executable Demo Package: a combination of a descriptive article, tutorial video, and Windows software demonstration (www.embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/introduction-computer-vision-using-op).
- A three-part video interview with Jitendra Malik, Arthur J. Chick Professor of EECS at the University of California at Berkeley and a computer vision academic pioneer (www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/malik-uc-interview-1).
