This article was originally published at Texas Instruments' website. It is reprinted here with the permission of Texas Instruments.
Today, automotive manufacturers are introducing new vehicle control features to save fuel, add security and convenience, and above all make cars much safer to drive. These changes, designed to assist drivers and make vehicles more autonomous in operation, are based on new developments in sensing and high-performance processing enabled by advanced integrated circuit technology. Texas Instruments (TI) is driving innovation in analog and embedded processing to shape the future of automotive electronics today.
Introduction
Cars that drive themselves have long been a dream of futurists. Experiments with remote control date back nearly a century, and automated highways were talked about in the 1939 New York World’s Fair as part of “The World of Tomorrow.” But hopes for fully autonomous vehicles remained out of reach until recently, when the availability of new electronic technologies suddenly turned the fantasies of the past into present-day realities.
The general public is aware of Google’s development of self-driving technology, which has been widely reported. Less well-known, however, is the extensive work on assisted driving by major auto makers worldwide, and the semiconductor innovations enabling them. Numerous developments are rapidly changing car design, providing an evolution in automotive control that will put semi-autonomous, then fully- autonomous vehicles on the roads in just a few years.
Semi-autonomous and fully-autonomous vehicle control, based on advanced electronic sensing and processing, are valuable for more than just the excitement that comes with a technological break- through. They will deliver real benefits in fuel savings, mobility and convenience, travel time, and the efficient use of roadways. Most important, however, are new forms of vehicle control that will work actively to promote safety.
Addressing the need for automotive safety
According to governmental agencies, every year some 34,000 people in the United States and about 1.24 million people worldwide die in traffic accidents. Automotive accidents remain the leading cause of death for young people in the U.S. and are high on the list of causes of death for the overall population. Above and beyond the fatalities are the even greater number of injuries and the high cost of repairs associated with car mishaps. Since traffic accidents are overwhelmingly caused by human error—as much as 90 percent, according to some estimates—assisting drivers to control their vehicles more safely is an obvious point of attack for reducing these deaths and damages.
Vehicle control represents not only a remarkable opportunity to enhance road safety, but also a thriving market for firms that offer the enabling electronic technology. Active safety systems represent a fast-growing portion of the $26 billion spent today for electronic components in automobiles worldwide. Leading-edge semiconductor solutions will help speed the introduction of these new capabilities, ushering in greater safety while sharing in this significant market.
Safety enhancements depend on advanced technology
Active safety depends on, among other things, Advanced Driver Assistance Systems (ADAS), a set of electronics- based technologies that are designed to aid in safe vehicle operation. ADAS innovations help prevent accidents by keeping cars at safe distances from each other, alerting drivers to dangerous conditions, protecting those in the car and on the street from bad driving habits, and performing other safety-related operations. ADAS also provides functions that will serve as important elements of computer-controlled autonomous operation in the future. If self-driving cars promise to free operators to use their time more effectively during commutes and longer trips, ADAS features will help them minimize collision repairs, prevent injuries and save lives.
Auto manufacturers are racing to introduce driver assistance capabilities into new cars, with the market availability of many features increasing significantly every year. To provide ADAS and automated vehicle control, auto
makers must rely on leading semiconductor suppliers for a range of advanced integrated circuit technologies that can accurately and reliably support a variety of external sensors, communicate among the car’s different systems, and provide high-performance, heterogeneous processing for the computer vision and decision-making that are needed in vehicle control. Among the leading suppliers car manufacturers depend on is TI, which provides a wide portfolio of analog and digital products for ADAS and automated control and is innovating to provide solutions for future developments.
Figure 1. Analog and embedded processing technologies enable a range of active safety capabilities today, and will be crucial in future safety applications.
Vehicle safety automation evolves
Self-driving cars remain the ultimate “gee whiz” vision, yet some degree of safety automation has been with us for quite a while in features such as stability control, anti-lock braking, airbags, occupant detection and various kinds of alarms. Building on these traditional capabilities, active safety technology, including ADAS, is evolving through four stages. First comes passive warning and convenience systems. Examples include cameras and display for rear view, radar for blind spot detection, and cross-traffic warning that aids in, say, backing out of a parking space between larger vehicles. Some warning systems may include camera image processing for traffic sign recognition, surround views of the car on the road, in-cabin monitors to alert distracted drivers or inattentive drivers who are falling asleep, and other advanced features.
In the second stage of development, these systems can briefly take active control of the car to assist in parking, prevent backing over unseen objects, and avoid collisions by braking or swerving. Sometimes the system actively controls an individual car function, such as adapting front headlights automatically to upcoming curves and other changing conditions.
The third stage involves semi-autonomous operation, when the car takes over driving in certain circumstances, though someone has to stay in the driver’s seat ready to resume control. One example is traffic assistance or highway lane self-driving, including adaptive cruise control, which changes speeds automatically to keep pace with traffic on expressways. Both adaptive cruise control and lane keep assistance will be required for expressway driving to keep the car centered in the lane and at a safe distance from other drivers. Lane keep assistance is another example, where a front or rear camera is used to guide the car along the middle of the lane. Park assist will take full control during parking in crowded parking lots and garages. Additionally, in-cabin driver monitoring, when detecting an incapacitated driver, may initiate a fully automatic stopping maneuver to pull a car to the side of the road and stop it safely.
Finally, when cars move to fully autonomous operation in the fourth stage of development, there may be no one in the driver’s seat at all. Instead, the only occupant may be an elderly or handicapped person in the back seat, or the car may be empty as it goes to pick up someone at school or the airport.
Each of these four stages builds on the ones before it, fusing existing safety systems into new ones that are more complex. Today, most new cars appear with passive and even some active ADAS safety features, and availability is increasing rapidly. For instance, according to the market research firm IHS Inc., adaptive cruise control is available in almost 25 percent of new cars worldwide, side object detection (also known as blind spot detection) in more than 20 percent, lane departure warning (lane keep assistance) in nearly 20 percent, and autonomous park assist (assisted parking) in ten percent. Features for semi-autonomous operation are starting to appear on high-end vehicles today, while more advanced and more capable semi-autonomous systems are projected to appear in the latter half of this decade. Fully autonomous cars, still experimental today, are expected to follow during the early to middle years of the next decade.
As with almost all innovations, ADAS features tend to be introduced in high-end vehicles first, then migrate down to medium-priced and economy cars. In some cases, such as rear-view cameras, commercial vehicles have pioneered adoption because these are features especially valuable in the safe operation of large trucks.
Driving a social transformation
While ADAS features are steadily finding acceptance today, moving to semi-automated and fully automated operation faces social, legal, as well as technological challenges. as well as technological challenges. In the past, safety measures have often been driven by legislative and insurance mandates, but automated operation presents a new issue in that the car can take control away from the driver. As a result, law makers and the courts will be forced to determine liability issues, a factor that may slow introduction. Add to this a period of adjustment as the public becomes accustomed to the changes in control and safety and learns to trust automated driving, plus the long lag time as cars equipped with the new systems will have to share the roads with older, unequipped cars. Eventually, smart roadways that communicate with cars will also be deployed, adding yet another factor to that will make for a gradual transition.
If these legal and social issues require time to resolve, so do the challenges facing technologists. Cars require electronic safety systems to be small, light-weight and inexpensive, yet offer high performance and reliability. The trunks of the self-driving automobiles that are so widely publicized today are packed with advanced electronics that are worth much more than the host vehicles themselves. To change these experimental cars into production units will require reliable and powerful, yet affordable electronics that are much smaller in size.
Just as important are system safety and reliability, not only for the benefits of safe driving, but also because accidents that are caused by electronics failure can delay innovation and acceptance of the new technologies. Reliability is especially critical in a constrained automotive environment, where high temperatures, large voltage swings and vibrations can unduly stress electronic components. In such a demanding environment, systems must have a failure recovery mode that remains safe to persons in and around the vehicle if something goes wrong.
These design challenges take time to address, and the resulting systems require additional time to test effectively on the road. Technology providers need to be prepared to work closely with auto manufacturers throughout these lengthy phase-in periods, a commitment that may be difficult for vendors who are new to the automotive field.
Innovating to deliver the enabling IC technologies
Assisted and automated driving relies on multi-modal systems with input from a variety of sensors, including ultrasound, radar, LIDAR (LIght Detection and Ranging) and cameras (color, monochrome, stereo, and infrared- night vision). Satellite communications, as well as radio communications with nearby cars (vehicle-to-vehicle) and terrestrial installations (vehicle-to-infrastructure), are also necessary for positioning, localization, highway conditions and other information.
Working in real time, as events on the street occur, ADAS and automated driving systems will have to convert these varied inputs into useful data forms and extract whatever information is necessary. Then the systems must merge information from different sources, decide the correct control action, and communicate appropriately to the driver or automatic control output. In addition to requiring high fidelity in the inputs themselves, the systems will rely on high- performance computation that can run a variety of algorithms concurrently. Given the pressures of developing new systems under intense market competition, the software used will have to be abstracted, refined and reused in later systems as active safety capabilities evolve.
Figure 2. Major ADAS sensor types and typical vehicle positions.
Computer vision and heterogeneous processing
Such a range of electronic functions, from sensing, through conversion and transmission, to high-performance processing, requires the full spectrum of analog and embedded processing ICs. Vision processing and digital signal processing expertise, along with the necessary foundation software, are necessary for supporting rapid algorithm and application development. For the signal chain and power supply, an extensive portfolio of power management ICs, sensor signal conditioning, interfaces and transceivers are also needed to support virtually all forms of ADAS sensing.
With its heavy reliance on cameras and other imaging sensors, assisted or autonomous driving requires a great deal of high-performance vision processing, which by nature is heterogeneous. Typically, low-level processing concentrates on pixel data to create useful images for further processing. Mid-level processing identifies objects of interest in the images, high-level processing uses the resulting information to recognize these objects, and microcontroller decides what the system should do.
For example, while low-level processing provides a steady stream of pre-filtered or pre-conditioned video images
of a road, mid-level processing identifies sections of the images where important objects could be located. Next, high-level processing determines what types of objects these are, such as other vehicles, people, animals, signs or traffic lights, plus how fast someone or something is moving. Finally, the microcontroller decides whether to go, stop, or wait until the pedestrian leaves, the light changes, or a nearby car passes. Concurrently, data streams from other input sensors, such as radar, are being examined for information in case there are overriding conditions, such as poor visibility due to fog. Because any given sensor may face challenging external conditions, inputs from different data streams are fused to greatly increase precision and reliability.
Low-level processing tends to use relatively simple repetitive algorithms that operate in parallel to handle massive amounts of input data. High-level processing has comparatively little data but complicated algorithms, and mid-level processing is between these extremes in both data size and algorithm complexity. Each level is optimally performed by a different processing architecture, with SIMD/MIMD (single-/multiple-instruction, multiple data), VLIW (very long instruction word), and RISC (reduced instruction set computing) processor types corresponding to the low- to high-level progression. Control logic is best implemented with RISC processors.
Figure 3. As complexity increases, dedicated, specialized vision processing allows the most performance and power efficient vision processing.
TI’s ADAS TDA2x system-on-chip (SoC) technology, for example, spans this range with a programmable Vision Accelerator containing one ore more Embedded Vision Engines (EVE), specialized for handling the massive data of video systems, ISP (image signal processor) for camera preprocessing, DSP (digital signal processor) for more general signal processing, and RISC options that include several ARM® microprocessors. In addition to its own software frameworks, TI is a key contributor to the Khronos OpenVX standard for computer vision acceleration that addresses the need for low-power, high-performance processing on embedded heterogeneous processors. TI’s system solution is scalable throughout.
Automotive expertise for building safety systems
TI’s SoC portfolio, including microcontrollers, multicore DSPs and heterogeneous vision processors.
In their ongoing development of ADAS and automated driving features, car manufacturers look for technology and logistics strength in choosing a semiconductor vendor. Meeting these requirements well, TI became an early sup- plier for active safety semiconductor systems. The company’s role in supplying active safety technology builds on a decades-long relationship with auto makers, dating back to development of the earliest anti-lock brakes.
Supply logistics can be a barrier to new entrants in the automotive industry, since they lack in-depth compliance experience with relevant automotive safety standards such as ISO/TS16949 for quality management and ISO 26262 for functional safety. Hundreds of millions of safety MCUs shipped to date testify to TI’s safety expertise, which extends through all stages of analog and digital design and manufacture. Automotive customers can also rely on the company’s worldwide design support and manufacturing footprint to help speed system development and ramping to volume production. Multi-sourced global fabrication assures customers of reliable, long-term product supply.
With these capabilities, TI is well-positioned to deliver innovative technology solutions for active safety development and manufacturing, from the simplest driver assistance to the fully autonomous vehicles of the future.
Technology for future safety, appearing today
Fully autonomous cars will not appear in significant numbers on the roads for a decade or so, and more time will pass before virtually all new vehicles are capable of driving themselves. But in the meantime, a steadily increasing number of vehicles will offer some level of assisted driving to help make them safer while reducing fuel consumption and adding convenience for operators.
Enabling these active safety features depends on innovation in advanced electronics, including sensing advancements for external input, heterogeneous high-performance processing to evaluate driving conditions and aid decision- making, and a variety of analog components for signal conditioning, communications and system power control. Auto manufacturers need reliable supplies of these IC devices from semiconductor vendors, as well as strong design support and the expertise required to advance technology in the years ahead.
Car makers know that they can turn to TI for the solutions they need for active safety systems. What the world will be like with self-driving cars is sometimes difficult to imagine, but one thing about it is certain: TI’s advanced
IC technology and culture of innovation will play a significant role in bringing it about.
For more information on how TI is innovating to ensure safety and security, visit www.ti.com/innovation-safety.
By Roman Staszewski
Distinguished Member Group Technical Staff Texas Instruments
Hannes Estl
Worldwide Systems Marketing for Advanced Driver Assistance Systems (ADAS) Texas Instruments
