Edge AI and Vision Insights: November 6, 2024

LETTER FROM THE EDITOR

Dear Colleague, 

On Tuesday, December 3, 2024 at 9 am PT, the Yole Group will present the free webinar “The CMOS Image Sensor Industry: Technology Trends and Emerging Applications” in partnership with the Edge AI and Vision Alliance. The CMOS image sensor (CIS) market, which is projected to grow at a 4.7% compound annual growth rate from 2023 to 2029, growing to $28.6 billion, is undergoing a transformation. Declining smartphone sales, along with weakening demand in devices such as laptops and tablet computers, are key challenges to growth. The Yole Group forecasts that automotive cameras and other emerging applications will instead be the key drivers of future CIS market growth. Technology innovations such as triple-stacked architectures and single-photon avalanche diode-based sensors are improving performance, enabling new applications in low light and 3D imaging, for example, while high dynamic range and LED flicker mitigation are key requirements for automotive image sensors.

This webinar, co-presented by the Yole Group’s Florian Domengie, Principal Analyst for Imaging, and Anas Chalak, Technology and Market Analyst for Imaging, will discuss how CIS suppliers are focusing on enhancing sensor capabilities, along with shifting their product mixes towards higher potential value markets. Domengie and Chalak will also explore how emerging sensing modalities such as neuromorphic, optical metasurfaces, short-wave infrared and multispectral imaging will supplement, and in some cases supplant, CMOS image sensors in the future. A question-and-answer session will follow the presentation. For more information and to register, please see the event page.

Brian Dipert
Editor-In-Chief, Edge AI and Vision Alliance

GENERATIVE AI CAPABILITIES

What’s Next in On-device Generative AI

The generative AI era has begun! Large multimodal models are bringing the power of language understanding to machine perception, and transformer models are expanding to allow machines to understand using multiple types of sensors. This new wave of approaches is poised to revolutionize user experiences, disrupt industries and enable powerful new capabilities. For generative AI to reach its full potential, however, we must deploy it on edge devices, providing improved latency, pervasive interaction and enhanced privacy. In this talk, Jilei Hou, Vice President of Engineering and Head of AI Research at Qualcomm Technologies, shares Qualcomm’s vision of the compelling opportunities enabled by efficient generative AI at the edge. He also identifies the key challenges that the industry must overcome to realize the massive potential of these technologies. And he highlights research and product development work that Qualcomm is doing to lead the way via an end-to-end system approach—including techniques for efficient on-device execution of LLMs, LVMs and LMMs, methods for orchestration of large models at the edge and approaches for adaptation and personalization.

Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics

At a high level, sport analytics systems can be broken into two components: sensory data collection and analytical models that turn sensory data into insights for users. In this presentation, Mehrsan Javan, Chief Technology Officer at Sportlogiq, focuses on the latter, and more specifically on the challenges his company has encountered in adapting advanced analytics originally developed for professional leagues to create a new product for use in a new market—youth sports. These challenges arise due to the unfamiliarity of end users with sophisticated analytical metrics, incomplete and partially accurate underlying visual data and the inherent limitations of vision-based data collection systems. Javan explains how Sportlogiq uses a combination of vision systems, generative models and reinforcement learning techniques to develop compelling products for youth sports, and shares what they’ve learned in this process.

RETAIL OPPORTUNITIES

Enabling Smart Retail with Visual AI

Automated checkout systems are on the rise—preferred by customers and businesses alike. However, most systems rely on the customer scanning one product at a time and identify the product using a barcode. This adds friction to the process. The Stockwell cabinet from 365 Retail Markets eliminates this friction by using visual AI to recognize items from a catalog. In this talk, Himanshu Vajaria, Engineering Manager at 365 Retail Markets, explains how the system works, the design decisions his company faced, and the operational and technical challenges and opportunities in the “Visual AI for Retail” space. After a brief overview of the business space, he delves into key technical aspects, including choice of camera and illumination, compute at the edge vs. the cloud, the AI inference pipeline and the data-labeling infrastructure for onboarding new products. He concludes with a look at collaboration opportunities in hardware, AI and data management.

Embedded Vision Opportunities and Challenges in Retail Checkout

In this interview, Anatoly Kotlarsky, Distinguished Member of the Technical Staff in R&D at Zebra Technologies, discusses computer vision challenges and opportunities in grocery stores and other retail environments, especially in traditional checkout and self-checkout applications. He highlights key challenges that have inhibited the use of computer vision, including performance, accuracy, cost, ergonomics, power consumption and the use of plastic produce bags. Kotlarsky explains some of the approaches that are used to address these challenges, discusses edge-vs.-cloud trade-offs and provides insights for technology suppliers on what they can do to help make computer vision ubiquitous in these applications.

UPCOMING INDUSTRY EVENTS

The CMOS Image Sensor Industry: Technology Trends and Emerging Applications – Yole Group Webinar: December 3, 2024, 9:00 am PT

Embedded Vision Summit: May 20-22, 2025, Santa Clara, California

More Events

FEATURED NEWS

MIPI Alliance Releases Camera Security Specifications for Flexible End-to-end Protection of Automotive Image Sensor Data

NXP Semiconductors Expands Edge AI Capabilities with eIQ Software Enablement

Basler Presents Small and Fast Line Scan Cameras for Mainstream Applications

Chips&Media Unveils New Multi Video Codec IP, WAVE6 Gen2+

e-con Systems Launches a 4K RGB-IR USB Camera Powered by Proprietary RGB-IR Separation Technology for Diverse Embedded Vision Applications

More News

EDGE AI AND VISION PRODUCT OF THE YEAR WINNER SHOWCASE

Ambarella Central 4D Imaging Radar Architecture (Best Edge AI Software or Algorithm)

Ambarella’s central 4D imaging radar architecture is the 2024 Edge AI and Vision Product of the Year Award Winner in the Edge AI Software and Algorithms category. It is the first centralized 4D imaging radar architecture that allows both central processing of raw radar data and deep low-level fusion with other sensor inputs—including cameras, lidar and ultrasonics. The central 4D imaging radar architecture combines Ambarella’s highly efficient 5 nm CV3-AD AI central domain controller system-on-chip (SoC) and the company’s Oculii adaptive AI radar software. This architecture’s optimized hardware and software provides the industry’s best AI processing performance per watt, for the lowest possible energy consumption, along with the most accurate and comprehensive AI modeling of a vehicle or robot’s surroundings. Ambarella’s Oculii AI radar algorithms uniquely adapt radar waveforms to the environment, achieving high angular resolution (0.5 degrees), an ultra-dense point cloud (10s of thousands of points/frame), and a long 500+ meters detection range, while using an order-of-magnitude fewer antennas for reduced data bandwidth and power consumption versus competing 4D imaging radars. Likewise, this architecture enables processor-less edge radar heads, further reducing both upfront costs and post-accident expenses (most radar modules are located behind the vehicle’s bumpers).

Please see here for more information on Ambarella’s central 4D imaging radar architecture. The Edge AI and Vision Product of the Year Awards celebrate the innovation of the industry’s leading companies that are developing and enabling the next generation of edge AI and computer vision products. Winning a Product of the Year award recognizes a company’s leadership in edge AI and computer vision as evaluated by independent industry experts.