Vision Algorithms for Embedded Vision
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language. Some of the pixel-processing operations (ex: spatial filtering) have changed very little in the decades since they were first implemented on mainframes. With today’s broader embedded vision implementations, existing high-level algorithms may not fit within the system constraints, requiring new innovation to achieve the desired results.
Some of this innovation may involve replacing a general-purpose algorithm with a hardware-optimized equivalent. With such a broad range of processors for embedded vision, algorithm analysis will likely focus on ways to maximize pixel-level processing within system constraints.
This section refers to both general-purpose operations (ex: edge detection) and hardware-optimized versions (ex: parallel adaptive filtering in an FPGA). Many sources exist for general-purpose algorithms. The Embedded Vision Alliance is one of the best industry resources for learning about algorithms that map to specific hardware, since Alliance Members will share this information directly with the vision community.
General-purpose computer vision algorithms
One of the most-popular sources of computer vision algorithms is the OpenCV Library. OpenCV is open-source and currently written in C, with a C++ version under development. For more information, see the Alliance’s interview with OpenCV Foundation President and CEO Gary Bradski, along with other OpenCV-related materials on the Alliance website.
Hardware-optimized computer vision algorithms
Several programmable device vendors have created optimized versions of off-the-shelf computer vision libraries. NVIDIA works closely with the OpenCV community, for example, and has created algorithms that are accelerated by GPGPUs. MathWorks provides MATLAB functions/objects and Simulink blocks for many computer vision algorithms within its Vision System Toolbox, while also allowing vendors to create their own libraries of functions that are optimized for a specific programmable architecture. National Instruments offers its LabView Vision module library. And Xilinx is another example of a vendor with an optimized computer vision library that it provides to customers as Plug and Play IP cores for creating hardware-accelerated vision algorithms in an FPGA.
Other vision libraries
- Halcon
- Matrox Imaging Library (MIL)
- Cognex VisionPro
- VXL
- CImg
- Filters
Computer Vision Quarterly Snapshot – Q3 2024
Woodside Capital Partners (WCP) is pleased to share its Computer Vision and Vision AI Market Report Q3 2024, authored by Managing Partner Rudy Burger, and Associate Akhilesh Shridar. Hardware startups generally take longer to get a product to market and require more investment than software companies. Hardware has a more complicated distribution and sales structure
How Qualcomm Technologies’ New Platforms Power the Most Demanding Applications in Extreme Environments
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. The convergence of physical and digital technologies is transforming industries and driving the need for more advanced platforms to power the next generation of industrial IoT (IIoT) devices. The result is a dramatic leap in demand for
“Cost-efficient, High-quality AI for Consumer-grade Smart Home Cameras,” a Presentation from Wyze
Lin Chen, Chief Scientist at Wyze, presents the “Cost-efficient, High-quality AI for Consumer-grade Smart Home Cameras” tutorial at the May 2024 Embedded Vision Summit. In this talk, Chen explains how Wyze delivers robust visual AI at ultra-low cost for millions of consumer smart cameras, and how his company is rapidly… “Cost-efficient, High-quality AI for Consumer-grade
“Edge AI Optimization on Rails—Literally,” a Presentation from Wabtec
Matthew Pietrzykowski, Principal Data Scientist at Wabtec, presents the “Edge AI Optimization on Rails—Literally” tutorial at the May 2024 Embedded Vision Summit. In this talk, Pietrzykowski shares highlights from his company’s adventures developing computer vision solutions for the rail transportation industry. He begins with an introduction to the types of… “Edge AI Optimization on Rails—Literally,”
Optimizing the CV Pipeline in Automotive Vehicle Development Using the PVA Engine
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. In the field of automotive vehicle software development, more large-scale AI models are being integrated into autonomous vehicles. The models range from vision AI models to end-to-end AI models for autonomous driving. Now the demand for computing
“How Large Language Models Are Impacting Computer Vision,” a Presentation from Voxel51
Jacob Marks, Senior ML Engineer and Researcher at Voxel51, presents the “How Large Language Models Are Impacting Computer Vision” tutorial at the May 2024 Embedded Vision Summit. Large language models (LLMs) are revolutionizing the way we interact with computers and the world around us. However, in order to truly understand… “How Large Language Models Are
“Implementing AI/Computer Vision for Corporate Security Surveillance,” a Presentation from VMware
Prasad Saranjame, former Head of Physical Security and Resiliency at VMware, presents the “Implementing AI/Computer Vision for Corporate Security Surveillance” tutorial at the May 2024 Embedded Vision Summit. AI-enabled security cameras offer substantial benefits for corporate security and operational efficiency. However, successful deployment requires thoughtful selection of use cases and… “Implementing AI/Computer Vision for Corporate
“Continual Learning thru Sequential, Lightweight Optimization,” a Presentation from Vision Elements
Guy Lavi, Managing Partner at Vision Elements, presents the “Continual, On-the-fly Learning through Sequential, Lightweight Optimization” tutorial at the May 2024 Embedded Vision Summit. In this presentation, Lavi shows how techniques of sequential optimization are applied to enable continual learning during run-time, as new observations flow in. The lightweight nature… “Continual Learning thru Sequential, Lightweight
Qualcomm and Mistral AI Partner to Bring New Generative AI Models to Edge Devices
Highlights: Qualcomm announces collaboration with Mistral AI to bring Mistral AI’s models to devices powered by Snapdragon and Qualcomm platforms. Mistral AI’s new state-of-the-art models, Ministral 3B and Ministral 8B, are being optimized to run on devices powered by the new Snapdragon 8 Elite Mobile Platform, Snapdragon Cockpit Elite and Snapdragon Ride Elite, and Snapdragon
Why Qualcomm AI Orchestrator is the Key to Next Generation AI Experiences
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. The software will help facilitate your personal preferences, apps and gen AI models to work together and provide revolutionary on-device AI capabilities In the rapidly evolving landscape of generative artificial intelligence (AI) and how we interact with
“Multi-object Tracking Systems,” a Presentation from Tryolabs
Javier Berneche, Senior Machine Learning Engineer at Tryolabs, presents the “Multiple Object Tracking Systems” tutorial at the May 2024 Embedded Vision Summit. Object tracking is an essential capability in many computer vision systems, including applications in fields such as traffic control, self-driving vehicles, sports and more. In this talk, Berneche… “Multi-object Tracking Systems,” a Presentation
“Improved Navigation Assistance for the Blind via Real-time Edge AI,” a Presentation from Tesla
Aishwarya Jadhav, Software Engineer in the Autopilot AI Team at Tesla, presents the “Improved Navigation Assistance for the Blind via Real-time Edge AI,” tutorial at the May 2024 Embedded Vision Summit. In this talk, Jadhav presents recent work on AI Guide Dog, a groundbreaking research project aimed at providing navigation… “Improved Navigation Assistance for the
Qualcomm Announces Multi-year Strategic Collaboration with Google to Deliver Generative AI Digital Cockpit Solutions
Highlights: Qualcomm and Google will leverage Snapdragon Digital Chassis and Google’s in-vehicle technologies to produce a standardized reference framework for development of generative AI-enabled digital cockpits and software-defined vehicles (SDV). Qualcomm to lead go-to-market efforts for scaling and customization of joint solution with the broader automotive ecosystem. Companies’ collaboration demonstrates power of co-innovation, empowering automakers
Flux and Furious: New Image Generation Model Runs Fastest on RTX AI PCs and Workstations
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Black Forest Labs’ latest models generate high-quality images and are highly performant on NVIDIA RTX GPUs. Editor’s note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and
“Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics,” a Presentation from Sportlogiq
Mehrsan Javan, Chief Technology Officer at Sportlogiq, presents the “Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics” tutorial at the May 2024 Embedded Vision Summit. At a high level, sport analytics systems can be broken into two components: sensory data collection and analytical models that turn sensory… “Using Vision Systems, Generative Models
Exploring the Next Frontier of AI: Multimodal Systems and Real-time Interaction
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Discover the state of the art in large multimodal models with Qualcomm AI Research In the realm of artificial intelligence (AI), the integration of senses — seeing, hearing and interacting — represents a frontier that is rapidly
