Vision Algorithms for Embedded Vision
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language. Some of the pixel-processing operations (ex: spatial filtering) have changed very little in the decades since they were first implemented on mainframes. With today’s broader embedded vision implementations, existing high-level algorithms may not fit within the system constraints, requiring new innovation to achieve the desired results.
Some of this innovation may involve replacing a general-purpose algorithm with a hardware-optimized equivalent. With such a broad range of processors for embedded vision, algorithm analysis will likely focus on ways to maximize pixel-level processing within system constraints.
This section refers to both general-purpose operations (ex: edge detection) and hardware-optimized versions (ex: parallel adaptive filtering in an FPGA). Many sources exist for general-purpose algorithms. The Embedded Vision Alliance is one of the best industry resources for learning about algorithms that map to specific hardware, since Alliance Members will share this information directly with the vision community.
General-purpose computer vision algorithms
One of the most-popular sources of computer vision algorithms is the OpenCV Library. OpenCV is open-source and currently written in C, with a C++ version under development. For more information, see the Alliance’s interview with OpenCV Foundation President and CEO Gary Bradski, along with other OpenCV-related materials on the Alliance website.
Hardware-optimized computer vision algorithms
Several programmable device vendors have created optimized versions of off-the-shelf computer vision libraries. NVIDIA works closely with the OpenCV community, for example, and has created algorithms that are accelerated by GPGPUs. MathWorks provides MATLAB functions/objects and Simulink blocks for many computer vision algorithms within its Vision System Toolbox, while also allowing vendors to create their own libraries of functions that are optimized for a specific programmable architecture. National Instruments offers its LabView Vision module library. And Xilinx is another example of a vendor with an optimized computer vision library that it provides to customers as Plug and Play IP cores for creating hardware-accelerated vision algorithms in an FPGA.
Other vision libraries
- Halcon
- Matrox Imaging Library (MIL)
- Cognex VisionPro
- VXL
- CImg
- Filters
Deploying Accelerated Llama 3.2 from the Edge to the Cloud
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Expanding the open-source Meta Llama collection of models, the Llama 3.2 collection includes vision language models (VLMs), small language models (SLMs), and an updated Llama Guard model with support for vision. When paired with the NVIDIA accelerated
“Advancing Embedded Vision Systems: Harnessing Hardware Acceleration and Open Standards,” a Presentation from the Khronos Group
Neil Trevett, President of the Khronos Group, presents the “Advancing Embedded Vision Systems: Harnessing Hardware Acceleration and Open Standards” tutorial at the May 2024 Embedded Vision Summit. Offloading processing to accelerators enables embedded vision systems to process workloads that exceed the capabilities of CPUs. However, parallel processors add complexity as… “Advancing Embedded Vision Systems: Harnessing
When, Where and How AI Should Be Applied
Phil Koopman dissects strengths and weaknesses of machine learning based AI AI does amazing stuff. No question about it. But how hard have we really thought about “machine-learning capabilities” for applications? Phil Koopman, professor at Carnegie Mellon University, delivered a keynote on Sept. 11, 2024 at the Business of Semiconductor Summit, (BOSS 2024), concentrating on
“Using AI to Enhance the Well-being of the Elderly,” a Presentation from Kepler Vision Technologies
Harro Stokman, CEO of Kepler Vision Technologies, presents the “Using Artificial Intelligence to Enhance the Well-being of the Elderly” tutorial at the May 2024 Embedded Vision Summit. This presentation provides insights into an innovative application of artificial intelligence and advanced computer vision technologies in the healthcare sector, specifically focused on… “Using AI to Enhance the
AI Model Training Cost Have Skyrocketed by More than 4,300% Since 2020
Over the past five years, AI models have become much more complex and capable, tailored to perform specific tasks across industries and provide better efficiency, accuracy and automation. However, the cost of training in these systems has exploded. According to data presented by AltIndex.com, AI model training costs have skyrocketed by more than 4,300% since
BrainChip Demonstration of LLM-RAG with a Custom Trained TENNs Model
Kurt Manninen, Senior Solutions Architect at BrainChip, demonstrates the company’s latest edge AI and vision technologies and products at the September 2024 Edge AI and Vision Alliance Forum. Specifically, Manninen demonstrates his company’s Temporal Event-Based Neural Network (TENN) foundational large language model with 330M parameters, augmented with a Retrieval-Augmented Generative (RAG) output to replace user
Advex AI Demonstration of Accelerating Machine Vision with Synthetic AI Data
Pedro Pachuca, CEO at Advex AI, demonstrates the company’s latest edge AI and vision technologies and products at the September 2024 Edge AI and Vision Alliance Forum. Specifically, Pachuca demonstrates Advex’s ability to ingest just 10 images and produce thousands of labeled, synthetic images in just hours. These synthetic images cover the distribution of variations
“Testing Cloud-to-Edge Deep Learning Pipelines: Ensuring Robustness and Efficiency,” a Presentation from Instrumental
Rustem Feyzkhanov, Staff Machine Learning Engineer at Instrumental, presents the “Testing Cloud-to-Edge Deep Learning Pipelines: Ensuring Robustness and Efficiency” tutorial at the May 2024 Embedded Vision Summit. A cloud-to-edge deep learning pipeline is a fully automated conduit for training and deploying models to the edge. This enables quick model retraining… “Testing Cloud-to-Edge Deep Learning Pipelines:
AI PCs to Make Up 60% of Total PC Sales in 2027, 3x More than This Year
Although traditional PCs remain the number one choice for most consumers buying a new device, the surging demand for AI-driven applications has boosted the popularity of AI PCs, both in consumer and professional sectors. This trend is expected to continue in the following years, with AI PC gaining a much bigger market share than this
How AI and Smart Glasses Give You a New Perspective on Real Life
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. When smart glasses are paired with generative artificial intelligence, they become the ideal way to interact with your digital assistant They may be shades, but smart glasses are poised to give you a clearer view of everything
“Real-time Retail Product Classification on Android Devices Inside the Caper AI Cart,” a Presentation from Instacart
David Scott, Senior Machine Learning Engineer at Instacart, presents the “Real-time Retail Product Classification on Android Devices Inside the Caper AI Cart” tutorial at the May 2024 Embedded Vision Summit. In this talk, Scott explores deploying an embedded computer vision model on Android devices for real-time product classification with the… “Real-time Retail Product Classification on
Using Generative AI to Enable Robots to Reason and Act with ReMEmbR
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Vision-language models (VLMs) combine the powerful language understanding of foundational LLMs with the vision capabilities of vision transformers (ViTs) by projecting text and images into the same embedding space. They can take unstructured multimodal data, reason over
“Ten Commandments for Building a Vision AI Product,” a Presentation from Hayden AI
Vaibhav Ghadiok, Chief Technology Officer of Hayden AI, presents the “Ten Commandments for Building a Vision AI Product” tutorial at the May 2024 Embedded Vision Summit. Over the past three decades, the convergence of machine learning, big data and enhanced computing power has transformed the field of computer vision from… “Ten Commandments for Building a
Visionary.ai Demonstration of AI Bayer Denoising Software Delivering Crisp Images at 0.1 LUX
David Jarmon, SVP of Worldwide Sales for Visionary.ai, demonstrates the company’s latest edge AI and vision technologies and products at the September 2024 Edge AI and Vision Alliance Forum. Specifically, Jarmon demonstrates the company’s AI denoising solution—often described as “steroids for cameras”—on Qualcomm’s Snapdragon Gen-3 Android platform. In the demo, Visionary.ai’s real-time AI denoiser allows
Gimlet Labs Demonstration of Instant Deployment of Custom AI to a Device
Natalie Serrino, cofounder of Gimlet Labs, demonstrates the company’s latest edge AI and vision technologies and products at the September 2024 Edge AI and Vision Alliance Forum. Specifically, Serrino demonstrates Gimlet’s capabilities for deploying and monitoring edge AI workloads. She deploys a custom pipeline for hardhat compliance detection to an Intel NUC, and shows live
Ceva and Edge Impulse Join Forces to Enable Faster, Easier Development of Edge AI Applications
Partnership enables AI developers to train, optimize and deploy embedded ML models on the Ceva-NeuPro-Nano NPU IP, pre-silicon, via Edge Impulse Platform ROCKVILLE, Md., Sept. 24, 2024 /PRNewswire/ — Ceva, Inc. (NASDAQ: CEVA), the leading licensor of silicon and software IP that enables Smart Edge devices to connect, sense and infer data more reliably and
