Vision Algorithms for Embedded Vision
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language. Some of the pixel-processing operations (ex: spatial filtering) have changed very little in the decades since they were first implemented on mainframes. With today’s broader embedded vision implementations, existing high-level algorithms may not fit within the system constraints, requiring new innovation to achieve the desired results.
Some of this innovation may involve replacing a general-purpose algorithm with a hardware-optimized equivalent. With such a broad range of processors for embedded vision, algorithm analysis will likely focus on ways to maximize pixel-level processing within system constraints.
This section refers to both general-purpose operations (ex: edge detection) and hardware-optimized versions (ex: parallel adaptive filtering in an FPGA). Many sources exist for general-purpose algorithms. The Embedded Vision Alliance is one of the best industry resources for learning about algorithms that map to specific hardware, since Alliance Members will share this information directly with the vision community.
General-purpose computer vision algorithms
One of the most-popular sources of computer vision algorithms is the OpenCV Library. OpenCV is open-source and currently written in C, with a C++ version under development. For more information, see the Alliance’s interview with OpenCV Foundation President and CEO Gary Bradski, along with other OpenCV-related materials on the Alliance website.
Hardware-optimized computer vision algorithms
Several programmable device vendors have created optimized versions of off-the-shelf computer vision libraries. NVIDIA works closely with the OpenCV community, for example, and has created algorithms that are accelerated by GPGPUs. MathWorks provides MATLAB functions/objects and Simulink blocks for many computer vision algorithms within its Vision System Toolbox, while also allowing vendors to create their own libraries of functions that are optimized for a specific programmable architecture. National Instruments offers its LabView Vision module library. And Xilinx is another example of a vendor with an optimized computer vision library that it provides to customers as Plug and Play IP cores for creating hardware-accelerated vision algorithms in an FPGA.
Other vision libraries
- Halcon
- Matrox Imaging Library (MIL)
- Cognex VisionPro
- VXL
- CImg
- Filters
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO,” a Presentation from Intel
Leila Sabeti, Americas AI Technical Sales Lead at Intel, presents the “Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO” tutorial at the May 2024 Embedded Vision Summit. In this talk, Sabeti focuses on the transformative impact of AI at the edge, highlighting… “Transforming Enterprise Intelligence: The Power
“Challenges and Solutions of Moving Vision LLMs to the Edge,” a Presentation from Expedera
Costas Calamvokis, Distinguished Engineer at Expedera, presents the “Challenges and Solutions of Moving Vision LLMs to the Edge” tutorial at the May 2024 Embedded Vision Summit. OEMs, brands and cloud providers want to move LLMs to the edge, especially for vision applications. What are the benefits and challenges of doing… “Challenges and Solutions of Moving
“A Cutting-edge Memory Optimization Method for Embedded AI Accelerators,” a Presentation from 7 Sensing Software
Arnaud Collard, Technical Leader for Embedded AI at 7 Sensing Software, presents the “Cutting-edge Memory Optimization Method for Embedded AI Accelerators” tutorial at the May 2024 Embedded Vision Summit. AI hardware accelerators are playing a growing role in enabling AI in embedded systems such as smart devices. In most cases… “A Cutting-edge Memory Optimization Method
Decoding How NVIDIA AI Workbench Powers App Development
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Free tool lets developers experiment with, test and prototype AI applications. Editor’s note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools
“Implementing Transformer Neural Networks for Visual Perception on Embedded Devices,” a Presentation from VeriSilicon
Shang-Hung Lin, Vice President of Neural Processing Products at VeriSilicon, presents the “Implementing Transformer Neural Networks for Visual Perception on Embedded Devices” tutorial at the May 2024 Embedded Vision Summit. Transformers are a class of neural network models originally designed for natural language processing. Transformers are also powerful for visual… “Implementing Transformer Neural Networks for
“Optimized Vision Language Models for Intelligent Transportation System Applications,” a Presentation from Nota AI
Tae-Ho Kim, Co-founder and CTO of Nota AI, presents the “Optimized Vision Language Models for Intelligent Transportation System Applications” tutorial at the May 2024 Embedded Vision Summit. In the rapidly evolving landscape of intelligent transportation systems (ITSs), the demand for efficient and reliable solutions has never been greater. In this… “Optimized Vision Language Models for
Understanding What the Machines See: State-of-the-art Computer Vision at CVPR 2024
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Qualcomm’s accepted papers, demos and workshops at CVPR 2024 showcase the future of generative AI and perception The Computer Vision and Pattern Recognition Conference (CVPR) 2024 begins on Monday, June 17, and Qualcomm Technologies is excited to
“Image Signal Processing Optimization for Object Detection,” a Presentation from Nextchip
Young-Jun Yoo, Executive Vice President at Nextchip, presents the “Image Signal Processing Optimization for Object Detection” tutorial at the May 2024 Embedded Vision Summit. This talk delves into the challenges and optimization strategies in image signal processing (ISP) for enhancing object detection in advanced driver-assistance systems (ADAS). Through real-world examples,… “Image Signal Processing Optimization for
Understanding Spatial Noise and Its Reduction Methods Using Convolution
This blog post was originally published at e-con Systems’ website. It is reprinted here with the permission of e-con Systems. Convolution is a mathematical operation used in image processing to apply filters to images. These filters are used for spatial noise reduction in images with variations or irregularities in the pixel values that are unrelated
“Temporal Event Neural Networks: A More Efficient Alternative to the Transformer,” a Presentation from BrainChip
Chris Jones, Director of Product Management at BrainChip, presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit. The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent… “Temporal Event Neural Networks: A
How Edge Devices Can Help Mitigate the Global Environmental Cost of Generative AI
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Exploring the role of edge devices in reducing energy consumption and promoting sustainability in AI systems The economic value of generative artificial intelligence (AI) to the world is immense. Research from McKinsey estimates that generative AI could add the
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Number Four Will Amaze You!),” a Presentation from BDTI
Phil Lapsley, Co-founder and Vice President of BDTI, presents the “Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Number Four Will Amaze You!)” tutorial at the May 2024 Embedded Vision Summit. For over 30 years, BDTI has provided engineering, evaluation and advisory services to processor suppliers and companies… “Silicon Slip-ups: The Ten Most
Upcoming Webinar Explores AI for Audio
On June 25, 2024 at 2:00 pm PT (5:00 pm ET), Alliance Member company MACSO Technologies will deliver the free webinar “AI for Audio.” From the event page: Come join the MACSO team and our partner community to dive into the intricacies of AI for audio. MACSO has a wealth of experience to share in
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision,” a Presentation from Axelera AI
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit. As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability… “How Axelera AI Uses Digital
Power Cloud-native Microservices at the Edge with NVIDIA JetPack 6.0, Now GA
This article was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. NVIDIA JetPack SDK powers NVIDIA Jetson modules, offering a comprehensive solution for building end-to-end accelerated AI applications. JetPack 6 expands the Jetson platform’s flexibility and scalability with microservices and a host of new features. It’s the most downloaded
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,” a Presentation from Arm
Stephen Su, Senior Segment Marketing Manager at Arm, presents the “How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge” tutorial at the May 2024 Embedded Vision Summit. AI at the edge has been transforming over the last few years, with newer use cases running more efficiently and securely.… “How Arm’s Machine Learning Solution
