Timofey Uvarov, Camera System Lead at Pony.AI, presents the “Using an ISP for Real-time Data Augmentation” tutorial at the September 2020 Embedded Vision Summit.
Image signal processors (ISPs) are tasked with processing raw pixels delivered by image sensors in order to optimize the quality of images. In computer vision applications, much attention is focused on tuning the ISP, both to obtain good training data and to optimize image quality in deployed systems when faced with widely varying, dynamic imaging conditions (e.g., changes in lighting).
In this presentation, Uvarov describes a data-driven solution to these challenges developed in a collaboration between Pony.AI and On Semiconductor. His company’s approach manipulates the ISP configuration during training data collection in order to perform real-time data augmentation. By training deep neural networks (DNNs) with the resulting augmented data, Pony.AI is able to create DNNs that are robust to variations in imaging conditions and ISP settings.
See here for a PDF of the slides.
