Nishita Sant, Computer Vision Manager, and Greg Chu, Senior Computer Vision Scientist, both of GumGum, present the “Creating Efficient, Flexible and Scalable Cloud Computer Vision Applications: An Introduction” tutorial at the May 2019 Embedded Vision Summit.
Given the growing utility of computer vision applications, how can you deploy these services in high-traffic production environments? Sant and Chu present GumGum’s approach to the infrastructure for serving computer vision models in the cloud. They elaborate on a few aspects, beginning with modularity of computer vision models, including handling images and video equivalently, creating module pipelines, and desiging for library agnosticism so we can leverage open source developments.
They also discuss inter-process communication—specifically, the pros and cons of data serialization, and the importance of standardized data formats between training and serving data, which lends itself to automated feedback from serving data for retraining and automated metrics. Finally, they discuss GumGum’s approaches to scaling, including a producer/consumer model, scaling triggers and container orchestration. They illustrate these aspects through examples of image and video processing and module pipelines.
