Building vision AI applications for the edge often comes with notoriously long and costly development cycles. At the same time, quickly developing edge AI applications that are cloud-native, flexible, and secure has never been more important. Now, a powerful yet simple API-driven edge AI development workflow is available with the new NVIDIA Metropolis microservices.
NVIDIA Metropolis microservices is a suite of customizable, cloud-native building blocks for developing vision AI applications and solutions. This release introduces an expanded set of APIs and microservices on the NVIDIA Jetson platform to further accelerate the development and deployment of vision AI applications at the edge.
These new Jetson microservices empower developers to modernize their AI application stack, streamline processes, and safeguard applications for the future. You can easily incorporate the latest in generative AI advancements through APIs and microservices such as video storage and management, prebuilt AI perception pipelines, tracking algorithms, system monitoring, IoT services for secure edge-to-cloud connectivity, and more.
Download NVIDIA Metropolis microservices for Jetson.
Figure 1. Develop edge AI applications faster with NVIDIA Metropolis microservices
Release highlights
Production-ready edge AI applications require numerous components, including AI models, optimized processing and inference pipelines, glue logic, security measures, cloud connectivity, and more. NVIDIA Metropolis microservices for Jetson streamline the application development process by offering pre-built microservices for the most ubiquitous components using a cloud-native, modular, and extensible architecture.
The flexibility of the platform enhances development efficiency, with standard APIs seamlessly integrating with other applications and services. The platform also provides essential services such as IoT, security, and monitoring, offering ready-to-use core components for production applications.
With access to more than 15 microservices across Application, Platform services, and Cloud services, developers are freed to concentrate on building intellectual property and achieving differentiation in the market.
NVIDIA partners are incorporating NVIDIA Metropolis microservices into their offerings, including AAEON, Aetina, Advantech, Allxon, CRG, CVEDIA, Namla, Rebotnix, RidgeRun, Seeed Studio, and Silicon Highway. More are added daily.
Reference workflows and applications
Two reference applications are included to help get you started with NVIDIA Metropolis microservices for Jetson: the AI-enabled network video recorder and a generative AI application with zero-shot detection. These workflows show how the microservices and APIs come together to build complete applications from video ingestion, AI inference, analytics, and monitoring to securely connecting to the cloud.
AI-enabled network video recorder
The AI-enabled network video recorder (AI-NVR) reference workflow brings nearly all the microservices together in one comprehensive app. It includes:
- Video ingestion and storage using the Video Storage Toolkit (VST) microservice
- People detection and tracking with the AI Perception service with NVIDIA DeepStream
- Line crossing and Region of Interest (ROI) insights and alerts using the Analytics service
An Android reference mobile application is provided to demonstrate the use of APIs to build client applications. To learn more, check out the NVIDIA On-Demand playlist, AI-NVR Using Metropolis Microservices for Jetson.
Figure 2. The AI-enabled network video recorder (AI-NVR) application architecture
Zero-shot detection using generative AI
Metropolis microservices for Jetson enables developers to prototype and productize generative AI applications for the edge. The generative AI reference application enables zero-shot detection of live streaming data. Models can detect any objects specified with a prompt.
Prompts can be made remotely over REST APIs to the AI Perception service to dynamically change classes to detect. Generative AI enables a new breed of AI-powered applications for the edge. To learn more about generative AI with Metropolis microservices, see Bringing Generative AI to the Edge with NVIDIA Metropolis Microservices for Jetson.
Figure 3. Generative AI reference application for zero-shot object detection
Powerful microservices and APIs
Metropolis microservices for Jetson is a collection of feature-rich microservices and APIs, including application services, platform and Board Support Package (BSP) services, and cloud services. The modular and extensible microservices make it easy to build modern cloud-native applications for the edge.
As a developer, you have the flexibility to choose one, several, or all of the services depending on the maturity of your product.
Figure 4. Metropolis microservices for Jetson software stack
Application services
- Video Storage Toolkit: Service for video ingestion and storage
- AI Perception service using NVIDIA DeepStream: For AI inference, object tracking and metadata creation
- AI Perception service for zero-shot detection: For generative AI inference with the NanoOWL model and visual prompting
- Analytics service: Object counting analytics such as line crossing, Region of Interest, and Field of View
Platform services
- Redis: Global message bus for inter-process communication
- API Gateway: Maps incoming API requests to the subsequent services
- Monitoring: Monitor and visualize edge device status such as utilization and app KPIs
- IoT Gateway: Secure bidirectional communication between edge and cloud
Cloud services
- IoT Cloud: Create secure connection from cloud to edge, including authentication and authorization
Summary
NVIDIA Metropolis microservices fast-tracks vision AI development for the edge, providing ready-to-use applications, over 15 microservices for platform services, pixel perception, video storage, analytics, and more. Download NVIDIA Metropolis microservices for Jetson.
To learn about the technical details of Metropolis microservices for Jetson, read the NVIDIA Metropolis Microservices for Jetson whitepaper.
For a tutorial on building applications using Metropolis APIs, see Build Vision AI Applications at the Edge with NVIDIA Metropolis Microservices and APIs. To learn even more, register to join us for the two-part webinar, Accelerate Edge AI Development With Metropolis APIs and Microservices for Jetson (Part 1) and How to Build With Metropolis Microservices for Jetson (Part 2).
Related resources
- GTC session: A New Class of Cloud-Native Applications at the Far Edge with Generative AI
- GTC session: Accelerate Smart City Edge AI Deployment with Open-Source Cloud-Native Infrastructure
- SDK: Live Portrait Microservice
- SDK: Video Effects Microservice
- Webinar: Accelerate Edge AI Development With Metropolis APIs and Microservices on Jetson
- Webinar: Transform Your Vision AI Applications With Gen AI
Chintan Shah
Senior Product Manager, NVIDIA
Debraj Sinha
Product Marketing Manager for Metropolis, NVIDIA
