Introducing Qualcomm Custom-built AI Models, Now Available on Qualcomm AI Hub

This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.

We’re thrilled to announce that five custom-built computer vision (CV) models are now available on Qualcomm AI Hub! Qualcomm Technologies’ custom-built models were developed by the Qualcomm R&D team, optimized for our platforms and designed with end-user applications in mind. These models offer high performance and efficiency, thanks to their platform-specific design and our deep understanding of customer needs.

Explore Qualcomm Technologies’ custom-built models

These models are available via the Qualcomm AI Hub, HuggingFace, and GitHub fostering innovation and collaboration within the developer community while providing a comprehensive solution for AI development and deployment.

To date, we have published 5 custom-built models through the AI Hub:

• Person Foot Detection model is a multi-task detector that not only locates people but also their key body parts, such as face and feet, in each image.
• PPE Detection model detects personal protective equipment (PPE) from a cropped person image. The supported PPE classes include hard-hat and safety vest, with more classes to be added in the future.
• Lightweight Face Detection model is an optimized network for face detection with high accuracy and high speed. Since face detection is often the first building block for many computer vision use cases, its accuracy and efficiency are equally important.
• Facial-Landmark-Detection model predicts 68 facial landmark locations from a single image using a 3D morphable model (3DMM). These facial landmarks can be further used to analyze facial expressions or even to drive animation.
• Facial Attribute Detection model provides features for comprehensive facial analysis, such as liveness, eye-openness, wearing a facial mask and wearing glasses or sunglasses. It also generates a facial descriptor that can be used for face verification.

Qualcomm AI Hub Sample model use

These models are highly accurate for real-world applications. They have been developed with a focus on meeting the needs of our end users. For example, a developer can easily combine the above models to create advanced solutions for workplace security and safety.

• PPE compliance for safety: this potential application daisy chains the person detection model and the PPE detection model to monitor if every person follows the safety protocol to wear hard helmets and safety vests when working at places where PPEs are required.

Figure 1. Person detection using Qualcomm AI Hub

• Restricted Zone Monitoring: this sample use case combines the detected persons and their feet locations with a person tracker to determine if a person enters a restricted zone. Assuming people always walk on the ground, the accurate detection of feet allows a developer to turn a seemingly complicated 3D reasoning problem into a 2D point check to tell if the zone is stepped into or simply occluded by people.

Figure 2. Person tracking using Qualcomm AI Hub

• Access control: in this example pipeline, we leverage three of our custom-built models released on Qualcomm AI Hub, the Lightweight Face Detection, the Facial-Landmark-Detection, and the Facial Attribute Detection models, to build a face-based access control system. Imagine that your doorbell can greet you when you arrive home and unlock the door when your hands are full of groceries.

Figure 3. Face and pose estimation diagram

Pre-optimized AI models

All of our custom-build models are tailored specifically for Qualcomm Technologies’ and Snapdragon platforms. These models leverage our hardware capabilities to deliver optimal performance. The 5 specified models support inference with 16bit float point allowing it to run on any platform. We also provide quantized 8bit integer version of these models where possible, which take complete advantage of Qualcomm Technologies’ and Snapdragon NPU hardware to offer lower latency and lower power consumption.

For example, Person-Foot-Detection model’s inference time is improved by 64% from 2.51ms in FP16 to 0.902ms in INT8 on Snapdragon 8 Elite NPU. Similarly, about 30% latency improvement is observed for the PPE model going from FP16 to INT8 inference on the Qualcomm Technologies QSC8550 chipset.

Figure 4-7. Person-Foot-Detection model’s inference time benchmarking

Qualcomm Technologies’ custom-built models offer high performance and efficiency, tailored specifically for our platforms and designed with end-user applications in mind. By making these models available via Qualcomm AI Hub, we empower the developer community to innovate and collaborate. Each have published five custom-built models to date, each providing unique benefits such as person foot detection, PPE detection, and advanced facial analysis. These models are pre-optimized for Qualcomm Technologies’ and Snapdragon platforms, ensuring lower latency and power consumption.

Next steps

Try out these models today by testing their performance across devices on Qualcomm AI Hub, downloading and integrating them in your application. Look out for more custom-built models, coming soon to Qualcomm AI Hub!

As we continue to release more models to Qualcomm AI Hub, we invite developers to leverage these models to create innovative on-device AI solutions. Reach out to us in our Slack community with any questions or feedback!

Dashan Gao
Principal Engineer/Manager, Qualcomm Technologies