This blog post was originally published at Deci’ website. It is reprinted here with the permission of Deci.
Computer vision is a fast-growing subfield of AI and deep learning. From cashierless stores in retail to crop detection in agriculture, there’s an increasing interest in CV applications. This has created a vibrant community that gladly shares architectures, codes, pre-trained models, and even tips for every stage of the development cycle.
Starting a CV project from scratch takes time. So, the usual process is, given a problem or a use case, you look for models that partially solve it. If one already exists, then training a new model is no longer necessary. Otherwise, you use the publicly available models to guide you through which architecture to choose as a base for fine-tuning or transfer learning.
In this post, we list 13 free resources and model zoos for deep learning and computer vision models that can hopefully help to simplify your work.
1. Papers with Code
Papers With Code is a community-led open resource that puts together machine learning papers, codes, datasets, methods, and evaluation tables. While the core team of the project is based in Meta AI Research, everyone across academia and industry can contribute.
Source: Papers with Code
You can filter the database and browse state-of-the-art models, datasets, tasks, methods, and trending research. Moreover, you can see popular benchmarks, submissions, and developments made on each task and dataset.
Source: Papers with Code
In addition to computer vision, the resource is updated regularly with the latest papers and implementations in other subfields of AI such as natural language processing (NLP), reinforcement learning, audio, and more. There are also portals for other disciplines including physics, astronomy, and statistics.
2. Open Model Zoo
OpenVINO™ Toolkit’s Open Model Zoo is a repository of pre-trained deep learning models and demos that developers can use to speed-up the development and production deployment process.
The models in the resource can be divided into two categories: Intel pre-trained models and public pre-trained models. In total, there are over 200 models in the repository covering a variety of tasks such as object detection, place recognition, text to speech, colorization, and more. Users can also leverage dozens of demo application templates.
Source: OpenVINO Toolkit GitHub
To make it fast and easy, the community can use automation tools to download and convert the models into the OpenVINO™ IR format. There are tools for quantizing models, printing information, and copying datasets from an installed location. Open Model Zoo also provides an accuracy checker to validate model accuracy.
3. Deci’s Model Zoo
Deci is an end-to-end deep learning development platform that enables data scientists and deep learning engineers to build, optimize and deploy models to any environment and hardware.
It includes a model zoo with PyTorch-based computer vision models for all the main tasks. Unlike other model zoos, the models in Deci’s zoo are pre-trained and optimized (compiled and quantized) for a wide range of hardware types.
The model zoo is designed to help select the best deep neural network (DNN) for your task and ensure that it works well on your inference hardware, even before you start training it. Having this information can help you not only to save time and resources for model training, but also ensure that there won’t be any friction deploying the model once it’s ready. By selecting the right architecture with the optimal performance that fits your task, dataset, and inference hardware, there won’t be the expected-versus-actual-performance gap at the deployment stage.
In addition to displaying the accuracy per model, you can easily select a target inference hardware and see the model’s runtime performance results such as throughput, latency, model size, and memory footprint for any given hardware.
The model’s performance results are plotted on an “Efficient Frontier” graph which enables you to easily see the expected “accuracy-latency” trade-off on the selected hardware. You can also click each model and compare it to others.
You can download the models for free or deploy with Infery, a Python runtime engine, when you sign up to Deci’s deep learning platform.
4. Jetson Model Zoo and Community Projects
The Jetson Zoo is an open-source catalog containing guidelines for installing add-on packages and frameworks on NVIDIA Jetson. It also includes DDN models for inferencing on Jetson with support for TensorRT.
Source: eLinux.org
The Model Zoo includes classification, object detection, segmentation, and pose estimation models, code samples, and links to the original sources. You can also see resources for performance benchmarks.
Source: NVIDIA
In addition, NVIDIA features community projects created for Jetson developer kits. These projects contain codes, videos, and instructions, which you can use for inspiration or modify according to your own needs.
5. Model Zoo
Model Zoo curates pre-trained deep learning models for a variety of frameworks and domains. Built by Jing Yu Koh, who is now a machine learning researcher at Google, the models are well-tagged, making it easy for researchers and practitioners to filter and find models that suit what they’re looking for.
Source: ModelZoo
Some of the frameworks include TensorFlow, PyTorch, Keras, Caffe, and more. You can browse computer vision, NLP, reinforcement learning, unsupervised learning, audio and speech, and generative models. In addition, each model or library has in-depth implementation details.
6. TensorFlow Hub
TensorFlow is a popular library for training and inference of deep neural networks. With the TensorFlow Hub, deep learning practitioners can simply download and reuse assets for machine learning.
Source: TensorFlow
The pre-trained SavedModels are ready for solving new tasks with less training time and data. Downloading them involves detailed documentation and minimal coding.
7. TensorFlow Model Garden
Maintained by the TensorFlow team, the Model Garden contains a variety of different implementations of SOTA models, example implementations, and best practices for modeling.
The repository features four main directories:
- Official is the collection of SOTA model implementations using the latest TensorFlow 2 APIs.
- Research contains model implementations in TensorFlow 1 or 2 that are in the research phase.
- Community includes a curated list of TensorFlow 2-powered GitHub repositories.
- Orbit is a library that enables users to write customized training loop code in TensorFlow 2.
Source: TensorFlow GitHub
Unlike TensorFlow Hub, the Model Garden provides a solid foundation of configurations and datasets for model training or fine-tuning.
8. MediaPipe Models
MediaPipe is an open-source cross-platform that offers customizable machine learning solutions for live and streaming media. Some solutions include instant motion tracking, face mesh, KNIFT, and selfie segmentation. While still in alpha, products such as Google Nest and Ariel AI (acquired by Snap) are already using MediaPipe solutions.
Source: MediaPipe
On its GitHub repository, developers can see how to implement the models in Python, JavaScript, C++, Android, and iOS. Each model also has a card that includes all details regarding its specifications, applications, limitations, and ethical considerations.
9. Awesome TensorFlow Lite
TensorFlow Lite is specifically built for running inference on devices with limited compute power such as phones and embedded devices.
Source: Magenta GitHub
Awesome TensorFlow Lite is a comprehensive repository for TensorFlow Lite. It includes models with samples ranging from computer vision to recommendation systems. It also gathers resources for tutorial projects, plugins and SDKs, and other learning resources such as books, podcasts, and videos, which can be helpful for beginners.
10. Awesome CoreML Models
Core ML is a framework released by Apple to enable developers to integrate machine learning into applications. With Awesome CoreML Models, you get the largest list of models in Core ML (iOS 11+) that you can use to experiment and implement on iOS, macOS, tvOS, and watchOS.
Source: Awesome CoreML GitHub
In addition to the different models that process image and text, the repository also provides users with visualization tools. Supported model formats that are convertible to Core ML with examples include Caffe, Keras, XGBoost, Scikit-learn, and more.
11. Pinto Model Zoo
Created by Katsuya Hyodo, the Pinto Model Zoo is a repository that shares tuning results of trained models generated by TensorFlow or Keras. To date, it contains over 260 optimized models. Post-training quantization techniques used include weight quantization, integer quantization, full integer quantization, Float16 quantization), and quantization-aware training.
Source: Pinto Model Zoo GitHub
The models are also available for TensorFlow Lite, OpenVINO, CoreML, TensorFlow.js, TF-TRT, MediaPipe, and ONNX format. Moreover, you can see various sample implementations of the trained models with specifications, codes, and video files.
12. ONNX Model Zoo
Open Neural Network Exchange (ONNX) is an open-source format for representing machine learning models. With the ONNX Model Zoo, you can explore and use pre-trained, SOTA models in ONNX that the community contributed. There are models for various vision, language, and other interesting tasks.
Source: ONNX GitHub
Each model has a corresponding Jupyter notebook for model training and running inference. Written in Python, the notebooks contain links to the datasets and original papers. There are also instructions for using Git LFS command line when downloading multiple ONNX models, as well as for configuring starter Python code when validating your ONNX model with test data.
13. CatalyzeX
CatalyzeX is a small browser extension that allows you to get implementation code for machine learning research papers directly on Google, ArXiv, GitHub, Twitter, and other websites.
Source: CatalyzeX
As you browse online, the extension provides you with an in-line link next to papers in the fields of AI, NLP, computer vision, deep learning, and reinforcement learning. On the CatalyzeX website, you can also see a collection of models, code, and papers for popular deep learning tasks.
Conclusion
Beyond providing the building blocks for deep learning projects, the vibrant, open-source community is instrumental in the continuous progress and increasing accessibility of AI. Take advantage of these free resources and deep learning model zoos so you can speed up your development and production deployment process.
If the Deci hardware-aware model zoo sparked your interest, you can sign up for the platform for free, go straight to the Model Zoo tab, and start comparing and downloading your pre-trained, optimized, hardware-aware models today.
