This blog post was originally published at Arm’s website. It is reprinted here with the permission of Arm.
The integration of ML into IoT is sparking exponential growth and shaping a dynamic ecosystem
The pace of Internet of Things (IoT) innovation has quickened in recent years, but it’s now being kicked into hyper drive. The first significant IoT transformation happened when edge devices just got connected to each other; now they are getting connected to the cloud. Suddenly vast amounts of captured data could be processed much more cost-effectively in the cloud, and powerful algorithms took the interpretation of the data to new planes of understanding.
That led to the development of systems in rainforests that can hear and geo-locate illegal logging or identify the sounds of endangered species in distress or sense mechanical faults and automatically alert plant managers before equipment breaks down. All this has happened in just the past few years.
A new transformation in IoT with machine learning
But a new, even more powerful transformation is underway. Devices will no longer rely just on cloud computing to interpret data; devices are evolving into systems capable of making decisions themselves based on the data they generate. It promises to speed decision-making, improve privacy and security by handling data closer to where it’s captured.
Consider vision applications: A few years ago, home-security cameras streamed videos into the cloud in order to notify homeowners about package delivery. Now, many of the newer home-security cameras have integrated machine learning (ML) capabilities. This cuts the latency by not having to ship data to the cloud and back and improves privacy by keeping personal information – images of a person’s home or belongings for example – locally on the device, fully secure. And with ML on the device – thanks to more powerful computing – the features in such systems could expand. A camera, for example, could be programmed to capture images only if there are unknown or new people at the front door.
Sweet as Pi
Not convinced innovation in IoT is going warp speed? Just a few weeks ago, some developers ran a large language model (LLM) on a Raspberry Pi device – one of the most popular IoT devices on the planet – and created their own AI chat bot servers. Let that sink in: An LLM – which has billions of parameters – is running on a Raspberry Pi. Tom’s Hardware published a how-to on the topic this spring.
This is just the beginning. Various iterations of language and transformer models will soon find their place in IoT edge devices which will have new compute capabilities. This will unlock new possibilities that were once confined to the realm of imagination. Let’s think about a few.
Strengthening security and privacy
As technology has proliferated rapidly and consumers have embraced digital solutions in all aspects of their lives, their concern for privacy and security has risen as well. Not a week goes by, it seems, when we read about some type of hacking whether it’s large scale – datacenters – or IoT scale. The world took notice when it was reported that hackers got into someone’s baby monitor. Many healthcare devices currently monitor important vitals and provide feedback about an individual’s health. These applications need to evolve with rock-solid security to ensure patient privacy. With ML advancements, these devices will analyze data directly on the device, detect patterns and even customize for one’s preferences and objectives, thus improving the average life expectancy.
Local decision-making with faster response time
The potential to transform communities by embedding intelligence locally will change decision-making. Smart cities use IoT sensors and devices to collect and analyze data in real-time to make informed decisions. ML algorithms can be used to analyze the data to provide insights for making decisions in real time. A smart transportation system can use ML capabilities to monitor traffic flow, detect accidents, provide alternative routes, predict traffic flow patterns, and suggest optimal routes.
Driven by necessity during the pandemic, the world increasingly used the convenience online home delivery services. But where the rubber meets the road – in warehouses where goods are stored – smart robots and sensors will identify packages in new orders and arrange delivery to customers in an optimal way. And in a few years, ML-enabled IoT devices will solve the “last-mile” problem with drones delivering packages to households in areas that were out of reach.
Improvements in operational efficiency
Industrial factories are the backbone of many countries and contribute a large percentage to the world’s GDP. Many industrial factories have become smart over the last few years and have embraced ML to improve operational efficiency by many orders of magnitude. Predictive maintenance is crucial in industrial automation, as it can significantly reduce downtime, increase productivity, and slash maintenance costs. ML can be used in this context to monitor equipment performance and predict potential problems. By analyzing data generated by edge devices in real-time, ML algorithms can identify potential issues, allowing maintenance teams to proactively take corrective action.
Amazon Alexa and Google Nest devices have revolutionized how homes can be managed by making them smarter, enabling newer, customized use-cases based on families’ needs. For example, smart lighting systems can learn user preferences and adjust settings accordingly. In addition to improving user experience, ML on IoT devices can also help reduce energy consumption by optimizing heating and lighting based on user behavior.
A thriving, dynamic ecosystem
The transformation of IoT with ML paradigm will exponentially grow and influence through a thriving ecosystem. ML in IoT has enabled a new value chain, with companies inventing AutoML tools, models specific for IoT devices, tools to translate models from different frameworks and data to be able to run on IoT devices.
But there are concerns related to ML that the ecosystem as a whole needs to come together to solve.
Chief among them: Models are developed using many frameworks which often need to target many hardware devices. This hampers scaling. This is the reason Arm has been working on a new standardized operator architecture called TOSA (Tensor Operator Set Architecture) so that devices can run future models developed on the billions of devices. This is also the reason Arm has been a proponent of TVM (Tensor Virtual Machine), open-source machine learning compiler framework supported on Arm’s CPUs, and specialized ML accelerators.
Another example: Data drift becomes an increasing problem as IoT devices proliferate in the field. Data drift occurs when the characteristics or patterns of the data generated by IoT devices change over time, due to things like device degradation or environmental factors. This can change statistical properties or distribution of models and affect applications. But, designed properly, IoT devices can detect drifts and classify them as natural or abnormal based, maintaining reliability, system performance and safety.
Arm has been working with many ecosystem partners to enabling a thriving ecosystem across the entire spectrum of machine learning deployments ranging from AutoML partners, model providers partners, end-end solution deployment partners and many more.
Unlocking the potential of ML and IoT
Over time, it is expected that all IoT devices will run some sort of ML on the device. This is the reason Arm enables a wide range of ML devices – ranging from silicon deployed to run tiny sensors with Cortex-M CPUs, Cortex-M CPUs paired with custom ML accelerators, Cortex-A CPUs, Cortex-A CPUs paired with custom ML accelerators including GPUs with more and more devices coming in the future.
The integration of ML into the IoT is revolutionizing industries and livelihoods, ushering in a new era of transformation. This convergence fuels exponential growth and shapes a dynamic ecosystem, where companies are creating AutoML tools, specialized models for IoT devices, and frameworks for seamless deployment. Monitoring the vast number of IoT devices running ML algorithms is another emerging facet of this ecosystem, allowing for the detection of data drifts. Leveraging this information, models are continuously retrained and redeployed, leading to enhanced accuracy and performance. Together, ML and IoT are poised to reshape the world and unlock unprecedented possibilities across various sectors.
To learn about the opportunities in this evolving age of innovation, click here for more information.
Parag Beeraka
Senior Director of Segment Marketing for IoT, Arm
