-
Can anyone make ‘smart home’ devices useful to consumers? A Calif. startup, Useful Sensors, believes it has the magic potion.
-
What’s at stake? Talk of edge AI, particularly machine learning, has captivated the IoT market. Yet, actual consumer products with local machine learning capabilities, are rare. Who’s ready to pull that off? Will it be a traditional MCU supplier or an upstart — like Useful Sensors?
Tech jargons like “smart home” and “smart sensor” have been overused to the point where real value that might be delivered by the related technologies reaches most non-techie consumers largely as fog.
Why, for instance, would any sensible person fiddle with apps, options and swipes on a smartphone to turn off the light when there’s a simple switch within reach?
Here’s the good news. We found an executive who shares our skepticism about smart homes.
Pete Warden, Useful Sensors CEO
Pete Warden is CEO and co-founder of Useful Sensors (Mountain View, Calif.).
Warden pointedly named his 2022 startup Useful — rather than “Smart” — Sensors. When he spins his dream of a home equipped with Useful Sensors’ hardware and software solutions, Warden calls it “a magical house,” not a smart home.
In Warden’s magical house, one simply enters a room, looks at a light and says “On.” And then there is light. “Or your TV will pause when you get up to have a cup of tea,” said Warden.
Useful Sensors’ brainstorm is “low-cost, easy-to-integrate hardware modules that bring machine learning capabilities like gesture recognition, presence detection, and voice interfaces to TVs, laptops, and appliances while preserving users’ privacy,” he explained.
Warden said, “Consumers might not even know that there is AI in everyday consumer devices. But those devices just do the right thing. That’s why I like the idea of ‘useful,’ instead of ‘smart.’ Smart is kind of like showing off.”
Emphasizing the importance of usefulness in technologies, he added, “I hope it to be in the background and doing what you want.”
Deep learning inference on embedded devices is a rapidly growing field with many potential applications. Edge AI is the buzz at every technology conference. Virtually every MCU supplier from STMicroelectronics and Renesas to Silicon Labs is promoting a new MCU with AI capabilities.
Ex-Google
Many MCU suppliers try to tackle from a hardware angle the broad challenge of implementing machine learning on resource-constrained processors. They see the name of the game is in running AI stack as efficiently as possible on their tailored hardware, while substantially reducing power consumption.
Warden, in contrast, sees these challenges and thinks software. After all, he takes credit for coining the term “TinyML,” the application of machine learning in a tiny footprint within embedded platforms.
Prior to Useful Sensors, Warden was at Google, as one of the members of the original TensorFlow team. But after his team got TensorFlow running well on phones, Warden got busy with TensorFlow Lite Micro, a machine learning framework for embedded systems.
His obsession with running machine learning on embedded devices only intensified. With Qualcomm senior director Evgeni Gousev, Warden launched the TinyML conference four years ago.
Eager to evangelize TinyML, Warden also wrote the standard embedded ML text book, called TinyMl, published by O’Reilly.
He also teaches embedded machine learning at Stanford University.
‘Don’t make us think’
Naturally, Warden and his team were very excited about the potential of ML applications in various consumer devices. They figured they were doing everything necessary for system OEMs to embrace TensorFlow Lite Micro.
But the real-world reaction to Warden’s pitch was tepid.
“When I went to talk to light switch manufacturers or TV manufacturers to try and get them to adopt this open-source framework made available in TensorFlow Lite Micro, they listened to me” politely, said Warden. He offered “free code, examples, courses, books, everything else available to system companies.”
But at the end, Warden’s audience typically said, “Look, we barely have a software engineering team. We do not have a Machine Learning Team. Can you just give us the thing that does a voice interface?” Or, “something that’s ready-made?”
Warden: “They literally told us, ‘Don’t make us think. Don’t make us code.’”
After a series of rebuffs, Warden said he realized, “What we needed to do was to put together something that was as easy to integrate as a temperature or pressure or an accelerometer sensor. Those sensors should give people machine learning capabilities.”
This was a scheme that would require Google to release hardware, an unlikely prospect. “Google wouldn’t want to risk the company’s whole reputation behind hardware products.”
So, bye bye, Google, and on to Useful Sensors.
Building blocks of Useful Sensors’ offerings
Typically, Useful Sensors offers a small module with a sensor on one side and a little microcontroller opposite.
A “Person Sensor” comes with a small camera and a small MCU. The module is preloaded with software to detect people. It includes a I²C interface. The processing cores used in the module are all off-the-shelf, said Warden, such as Synopsys’ Arc processing core or Arm Cortex M, “to drive down the cost.”
Useful Sensors are also mindful of privacy issues. “All one can get on our module is the metadata,” said Warden. “So, it can tell, for example, there are three people in the room.”
As with any consumer electronics device, the tricky part is figuring out the user experience, explained Warden. It’s a challenge to make Useful Sensors’ features “both discoverable and seamless” in consumer appliances. Warden’s theoretical example of subtle consumer symbiosis: “Imagine your TV … when you sat down, it just booted what you were watching last.”
‘We do Transformer’
Useful Sensors’ software expertise shines especially when squeezing software into off-the-shelf hardware.
Warden boasts that his team knows how to do stuff on $2 hardware that would require someone else a $20 chip.”
Another advantage of Useful Sensors’ software expertise is its nimbleness in keeping up with rapid advancements in machine learning. “For example, we use Transformers,” said Warden. Transformers is a new building block for machine learning models, different from convolutional learning models familiar to many people already using AI.
“T” in Chat GPT stands for Transformers, as it deploys the Generative Pre-trained Transformer 3.5 (GPT-3.5).
Because Transformer employs a different kind of compute, “a lot of the fixed function neural network accelerators that people have brought out in the last year or two, do not support Transformers,” explained Warden. “So it’s an example of why having some degree of programmability is really important for these kinds of hardware, because what people want to run is changing so fast.”
Who has signed up so far?
Useful Sensors has signed Non-Disclosure Agreements and Evaluation agreements with several companies whose names Warden declined to reveal. But the company hopes to announce partnerships “over the next few months,” said Warden.
For AI hardware startups to sign agreements with automotive companies is unimaginably tough – largely because so many startups don’t last. Working with consumer electronics companies is equally tough. Warden agreed. “CE companies have very high standards,” he said, adding: “I am really passionate about trying to do something that actually helps people’s everyday lives, instead of gimmicks. I believe consumer electronics is such a good way to reach people.”
Warden noted that among the different Useful Sensors’ applications, the one gaining momentum most is “having local voice interfaces” that don’t require sending the voice request to the cloud. Functions from lighting to audio equipment and TVs work better with local voice control, but don’t have the capability today. “That’s why so few people actually use Alexa or Siri,” Warden noted.
Undoubtedly, local voice will represent a big user experience challenge. Warden, said, optimistically, “There’s something magical about just being able to talk to all of the objects around your home … like a Disney house where you walk in and say. ‘hello, coffee pot!’”
Bottom line
Enabling edge AI is easier said than done. The crux of the issue is the lack of talent. The hardest thing, either for an MCU supplier or a startup like Useful Sensors, is to find skilled people good at machine learning, who also know how to build hardware in the embedded space.
Junko Yoshida
Editor in Chief, The Ojo-Yoshida Report
This article was published by the The Ojo-Yoshida Report. For more in-depth analysis, register today and get a free two-month all-access subscription.
