Pete Warden, CEO of Useful Sensors, presents the “Deploying Large Language Models on a Raspberry Pi,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, Warden outlines the key steps required to implement a large language model (LLM) on a Raspberry Pi. He begins by outlining the motivations for running LLMs on the edge and exploring practical use cases for LLMs at the edge. Next, he provides some rules of thumb for selecting hardware to run an LLM.
Warden then walks through the steps needed to adapt an LLM for an application using prompt engineering and LoRA retraining. He demonstrates how to build and run an LLM from scratch on a Raspberry Pi. Finally, he shows how to integrate an LLM with other edge system building blocks, such as a speech recognition engine to enable spoken input and application logic to trigger actions.
See here for a PDF of the slides.
