AI Assistant using LLM on Raspberry Pi

Abstract

This project introduces an offline AI assistant system built on Raspberry Pi, integrating speech-to-text (STT), large language model (LLM) inference, and text-to-speech (TTS) components into a unified, low-latency pipeline. The assistant operates entirely without internet connectivity, thereby preserving user privacy while enabling real-time, human-like voice interaction. This approach addresses critical limitations in existing cloud-based voice assistants, such as latency, privacy risks, and dependence on reliable network infrastructure.The system is activated via wake word detection and leverages open-source technologies
to perform each stage of the interaction. Whisper and Vosk are employed for accurate and lightweight speech recognition; LLaMA 3.2, running locally via the Ollama framework, powers the natural language understanding; and RHVoice generates natural-sounding voice responses. This pipeline ensures modularity and adaptability across a range of edge devices.Designed to run efficiently on resource-constrained hardware like Raspberry Pi 5, the assistant is optimized for offline use cases in smart homes, remote environments, and privacy-sensitive domains such as education and healthcare. Experimental evaluation
demonstrates a word error rate (WER) of approximately 2.5%, a response latency of 2.2 seconds, and a Mean Opinion Score (MOS) of 3.2 for speech synthesis quality. These results confirm the system’s feasibility for real world applications, offering a scalable and cost-effective alternative to commercial AI assistants. By combining the capabilities of LLMs with edge computing, this work contributes a practical framework for developing privacy first conversational agents that are accessible, affordable, and highly customizable. The system sets a precedent for future developments in offline AI-powered interaction across diverse use cases.