r/HomePod • u/Ffiren • 10m ago
Question/Support Using Siri / HomeView as a front end for a local HA voice pipeline: anyone tried the Assist App Intent route
I run a fully local voice stack behind Home Assistant: qwen3:8b on Ollama as the conversation agent, mlx-whisper for STT and Kokoro for TTS, all on an M4 Mac mini, with four Voice PE units as the front end. It works, but the Voice PE firmware has a 2 s playback timeout that reopens the mic mid-reply and causes echo loops, and whisper struggles with my kids’ voices.
With HomeView shipping to AU next month, I want to try Apple hardware as the ears and mouth and keep the brain local. The plan:
- Use the Companion app’s “Assist prompt” App Intent instead. Siri does on-device STT, a Shortcut hands the transcript to a dedicated HA pipeline (no STT or TTS stage, same Ollama agent), the text reply comes back and Siri speaks it. Works on iPhone and Watch today, and on HomePod via Personal Requests.
- Test whether HomeView runs that Shortcut by voice. It has no App Store and projects apps from a paired iPhone CarPlay-style, so Personal Requests is the only likely path.
- Watch the iOS 27 Siri Extensions story. It is partner-gated (ChatGPT, Gemini, Claude) with no custom endpoint reported. If Apple ever allows an OpenAI-compatible URL, a local proxy is already the right shape to receive it.
What I expect to gain: Siri owns barge-in so the echo problem disappears, and Apple’s far-field recognition for the kids. What it will not fix: the LLM stage itself, which is still my mini.
Questions for anyone who has gone down this road:
- Has anyone got “Hey Siri, Assist” working without the invocation-phrase dance, with or without the AppShortcutsProvider PR?
- Real-world latency for the Assist prompt action via HomePod Personal Requests, as opposed to directly on the phone?
- Any way to keep a multi-turn conversation going, given the Shortcut has no continue_conversation equivalent?
- Per-user voice recognition mapping to the right phone for kids, how painful is it in practice?
Happy to share numbers once the HomeView arrives.