Home Assistant Local Voice Hub — Deployment Guide

Preset

ARM64 Linux Host + Voice Host (J40, RK3588 + RK1828 or J30)

An ARM64 Linux host runs Home Assistant, and a voice host understands speech and generates the spoken replies; questions other than device control are answered by an LLM. The two devices work together on the same local network.

DevicePurpose
ARM64 Linux host (64-bit OS, 8GB RAM)Runs Home Assistant
Voice host (any one from the table below)Local speech recognition and speech synthesis
Home Assistant Connect ZBT-2Connects Zigbee devices
Home Assistant Voice Preview EditionVoice terminal in the room
Voice hostAnswers questions other than device control with
reComputer J40 (Jetson Orin NX 16GB)An LLM running on the device
reComputer RK3588 (16GB) + RK1828 accelerator cardAn LLM running on the card
reComputer J30 (Jetson Orin Nano 8GB)The cloud LLM service you enter

What you'll get:

  • Control lights and other devices by voice in Chinese or English, with a median 0.12 s from end of speech to recognized text
  • Ask questions such as "Is it a good day to open the windows?" and get an answer from the LLM; device commands are still executed locally by Home Assistant
  • Zigbee sensors and switches in Home Assistant
  • The same devices in the Home app and Siri on iPhone

Requirements: Both devices on the same local network · Internet access on the voice host's first start to download the models · For a cloud LLM: the address, model name and API key of an OpenAI-compatible LLM service

1

Deploy Home Assistant

Install Home Assistant on the ARM64 Linux host.

Prerequisites
  • The host runs a 64-bit Linux OS and has at least 8 GB of free disk
  • The ZBT-2 can be added later: leave "ZBT-2 device path" empty, then fill in the path and deploy again once the ZBT-2 is plugged in
Wiring
  1. Connect the ARM64 Linux host to the local network and power it on
  2. Plug the ZBT-2 into a USB port on the host; run ls /dev/serial/by-id/ on the host and enter the listed path as "ZBT-2 device path"
  3. The default port is 8123; if another program on the host already uses 8123, enter a different port
  4. Click Deploy
Troubleshooting
IssueSolution
Port already in useEnter another port in "Home Assistant port" and deploy again
Changed the port but Home Assistant still uses the old oneAn existing Home Assistant installation keeps the port from its first install and ignores the deploy setting; keep using the original port
Home Assistant was installed by an earlier version of this solution and starts with a new, empty configuration after redeployingEarlier versions stored the configuration in /opt/ha-whole-home/ha-config; it now lives in ~/ha-whole-home/ha-config and is not moved automatically. On the host, run docker stop $(docker ps -q --filter label=com.docker.compose.project=ha_whole_home_rpi), then mkdir -p ~/ha-whole-home && sudo cp -a /opt/ha-whole-home/ha-config/. ~/ha-whole-home/ha-config/, then deploy again
Page does not loadWait a few minutes and refresh; make sure your computer and the host are on the same local network
ZBT-2 does not appear in Home AssistantMake sure "ZBT-2 device path" is the full path under /dev/serial/by-id/, then deploy again
Cannot connect to the hostCheck the IP address, username and password, and that the host is powered on and on the network
After Deployment
  1. Open http://<host-ip>:8123 in your browser (use your port if you changed it); the first start takes a moment
  2. Follow the prompts to create the administrator account
Deployment method

Install on the ARM64 Linux host over the network; you need its IP address, username and password.

What you will be asked for(filled in the SenseCraft Solution app)
  • ZBT-2 device path

    Plug in the ZBT-2, run ls /dev/serial/by-id/ on the host and enter the listed path. Leave empty to install Home Assistant first; fill it in and deploy again once the ZBT-2 is plugged in.

1 more with defaults you can keep
  • Home Assistant portdefault 8123

    Port of the Home Assistant web page. Change it only if 8123 is already used on this host, for example by another Home Assistant. Takes effect on the first install; an existing installation keeps its port.

2

Deploy the voice host

Install the speech recognition and speech synthesis services on the voice host for Home Assistant to use. The reComputer J40 and the RK3588 with an RK1828 card also install a local LLM; the other voice hosts use the cloud LLM service you enter.

Prerequisites
  • Free disk: at least 40 GB on the reComputer J40, 20 GB on the J30, 16 GB on the RK3588 + RK1828, 10 GB on the RK3588 and RK3576
  • Internet access on first start to download the models; the services are not available until the download finishes
  • RK3588 + RK1828: the card is powered, its driver and firmware are installed (a /dev/pcie-rkep-* device exists), and no other LLM runs on the card
  • Cloud LLM: the address of an OpenAI-compatible service (for example https://api.deepseek.com), a model name and an API key; the API key is entered only in Home Assistant in step 3
  • Local LLM (J40, RK3588 + RK1828): the IP address of the Home Assistant host; the local LLM accepts connections only from that host and from the voice host itself
Wiring
  1. Connect the voice host to the same local network as the Home Assistant host and power it on
  2. Keep the default ports: speech-to-text 10300, text-to-speech 10200, voice service 8623, local LLM 8000 (J40) or 1828 (RK1828); change one only if it is already in use
  3. Local LLM: enter the Home Assistant host's IP address; cloud LLM: enter the LLM API address, and deployment first checks that the voice host can reach it
  4. Note the voice host's IP address and ports for step 3
  5. Click Deploy
Troubleshooting
IssueSolution
Not enough disk spaceFree the space listed under Prerequisites and deploy again
Port already in useChoose an unused port and deploy again; the ports must be different
Services do not become ready for a long timeThe first start downloads the models first, and the time depends on your connection; make sure the voice host can reach the internet
NVIDIA container runtime missingMake sure the reComputer J40 / J30 runs a full JetPack installation that includes the NVIDIA container runtime, then retry
RK1828 card not foundCheck the card's power, driver and firmware; without the card, choose "RK3588 (cloud LLM)" instead
The LLM service cannot be reachedCheck the API address and that the voice host can reach the internet
The address is not an OpenAI-compatible APIEnter the provider's "OpenAI-compatible" address from its documentation
curl is missingInstall curl on the voice host and deploy again
iptables is missingInstall iptables on the voice host and deploy again; it limits the local LLM to the Home Assistant host
The local LLM does not start after the voice host or Docker restartsThe local LLM starts only after its access rule is in place (re-applied automatically when the voice host or Docker restarts); on the voice host run sudo systemctl status ha-whole-home-llm to see why, fix it, then run sudo systemctl restart ha-whole-home-llm
Cannot connect to the deviceCheck the IP address, username and password, and that the device is powered on and on the network
Deployment method

Install on the reComputer J40 over the network; the LLM runs on the device. You need its IP address, username and password.

What you will be asked for(filled in the SenseCraft Solution app)
  • Home Assistant host IP*

    IPv4 address of the Home Assistant host from step 1. Only this host and this device can reach the local LLM.

4 more with defaults you can keep
  • Voice service portdefault 8623

    Port used only inside the voice host by the voice services. Change it only if 8623 is already in use.

  • Speech-to-text portdefault 10300

    Port Home Assistant connects to for speech-to-text; enter it when adding the Wyoming Protocol integration in step 3. Change it only if 10300 is already in use.

  • Text-to-speech portdefault 10200

    Port Home Assistant connects to for text-to-speech; enter it when adding the second Wyoming Protocol integration in step 3. Change it only if 10200 is already in use.

  • Local LLM portdefault 8000

    Port of the local LLM; enter it in the LiteLLM integration address in step 3. Change it only if 8000 is already in use.

3

Connect local voice and verify

Add the voice services and the LLM conversation agent to Home Assistant, then test.

Initial Setup

  1. Add Zigbee devices: go to Settings → Devices & services → Add integration, choose Zigbee Home Automation and select the ZBT-2 serial port; then click "Add device" in that integration and put the Zigbee device into pairing mode
  2. Add to Apple Home: a HomeKit Bridge pairing QR code appears in the Home Assistant notifications; scan it with the Home app on iPhone
  3. Add the voice terminal: follow the Home Assistant Voice Preview Edition instructions to connect it to Wi-Fi and add it to Home Assistant, then select the voice assistant configured in step 3 on its device page

Quick Verification

  • Say "Turn on the living room light" to the voice terminal; the light turns on and you hear a reply
  • The same devices are visible and controllable in the Home app on iPhone
  • Zigbee sensor readings update in Home Assistant
Troubleshooting
IssueSolution
Adding the integration fails to connectMake sure step 2 is ready, the IP address and port are correct, and the Home Assistant host can reach the voice host
Adding LiteLLM fails to connect or reports an invalid keyLocal LLM: wait until step 2 is ready and retry, and check that the Home Assistant host IP entered in step 2 is correct (deploy step 2 again after an IP change); cloud LLM: check the address and key, and that the Home Assistant host can reach the internet
Device commands are answered by the LLMTurn on "Prefer handling commands locally" in the voice assistant
A light command was not carried out but the LLM says it wasThe device name was misheard and the command went to the LLM; set the "Instructions" as in item 4 and the LLM asks you to repeat the exact name
Other questions only get "I don't understand"Choose the LiteLLM agent as the voice assistant's "Conversation agent"
The voice services are not listed in the voice assistantMake sure both Wyoming Protocol integrations were added successfully
Voice stopped working after redeploying step 2 with different portsDelete both Wyoming Protocol integrations and add them again with the new ports
The command is recognized but nothing happensMake sure the name in the command matches the device or area name in Home Assistant
After Deployment

  1. Open Home Assistant, go to Settings → Devices & services → Add integration and search for Wyoming Protocol; enter the voice host's IP address as host and the speech-to-text port from step 2 (default 10300) as port

  2. Add a second Wyoming Protocol integration with the same host and the text-to-speech port (default 10200)

  3. Add a LiteLLM integration:

    • reComputer J40 or RK3588 + RK1828: enter http://<voice-host-ip>:<local LLM port> (default 8000 or 1828) as the URL and leave the API key empty
    • Other voice hosts: enter the LLM API address from step 2 and the API key from your LLM provider

  4. In the LiteLLM integration, add a conversation agent: choose the model name from step 2 (a local LLM lists only one model), leave "Control Home Assistant" unchecked, replace "Instructions" with the text below, and save:

    You are the home voice assistant. Answer briefly in spoken language. You cannot control devices in the home and cannot see their state. When the user asks you to switch or adjust a device, do not say it is done; ask them to say it again using the device's exact name.

  5. Go to Settings → Voice assistants, open the assistant and set the language to Chinese or English; choose the new LiteLLM agent as "Conversation agent" and turn on "Prefer handling commands locally"; select the two new Wyoming services for "Speech-to-text" and "Text-to-speech", and save

  6. Click the conversation button at the top right and say or type "Turn on the living room light"; the light turns on and you get a reply. Then ask "Is it a good day to open the windows?" and you get an answer from the LLM

Home Assistant and the local voice services are ready.

Deployment Complete

Home Assistant and the local voice services are ready.

Initial Setup

  1. Add Zigbee devices: go to Settings → Devices & services → Add integration, choose Zigbee Home Automation and select the ZBT-2 serial port; then click "Add device" in that integration and put the Zigbee device into pairing mode
  2. Add to Apple Home: a HomeKit Bridge pairing QR code appears in the Home Assistant notifications; scan it with the Home app on iPhone
  3. Add the voice terminal: follow the Home Assistant Voice Preview Edition instructions to connect it to Wi-Fi and add it to Home Assistant, then select the voice assistant on its device page ("Home Voice" if Home Assistant was connected automatically)

Quick Verification

  • Say "Turn on the living room light" to the voice terminal; the light turns on and you hear a reply
  • The same devices are visible and controllable in the Home app on iPhone
  • Zigbee sensor readings update in Home Assistant