Voice-Controlled Grasping Arm (reBot B601-DM) — Deployment Guide

Preset

Voice Grasping on Jetson

Deploy a voice-controlled grasping arm: say "Hey Jarvis, grab the water bottle" and the reBot B601-DM finds the object with its wrist RGB-D camera and picks it up. Wake word, speech recognition, LLM, object detection, grasp planning, arm control and TTS reply all run locally on the Jetson, with no online API.

DevicePurpose
reBot B601-DM6-DoF arm with parallel gripper (0.100 m max jaw) — USB serial
Orbbec Gemini 2wrist-mounted RGB-D camera (eye-in-hand) — USB 3.0
reComputer J40 seriesJetson Orin NX 16GB, runs all services
reSpeaker USB mic + speakerfar-field voice in, TTS reply out

What you'll get:

  • Voice-commanded grasping of boxes, standing (opaque) bottles, bananas, cups and oranges
  • The object list is editable config, no retraining
  • Live dashboard with the wrist-camera view and arm state (:8776)
  • Cartesian observation API (:8775/observation) for integration with other solutions

Before you start (hardware checklist):

  1. Arm powered on and connected — ls /dev/ttyACM* shows it (usually /dev/ttyACM0)
  2. Gemini 2 on a USB 3.0 port (blue connector — USB 2 starves the depth stream)
  3. reSpeaker mic + speaker connected; note your desktop user's uid (id -u, usually 1000)
  4. Docker + NVIDIA runtime (standard on JetPack 6); ~10 GB free disk
  5. Internet on first boot to download ~1.4 GB of container images and ~8.5 GB of models

China networks: set the HuggingFace Endpoint input to https://hf-mirror.com in Step 1 — the LLM engine, speech models and grasp detector all download through it.

1

Deploy the Stack

Deploy the voice, LLM, arm-control and inventory services to the Jetson.

Services

Deployment starts three services — rebot-arm (arm control), seeed-voice (speech recognition and synthesis) and edge-llm (LLM) — and downloads the grasp detector.

Troubleshooting
SymptomFix
Grasp detection is slowIf the TensorRT engine fails to load, the stack switches to ONNX Runtime — same detections, slower. The first line of docker logs voice-rebot-arm shows which one is active.
SSH connection failsCheck the Jetson IP address, SSH username/password and that port 22 is reachable.
Arm serial device is missingConfirm ls /dev/ttyACM* on the Jetson and update the Arm Serial Device field.
edge-llm stays unhealthy on first bootThe TensorRT engine is still downloading or warming up; watch docker logs edge-llm.
Deployment method

Deploy to a Jetson over SSH. Enter the Jetson IP address and SSH credentials, then set the arm serial device, audio user id and HuggingFace endpoint below.

What you will be asked for(filled in the SenseCraft Solution app)
  • Jetson IP Address*

    IP address or hostname of the Jetson Orin NX target.

  • SSH Username*default seeed

    SSH login username on the Jetson.

  • SSH Password*

    SSH login password for the Jetson.

3 more with defaults you can keep
  • SSH Portdefault 22

    SSH port, default 22.

  • HuggingFace Endpointdefault https://huggingface.co

    Model download endpoint. On China networks use https://hf-mirror.com. First boot pulls ~8.5 GB from here: the Qwen3.5-4B LLM engine, the MOSS-TTS-Nano and ASR speech engines, and the grasp detector.

  • Host Audio User IDdefault 1000

    Numeric uid of the Jetson desktop user that owns PulseAudio (run: id -u). Default 1000 fits a standard single-user JetPack install.

2

Open the Dashboard

Open the dashboard and confirm the live camera feed, arm state and voice path are healthy.

Troubleshooting
SymptomFix
No camera imageGemini 2 on a USB 2 port, or another process holds the camera — replug into USB 3.0, restart the rebot-arm container
No voice responsedocker logs voice-rebot-arm | grep -i wake; check the audio uid input matches id -u
edge-llm unhealthy for longEngine still downloading/warming — normal on first boot
Disk slowly fills with tegra-xusb: buffer overrun kernel logsJetPack driver logs; harmless but can take gigabytes over weeks. Filter those lines: echo ':msg, contains, "buffer overrun event for slot" stop' | sudo tee /etc/rsyslog.d/30-tegra-xusb-spam.conf && sudo systemctl restart rsyslog
After Deployment

Enter the same Jetson IP address used in Step 1 for remote deployment, or localhost for local deployment. The dashboard URL is http://<jetson>:8776.

  • Camera frames are refreshing: perception is up
  • State JSON is present: the serial link is up

Then the end-to-end voice test — say near the mic:

"Hey Jarvis, wave"

The arm waves and the speaker confirms. Voice + LLM + arm control all work now. Grasping needs one more step: calibration.

3

Hand-Eye Calibration — unlocks grasping

Finish one-time hand-eye calibration before using grasp commands.

First grasp

Place a small cardboard box (each face under 9.5 cm) about 25–30 cm in front of the arm, roughly centered, and say:

"Hey Jarvis, grab the box"

The arm scans, announces what it found, grasps, lifts and carries it home. Then try a cup, a banana, an orange, then an opaque bottle (standing).

Known-good placements: straight ahead or moderately left/right of center. Use opaque objects — the depth camera cannot see transparent bottles.

Prerequisites

Each unit needs its own calibration. Until /opt/rebot-models/hand_eye.npz exists, grasp commands detect objects but do not move the arm. One-time, ~30 minutes:

  1. Download and print the official ArUco calibration PDF (DICT_4X4_50, ID 0, nominal 100 mm), then measure the printed black outer square with a ruler — printers rescale. A 1 mm error in that value corresponds to about a 1 cm grasp offset.
  2. Tape the board flat on the table ~65 cm in front of the arm base.
  3. Follow the collection + solve procedure in the repository RUNBOOK §3.2 — the arm sweeps ~16 poses over the board, then solves the transform (target mean error < 5 mm).
  4. Copy the resulting hand_eye.npz to /opt/rebot-models/ and restart the rebot-arm container.
Troubleshooting
SymptomFix
"I couldn't find the …" occasionallyDetection confidence is marginal at some angles — repeat the command; move the object toward the center
"The box is too big for me to grip"Every visible face exceeds the 0.100 m jaw — expected; use a smaller object or turn a narrow face toward the arm
First attempt fails, retry worksOccasional; retry the command
Grasp lands centimeters offRecalibrate — and re-measure the printed marker size (step 1 above)
Arm joints fault (status_code=12)The stack clears this latched fault automatically at startup; if it persists after a container restart, power-cycle the arm
Arm was power-cycled and now ignores commandsRun docker restart voice-rebot-arm; a changed /dev/ttyACM* number is picked up automatically.
Detection is slowThe stack fell back to ONNX Runtime. Check the first line of docker logs voice-rebot-arm, then docker logs voice-rebot-arm-model-init-1 for the reason
Restarting the stack re-downloads several GBDo not use docker compose down -v (it deletes the model data); use down or restart