AI Lab — Edge AI Model Hub on Rockchip — Deployment Guide

Preset

Object Detection

Deploy YOLO 11 object detection to a reComputer RK3576 or RK3588, with an image detection REST API and an MJPEG live video feed covering the 80 COCO object classes.

Devices: reComputer RK3576 or RK3588; a USB camera for live video detection (optional).
Software: Docker installed on the device, reachable over SSH.
Models: choose Nano, Small or Medium at deployment.

Deploy YOLO 11

Deploy the object detection container to the device.

Deployment method

Device

Deploy to the RK3576 over SSH.

Wiring

Connect the RK3576 to the same network as your computer
Plug in a USB camera for live video detection
Select the model size (start with Nano)
Fill in device IP, SSH username, and password
Click Deploy

What you will be asked for(filled in the SenseCraft Solution app)

Model*default yolo11n
3 options
- · YOLO 11n (Nano - fastest, ~5MB)
- · YOLO 11s (Small - balanced, ~10MB)
- · YOLO 11m (Medium - most accurate, ~20MB)

Troubleshooting

Symptom	Fix
SSH connection failed	Verify IP address, username, password
NPU not detected	Ensure device is RK3576 with RKNPU kernel module loaded
No camera detected	Check USB camera is connected. Detection still works with image upload API
Image pull slow	Check the network. Image is about 1-2GB

After Deployment

Detection API: http://<device-ip>:8000/api/models/yolo11/predict
Live video feed: http://<device-ip>:8000/api/video_feed (requires camera)

Try Detection

Verify the detection service is working.

ModeMode

Upload an image to test object detection.

Troubleshooting

Symptom	Fix
No detections	Use an image with people or vehicles
Connection refused	Wait 15-30 seconds for service to start

Preset

LLM Chat

Deploy the DeepSeek-R1 large language model to a reComputer RK3576, with an OpenAI-compatible chat API running on the device.

Devices: reComputer RK3576; 7B variants need 8GB+ RAM.
Software: Docker installed on the device, reachable over SSH.
Models: choose from 5 variants (1.5B/7B, different quantizations) at deployment.

Deploy DeepSeek-R1

Deploy the LLM container to the RK3576.

Deployment method

Deploy to the RK3576 over SSH.

Wiring

Connect the RK3576 to the same network as your computer
Select the model variant
Fill in device IP, SSH username, and password
Click Deploy

What you will be asked for(filled in the SenseCraft Solution app)

Model*default 1.5b-w4a16-latest
5 options
- · DeepSeek-R1 1.5B W4A16 (1.2GB, recommended)
- · DeepSeek-R1 1.5B FP16 (2.7GB, highest accuracy)
- · DeepSeek-R1 1.5B W4A16-G128 (1.2GB, most compressed)
- · DeepSeek-R1 7B W4A16 (4.1GB, stronger reasoning)
- · DeepSeek-R1 7B W4A16-G128 (4.1GB, compressed)

Troubleshooting

Symptom	Fix
SSH connection failed	Verify IP address, username, password
NPU not detected	Ensure device is RK3576 with RKNPU kernel module loaded
Out of memory (7B model)	7B variants require 8GB+ RAM. Try a 1.5B variant instead
Image pull slow	Check the network. Image size is 1-4GB depending on variant

After Deployment

Chat API: http://<device-ip>:8001/v1/chat/completions. Connect with any OpenAI-compatible client.

Try Chat

Test the LLM by sending a message.

Troubleshooting

Symptom	Fix
Connection refused	Wait 30-60 seconds for model to load
Timeout	7B models take longer. Wait up to 2 minutes
Empty response	Check container logs: `docker logs ai_lab_llm`

After Deployment

Object Detection — Image Upload

curl -X POST http://<device-ip>:8000/api/models/yolo11/predict \
  -F "file=@photo.jpg" \
  -F "conf=0.5"

Object Detection — Live Video Feed

Open in browser: http://<device-ip>:8000/api/video_feed

LLM Chat — Quick Call

curl http://<device-ip>:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "rkllm-model", "messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 256}'

Vision Chat — Image + Question

curl -X POST http://<device-ip>:8002/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}
      ]
    }],
    "max_tokens": 256
  }'

Python (OpenAI client) — works for LLM Chat and Vision Chat

import openai
client = openai.OpenAI(base_url="http://<device-ip>:8001/v1", api_key="dummy")
response = client.chat.completions.create(
    model="rkllm-model",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=256
)
print(response.choices[0].message.content)

Preset

Vision Chat

Deploy the Qwen2.5-VL vision-language model to a reComputer RK3576, with an OpenAI-compatible vision API on the device for image captioning and visual Q&A.

Devices: reComputer RK3576 with 8GB+ RAM.
Software: Docker installed on the device, reachable over SSH.

Deploy Qwen2.5-VL

Deploy the vision-language model container to the RK3576.

Deployment method

Deploy to the RK3576 over SSH.

Wiring

Connect the RK3576 to the same network as your computer
Fill in device IP, SSH username, and password
Click Deploy

Troubleshooting

Symptom	Fix
SSH connection failed	Verify IP address, username, password
NPU not detected	Ensure device is RK3576 with RKNPU kernel module loaded
Out of memory	VLM requires 8GB+ RAM. Close other services to free memory
Image pull slow	Check the network. Image is about 3GB

After Deployment

Vision chat API: http://<device-ip>:8002/v1/chat/completions
API docs: http://<device-ip>:8002/docs

Try Vision Chat

Test the VLM by sending an image or text.

ModeMode

Upload an image and ask a question about it.

Troubleshooting

Symptom	Fix
Connection refused	Wait 60-120 seconds for model to load
Timeout	VLM model is large, initial load takes time