Assembly Inspection — Missing Parts and Dimensions — Deployment Guide

Preset

Camera + reComputer J30 / J40 (Orin)

Detection, assembly comparison, dimension measurement, Modbus TCP and MQTT all run on one Jetson Orin. The camera can be any RTSP or ONVIF camera, a USB camera or a recorded file.

  • Model: The shipped model is trained on DeepPCB, a bare-board copper-defect dataset, and only proves the chain end to end; a real station needs a model trained on your own images.
  • Known limits: Expected-item ROIs are picture coordinates, so moving the camera invalidates the template; a tilted part or a calibration reference outside the measured plane biases measurements; all cameras share one Modbus register bank, so per-stream results come from MQTT only.
  • Network: The camera is reachable from the Jetson, and the PLC can reach port 502 on the Jetson.
1

Deploy the Inspection Runtime

Downloads the model, builds the TensorRT engine on the device (about five minutes on first deploy), then starts the runtime and its MQTT broker.

Prerequisites
  • Ports 1883, 502 and 8080 free on the device.
  • The RTSP address tested in VLC.
Troubleshooting
SymptomFix
Image ... is not on this deviceConfirm the device can reach sensecraft-missionpack.seeed.cn, then deploy again
Engine build failsConfirm /usr/src/tensorrt/bin/trtexec exists and 10 GB is free; delete a half-built *.engine.part before retrying
ONNX checksum mismatchDelete the model file and deploy again so it is downloaded again
No video from the cameraTest the RTSP URL in VLC and check the path and credentials
Container restarts every ~30 sA recorded file without looping exits at its end; this is expected
Deploy cannot connectConfirm SSH is reachable and the username is right (usually recomputer or nvidia)
Modbus port 502 refusedAnother Modbus server holds the port, or the container did not start; run docker logs edge-inspection-assembly-app
Deployment method

Deploy over SSH to a Jetson on the network. The device needs JetPack 6.x with the TensorRT dev packages, Docker with the NVIDIA runtime, and at least 10 GB free disk.

What you will be asked for(filled in the SenseCraft Solution app)
  • Jetson IP*
  • SSH Username*default recomputer

    Common defaults: recomputer (Seeed reComputer J30 / J40 series), nvidia, or ubuntu.

  • SSH Password*
  • Station Name*default line1

    First segment of the MQTT topic — name the station (line, cell or site) so several stations stay separable on one broker

  • Stream ID*default line1-pcb

    Identifies this camera in the MQTT topic and in every event

  • Inspection Camera URI*

    The camera looking at the board or assembly. An RTSP URL, a /dev/videoN device, or a path to a recorded file inside the container's /data. Test an RTSP address in VLC first.

  • Dimension Camera URI

    Optional second camera for the dimension measurement, framed on the part together with its calibration reference. Leave empty to deploy the missing-part check only.

2

Set Up the Dimension Calibration

Optional, only needed for the dimension check. When skipped, the dimension section reports uncalibrated and Modbus HR 11 = 4; the assembly comparison is unaffected.

Prerequisites
  • A calibration reference in the same plane as the measured surface: an ArUco marker (the example uses DICT_4X4_50, id 7, 25 mm wide) or an object of known width. Measure a printed marker with a caliper.
  • A known-good part to confirm the result against.
  • Step 1 completed.
Troubleshooting
SymptomFix
check_aruco.py exits non-zeroRebuild the image with opencv-contrib-python-headless
status: uncalibratedCheck that calibration.roi bounds the reference and that aruco_dict and aruco_id match the printed marker
status: not_foundRaise the contrast, or draw the measurement ROI around the right feature
Measurement is consistently off by a few percentPut the reference in the plane of the part and set ref_object_width_mm to the caliper width
Everything reads NG at nominal sizeMeasure several known-good parts and set tolerance_mm above their spread
3

Watch the Verdicts

Opens the panel on port 8080 with live counters, recent events and a preview with detection boxes and assembly ROIs.

Troubleshooting
SymptomFix
The panel does not openCheck that the host firewall allows port 8080
Panel loads but the preview is blackCheck /healthz for a rising frames_processed, then the camera URI in the container logs
The coil and the registers disagreeRead the registers first and treat the coil as the trigger
Everything is NG the moment the line startsThe expected list is still the shipped example; rebuild it for your station
dimension.enabled is false in every eventThat source has no dimension block; check the events from the calibration camera's source
After Deployment

The station publishes one MQTT event per frame and keeps the Modbus registers and coils current.

Quick verification

  1. Open http://<jetson-ip>:8080/healthz: frames_processed increases and mqtt_rejected stays at 0.
  2. Run mosquitto_sub -h <jetson-ip> -t '<station-name>/inspection/#' -C 5: each event carries assembly, dimension and verdict_reasons.
  3. Take one expected part off the fixture: missing_count rises and verdict becomes NG.
  4. Connect a Modbus client to port 502, unit 1: Coil 0 (NG) and Coil 1 (OK) are mutually exclusive, and HR 8 equals the missing count.

Configuring the expected-item list

Configure assembly and dimension per camera under sources[] in config/config.json:

  • assembly.expected[]: one entry per part slot with class (one of the model classes), roi ([x1, y1, x2, y2] normalised to this camera's picture), min_count and label (the name shown when the part is missing).
  • dimension: on the source that sees the calibration reference; calibration bounds the reference, and each measurements[] entry has an ROI, nominal size and tolerance_mm.
  • rules.ng_on_defect, ng_on_missing, ng_on_extra and ng_on_dimension decide which reasons fail a board.

Reading the outputs

Modbus TCP, unit 1, port 502:

RegisterMeaning
Coil 0 / Coil 1NG / OK, mutually exclusive
HR 0 / HR 1Primary defect class id / defect count
HR 2–5Primary bbox cx, cy, w, h, normalised x10000
HR 6–7Heartbeat, Unix seconds as a uint32 high/low word
HR 8 / HR 9Missing count / extra count
HR 10Primary measurement in millimetres x100 (long edge)
HR 11Tolerance code: 0 ok / 1 undersize / 2 oversize / 3 not_found / 4 uncalibrated

When HR 10 = 0, read HR 11 first. MQTT publishes on <station-name>/inspection/<stream-id>/results; verdict = NG does not imply defect_count > 0.

Next steps

  • Replace the example expected list with your own slots and retrain the model on your own images.
  • Point mqtt.host at a broker with credentials; the bundled mosquitto allows anonymous local access.
  • Add cameras by appending to sources[]. A reComputer J30 series unit (Orin Nano 8GB) ran 8 streams at 10 fps stably with MQTT and Modbus disabled; plan for fewer with the full I/O path.
4

Generate Expected List and ROIs with SAM2 (Optional)

Optional. Runs the upstream semi-automatic annotation tool (tools/annotation/) on a GPU workstation to generate the assembly.expected[] template and ROI profile from your own images instead of hand-writing ROIs. Nothing in this step runs on the inspection device.

Prerequisites
  • A GPU workstation with the upstream edge-inspection-assembly repository (--backend otsu needs no GPU but gives weaker results).
  • Your own station images and a COCO-style category list.
Troubleshooting
SymptomFix
The SAM2 backend fails to start or is slowConfirm the workstation has a GPU, or re-run with --backend otsu
The runtime on the device rejects the generated templateCopy the assembly.expected[] template and ROI profile from this step over together and reload
SAM2 proposes no useful boxesAdd your parts to the category list, label a few classes by hand, then re-run