GPT OSS 20B on Jetson — Deployment Guide

Preset

Jetson GPT OSS 20B Service

Deploy GPT OSS 20B to your Jetson device with one click from this platform.

DevicePurpose
NVIDIA Jetson (reComputer)Runs GPT OSS 20B in Docker
1

Deploy GPT OSS 20B Service

Deploy the containerized GPT OSS 20B runtime to your Jetson over SSH.

Deployment method

Deploy to your Jetson over SSH with one click.

Wiring
  1. Connect Jetson and your computer to the same network.
  2. Fill in Jetson IP, SSH username, and password.
  3. Click Deploy.
What you will be asked for(filled in the SenseCraft Solution app)
2 more with defaults you can keep
  • Llama NGLdefault 20

    GPU layers for llama-server (-ngl)

  • Llama Contextdefault 1024

    Context window for llama-server (-c)

Troubleshooting
IssueSolution
SSH connection failedVerify Jetson IP, username, password, and SSH service status
Docker runtime check failedEnsure Docker is installed and NVIDIA runtime is available
Docker Compose unavailableEnsure docker compose or docker-compose is installed
Service start failedInspect logs on Jetson: docker compose logs --tail=200
503 {"message":"Loading model"} on /v1/modelsModel is still warming up; first run can take several minutes
Out-of-memory at startupReduce settings, for example set Llama NGL=16 and Llama Context=512
After Deployment
  1. The GPT OSS 20B container is running on your Jetson.
  2. llama-server is started inside the container.
  3. The service endpoint is available at http://<jetson-ip>:8080.
  4. Readiness endpoint is available at http://<jetson-ip>:8080/v1/models.
2

Open Service Link

Use this step to open the Jetson service URL directly in a new browser tab.

Wiring
  1. Enter Jetson IP in this step.
  2. Click Connect.
  3. The platform opens http://<jetson-ip>:8080 in a new tab.
Troubleshooting
IssueSolution
Invalid host inputEnter a valid IP or hostname, for example 192.168.1.100
New tab not openedAllow pop-ups for this site and retry
Service page not reachableConfirm Jetson service is listening on 8080 and network is reachable
After Deployment
  1. The service page opens in your browser.
  2. You can return here and click Connect again to reopen it.

GPT OSS 20B runtime has been deployed successfully on your Jetson.

Validation Checklist

  1. Step 1 deployment status shows success.
  2. The GPT OSS 20B container stays in running state.
  3. Clicking Connect in Step 2 opens http://<jetson-ip>:8080.