JavaScript SDK runtime reference

Building a Reachy Mini JS app? The single source of truth is ts/APP_CREATION_GUIDE.md at the repo root. It covers scaffolding, public/icon.svg, the host shell, sdk: static deploy, mountHost() / connectToHost() API, local dev, FAQ, and the host ↔ embed contract. Pin the SDK to @pollen-robotics/reachy-mini-sdk@1.8.0 (the stable release validated against the host shell + daemon).

This file is the runtime API surface of the ReachyMini class you receive from handle.reachy once connectToHost() resolves: methods, events, properties, state machine, and the daemon-side recorded-move playback API. Bookmark it after you’ve shipped a first app from the guide.

Reachy Mini ships a browser SDK that drives a robot over WebRTC. The npm package @pollen-robotics/reachy-mini-sdk exposes:

• The ReachyMini class (the SDK runtime documented below).
• The host shell + embed adapter under the ./host* subpath exports (./host, ./host/auto, ./host/embed, ./host/protocol). See ts/APP_CREATION_GUIDE.md for the integration recipe.

Architecture

┌─────────────────────────────────┐
│  Browser                        │
│  (your app + reachy-mini-sdk.js)│
└───────┬────────────┬────────────┘
        │ SSE/HTTP   │ WebRTC (peer-to-peer)
        │ signaling  │ video + audio + data
┌───────▼──────┐     │
│  Signaling   │     │
│  Server      │     │
│  (HF Space)  │     │
└───────┬──────┘     │
        │            │
┌───────▼────────────▼────────────┐
│  Robot                          │
│  GStreamer WebRTC daemon        │
│  camera · mic · motors          │
└─────────────────────────────────┘

1. Your app is a static HTML/JS page hosted on Hugging Face Spaces.
2. The SDK handles authentication, signaling, and WebRTC negotiation.
3. The signaling server relays SDP offers/answers and ICE candidates and validates Hugging Face OAuth tokens.
4. Once the WebRTC connection is established, video, audio, and commands flow peer-to-peer; the signaling server is no longer in the path.

When you use the host shell (mountHost() + connectToHost(), documented in ts/APP_CREATION_GUIDE.md), the steps below are handled for you. The class-level API documented here is what you use after connectToHost() resolves, or what you call directly if you opted out of the host shell.

API Reference

Constructor

new ReachyMini({
    signalingUrl: "https://pollen-robotics-reachy-mini-central.hf.space",  // default
    enableMicrophone: true,  // default — request mic on startSession()
})

State Machine

'disconnected' ──connect()──▸ 'connected' ──startSession()──▸ 'streaming'
     ▴ disconnect()                ▴ stopSession()
     └─────────────────────────────┘

Properties (read-only)

Methods

setTarget head-vs-body coupling. The head matrix is in the world frame. Sending setTarget({ body_yaw }) alone rotates the body but not the head’s commanded world yaw — the head’s gaze stays fixed in world frame, so visually it appears to counter-rotate as the body turns. For tank-style “head follows body”, add the body yaw delta to the head RPY’s yaw and ship head + body_yaw in the same setTarget call. The baseline for the head yaw must be the last commanded value you tracked yourself, not state.head from the telemetry event — telemetry lags one WebRTC RTT and cumulative deltas computed against it stall under rapid input.

Events

Use robot.addEventListener(name, handler) — the SDK extends EventTarget.

Math Utilities

import { rpyToMatrix, matrixToRpy, degToRad, radToDeg } from "@pollen-robotics/reachy-mini-sdk";

rpyToMatrix(roll, pitch, yaw)  // degrees → 4×4 rotation matrix (ZYX)
matrixToRpy(matrix)            // 4×4 matrix → { roll, pitch, yaw } in degrees

Daemon-side recorded-move playback

Long recorded moves (and any move with audio) should play server-side on the daemon’s local clock, not by streaming set_target frames from the browser. The browser uploads the move once over the WebRTC data channel and the daemon ticks the inner loop at the requested frequency — no per-frame round-trip, smooth on wireless robots. When audio is attached the daemon plays it on the same GStreamer pipeline, so motion and audio share a single clock (no cross-network drift).

Combined motion + audio

const result = await robot.playMove(motion, {
    audioBlob,                    // optional, 16 kHz mono PCM WAV
    audioLeadMs: -100,            // system-wide default
    description: "happy wave",
    onProgress: (p) => console.log(p.phase, p.sent, p.total),
    onStarted: ({ duration_s, has_audio }) => { /* sync anchor */ },
});
// result is { finished: true } | { cancelled: true } | { error: "..." }

// Cancel at any time from another code path:
robot.cancelMove();

motion is the shape the Python RecordedMove parser expects:

{ time: [0, 0.01, 0.02, …], set_target_data: [{ head, antennas, body_yaw }, …] }

audioLeadMs shifts audio relative to motion at the daemon:

• Positive — audio fires N ms BEFORE motion (compensates motor pickup).
• Negative — motion fires N ms BEFORE audio (compensates GStreamer playbin warmup).
• Default -100 is the empirical system-wide constant (combined motor + pipeline). Tune only after measuring.

The encoded wire form defaults to gzip+base64 (typically ~3× smaller for recorded-move JSON). Falls back to plain JSON if the browser lacks CompressionStream.

Record-time audio (sync anchor)

For recording flows that want the SAME audio pipeline at capture AND replay (so pipeline latency cancels out and one audioLeadMs works for all recordings):

// 1. During the countdown — upload the source audio.
const audioId = await robot.uploadAudio(audioBlob, { description: "song" });

// 2. At the GO! moment — kick off daemon-side playback, await the
//    started broadcast, then start motion capture.
await robot.playUploadedAudio(audioId);
const captureT0 = performance.now();
startMyMotionCapture();

// 3. On stop / cancel / restart — stop the audio.
robot.cancelAudio();

The daemon does NOT emit a finished event for standalone audio; callers know the duration from the WAV header and call cancelAudio() when done.

Audio format

Audio must be canonical 16 kHz mono 16-bit PCM WAV. Apps are responsible for normalizing before upload — the daemon does not transcode. Format mismatch is a frequent cause of “audio is silent / wrong speed” on inherited datasets.

Backpressure & cancellation

playMove and uploadAudio pace chunk sends on the data channel’s bufferedAmount so multi-megabyte uploads (a 3-min song’s WAV is ~6 MB base64) don’t degrade other channels on the same peer connection. There’s no separate pause — to stop a long upload mid-way, close the session.

Security

• Authentication goes through Hugging Face OAuth — only users logged in to HF can access the signaling server.
• By default, you can only connect to robots registered under your own HF account.
• WebRTC connections are encrypted (DTLS/SRTP).

Prerequisites

• Your robot must be running the wireless firmware and connected to the central signaling server.
• The robot must have a valid Hugging Face token configured (see Usage).
• Currently supported on wireless versions only.

Working examples

The three reference apps maintained alongside the SDK are the canonical worked examples. They all use the host shell pattern and the current SDK pin:

• pollen-robotics/reachy_mini_minimal_conversation — vanilla TS + Vite.
• pollen-robotics/reachy_mini_emotions — React 19 + MUI 7 + Vite.
• pollen-robotics/reachy_mini_telepresence — React 19 + MUI 7 + Vite with camera + media streams.

Clone the closest one and trim. See ts/APP_CREATION_GUIDE.md for the step-by-step.