Imitation learning

The LeRobot integration is the follower plugin and its policy interface; training datasets come from the simulator (teleoperated recording was removed in 2026-09). Code lives in python/lerobot_robot_tatbot/.

Wrist observations come from cameras assigned to the receiving physical arm in the vision registry. A rollout reads one or more local views; it does not open the other arm’s camera. tatbot rollout run uses the physical left arm; the follower plugin’s physical_arm selects the controller preset, fitted tool, URDF chain and floor receipt independently of its LeRobot action interface. The launcher resolves the address and controller file from config/arms.json and the hardware profile. Async rollout checks the checkpoint’s complete RGB/depth key set against those local views before acquiring motion authority. A checkpoint trained on two wrist views must be replaced or retrained for a one-view setup; the missing view is never filled with the other arm’s image.

The required floor receipt belongs to that same physical arm. A missing pad touch, stale receipt or mismatched generated tool block refuses startup. Motion aborts terminate the policy session; the retired recording-resume API is absent. Repeated staging in one connection preserves the flight CSV and the existing warning throttle.

Dataset contract

An episode should include task text, action and observation schemas, sampling rate, tool/schema version, source revision, and a clear simulated-versus-real label. Keep personal data, raw recordings, credentials, and private model artifacts out of the public repository.

Reproducibility

Pin the code revision and dependency lockfile. Validate shapes and units offline before training. Compare policy results on held-out fixtures and report failed or rejected runs rather than selecting only successful examples.

Training and physical evaluation policies are private acceptance material; this page documents only the public adapter boundary.

Evaluation ladder

Offline dataset loss checks tensor and preprocessing compatibility. tatbot sim eval policy is the next screen: a feature-only Tatbot follower client sends exact simulated RGB/depth/state through the deployed async LeRobot policy server, executes the returned chunks in ManiSkill with the rollout filter/slew semantics, then compares pigment against each episode’s exact generated design. It retains checkpoint and protocol identity, chunk accepts/rejects, intended/drawn/overlay evidence, and confidence intervals. It refuses to compare contaminated seed splits, dirty producers, or mixed tool/task contracts. A physical rollout remains a separate, operator-observed acceptance stage; sim evaluation never advances or authorizes it.