TIDE

Make deployment a learning process.

TIDE

Training in Deployment:
Human-in-the-loop Online Post-Training
for Robot Manipulation

Human-in-the-loopOnline Post-TrainingProvenance-Conditioned Learning
Towel Folding TIDE · Bimanual UR5e · Autonomous rollout
Explore the project

Real-Robot Learning

4 bimanual manipulation tasks

Precision contact · Deformable objects · Recovery

Training in Deployment

Less than 1 hour per task

From deployment rollouts to the final deployable checkpoint

Structured Online Experience

Human + policy provenance

Source-conditioned learning · Episode-boundary updates

Robot manipulation / Online post-training

TIDE

Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation

Chun Gu1,2,3*, Xiaofei Wei1,2,3*, Yiming Chen2,3, Bozhou Zhang1,3, Juntong Zhang2,3, Xiatian Zhu4, Li Zhang1,2,3†

1School of Data Science, Fudan University 2Shanghai Innovation Institute 3Logos Robotics Group 4University of Surrey

*Equal contribution. †Corresponding author (lizhangfd@fudan.edu.cn).

PDF GitHub BibTeX
TIDE couples robot deployment, human intervention, source-labeled data, online learning, and updated policy deployment.

Learn from deployment, during deployment. TIDE brings real-robot execution, human takeover, online learning, and checkpoint redeployment into one session. Execution provenance makes both human and policy experience useful for post-training.

01 / The problem & our approach

Abstract

Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment. Existing methods mitigate this issue by augmenting offline demonstrations with human-in-the-loop data or online reinforcement learning. Despite this, offline post-training adapts slowly because it cannot immediately absorb deployment experience, while online RL typically relies on value functions or auxiliary actor-critic models, increasing both training complexity and instability.

We present TIDE (Training In DEployment), a human-in-the-loop online post-training system. TIDE introduces a provenance-conditioned supervised learning framework that updates the policy online from human interventions and policy-generated rollouts collected during deployment, without a learned critic, reward model, or action scorer. Across four real-world bimanual manipulation tasks, TIDE improves milestone-normalized task progress and binary full-task completion over the warm-start policy and matched additional offline demonstrations, and performs comparably to the fixed-policy collection baseline while making updated checkpoints available during deployment. Each task-specific online post-training session, from the start of deployment rollouts until the final checkpoint is available, completes in less than one hour while learning and checkpoint handling run asynchronously.

02 / A single deployment loop

Act. Intervene. Learn. Redeploy.

A warm-started world-action policy acts on the real robot. Human interventions expose recovery behavior, while policy-executed segments show what the model can already accomplish. TIDE keeps these sources distinct as the learner absorbs deployment experience.

The TIDE runtime connects a bimanual robot client, an inference and recording server, and an asynchronous online learner.
Robot execution, provenance recording, learning, and checkpoint handoff run in the same deployment session.

Keep the source of supervision.

A chunk receives a human or policy label only when all 50 actions share that source. Chunks crossing a takeover boundary are excluded. Provenance records who executed the actions; it does not score their quality.

  • 01
    Execute & intervene
    The robot acts autonomously; the operator takes over when needed.
  • 02
    Record & condition
    Pure-source chunks condition joint training of all trainable policy modules.
  • 03
    Learn & redeploy
    New checkpoints are published every five committed episodes and adopted at episode boundaries.
Bimanual UR5e platform with a third-person camera and wrist cameras.
Bimanual UR5e · One D455 scene camera · Two D405 wrist cameras

Only episodes manually confirmed as successful enter the online dataset. The video backbone, adapter, action expert, and source embeddings are updated; the text encoder and video VAE remain frozen.

The reported session time of less than one hour includes robot execution, human takeover, episode resets, asynchronous data processing, training, and checkpoint handling, from the first deployment rollout until the final checkpoint is deployable. Initial setup and final autonomous evaluation are excluded.

03 / Four tasks, one evaluation protocol

From precision contact to deformable objects.

Each method–task pair is evaluated on 20 autonomous rollouts from the same predefined initial-state set, with no human takeover. We report milestone-normalized task progress and full-task completion separately. TIDE uses the human source condition (ℓ = 0) during these evaluations.

Four real-world tasks: Bit Insertion, Towel Folding, Towel Hanging, and Chain Packing.
Bit Insertion · Towel Folding · Towel Hanging · Chain Packing

Task progress

Completed milestones divided by the task's milestone count, averaged over 20 rollouts.

Full-task completion

All milestones must be completed. Counts are reported as k/20.

Matched comparisons

Warm-start, additional offline demonstrations, Fixed Policy, and TIDE share the task-specific starting checkpoint.

Fixed Policy is the fixed-policy collection baseline: the initial warm-start policy remains fixed throughout human-in-the-loop collection, matched to TIDE by duration. Training follows collection using the same objective, buffer configuration, filtering rules, and total optimizer-update budget as TIDE.

04 / Autonomous performance & deployment diagnostics

Better policies, available during deployment.

TIDE improves autonomous task performance over Warm-start and matched additional Offline demos. Its final performance is comparable to Fixed Policy; updated checkpoints become available within the deployment session.

Task progress and full-task completion for Warm-start, Offline demos, Fixed Policy, and TIDE across four manipulation tasks.
Top: milestone-normalized task progress. Bottom: binary full-task completion. Each method–task pair uses 20 autonomous rollouts.

Two complementary outcomes.

Progress credits intermediate milestones. Completion requires the entire task.

TIDE matches Fixed Policy on Bit Insertion and Towel Hanging progress, and is higher on Towel Folding and Chain Packing. Completion is tied on Bit Insertion and higher by one or two rollouts on the remaining tasks.

Watch the real-robot examples ↓
Full-task completions out of 20 autonomous rollouts. The paper reports milestone-normalized progress separately.
TaskWarm-startOffline demosFixed PolicyTIDE
Bit Insertion5/206/2016/2016/20
Towel Folding4/206/2012/2013/20
Towel Hanging4/204/2016/2017/20
Chain Packing6/204/2014/2016/20
Bit Insertion deployment diagnostics: observed intervention ratio over episodes, and task progress of 80% for TIDE, 65% without provenance conditioning, and 67.5% using human chunks only.
Bit Insertion: observed intervention ratio and provenance-conditioning ablation.

Learning from both sources.

On Bit Insertion, TIDE reaches 80% task progress, compared with 65% without provenance conditioning and 67.5% with human-executed online chunks only.

The observed frame-weighted intervention ratio decreases after the early deployment phase. It measures supervision burden within this task; it is not an autonomous-performance metric or, by itself, evidence of a causal learning effect.

05 / Real-robot rollouts

Before and after training in deployment.

Watch the warm-start policy and the final TIDE checkpoint on four bimanual manipulation tasks. These are illustrative autonomous rollouts with no human takeover.

Clips are independent recordings with their own playback controls. They retain the supplied playback timing and do not replace the 20-rollout quantitative evaluation.

06 / Citation

BibTeX

@misc{gu2026tide,
  title         = {Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation},
  author        = {Gu, Chun and Wei, Xiaofei and Chen, Yiming and Zhang, Bozhou and Zhang, Juntong and Zhu, Xiatian and Zhang, Li},
  year          = {2026},
  url           = {https://logosroboticsgroup.github.io/TIDE/}
}