01Motivation
From prediction to control
What training does the knowledge acquired by a video generation model need before an action policy can use it effectively?
When humans learn to manipulate an unfamiliar object, they may already adjust their grip force in anticipation of upcoming load changes while their hand still deviates from the target path. Prediction can develop well before skilled control. This motor-learning result suggests a question for video-driven robot learning: what training does the knowledge acquired by a video generation model need before an action policy can use it effectively?
Video generation models learn from large-scale visual data how scenes and objects change over time. Predicting how images change and learning the actions needed to complete a task, however, have different training objectives. A lower video-prediction error, on its own, says little about how well an action policy can use what the model has learned.
In controlled experiments, we fix the demonstrations and the total number of updates and compare two recipes: learning prediction and action jointly from the start (direct training), and spending the first 3,000 of 30,000 updates on video predictive training before joint learning. Across three geometric-conditioning settings with three training seeds each, the second recipe raises LIBERO-Plus success by 17.55 to 19.16 percentage points (Section 3.3).
02Method
Two modules, two stages
The video generation model is adapted first, then the action policy is attached. The two communicate only through attention keys and values.
2.1Architecture and training stages
ZERITH V2.5 keeps the video generation model and the action policy as separate modules that communicate only through exported attention keys and values, which is what makes the staged recipe possible. Because this interface decouples visual dynamics modeling from action readout, the video generation model can first be adapted alone to ground its representation in the robot’s visual domain before any policy parameters are introduced. At both Stage 2 and deployment, a single forward pass of the video generation model supplies the memory queried by the action policy, without fully decoding future video frames. Moreover, because each policy block mirrors the transformer architecture of its source video-model block, the two branches can be initialized independently from either original or adapted video weights. At each decision step, ZERITH V2.5 takes images from up to three camera views, a natural-language instruction, and the robot’s proprioceptive state to predict a 16-step action chunk, executing a configurable number of steps before replanning; the reported deployment executes the first four (Figure 1).
Stage 1: visual adaptation
Only the video model is trained.
Both stages use conditional flow matching. The noise level σ ranges from zero for clean data to one for pure noise. Losses include only valid elements, excluding padded visual patches and action dimensions.
Stage 1. S1 keeps the current-frame latent clean and trains the video generation model to predict velocities for valid future visual patches. The four future slots share one noise level, drawn as σw = 0 with probability 0.1 and σw ∼ Unif(0, 1) otherwise, while the Gaussian noise itself is sampled independently for each slot. S1 trains the video generation model alone: the action policy and the geometry pathways do not yet exist. We refer to this distribution as the broad-noise schedule.
Stage 2. S2 initializes the video generation model from W1 and attaches the action policy, initialized from the selected original Wan layers (R0) in the deployed configuration, or from the corresponding S1-adapted video-model layers (R1) in the corresponding experimental variants. Future visual inputs are held at σw = 1, the same condition the deployed system encounters, so a single video-model forward pass yields both the visual-velocity predictions that supervise the video generation model and the features that the policy consumes. Training and deployment thereby share one forward computation, and neither fully decodes a future frame.
2.2Matched training budget
The standard endpoint controls compare schedules at 30,000 total downstream updates on the same demonstrations. In this comparison, staged training allocates the first 3,000 to S1 and the remaining 27,000 to S2; direct training allocates all 30,000 to joint updates. The two schedules use the same total number of updates, separating the effect of the staged recipe from that of additional training.
03Results
Staged adaptation at a matched budget
Same demonstrations, same 30,000 updates. Spending the first 3,000 on video prediction raises LIBERO-Plus success.
We first ask whether allocating part of a fixed target-task budget to video-model adaptation improves the policy obtained from the remaining joint-training updates. Static geometry fixes the conditioning pathway, so only the training procedure changes. Both schedules use the same LIBERO demonstrations and 30,000 total updates: direct training uses 30,000 S2 updates, whereas staging uses 3,000 S1 updates followed by 27,000 S2 updates (Table 1).
| Target training | LIBERO ↑ | Plus ↑ | ΔPlus (pp) ↑ |
|---|---|---|---|
| S2 (DL; 30k), baseline | 92.15 | 37.64 | +0.00 |
| S1 (DL; 3k) → S2 (DL; 27k) | 95.30 | 56.80 | +19.16 |
S1–S2 raises LIBERO-Plus success from 37.64% to 56.80%. The paired gain is 19.16 ± 0.26 points (mean ± SD); on standard LIBERO it is 3.15 ± 0.40 points, from 92.15% to 95.30%. The three recorded Plus gains are 19.16, 19.41, and 18.90 points. Allocating part of the same target-task budget to video-model adaptation therefore improves control in this setting, especially under the tested perturbations.
3.1Learning curves
A higher final score leaves open whether S1 repays the early updates it takes away from action learning. We therefore compare policy checkpoints against both total updates, including S1, and S2 updates.
At matched S2 updates, the staged-minus-direct differences are +0.55, −1.44, and +0.05 points at 500, 1,000, and 2,000 updates, during which both schedules remain below 5% success. From 5,000 S2 updates onward the differences are +9.84, +8.88, +8.15, and +8.06 points at 5,000, 10,000, 15,000, and 20,000 updates. After the first few thousand policy updates, the policy attached to the adapted video generation model thus achieves 8 to 10 points higher success than the policy attached to the original video generation model at every recorded checkpoint.
On the total-update axis the recorded point estimates indicate an early crossover. At 5,000 total updates, staged training reaches 4.22% after 3,000 S1 and 2,000 joint updates, versus 21.28% for direct training, a 17.06-point deficit. At 8,000 total updates, the staged checkpoint reaches 31.12%, slightly above the direct checkpoint at 10,000 updates (30.98%).
3.2Training budget sweep
Staged training achieves higher success than direct training at every budget for which both schedules are evaluated, on both benchmarks (Figure 4). The staged schedule at 30,000 updates exceeds direct training at 50,000 by 18.18 points on LIBERO-Plus, 56.70% against 38.52%, with 40% fewer updates. Across the five budgets, direct training varies between 37.20% and 38.52%, a range of 1.32 points.
LIBERO-Plus at matched budgets
Each pair is two independent runs at the same total budget
3.3Geometric-conditioning variants
This study measures the effect of changing the training schedule under each geometric-conditioning operator. The matched 2×3 comparison crosses direct S2 and S1–S2 with static, additive, and multiplicative conditioning. All runs use the same LIBERO demonstrations and 30,000 total target-task updates.
The S1–S2 gains on LIBERO-Plus are 19.16 ± 0.26, 17.93 ± 0.27, and 17.55 ± 0.27 points under static, additive, and multiplicative conditioning, respectively (mean ± paired SD). Product-over-static gains are 2.29 ± 0.30 points under direct S2 and 0.68 ± 0.30 points under S1–S2. Thus the staging benefit persists under every tested operator.
Gain from staging versus operator choice
Paired mean ± SD over three seeds, in points
04System evaluation
Benchmarks and real robots
Robot pretraining data, simulation benchmarks, Meta-World MT50, real-robot tasks, and inference cost.
4.1Robot pretraining data
The pretraining corpus contains 10,000 hours of real-robot recordings selected after cleaning: 6,000 hours of proprietary data and 4,000 hours of public data (Table 2). Proprietary data comprise our ZERITH H1 robot recordings. Public sources comprise ABC-130K, RoboMIND 2.0, AgiBot World Beta, and RoboChallenge.
| Data category | Sources | Selected hours |
|---|---|---|
| Proprietary | Our ZERITH H1 robot recordings | 6,000 |
| Public | ABC-130K; RoboMIND 2.0; AgiBot World Beta; RoboChallenge | 4,000 |
| Total | 10,000 |
4.2Effect of pretraining
The benchmark scores of Section 4.3 are obtained after robot-data pretraining. To show how they depend on this stage, Figure 6 compares a zero-robot-pretraining reference with five upstream checkpoints, each followed by an independent 30,000-update downstream adaptation using the same recipe. The zero-pretraining reference retains the original Wan initialization before downstream S1–S2 training. This experiment uses 10,000 hours of real-robot recordings selected after cleaning from public datasets and our ZERITH H1 platform.
The two benchmarks differ substantially along this trajectory. Among the robot-pretrained checkpoints from 10,000 steps through one epoch, standard LIBERO ranges from 98.10% to 98.95%, a span of 0.85 points, while LIBERO-Plus rises monotonically from 79.24% to 93.70%, a gain of 14.46 points. The one-epoch checkpoint has the highest recorded score on both benchmarks. Relative to the configuration without robot-data pretraining (94.70% and 57.21%), the LIBERO-Plus gain is 22.03 points at 10,000 pretraining steps and 36.49 points after one epoch.
Effect of robot-data pretraining
Every checkpoint followed by the same 30k-update recipe
4.3Simulation benchmarks
The deployed checkpoint achieves 98.95% on standard LIBERO and 93.70% on LIBERO-Plus (Table 3); both scores come from a single checkpoint. Among the listed baselines, Cosmos Policy has the highest standard LIBERO score, 98.5%, and π0.5 the highest LIBERO-Plus score, 85.7%; ZERITH V2.5 exceeds them by 0.45 and 8.00 points, respectively. On LIBERO-Plus, the largest margin for ZERITH V2.5 among the listed baselines is 44.56 points over Fast-WAM. Table 3 also distinguishes video-model inference treatments. Cosmos Policy jointly predicts future states in its direct-policy mode but discards those predictions; ZERITH V2.5 reads future keys and values in a single video-model pass per decision without fully decoding a frame.
| System | Backbone | LIBERO ↑ | Plus ↑ | Video inference |
|---|---|---|---|---|
| π0 | PaliGemma 3B+0.3B | 94.20 | 53.60 | — |
| π0.5 | PaliGemma+flow | 96.90 | 85.70 | — |
| OpenVLA-OFT | OpenVLA 7B | 97.10 | 69.60 | — |
| Fast-WAM | Wan2.2-5B+1B | 97.60 | 49.14 | Current-frame K/V |
| Cosmos Policy | Cosmos-Predict2 2B | 98.50 | 82.20 | Joint future-state prediction |
| ZERITH V2.5 (Ours) | Wan2.2, 7B | 98.95 | 93.70 | Future-slot K/V |
ZERITH V2.5 reaches 92.3% on Meta-World MT50 (Table 4), with success pooled over evaluation episodes. The reference scores, 47.9% for π0 as reported in SmolVLA and 81.6% for π0.5 as reported in LA4VLA, average four difficulty groups with equal weight. The 10.7-point difference from the listed π0.5 score is therefore a cross-study comparison. This extends the complete-policy evaluation to a separate suite of 50 tasks.
| System | Success (%) ↑ | Aggregation / source |
|---|---|---|
| π0 | 47.9 | Four difficulty groups, equal weight; SmolVLA |
| π0.5 | 81.6 | Four difficulty groups, equal weight; as reported in LA4VLA |
| ZERITH V2.5 | 92.3 | Pooled evaluation episodes; reported complete-policy result |
These comparisons evaluate the combined policy, including additional robot-data pretraining and future-slot readout. Because architectures, pretraining, and evaluation protocols differ across systems, the benchmark margin cannot be attributed to S1 or to the readout architecture separately.
4.4Real-robot evaluation
The quantitative evaluation measures task-level success on pick and place and bin lining.
Table 5 reports task-level success for ZERITH V2.5, π0.5, and Fast-WAM, each evaluated over 100 pick-and-place trials and 20 bin-lining trials.
ZERITH V2.5 attains the highest success on pick and place and bin lining. On pick and place it succeeds in 97 of 100 trials, compared with 95 for π0.5 and 84 for Fast-WAM, that is, 3 failures against 5 and 16. On bin lining it succeeds in 8 of 20 trials (40%), compared with 7 (35%) for π0.5 and 3 (15%) for Fast-WAM. The margins are larger on bin lining, 5 points over π0.5 and 25 over Fast-WAM, than on pick and place, 2 and 13 points, where ZERITH V2.5 and π0.5 both succeed in at least 95 of 100 trials.
| System | Pick and place ↑ | Bin lining ↑ |
|---|---|---|
| 100 trials | 20 trials | |
| Fast-WAM | 84% | 15% |
| π0.5 | 95% | 35% |
| ZERITH V2.5 (Ours) | 97% | 40% |
4.5Inference
On one H800 at batch size one, mean decision latency is 331.25 ms and the 95th percentile is 365.30 ms. The measured planning rate is 3.02 plans per second. Executing four action steps per plan corresponds to an effective rate of 12.08 action steps per second.
The video-model pass enters once per decision and is amortized across all ten solver steps and then across four executed actions, so the cost of the large module is incurred once per plan, and the term that scales with the number of solver steps is that of the ten-layer policy.
| Trainable parameters | Approximately 7B |
| Video generation model | 30-block Wan2.2-TI2V-5B |
| Action policy | 10-block flow-matching policy attending to keys and values exported from ten video-model blocks |
| Geometry pathway | 6.3M-parameter geometry bridge and 23.6M-parameter multiplicative binding module |
| Decision latency | 331.25 ms mean, 365.30 ms 95th percentile, one H800 at batch size one |
| Rates | 3.02 plans per second; 12.08 action steps per second |
05Limitations and future work
Scope and next steps
One video generation model, one adaptation length, and the LIBERO benchmark family: whether the gain carries over is untested.
Scope. All controlled comparisons use one video generation model (Wan2.2-TI2V-5B), one adaptation length (τ = 3,000 updates), and the LIBERO benchmark family. Whether the gain carries over to other backbones, task domains, or adaptation lengths is untested, so the findings above apply to this setting. The real-robot evaluation compares complete systems on pick and place and bin lining, and real-robot trials involve some randomness. Baseline scores in Table 3 are reported numbers, each from its own evaluation protocol.
Budget crossover. On LIBERO-Plus, staged training trails direct training at 5,000 total updates and leads at every evaluated budget from 30,000 updates onward. The crossing lies between these budgets, but the evaluated checkpoints cannot pin it down. The staged 8,000-update checkpoint exceeds the direct 10,000-update checkpoint by only 0.14 points, and direct training was not evaluated at 8,000 updates.
Mechanism. The initialization probes describe only the state at the start of Stage 2. The claim that this state makes action updates more effective remains a hypothesis; testing it would require an intervention.
We see two directions for future work. The first concerns generality: evaluating staged visual grounding across additional video backbones and task families, and measuring how the budget crossover and downstream gain vary with the adaptation length τ, would test how broadly the recipe applies. The second concerns multi-task reuse: the amortization threshold in the report indicates that the cost of Stage 1 adaptation can be shared across many downstream policies, making a domain-adapted video generation model a natural candidate for a shared, task-independent foundation in multi-task robot learning.
06Report and citation
Report and citation
ZERITH V2.5: From Predictive Power to Policy Learnability
The full report, including ablation studies, analysis, related work and appendices, is available as a 50-page PDF.
@techreport{zerith2026zerith25,
title = {ZERITH V2.5: From Predictive Power to Policy Learnability},
author = {{ZERITH Z-Lab}},
institution = {ZERITH Z-Lab},
year = {2026},
month = nov
}