Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization

Abstract

While Vision-Language-Action (VLA) models show strong promise for generalist robot control, it remains unclear whether, and under what conditions, the standard "scale data" recipe translates to robotics, where training data is inherently heterogeneous across embodiments, sensors, and action spaces.

We present a systematic, controlled study of VLA scaling that revisits core training choices for pre-training across diverse robots. Using a representative VLA framework that combines a vision-language backbone with flow-matching, we ablate key design decisions under matched conditions and evaluate them in extensive simulation and real-robot experiments.

To improve the reliability of real-world results, we introduce a Grouped Blind Ensemble protocol that blinds operators to model identity and separates policy execution from outcome judgment, reducing experimenter bias.

Our analysis targets three dimensions of VLA scaling:

1. Physical Alignment: We show that a unified end-effector (EEF)-relative action representation is critical for robust cross-embodiment transfer.
2. Embodiment Mixture: We find that naively pooling heterogeneous robot datasets often induces negative transfer rather than gains, underscoring the fragility of indiscriminate data scaling.
3. Training Regularization: We observe that intuitive strategies, such as sensory dropout and multi-stage fine-tuning, do not consistently improve performance at scale.

This study challenges common assumptions about embodied scaling and provides practical guidance for training large-scale VLA policies from diverse robotic data.

Real-World Demonstration

To make the outcome explicit up front, we present paired rollouts for each task: a success case and a failure case. The comparison shows that our rethinking choices lead to more consistently successful real-world execution:

1. Stack Bowls: Precision manipulation.
2. Pick-to-Drawer: Long-horizon planning.
3. Wipe Board: Dynamic motion and surface contact.
4. Water Plant: Interaction with non-rigid objects.

Methodology & Evaluation

1. Mixture-of-Transformers with Flow-Matching

To combine high-frequency, precise control with strong semantic reasoning, we adopt a Mixture-of-Transformers (MoT) design.

• Dual Experts: The model comprises a Semantic Expert (initialized from a VLM to preserve priors) and an Action Expert (trained from scratch). They are connected via Layerwise Shared Attention, allowing the Action Expert to directly attend to the full visual-semantic context.
• Flow Matching: We model action generation as a conditional distribution over action chunks using flow matching, producing smooth trajectories.

2. Physically Grounded Unified Action Space

To support scalable pre-training across heterogeneous robots, we define a physically grounded unified action space $\mathcal{A}_{\text{uni}}$. This space acts as a superset of all supported physical degrees of freedom (DoF):

This includes bimanual end-effector poses, joint positions, grippers, dexterous hands, and auxiliary mechanisms. This construction allows the model to learn shared physical priors in overlapping subspaces while retaining embodiment-specific control.

3. Grouped Blind Ensemble Evaluation

Require: Model Pool M, Task Set T, Group Size K, Trials per model N

1.  Divide M into random non-overlapping groups {G1, ..., GM}
2.  For each task tau in T:
3.    For each model group Gj:
4.      H <- Anonymize(Gj)                     # Map models to aliases
5.      Q <- Shuffle({h1 x N, ..., hK x N})    # Randomized trial queue
6.      Initialize results Rj <- empty
7.      While Q is not empty:
8.        h <- Q.pop()                         # Next anonymous model
9.        Operator executes task tau with policy pi_h
10.       Record outcome r in {0, 1} to Rj
11.     Break                                  # Allow operator rest between groups
12. Return deanonymized aggregate statistics

Evaluating robotic foundation models is susceptible to experimenter bias. We propose the Grouped Blind Ensemble Protocol, a double-blind procedure where operators execute tasks without knowing which model variant is running, and outcome judgment is decoupled from execution.

Experiments

We conduct a systematic study centered on three key design dimensions: Physical Alignment, Embodiment Mixture, and Training Regularization.

1. Exploration of Physical Alignment

We evaluate four coordinate parameterizations: World-Relative, World-Delta, EEF-Relative, and EEF-Delta.

• Simulation Results: The coordinate frame strongly influences transfer behavior. On LIBERO, world-frame coordinates can look competitive in scratch training, but pre-training transfer is negative for world-frame variants and consistently positive for EEF-frame variants. In the frozen-VLM setting, EEF-Relative reaches the highest average success rate (75.1%) with the largest improvement (+8.2%).
• Real-World Results: EEF-Delta actions exhibit severe jittering and become unusable in real-robot deployment (0% in blind evaluation), while relative-coordinate variants execute effectively; in this setting, World-Relative and EEF-Relative show no significant performance gap.

Table II: Action Space Ablation on LIBERO

Table III: Action Space Ablation on RoboCasa

2. Exploration of Embodiment Mixture

With the action space fixed to EEF-Relative, we study the effect of scaling heterogeneous pre-training data. We constructed four progressively richer data mixtures:

• D1 (OXE Only): Standard public datasets.
• D2 (+Real EEF): Adding diverse real-world proprietary data.
• D3 (+Sim EEF): Further including simulation EEF data.
• D4 (+Joint): Additionally incorporating joint-space data projected into the unified model.

Unlike language model scaling, increasing robotic corpus size does NOT yield monotonic gains.

• Negative Transfer: Naively pooling heterogeneous real and simulation data (D2, D3) often degrades performance. For instance, on RoboCasa, D2 drops performance to 48.8% compared to D1's 54.7%.
• Fragility of Scaling: While projecting joint-space data in D4 recovers some of the performance lost in D2 and D3, the final mixture often fails to surpass the high-quality D1 baseline. This highlights that simply pooling structurally disparate robot datasets can induce destructive interference.
• Real-World Results: In blind real-robot evaluation, all pre-training mixtures (D1-D4) outperform training from scratch, while D1 remains the most robust setting. Adding simulated EEF trajectories introduces task-dependent degradation, especially on Stack Bowls, Pick-to-Drawer, and Wipe Board.

Table IV: Data Mixture Ablation on LIBERO

Table V: Data Mixture Ablation on RoboCasa

3. Exploration of Training Regularization

We evaluate the impact of sensory dropout (view/proprioception masking) and multi-stage training curricula.

• Limited Benefit: Stochastic modality dropout does not consistently improve performance. Disabling visual dropout ($p_{\text{view}}=0$) actually achieved higher success (85.6%) than the balanced baseline.
• Simplicity Wins: A two-stage curriculum is not essential. Direct end-to-end fine-tuning of the full model reached the best average success (85.8%), indicating that the action expert quickly co-adapts with the VLM backbone.

Table VI: Analysis of Training Dynamics and Strategies

4. Comparison with Generalist Policies

To validate that our testbed is representative of modern VLA capabilities, we benchmarked it against $\pi_0$, $\pi_{0.5}$, and GR00T-N1 under a matched 50-shot fine-tuning protocol.

Our base model is highly competitive without task-specific tuning or specialized optimizations, achieving 97.9% average success on LIBERO and 50.0% on RoboCasa. This confirms that our experimental conclusions are drawn from a high-performance regime aligned with current state-of-the-art robot learning.

Table VII: Comparison with Representative Generalist Policies

Conclusion

In this work, we present a systematic study on scaling Vision-Language-Action (VLA) models with heterogeneous robot data. To ensure reliable evaluation, we introduce a Grouped Blind Ensemble protocol that minimizes human bias in real-world experiments.

This study provides a reality check on VLA scaling:

1. EEF-Relative action space establishes itself as a reliable default for handling diverse robot kinematics.
2. Scaling cross-embodiment data is challenging. Indiscriminately mixing heterogeneous robotic data (Real/Sim/Joint) frequently degrades performance rather than improving it, highlighting the fragility of naive data scaling.
3. Simple training recipes are sufficient. Complex regularization strategies do not reliably translate into gains at scale.

Overall, our work suggests that future VLA research must move beyond simple data accumulation and focus on addressing the interference caused by embodiment heterogeneity.