CoRL2026

Abstract:
A prevailing view in training generalist policies is that simulation alone is not enough; effective sim-to-real transfer is widely believed to require at least some real-world data collection or task-specific fine-tuning to bridge the gap between simulated and physical environments. In this paper, we challenge that assumption: with sufficiently large-scale and diverse synthetic training data, we show that zero-shot transfer to the real world is not only possible, but state-of-the-art for both static and mobile manipulation of rigid objects. We introduce MolmoBot-Engine, a fully open-source pipeline for procedural data generation across robots, tasks, and diverse simulated environments in MolmoSpaces. With it, we release MolmoBot-Data, a dataset of 1.7 million expert trajectories for articulated object manipulation and pick-and-place tasks. We train three policy classes: MolmoBot, a Molmo2-based multi-frame vision-language model with a flow-matching action head; MolmoBot-Pi0, which replicates the \pi_0 architecture to enable direct comparison; and MolmoBot-SPOC, a lightweight policy suitable for edge deployment and amenable to RL fine-tuning. We evaluate on two robotic platforms: the Franka FR3 for tabletop manipulation tasks and the Rainbow Robotics RB-Y1 mobile manipulator for door opening, drawer manipulation, cabinet interaction, and mobile pick-and-place. Without any real-world fine-tuning, our policies achieve zero-shot transfer to unseen objects and environments. On tabletop pick-and-place, MolmoBot achieves a success rate of 79.2% in real world evaluations across 4 settings, outperforming \pi_0.5 at 39.2%. Our results demonstrate that procedural environment generation combined with diverse articulated assets can produce robust manipulation policies that generalize broadly to the real world.

Abstract:
Robust dexterous grasp synthesis has made substantial progress in generating stable, diverse, and physically plausible hand poses. Yet grasping in real manipulation is rarely task-agnostic: a functional grasp must preserve the action possibility specified by the task, contact the appropriate object region, and remain executable by a high-DoF hand. We study open-vocabulary task-oriented dexterous grasp pose generation, where a robot observes multi-view visual inputs, 3D object geometry, and a free-form language instruction, and must generate a grasp that is both physically feasible and functionally aligned. The central challenge is to establish a generalizable perception-action mapping across open-ended task semantics, functional contact selection, and dexterous hand pose generation. We present O PEN D EX G RASP, a unified data, training, and generative modeling framework. At the data level, we build OpenDexVerse, a dualsource data ecosystem constructed by a C2A Recipe (Coverage-to-Alignment): OpenDex-Scale provides broad open-vocabulary coverage through large-scale optimization-generated grasps with vision-language semantic annotation, while OpenDex-Align supplies high-quality embodied functional alignment through human teleoperation and category-level transfer. At the model level, O PEN D EX G RASP learns a shared semantic-geometric latent space that couples multi-view language-conditioned perception with 3D geometric structure. Affordance prediction and grasp pose generation are treated as complementary supervision views over this latent space, allowing the model to ground functional regions while directly generating task-consistent dexterous grasps. Extensive simulation and realrobot experiments show that O PEN D EX G RASP improves open-vocabulary task generalization, functional-region alignment, physical feasibility, inference efficiency, and real-world execution success.

Abstract:
Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Project website: https://research.nvidia.com/labs/gear/robottt/

Abstract:
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone. Unlike VLAs, WAMs learn physical dynamics by predicting future world states and actions, using video as a dense representation of how the world evolves. By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data without relying on repetitive demonstrations. This results in over 2x improvement in generalization to new tasks in new environments compared to state-of-the-art VLAs in real-robot experiments. Crucially, through model and system optimizations, we enable a 14B autoregressive video diffusion model to perform real-time closed-loop control at 7Hz.

Abstract:
Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods traditionally rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how the recently proposed flow matching policy gradient framework—which bypasses likelihood computation entirely—can be made practical for training more expressive robot control policies, both from scratch and by fine-tuning. We introduce an improved objective that enables success in legged locomotion, humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved finetuning robustness over baselines.

Abstract:
Action-conditioned world models have emerged as a promising paradigm for robot learning, offering a scalable alternative to costly real-world experimentation by generating action-consistent video rollouts. However, a fundamental challenge remains in \emphpersistent world modeling: due to frequent occlusions from the end-effector and rapid egocentric motion of the wrist camera, the information required to predict future observations is often insufficiently captured by the current observation. As a result, world models that rely only on the current observation may forget or hallucinate scene information visible in earlier frames, leading to inconsistent future predictions. Existing approaches mitigate this issue by enlarging the temporal context, retrieving observations with similar joint poses, or using camera field-of-view overlap as a retrieval cue. However, they often fail to retrieve informative history in dynamic manipulation scenarios. To address this limitation, we propose Ctrl-Mem, a memory-augmented multi-view action-conditioned world model. At its core, we introduce W-VMem, a 4D wrist-view-centered surfel-indexed memory that anchors historical observations to temporally evolving surface elements. By explicitly modeling when and where scene elements are observed, W-VMem enables geometry-aware retrieval of relevant history frames conditioned on future wrist camera poses. During generation, relevant history frames are selected via surfel-based rendering and scoring, providing informative and non-redundant context for prediction. Extensive experiments show that Ctrl-Mem generates more temporally consistent rollouts in complex manipulation scenarios, outperforming prior methods under a challenging replay-based evaluation protocol. Moreover, Ctrl-Mem enables reliable policy evaluation with strong correlation to real-world performance and improves policy performance on long-horizon tasks.

Abstract:
We present Being-H0.5, a Vision-Language-Action foundation model for cross-embodiment generalization. To address morphological diversity and data scarcity, we propose a human-centric learning paradigm treating human interaction traces as a universal ``mother tongue'' for physical skills. We curate a 35,000-hour embodied pre-training corpus spanning human and 30 robotic embodiments. Central to our approach is a unified action space that maps human hand motion and heterogeneous robot controls into semantically aligned slots, enabling low-resource robots to bootstrap from human and high-resource platforms. Built on this foundation, we develop unified sequence modeling and multi-task pre-training to jointly learn from human demonstrations and robot trajectories. Architecturally, Being-H0.5 extends conventional Mixture-of-Transformers with a novel Mixture-of-Flow, which separates shared motor primitives from embodiment-specific action experts. For real-world stability, we introduce Manifold-Preserving Gating to mitigate sensory shift and Universal Async Chunking for chunked control across varying latencies and control rates. Being-H0.5 achieves state-of-the-art results on LIBERO and RoboCasa, and demonstrates strong cross-embodiment performance across five real robot platforms.

Abstract:
A benchmark score measures success under one fixed evaluation setup, yet is routinely treated as evidence of general manipulation capability. We identify four failure modes, each of which weakens or invalidates a benchmark's role as a valid proxy for that capability: shortcut solvability, lack of statistical significance, creeping overfitting, and data-source dependence. We propose one diagnostic per failure mode. We audit LIBERO, CALVIN, SimplerEnv, RoboCasa, and RoboTwin 2.0 under these diagnostics. LIBERO and CALVIN fail multiple diagnostics. RoboCasa and RoboTwin 2.0 fail fewer, despite appearing far less often in recent progress claims. On LIBERO, a 0.09B probe with no language encoder scores at or near reported SOTA, and most reported gains are not provably statistically significant. On CALVIN, randomizing block poses within the training range drops performance for every tested policy. We release the four diagnostics with reference implementations for authors and reviewers to apply before treating a benchmark score as evidence of progress.

Abstract:
Vision-language-action (VLA) models can learn to perform diverse manipulation skills “out of the box,” but achieving the precision and speed that real-world tasks demand requires further fine-tuning – for example, via reinforcement learning (RL). We introduce a lightweight method that enables sample-efficient online RL fine-tuning of pretrained VLAs using just a few hours of real-world practice. We (1) adapt the VLA to expose an “RL token,” a compact readout representation that preserves task-relevant pretrained knowledge while serving as an efficient interface for online RL, and (2) train a small actor-critic head on this RL token to refine the actions, while anchoring the learned policy to the VLA. Online RL with the RL token (RLT) makes it possible to fine-tune even large VLAs with RL quickly and efficiently. Across four real-robot tasks (screw installation, zip tie fastening, charger insertion, and Ethernet insertion), RLT improves the speed on the hardest part of the task by up to 3× and raises success rates significantly within minutes to a few hours of practice. It can even surpass the speed of human teleoperation on some of the tasks.

Abstract:
Contact-rich insertion requires robots to combine visual alignment with tactile adjustment. However, existing visuotactile chunk-based policies collapse tactile feedback into a single generation-time fusion pathway, leaving persistent contact states, abrupt contact events, and future contact consequences implicitly modeled in a single representation. We present PACE (Phase-Aware Contact-Event Conditioning), a temporal-role tactile conditioning framework for chunk-based insertion policies. PACE organizes a shared tactile stream into three temporal roles: persistent contact states condition action-chunk generation, abrupt contact events correct the generated chunk, and contact-consequence prediction shapes action representations during training. This design preserves chunk-level action continuity while assigning distinct pathways to different temporal roles of contact feedback. Across five real-robot insertion tasks, PACE achieves an average success rate of 90.7%, outperforming vision-only and visuotactile fusion baselines by over 44 percentage points; on the ManipulationNet benchmark, PACE maintains 88.7% success across six geometry-clearance configurations, including an industrial-grade 0.1-mm clearance. Ablations confirm the complementary contributions of the three temporal roles, mechanistic analyses show that PACE separates normal insertion progress from contact events requiring immediate correction, and cross-signal and cross-architecture experiments indicate that this temporal-role principle extends across different physical feedback signals and policy backbones. Videos and materials are available at https://pace-insertion.github.io/.

Abstract:
VLA models are conditioned on a single observation at inference time, a design that suffices for quasi-static manipulation but creates two representational failures in dynamic settings. The first is motion ambiguity: a snapshot encodes configuration, not how the scene is evolving, so the policy cannot anticipate where a moving target will be. The second is state aliasing: perceptually similar observations at different task phases can demand opposite actions, producing jitter and mistimed commitments even on tasks the policy nominally understands. We show that these two failures are separable and that each has a natural fix: a motion summary from a frozen video foundation model separates scene dynamics from appearance without task-specific supervision, while a compact proprioceptive history disambiguates sequential task phases without per-task engineering. TEMPO adds these as minimal inputs to a pretrained VLA without architectural modification. On four dynamic bimanual tasks, TEMPO improves Bottle Handover success from 44% to 74% and is the only system to succeed on tasks where state-of-the-art asynchronous baselines score 0% due to unresolved state aliasing. Probing and ablation analyses confirm that each signal is individually necessary, suggesting that what limits VLAs in dynamic settings is not model capacity but the right temporal context.

Abstract:
Human behavior is among the most scalable sources of data for learning physical intelligence, yet how to effectively leverage it for dexterous manipulation remains unclear. While prior work demonstrates human-to-robot transfer in constrained settings, it is unclear whether large-scale human data can support fine-grained, high-degree-of-freedom dexterous manipulation. We present EgoScale, a human-to-dexterous-manipulation transfer framework built on large-scale egocentric human data. We train a Vision–Language–Action (VLA) model on over 20,854 hours of action-labeled egocentric human video—more than 20× larger than prior efforts—and uncover a log-linear scaling law between human data scale and validation loss. This validation loss strongly correlates with downstream real-robot performance, establishing large-scale human data as a predictable supervision source. Beyond scale, we introduce a simple two-stage transfer recipe: large-scale human pretraining followed by lightweight aligned human–robot mid-training. This enables strong long-horizon dexterous manipulation and one-shot task adaptation with minimal robot supervision. Our final policy improves the average success rate by 54% over a no-pretraining baseline using a 22-DoF dexterous robotic hand, and transfers effectively to robots with lower-DoF hands, indicating that large-scale human motion provides a reusable, embodiment-agnostic motor prior.

Abstract:
Tracking large human-motion datasets has become the dominant route to general, expressive humanoid whole-body control, but scaling across many motions is expensive. Current pipelines use model-free reinforcement learning, either merging per-motion specialists through lossy distillation, or brute-forcing a single policy with cluster-scale compute. SuperMimic amortizes learning across all motions with one differentiable world model of their shared dynamics, fit from logged transitions of a standard rigid-body simulator. The world model and the tracking policy train together in one loop, and backpropagation-through-time over short imagined rollouts gives the policy analytic gradients. Fast imagined rollouts and analytic gradients make policy learning efficient. In 3 hours on a single GPU, SuperMimic surpasses the return of a state-of-the-art motion tracker baseline trained for 7 days on 128 GPUs. It reduces GPU-hours by more than 1000× and fails on only 25 of 129514 benchmark motions, versus 179 failures for the baseline. After 9.8 hours on two GPUs, SuperMimic achieves zero failures. From 1 to 264 hours of motion, the world model's prediction error stays stable, and GPU-hours to a target return grow slowly. Our policy transfers to a physical Unitree G1 across diverse and acrobatic motions. SuperMimic removes compute as the bottleneck to scaling humanoid whole-body control, making it practical to measure motion-tracking scaling laws.

Abstract:
We present a new robotic foundation model, called \pi_0.7, that can enable strong out-of-the-box performance in a wide range of scenarios. The main idea behind \pi_0.7 is to use diverse context conditioning during training. This conditioning information, contained in the prompt, makes it possible to steer the model precisely to perform many tasks with different strategies. It is conditioned not just on a language command that describes what it should do, but on additional multimodal information that also describes the manner or strategy in which it should do it, including metadata about task performance and subgoal images. This enables \pi_0.7 to use very diverse data, including demonstrations, potentially suboptimal (autonomous) data including failures, and data from non-robot sources. Our experiments evaluate \pi_0.7 across numerous tasks that require speed and dexterity, language following, and compositional task generalization.

Abstract:
Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.

Abstract:
Despite recent advances in policy learning, long-horizon manipulation remains difficult because learned policies must avoid compounding errors while preserving future feasibility. While search-based planning can explicitly reason over such future consequences, it quickly becomes expensive in high-dimensional continuous action spaces. This raises a central question: how can long-horizon reasoning from search be transferred into policies in continuous action spaces? To this end, we propose DISCO: Distilled Tree Search over Continuous Actions. DISCO uses progressive-widening MCTS in physics simulation to generate long-horizon plans. During search, a learned policy proposes continuous actions, while a learned value function scores actions by their expected progress toward task completion. Using GPU-parallelized simulation, DISCO periodically distills MCTS-generated plans into the policy-value model, improving the action proposals and value estimates used in subsequent searches. Finally, the learned policy value model can be directly deployed without the need for simulator-based search at test time. Across simulated and real-world rearrangement tasks, DISCO out performs behavior cloning, reinforcement learning, and search-based baselines.

Abstract:
Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal. GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, we train egocentric visual policies through a sim-to-real pipeline and deploy them on a Unitree G1 humanoid, achieving 84% real-world success on diverse object pick-up and 90% success on stair-climbing. Code and data will be publicly released.

Abstract:
Contact-rich manipulation requires force sensitivity, but many robot arms lack dedicated force sensors due to their high cost. We present Neural External Torque Estimation (NEXT), a data-driven method that estimates external joint torques without needing any dedicated force sensors. NEXT trains in 1 minute from only 10 minutes of free-motion data yet achieves estimates comparable to dedicated joint-torque sensors. NEXT enables force-feedback teleoperation on low-cost arms and improves policy learning through Force-Informed Re-Sampling Training (FIRST), which up-samples pre-contact segments during behavior cloning. Across five long-horizon tasks, FIRST outperforms prior force-aware policies by over 14% in task progress. Together, NEXT and FIRST bring force-aware teleoperation and policy learning to off-the-shelf robots without additional sensing hardware. Video results and code are available at https://jasonjzliu.com/factr2.

Abstract:
Robotics datasets for imitation learning typically consist of long-horizon trajectories of different lengths over states, actions, and high-dimensional observations (e.g., RGB video), making it non-trivial to quantify diversity in a way that respects the underlying trajectory structure and geometry. We extend Shannon and von Neumann entropy to this setting by defining signature transform-based entropy on the Gram matrix of a signature kernel over demonstrations, yielding entropy and diversity metrics that operate directly on the demonstration dataset. Building on these metrics, we study how dataset diversity affects performance in robot imitation learning and propose a simple, model-free way to curate diverse demonstrations. We introduce FAKTUAL (FAst trajectory Kernel enTropy cUration for imitation Learning), a data curation algorithm that selects a subset of demonstrations maximizing entropy given a subset-size budget. FAKTUAL is fully model-free, requires no access to the imitation policy or rollouts, and adds negligible overhead relative to policy training. We evaluate our approach on image and state-based RoboMimic and MetaWorld benchmarks, as well as four real-world manipulation tasks. Across tasks and architectures, diversity-aware curation with FAKTUAL consistently improves downstream success rates over random selection, while being substantially cheaper to run compared to recent robot data curation methods. Our results suggest that the entropy of demonstration datasets is a practical tool for understanding and improving dataset diversity in robot imitation learning.

Abstract:
Elite humanoid soccer shooting requires whole-body stability, high-impulse whole-body interactions, and accuracy to targets. Motion tracking-driven reinforcement learning (RL) provides stability in whole-body movement coordination, but a fixed reference makes it hard to adapt to varied ball positions and strike timings; in contrast, task reward-driven RL struggles to explore and discover valid kicks from scratch. We therefore introduce RoboNaldo, a three-stage motion-guided curriculum RL framework for high-impulse humanoid interaction. A single human-kick reference is used as a scaffold and progressively shifts optimization towards shooting performance. The curriculum first learns stable whole-body kicking prior, then adapts the kick to free-kick settings where the ball is stationary at random positions, and finally extends it to moving-ball shooting through a locomotion-command and kick-trigger interface. A high-level heuristic planner controls this interface during training, while alternative high-level controllers can drive the same low-level policy at inference. In simulation, RoboNaldo demonstrates free-kick shot error 50% lower and shoot velocity 2.96 times than prior work baselines. In real world on a Unitree G1 with onboard perception, RoboNaldo attains 0.73m and 0.86m average target shooting error from 3m away in free-kick and moving-ball cases, accordingly. And the post-contact ball velocity reaches 13.10m/s, which is 59-71% of reported professional open-play shot speed.

Abstract:
Accurate and interpretable motion prediction for heterogeneous traffic spaces, including pedestrians, bicycles, cars, and buses, is essential for safe autonomous navigation. Nevertheless, state-of-the-art approaches remain predominantly black-box, lacking explicit encoding of the regulatory and behavioral constraints of real-world mobility. We propose Trajectory Compliance-Shaping (TraCS), a neuro-symbolic framework that augments existing black-box forecasting backbones with interpretable and probabilistic first-order logic. To do so, TraCS employs an agentic code-generation pipeline to bridge the gap between natural-language descriptions of traffic regulations and probabilistic motion prediction. Furthermore, TraCS employs a reactive data-streaming inference engine that maintains and efficiently updates compliance landscapes as scenes evolve. To prevent TraCS from overconfidently steering the backbone's predictions in the wrong direction, we propose a neural confidence rating learned as a context-aware attenuation of the compliance signal. We demonstrate on the Argoverse 2 benchmark how TraCS consistently improves state-of-the-art prediction backbones, showing that probabilistic and symbolic compliance reasoning is a broadly applicable complement for aligning purely neural motion predictors in safety-critical applications.

Abstract:
Most robotic systems rely on fixed-frequency discrete-time controllers, creating a trade-off between the efficiency of low-frequency control and the responsiveness of high-frequency feedback. As a result, systems typically default to high control rates for robustness, at the cost of wasted inference bandwidth, actuation jitter, and energy consumption. Addressing this, we introduce Time-Adaptive Robotic Control (TARC), a reinforcement learning framework in which the policy jointly predicts a control action and its duration of application. TARC learns temporally extended actions by optimizing task performance under soft or hard constraints on the number of control switches, enabling adaptive modulation of control rates. We evaluate TARC on two robotic hardware platforms: a high-speed RC car and the Unitree Go1 quadruped, and on a vision-language action model in simulation, where each query incurs a costly transformer forward pass. Across all settings, TARC matches the performance of high-frequency discrete-time controllers while operating at less than half their control frequency. Unlike fixed-rate controllers, TARC adapts its control frequency online, allocating high-frequency feedback only when required.

Abstract:
Model-based reinforcement learning has shown superior sample efficiency by leveraging the learned latent dynamics either in the form of derivative-free planning/imagined rollout training or dynamics gradient-based policy updates. While these approaches achieve comparatively fast learning, they all suffer, to varying degrees, from issues of recurrent compounding errors, high inference-time latency, and instability in backward propagation through time (BPTT). In this work, we propose a model-based RL approach that uses the chunk world model as a differentiable training-time evaluator to optimize the chunk actor without recurrent model invocation. Through the collaborative chunk-wise prediction of the world model and policy, we can generate H-step hypothetical rollouts in a single forward pass and directly optimize value expansion, avoiding the costly and unstable BPTT. We evaluate our method on 19 challenging high-dimensional locomotion tasks from HumanoidBench and deploy our method to the real-world for quadruped bipedal walking. Our code will be made publicly available.

Abstract:
Many stages of the robotic lifecycle, from morphology synthesis to operation, rely fundamentally on the reachable workspace. However, current methods for approximating workspaces are slow, imprecise, or tied to a single morphology. We introduce Reachability Across Morphologies (RAM): a morphology-conditioned, implicit neural representation that acts as a fast, differentiable surrogate for pose reachability, generalising to unseen morphologies while inherently accounting for self-collisions. To train RAM, we publish a large-scale dataset of 3\cdot10^10 samples generated solely from forward kinematics. Experiments show that our model achieves an F_1-score of 86% at nanosecond inference, outperforming the baseline by 14% while reducing inference time by three orders of magnitude. We further demonstrate speed-ups of one and two orders of magnitude for gradient-based morphology and trajectory optimisation, respectively.

Abstract:
Current agentic robot systems can write executable Code-as-Policy programs, observe feedback, and revise behavior across multiple attempts, but they remain largely task-driven: reusable skills are acquired only after explicit instructions. We study Playful Agentic Robot Learning, where an embodied coding agent uses self-directed play as a continual skill-learning stage before downstream tasks arrive. We introduce RATs, a Robotics Agent Team designed for play-time skill acquisition. During play, RATs proposes novel yet learnable exploratory tasks, plans and executes robot-code policies, verifies intermediate progress, diagnoses failures, retries with feedback, and distills successful executions into a persistent code skill library. This structured agent team turns sparse task outcomes into dense feedback for learning reusable skills from autonomous interaction. Experiments in LIBERO-PRO and MolmoSpaces show that play-learned skills improve held-out downstream tasks over no play and random-play baselines, with a 20.6 and 17.0 percentage-point gains over CaP-Agent0 on LIBERO-PRO and MolmoSpaces, respectively. Moreover, the learned skills can be plugged into other inference-time Code-as-Policy agents by simply retrieving them into the context, improving RoboSuite transfer by 8.9 points without finetuning the model.

Abstract:
Diffusion models can represent complex, multimodal trajectory distributions, but extracting uncertainty from them typically requires costly Monte Carlo sampling. This limits their use in real-time control, where robots must rapidly assess risk and maintain safety margins. We introduce Score-Curvature for Online Precision Estimation (SCOPE), a lightweight module that augments diffusion trajectory models with control-ready uncertainty. SCOPE learns a structured precision matrix around each nominal trajectory by distilling score-curvature information and producing calibrated Gaussian tubes with low overhead and without repeated Monte Carlo sampling. These tubes provide per-timestep covariance estimates that can be used both as predicted occupancy for moving agents and as adaptive exploration guides for robot control. We evaluate SCOPE in pedestrian forecasting, crowd navigation, Maze2D control, and real-world Franka Panda manipulation. Across these settings, SCOPE provides fast uncertainty estimation, which leads to better closed-loop performance.

Abstract:
Vision-Language-Action (VLA) models inherit rich semantic representations from pretrained Vision-Language Models, yet fine-tuning on limited robot demonstrations degrades this structure and undermines generalization. A fundamental question therefore arises: what constitutes a good action representation? Inspired by the mirror neuron theory's insight that observation and execution share an intention-level encoding, we examine whether a robot's action representations preserve the semantic structure captured by pretrained encoders. Systematic probing confirms that this structure erodes during fine-tuning, and that its quality synchronizes with both task success and out-of-distribution generalization. We further introduce a plug-and-play method that anchors action representations to a semantic manifold while decomposing representations into a shared semantic channel and a private channel, all discarded at inference, leaving the deployed model unchanged. Validated on different VLA backbones across simulation and real-world benchmarks, our method yields up to +18.7% on real-world in-distribution tasks and +21.5% on out-of-distribution generalization.

Abstract:
Manipulation in submerged settings (e.g., washing dishes, bathing, collecting water samples) is difficult because of the air-water interface. Water makes visual cues unreliable and damages electronics. We learn visuo-tactile policies for this regime by fusing visual observations with tactile and immersion-depth signals from a sensorized waterproof gripper. We train imitation learning transformer policies that use submerged-domain visual augmentations to reduce overfitting to turbidity, reflection- and refraction-related artifacts. Across the evaluated submerged manipulation tasks, our full policy achieves a mean success rate of 71%, compared to 41% for the best no-tactile variant. This improvement is especially pronounced under difficult visual conditions, such as floating foam and strong glare. Overall, our results demonstrate combining visual robustness with multimodal tactile sensing improves submerged manipulation. See https://davatar-corl26.github.io/ for details.

Abstract:
Embodied foundation models have significantly propelled the advancement of artificial general intelligence, yet they rely heavily on large-scale data. Recent studies identify human behavior as one of the most scalable sources for fostering robotic physical intelligence. However, the inherent embodiment gap remains a fundamental challenge for effective human-to-robot learning. In this work, we model human intention to enable efficient learning from human demonstrations. Among human behavioral cues, gaze simultaneously encodes high-level task semantics and exhibits a synergistic coupling with low-level action. By modeling intention through gaze, we derive a mid-level representation that facilitates cross-embodiment knowledge transfer. We demonstrate that robots can internalize human intentions to enhance manipulation, even without robotic intention annotations. To capture the synergy between intention and action, we adopt a Chain-of-Thought reasoning paradigm during inference. Our approach enables the execution of long-horizon and fine-grained tasks across diverse embodiments. Extensive experiments in both simulation and real-world demonstrate that our method consistently outperforms baseline approaches and generalizes to unseen settings.

Abstract:
Deep unfolding (DU) accelerates iterative optimizers by introducing learnable components and training them through unrolled iterations, but extending DU to the large-scale semidefinite programs (SDPs) common in robotics has remained limited. Unrolling a full-update conic solver such as COSMO exposes two obstacles that prior work on learned conic solvers has not: backpropagating through the per-iteration linear-system solve incurs memory quadratic in the problem size once the coefficient matrix is formed explicitly, and backpropagating through the positive semidefinite (PSD) cone projection becomes numerically unstable when eigenvalues coincide. We address the first obstacle with a matrix-free implicit differentiation rule that operates entirely through matrix-vector products, reducing memory from O(n^2) to O(n) and enabling backpropagation at scales where direct factorization runs out of memory. We address the second with a backward rule based on the Dale\vckii--Krein representation of the Fr\'echet derivative, which remains well-defined under repeated eigenvalues. Together these make it possible to learn lightweight hyperparameter policies and warm-starts for a full-update conic solver. We evaluate on nonlinear covariance steering problems solved via sequential convex programming (SCP), as well as standalone SDPs and second-order cone programs ranging from max-cut and Lov\'asz \vartheta SDPs to robust estimation and control problems. The learned policies outperform state-of-the-art solvers across all problems, and can provide up to a 50× speedup depending on the class. When used as a subroutine in SCP, the learned approach delivers over a 30× speedup compared to COSMO.

Abstract:
Humanoid robots hold great promise for operating in human-centered environments, yet ladder climbing remains one of the most challenging tasks due to sparse footholds and handholds, complex whole-body coordination, and sensitivity to perception and control errors. We present LadderMan, a unified system that enables humanoid robots to robustly climb diverse ladders and perform manipulation under such constrained conditions. Our climbing policy is built on a scalable two-stage learning pipeline, where we use hybrid motion tracking to learn multiple climbing experts from a single reference motion, and distill these experts into a unified depth-based visuomotor climbing policy via hybrid imitation and reinforcement learning. To enable real-world deployment, we leverage vision foundation models to bridge the sim-to-real gap in depth perception. Building on the learned climbing policy, we further train a separate manipulation policy using a dual-agent formulation, allowing stable on-ladder manipulation via teleoperation. Experiments demonstrate that LadderMan achieves robust ladder climbing across a wide range of geometries, successfully transfers to real-world hardware in a zero-shot manner, and supports various manipulation tasks under challenging ladder constraints. Video results are available at https://ladderman-submission.github.io

Abstract:
Motion imitation is a promising approach for humanoid locomotion, enabling agents to acquire humanlike behaviors. Existing methods typically rely on high-quality motion capture datasets such as AMASS, but these are scarce and expensive, limiting scalability and diversity. Recent studies attempt to scale data collection by converting large-scale internet videos, exemplified by Humanoid-X. However, they often suffer from physical artifacts such as floating, penetration, and foot skating, which hinder stable imitation. To address this, we introduce PHUMA, a Physically Reliable HUMAnoid locomotion dataset produced by a two-stage pipeline combining physics-aware curation and physics-constrained retargeting, aggregating both motion capture and internet video into a physically reliable, 73-hour corpus. On motion tracking benchmarks, PHUMA-trained policies achieve higher success rates than those trained on AMASS and Humanoid-X, and successfully transfer zero-shot to a real Unitree G1.

Abstract:
Vision-Language-Action (VLA) models aim to provide a single generalist controller for robots, but today's systems fall short on the criteria that matter for real-world deployment. Frontier models are closed, open-weight alternatives are tied to expensive hardware, reasoning-augmented policies pay prohibitive latency for their grounding, and fine-tuned success rates remain below the threshold for dependable use. We present MolmoAct2, a fully open action reasoning model built for practical deployment, advancing its predecessor, MolmoAct along five axes. We introduce Molmo2-ER, a VLM backbone specialized for spatial and embodied reasoning, trained on a 3.3M-sample corpus with a specialize-then-rehearse recipe. We release three new datasets spanning low-to-medium cost platforms: MolmoAct2-BimanualYAM Dataset, 720 hours of teleoperated bimanual trajectories that constitute the largest open bimanual dataset to date; MolmoAct2-DROID Dataset, a quality-filtered Franka subset of DROID; and MolmoAct2-SO100/101 Dataset, a quality-filtered SO-100/101 subset. We provide OpenFAST Tokenizer, an open-weight, open-data action tokenizer trained on millions of trajectories across five embodiments. We redesign the architecture to graft a flow-matching continuous-action expert onto a discrete-token VLM via per-layer KV-cache conditioning. Finally, we propose MolmoAct2-Think, an adaptive-depth reasoning variant that re-predicts depth tokens only for scene regions that change between timesteps, retaining geometric grounding at a fraction of prior latency. In the most extensive empirical study of any open VLA to date, spanning 7 simulation and real-world benchmarks, MolmoAct2 outperforms strong baselines including π0.5, while Molmo2-ER surpasses GPT-5 and Gemini Robotics ER-1.5 across 13 embodied-reasoning benchmarks. We release model weights, training code, and complete training data.

Abstract:
A robotic world model should predict how objects evolve under robot actions and physical dynamics. While recent 3D world modeling approaches have made progress toward general robotic prediction, they often struggle to capture the heterogeneous dynamics underlying diverse object interactions. In this paper, we propose MoRE, a novel 3D world model that organizes robotic dynamics into rigid, deformable, and prehensile regimes using a mixture of motion experts built on a shared encoder backbone. This design enables a single model to capture diverse physical behaviors while preserving regime-specific inductive biases. We train a 1B-parameter version of MoRE on a mixture of simulation data generated in Isaac Sim and real-world robotic interaction data. We further integrate MoRE with Model Predictive Control (MPC), enabling a range of manipulation tasks with a unified model. Experiments show that MoRE consistently improves prediction accuracy across rigid, deformable, and prehensile interaction benchmarks, leading to stronger real-world MPC performance.

Abstract:
Deploying powerful Vision-Language-Action (VLA) models on edge devices is limited by their massive size. In this paper, we take a deployment-oriented view of VLA training: we target efficiency through model design and optimization, rather than relying solely on post-hoc compression. Thus, we propose BitVLA, a fully native 1-bit VLA model for robotic manipulation, where every parameters is ternary, i.e., -1,0,1. BitVLA is built on the publicly available 1-bit LLM BitNet b1.58 2B4T, and is trained as a vision-language-action policy that inherits the compactness of 1-bit pretraining while retaining strong task performance. To further reduce the memory footprint of the vision backbone, we introduce Quantize-then-Distill, a post-training quantization-aware training strategy that compresses a full-precision vision encoder to 1.58-bit weights, while a full-precision teacher guides representation alignment during training. Across simulation benchmarks and real-world tasks, BitVLA matches the performance of the full-precision OpenVLA-OFT baseline, while reducing model memory by 11.0x and end-to-end latency by 4.4x. These results suggest a practical path toward training-time efficiency–accuracy co-design for embodied policies, enabling competitive manipulation capability on memory-constrained edge robotic platforms.

Abstract:
Sequential robot manipulation requires policies to execute novel combinations of familiar instruction components. However, collecting demonstrations for all possible instruction tuples is combinatorially expensive, while sparsely covered datasets often fail under out-of-distribution recombination. This paper studies compositional generalization through the lens of instruction-space coverage. We decompose the generalization gap into three sources: marginal instruction shift, instruction-compositional shift, and context--action shift. This decomposition allows us to diagnose when sparse training coverage is sufficient, and what structure the training set must preserve for reliable action prediction. Our results show that exhaustive tuple enumeration is unnecessary: a structured subset, as small as one quarter of the full task space, can recover strong out-of-distribution performance when it covers action-relevant dependencies. We further find that sparse training often fails due to instruction steering rather than missing low-level skills; finetuning only one demonstration per task improves OOD success from 0.4% to 54.7%. For semantically dependent tasks, effective coverage must capture relational structure rather than only factor diversity. These findings suggest that efficient robot data collection should prioritize dependency coverage in instruction space over exhaustive task expansion. More results are available in the supplementary material. Code and datasets will be released soon.

Abstract:
Integrating contact information into visuomotor policies remains an open problem. Touch is essential to robust manipulation, yet most modern policies, including pretrained vision-language-action (VLA) models, operate from vision and proprioception alone. Existing approaches to closing this gap require specialized tactile hardware, add separate tactile encoders, or commit to non-image policy backbones, all incompatible with the modern paradigm of image-conditioned policies built on pretrained 2D visual representations. Our key insight is that the bottleneck is not the contact information itself, but how it is delivered: when contact signals are exposed in the same spatial frame as the scene the policy already attends to, they become directly usable by any image-conditioned policy without architectural changes. We operationalize this insight in Visible Touch, paired with a custom low-cost magnetic contact sensor that is open-sourced and fabricated from off-the-shelf parts via a parametric CAD-to-mold pipeline. Across the LIBERO benchmark, Visible Touch improves BC-Transformer success rates by 16 percentage points on average, with controlled comparisons showing that how contact is represented matters more than what contact information is provided. The pattern holds when fine-tuning pretrained VLAs: miniVLA on LIBERO gains 29 points on average, and \pi_0.5 on four real-world contact-rich tasks gains 30 points with our custom sensor.

Abstract:
Theories of cognition suggest that adaptive behavior may emerge through minimization of surprise associated with sensory states rather than explicit reward maximization. Motivated by this distinction, we present a framework for embodied control based on minimization of deviations from priors over preferred states. The framework explores two complementary regimes of adaptive control: (1) a model-based free-energy planner that performs predictive control through learned world models and trajectory optimization, and (2) a model-free free-energy controller trained through deep policy optimization. Importantly, actions are generated through inference over preferred states and maintenance of predictive continuity rather than optimization of scalar reward signals. Across continuous locomotion, autonomous driving, and underactuated control environments, the framework produces coherent and adaptive behavior without access to explicit environment-defined rewards. In a sparse-reward control setting, the framework learns goal-directed behavior solely from preferred-state priors. Stable locomotion emerges from homeostatic and kinematic priors alone, while model-based free-energy planning yields collision-averse and survival-oriented driving behavior. These results suggest that complex embodied behavior can emerge from minimal, self-supervised, free-energy-based objectives, providing an alternative paradigm to reward-centric reinforcement learning.

Abstract:
Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While recent visual navigation models offer a promising solution, the lack of a unified benchmark hinders quantitative and reproducible evaluation. To bridge this gap, we propose SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks. Built upon NVIDIA Isaac Sim, SidewalkBench brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes. We further populate the scenes with rich, reactive event-based pedestrian behaviors and flexible, efficient animation, enabling standardized model evaluation under realistic real-world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution. All benchmarking code and scenarios will be made publicly available.

Abstract:
Generalist robotics methods have recently seen a rapid growth in both popularity and capability, sparked by advancements in Vision Language Action models (VLAs), offline learning algorithms, and reinforcement learning (RL) techniques. These methods, though highly versatile and capable, are also extremely data hungry, requiring access to large and diverse training datasets to be successful. This has lead to the development of neural network based, predictive world models as a more lightweight means for generating synthetic training data or simulating real world environments, with many works proposing their integration into the robot learning supply chain. While highly practical, in this work we demonstrate that world models introduce a uniquely stealthy and effective data poisoning entry point into the robot learning supply chain that can result in the deployment of unsafe or otherwise compromised robotic policies despite training on seemingly safe ground truth training data. In contrast to traditional data poisoning techniques which directly implant dangerous trajectories into sold or uploaded datasets, our novel attack methods inject malicious prompts or compromising transition dynamics into visibly safe teleoporated datasets which are only activated once fed through a world model as input. This results in the generation of synthetic, dangerous robot training trajectories and subsequently unsafe or compromised robot policies. We demonstrate the effectiveness of our attacks against both state of the art action conditioned and text conditioned world models and provide practical examples of these attacks propagating to downstream VLA and RL policies. Overall these findings necessitate research into more secure world models and reevaluation their position within the robot learning supply chain.

Abstract:
We propose Diffusion ReRoll, a diffusion-based framework for robotic sequential prediction that enables revisable denoising over horizons. Existing diffusion-based sequence predictors typically perform a single monotonic denoising process. In contrast, Diffusion ReRoll selectively re-noises regions that have become locally stable while the remaining regions continue denoising, so the re-noised regions can be refined again using context from the rest of the horizon. This structured re-noising enables iterative cross-horizon revision, allowing earlier and later segments to revise one another, while maintaining local consistency. We evaluate Diffusion ReRoll against full-sequence diffusion and causal denoising based on Diffusion Forcing across long-horizon planning, policy learning, and unified video-action modeling. On OGBench PointMaze and AntMaze, Diffusion ReRoll achieves relative gains in average success rate of 21% over Diffusion Forcing in matched guidance-based planning and 23% over Diffuser in matched goal-inpainting. In diffusion-policy-style action prediction, Diffusion ReRoll improves average success by 56.5% relative to Diffusion Policy across different prediction horizons and history lengths on the LIBERO-10 multi-task benchmark. In unified video-action prediction, Diffusion ReRoll improves policy and inverse dynamics performance, especially under out-of-distribution evaluation, and achieves the best action-video consistency. These results support structured re-noising as an effective mechanism for revisable robotic sequence generation.

Abstract:
Simulating deformable objects is essential for a wide range of robotic manipulation applications, yet accurately predicting their dynamics remains challenging. We propose Physics-Guided Residual Dynamic (PGRD), a hybrid simulation framework that combines the advantages of physics-based and learning-based approaches. Specifically, PGRD combines an optimizable spring–mass simulator as a backbone with a learned neural network that predicts residual corrections to the physics-based predictions. We adopt a velocity-based formulation to ensure stable simulation and a sliding-window transformer architecture to capture temporal dependencies. We show that PGRD produces more accurate results than both purely physics-based and learning-based methods on a set of diverse real-world deformable objects. We further demonstrate the utility of PGRD in two applications: manipulation planning via Model Predictive Control, including a language-conditioned setting with a generated goal image; and interactive simulation via action-conditioned video prediction by 3D Gaussian Splatting.

Abstract:
Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online, a paradigm proven effective in rigid-body manipulation where state and contact are relatively tractable. We bring this paradigm to real-world cloth manipulation from a single RGB input through three pillars. (i) We design a scalable synthetic-data generation and inference-time rollout pipeline built on FLASH, a deformable-object simulator that provides a practical balance among physical fidelity, numerical stability, and rollout efficiency. (ii) We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state by fusing pretrained visual features with learnable canonical tokens. (iii) We perform online planning by coupling a sparse-mesh rollout backend with prior-guided MPPI, anchored at an offline-distilled policy trajectory, preserving manipulation-relevant deformation and contact while enabling sufficient parallel rollout batches. Real-robot experiments show higher success rates and stronger robustness than baseline methods.

Abstract:
Deformable Linear Objects (DLOs), such as wires and cables, are central to industrial assembly. Unlike rigid objects, whose state is captured by a 6-DoF pose, DLOs have an infinite-dimensional configuration space and deform continuously under contact with grippers, fixtures, and the workspace, making them a demanding benchmark for general dexterous manipulation. Despite their importance, policy development and comparison remain difficult: existing benchmarks are often tied to specific hardware setups, lack modular and customizable task assets, or study generic deformable-object tasks without the fixtures relevant to real-world industrial wire manipulation. Few benchmarks align simulation, real-world data, and shared evaluation protocols. To bridge this gap, we introduce WireCraft, a simulation benchmark for industrial DLO manipulation with configurable difficulty and assets, spanning three task families: connector insertion, clip routing, and channel seating. It supports two complementary DLO physics models, articulated and deformable, and the released trajectories come from both simulation and a physical UR5. We benchmark reinforcement learning (RL), imitation learning (IL), and vision-language-action (VLA) policies under shared metrics. Privileged state-based RL solves a representative setting in each task family with over 82% success, confirming the tasks are well-posed. For connector insertion, however, the transition from reaching the socket to contact-rich alignment remains a key bottleneck for vision RL, IL, and VLA policies. These results indicate that industrial DLO manipulation, though tractable under privileged state, remains an open challenge for current vision-based learning.

Abstract:
Fine-grained, contact-rich teleoperation remains slow, error-prone, and unreliable in real-world manipulation tasks, even for experienced operators. Shared autonomy offers a promising way to improve performance by combining human intent with automated assistance, but learning effective assistance in simulation requires a faithful model of human behavior, which is difficult to obtain in practice. We propose a real-to-sim-to-real shared autonomy framework that augments human teleoperation with learned corrective behaviors, using a simple yet effective k-nearest-neighbor (kNN) human surrogate to model operator actions in simulation. The surrogate is fit from less than five minutes of real-world teleoperation data and enables stable training of a residual copilot policy with model-free reinforcement learning. The resulting copilot is deployed to assist human operators in real-world fine-grained manipulation tasks. Through simulation experiments and a user study with sixteen participants on industry-relevant tasks, including nut threading, gear meshing, and peg insertion, we show that our system improves task success for novice operators and execution efficiency for experienced operators while preserving operator intent compared to direct teleoperation and shared-autonomy baselines that rely on expert priors or behavioral-cloning pilots. In addition, copilot-assisted teleoperation produces higher-quality demonstrations for downstream imitation learning.

Abstract:
Dexterous robotic manipulation requires perception that remains informative from pre-contact approach to contact initiation and post-contact control. We introduce FingerEye, a sensing and learning framework that strengthens robotic dexterity through continuous vision-tactile feedback throughout interaction. On the sensing side, FingerEye integrates binocular RGB cameras with a compliant contact interface to support perception both before and after contact. Before contact, the fingertip cameras provide close-range visual cues and implicit stereo for precise approach and object localization. After contact, marker-tracked deformation of the compliant ring provides a proxy for contact wrench sensing. On the learning side, we build real-and-sim infrastructure for data collection and evaluation, systematically study policy-interface designs for learning with multiple FingerEye sensors, and develop FingerEye Policy, which applies group-structured modality fusion to reduce modality shortcuts and better exploit distributed fingertip feedback. Across seven contact-sensitive task settings, FingerEye improves wrist-only policy by over 30 percentage points in mean success rate in both simulation and the real world.

Abstract:
Training non-prehensile manipulation policies in contact-rich settings is a core challenge in robotics. While Reinforcement Learning (RL) has demonstrated its strength in such settings, it may struggle to sufficiently explore and discover complex manipulation strategies. To address this, we combine two basic ideas: First, designing appropriate reset strategies (the start state distribution of episodes) has shown promise in improving RL exploration and effectiveness. Second, while model-based approaches to finding trajectories through manipulation are hard, recent work showed that model-based approaches to sampling states on constrained manifolds can be highly efficient. Based on these observations, we propose a novel state sampler that boosts the performance of goal-conditioned RL in complex contact-rich manipulation tasks. Our sampler explicitly takes into account the structure of contact in order to provide a rich covering of diverse contact modes. By combining constrained sampling resets with projected interpolation and curriculum learning, our novel approach outperforms RL without constraint sampling and alternative goal-conditioned methods, and effectively trains universal, non-prehensile, and dynamic manipulation policies in contact-rich settings.

Abstract:
Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects. This is difficult because failures often arise not from poor local skills, but from mistimed waiting, yielding, pulling, releasing, or repositioning. We study this problem with two bimanual mobile manipulators coupled through rigid and deformable objects. We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors without synchronized dual-operator demonstrations or explicit inter-robot communication. SAI trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures. This staged process exposes the policies to increasingly realistic partner behaviors, including delay, phase mismatch, insufficient yielding, and interaction conflict. Across real-world dual-robot manipulation tasks, SAI improves task success, phase synchronization, and partner-contingent yielding over independent imitation and curriculum-ablation baselines. These results suggest that physically coupled collaboration can be learned through the structure of the imitation curriculum, rather than through synchronized multi-operator demonstrations or explicit coordination mechanisms.

Abstract:
Semantic scene understanding has enabled robots to reason and perform affordance-driven object interaction. Despite this, real-world manipulation remains fundamentally limited by the inability to predict how objects move and are articulated. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We lift point trajectories to 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with articulations and detect contained objects by reasoning over parent-child relations at identified opening states. We also introduce the Arti4D-Semantic dataset, which provides parent-child relation annotations between container and contained objects together with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate robust manipulation of articulated objects in everyday home environments. We provide code and data at: anonymized-for-review.

Abstract:
We present RoboKit, a performant, general-purpose toolkit that extends the value of optimization to robot learning. It is built around three principles: (i) \emphPerformance and flexibility: implemented in a kernel-level language, it pairs multi-seed sampling with versatile optimizers. With well-tuned engineering, it runs over 10× faster than prior frameworks; (ii) \emphGeometry and rendering: it makes geometry and differentiable rendering first-class and accelerates both with hybrid and coarse-to-fine representations. It can optimize 1024 grasps in 2\,s; (iii) \emphIntegration with learning: every cost is differentiable and plugs directly into learning frameworks, letting us distill optimization into policies. The learned policies run significantly faster while matching or surpassing optimization performance.

Abstract:
Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and \Psi_0) using less than half the total demonstration time. We will release our code, data, and model checkpoints to support future research.

Abstract:
A mobile robot following a graph of known routes can make costly navigation errors when a temporary obstacle blocks a critical edge: waiting too long behind a parked cart wastes time, but immediately rerouting around a person who would move in a few seconds is also inefficient. Standard reactive obstacle avoidance addresses local motion around obstacles, while fixed wait-or-reroute rules ignore how long different obstacle types tend to persist. We propose OSCAR: an adaptive survival-modeling framework for graph-based navigation with temporary blockages. Assuming obstacle class labels are available at encounter time, the robot learns class-conditioned residual clearance-time distributions from online experience, including right-censored observations when it reroutes before observing clearance. These survival models are integrated into a time-dependent graph planner that maintains obstacle memory and computes a patience threshold at each blocked edge: how long to wait before taking an alternate route. The method continuously updates its clearance estimates across episodes and uses them to balance waiting against rerouting. We evaluate the approach in simulation and on a real mobile robot in a university atrium with obstacles including people, chairs, bins, and tubes. In simulation, the learned policy's time-to-goal converges to within 1% of an oracle with access to ground-truth clearance distributions after fewer than 20 observations per obstacle class, outperforming all heuristic baselines. Real-world deployment confirms that the policy improves online, adapting its patience thresholds from experience across 50 navigation episodes.

Abstract:
Conventionally, memory in end-to-end robotic learning involves inputting a sequence of past observations into the learned policy. However, in complex multi-stage real-world tasks, the robot’s memory must represent past events at multiple levels of granularity: from long-term memory that captures abstracted semantic concepts (e.g., a robot cooking dinner should remember which stages of the recipe are already done) to short-term memory that captures recent events and compensates for occlusions (e.g., a robot remembering the object it wants to pick up once its arm occludes it). In this work, our main insight is that an effective memory architecture for long-horizon robotic control should combine multiple modalities to capture these different levels of abstraction. We introduce Multi-Scale Embodied Memory (MEM), an approach for mixed-modal long-horizon memory in robot policies. MEM combines video-based short-horizon memory, compressed via a video encoder, with text-based long-horizon memory. Together, they enable robot policies to perform tasks that span up to fifteen minutes, like cleaning up a kitchen, or preparing a grilled cheese sandwich. Additionally, we find that memory enables MEM policies to intelligently adapt manipulation strategies in-context.

Abstract:
Learning generalizable robot manipulation policies requires large-scale and diverse demonstration data. Egocentric human manipulation videos offer rich scene and task diversity, and prior work has shown that retargeting and rendering such videos into robot-format data can yield effective per-task policies at small scale. However, whether this approach can provide pretraining benefits for vision-language-action models at scale remains unexplored. We present Ego2Robot, a scalable pipeline that converts egocentric human manipulation videos into robot training data through action retargeting, robot-arm visual synthesis, and multi-level quality curation. Ego2Robot supports both curated datasets and in-the-wild videos, producing 18,561 hours of robot training data spanning 15 robot morphologies, making it the largest ego-to-robot dataset to date. To evaluate generalization, we extend RoboTwin2.0 with disentangled perturbation axes covering visual appearance, scene layout, embodiment morphology, and task semantics. Experiments show that joint pretraining on Ego2Robot-synthesized and robot data consistently improves out-of-distribution generalization across multiple perturbation types, with benefits validated on real-robot deployment.

Abstract:
We propose to turn generalist multi-embodiment value functions into reusable models for robot design. Instead of running a new reinforcement learning co-design loop for each robot, we first train an embodiment-aware policy and value function across many robot designs. After training, the frozen value function is used as a differentiable surrogate to optimize candidate embodiments through value gradients. We evaluate our approach across different robot design settings, from perturbed single robots to held-out robots across morphology classes, with single models trained on up to 50 robots and design spaces of over 1100 continuous embodiment parameters. Beyond optimizing complete embodiments, we show that value gradients can identify performance-limiting design and control parameters, enabling both the optimization and the analysis of new robot designs.

Abstract:
Generating high-quality dexterous grasps remains challenging for learning-based methods, which often depend on carefully tuned contact losses or costly contact-based test-time refinement. We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data rather than relying on contact losses or contact-based test-time refinement. KPGrasp couples an all-Euclidean 3D hand-keypoint parameterization with a simple yet scalable Transformer flow model. The parameterization avoids the drawbacks of the conventional mixed SE(3) pose and joint-angle output space, expresses grasps in the same frame as the object point cloud, and thus enables native spatial reasoning; the Transformer flow model is trained with only the standard flow-matching loss and scales effectively with data, model capacity, and batch size. Experiments demonstrate state-of-the-art performance on two simulation benchmarks. On the Dexonomy benchmark, it reaches a 76.3% grasp success rate, improving over the strongest directly comparable baseline by 47.4% while reducing penetration depth to 2.4 mm. The same model also achieves the best average performance on the DexGrasp Anything benchmark without fine-tuning. For batched inference, KPGrasp requires only 0.032 s per grasp. Finally, real-world experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.

Abstract:
Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this "feasibility collapse". We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.

Abstract:
Interactive robot planning requires robots to infer and execute human intentions from natural language instructions that are often ambiguous, incomplete, or underspecified. Although large language models (LLMs) provide a powerful interface for clarification, relying on the generative model to drive an multi-turn conversation can introduce systematic failures. We propose a Bayesian framework that treats clarification as an active learning problem over grounded Signal Temporal Logic (STL) task specifications. Our method uses LLMs to initialize candidate formal specifications and translate informative contrasts into natural-language clarification questions, while Bayesian optimization maintains uncertainty estimation over user intent and selects queries that maximize information gain. After convergence, the inferred STL specification is passed to a formal planner to synthesize a verifiable robot trajectory. Across four simulated and real-world task domains, our approach generally achieves higher task satisfaction and requires fewer clarification rounds than LLM baselines, while helping smaller models close the performance gap against larger reasoning models.

Abstract:
Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.

Abstract:
Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent’s final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

Abstract:
Robust inertial odometry is essential for various carriers when external sensing is unreliable. Learning-based methods reduce integration drift by capturing local motion priors, but these methods often remain tied to a particular carrier, limiting generalization across heterogeneous platforms. We present MosaicIMU, a carrier-conditioned Mixture-of-Experts (MoE) pretraining-and-adaptation framework for generalizable neural inertial odometry. MosaicIMU uses a prototype-based router to compose carrier-specific expert features, decodes local velocity and uncertainty constraints, and integrates them with a history-aware EKF. For unseen domain adaptation, it freezes the pretrained base model and learns a new lightweight expert residual branch. For edge-deployment, it further reuses the router to select informative online samples for efficient incremental updates. Experiments show that MosaicIMU consistently outperforms learning-based baselines, reducing average ATE and RTE-10s by 40% and 34%, respectively. These results highlight that MosaicIMU provides a scalable pretraining-to-deployment paradigm for generalizable and adaptive neural inertial odometry.

Abstract:
Vision-Language-Action (VLA) models advance robotic control via strong visual-linguistic priors. However, existing VLAs predominantly frame pretraining as supervised behavior cloning, overlooking the fundamental nature of robot learning as a goal-reaching process that requires understanding temporal task progress. We present PRTS (Primitive Reasoning and Tasking System), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning. By treating language instructions as goals and employing contrastive reinforcement learning, PRTS learns a unified embedding space where the inner product of state-action and goal embeddings approximates the log-discounted goal occupancy, the probability of reaching the language-specified goal from the current state-action, quantitatively assessing physical feasibility beyond static semantic matching. PRTS draws this dense goal-reachability supervision directly from offline trajectories without reward annotations, and folds it into the VLM backbone via a role-aware causal mask, incurring negligible overhead over vanilla behavior cloning. This paradigm endows the high-level reasoning system with intrinsic goal reachability awareness, bridging semantic reasoning and temporal task progress, and further benefits goal-conditioned action prediction. Pretrained on 167B tokens of diverse manipulation and embodied-reasoning data, PRTS reaches state-of-the-art performance on LIBERO, LIBERO-Pro, LIBERO-Plus, SimplerEnv, and a real-world suite of 14 complex tasks, with particularly substantial gains on long-horizon, contact-rich, and zero-shot novel-instruction settings, confirming that injecting goal-reachability awareness significantly improves both execution success and long-horizon planning of general-purpose robotic foundation policies.

Abstract:
Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time. In this paper, we propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system-level accelerations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling. Extensive experiments on NVIDIA Jetson Orin and Thor demonstrate that Jetson-PI achieves 8.66x and 3.48x improvements in control frequency, respectively, while outperforming VLASH by 14.8% in average success rate on the LIBERO benchmark. Our code is available on Anonymous GitHub.

Abstract:
Zero-shot generalization to unseen embodiments is important for generalizable vision-language-action (VLA) models as robot hardware evolves and task-specific data collection remains costly. However, a systematic understanding of this problem remains limited, in part because the literature lacks a unified zero-shot transfer definition and controlled evaluation settings that isolate embodiment changes from differences in tasks, scenes, or protocols. To address this gap, we first distinguishstrict zero-shot transfer, where the target embodiment is absent from all training data, from pretrain-exposed zero-shot transfer, where it appears only during pretraining. We then introduce a controlled benchmark spanning 14 held-out target embodiments across simulation and real-world validation. Within this framework, we systematically study four factors: state-action representations, pretraining embodiment diversity, auxiliary co-training objectives, and target-embodiment exposure. Experimental results show that local end-effector (EEF) state-action representations, the source-diversity comparison, and auxiliary co-training improve cross-embodiment transfer by around 15, 18, and 7 percentage points, respectively. We further find that adding only 5% target-embodiment data during pretraining improves average target-embodiment progress by 13.4 percentage points, showing that strict and pretrain-exposed zero-shot transfer are distinct and should be reported separately. Together, these findings provide practical guidance for evaluating and improving cross-embodiment VLA transfer.

Abstract:
Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and world-action models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: https://flowdagger.github.io

Abstract:
Bimanual dexterous grasping is a fundamental and promising area in robotics, yet its progress is constrained by the lack of comprehensive datasets and powerful generation models. In this work, we propose BiDexGrasp, consisting of a large-scale bimanual dexterous grasp dataset and a novel learning-based framework. For dataset construction, we propose a novel bimanual grasp synthesis pipeline to efficiently annotate physically feasible data. This pipeline addresses the challenges of high-dimensional bimanual grasping through a two-stage synthesis strategy of efficient region-based grasp initialization and decoupled force-closure grasp optimization. Powered by this pipeline, we construct a large-scale bimanual dexterous grasp dataset, comprising 6351 diverse objects with sizes ranging from 30 to 80 cm, along with 9.7 million annotated grasp data. Based on this dataset, we further introduce a novel learning-based dexterous grasping generation framework. The framework lies in two key designs: a bimanual coordination module and a geometry-size-adaptive grasp generation strategy to generate coordinated and high-quality grasps on unseen objects. Extensive experiments conducted in both simulation and real world demonstrate the superior performance of our proposed data synthesis pipeline and learned generative framework.

Abstract:
Articulated object manipulation tasks such as opening doors, tightening bolts, and pulling drawers, impose kinematic constraints on a robot end-effector's motion that neural policies may fail to satisfy. Unlike hybrid force-position control that strictly separates force and position control into orthogonal subspaces, impedance control modulates a mechanical impedance system in the form of a virtual inertia, damping, and stiffness. However, selecting task-appropriate impedance gains remains challenging for robot policies. In this paper, we address this problem for the case of manipulating articulated objects. Our constraint-decomposed impedance control method uses the sampling diversity of a generative policy to estimate the feasible motion directions in twist space. The key idea is to infer the feasible and constrained subspaces of twists from an ensemble of trajectories generated by the policy from the same observation. Principal component analysis (PCA) of the end-effector twists determines the feasible twist subspace, which is then used to form 6 × 6 impedance gain matrices. For many common articulated objects, the feasible twist subspace is independent of the object configuration, so the subspace and resulting gains are estimated once and held fixed during manipulation, minimizing computational overhead. For less common cases where this independence does not hold, we instead estimate a configuration-independent twist subspace of higher dimension that contains the local feasible subspaces, allowing the same fixed-gain strategy to be used. Experiments across simulated and real-world single- and multi-DoF objects show that the proposed method consistently reduces constraint-violating contact loads relative to isotropic impedance control while maintaining task execution.

Abstract:
A policy tuned for one robot often behaves differently on another, whether due to the sim-to-real gap, unknown payloads, or the differing dynamics of two instances of the same robot. In contact-rich, dynamic manipulation, even small motion discrepancies can result in failure to track reference motion, since they disrupt the timing and modes of contact. Common remedies, such as domain randomization or system identification, either produce overly conservative task policies or require data that must be recollected for each robot or payload. We introduce the Torque Adaptation Module (TAM), a learned module that adapts the torque commands sent to the robot to match the behavior of an ideal robot. TAM operates between the low-level controller that tracks the policy's actions and the robot's torque interface. It includes a history encoder that embeds proprioceptive history into a compact latent state and a torque adaptor that computes residual torque corrections. Because TAM depends only on proprioceptive history and not on policy observations, rewards, or the action space, the same TAM weights can be reused to adapt policies with different action spaces (joint targets, end-effector targets, or direct torques). The policies themselves do not need to be trained with domain randomization of robot parameters. Instead, we offload the need for domain randomization to TAM by training it entirely in randomized simulation, using multi-robot pretraining followed by a robot-specific fine-tuning step that still requires no real-robot data. We evaluate TAM zero-shot on a real Franka Panda robot across dynamic manipulation tasks with unknown payloads that include a vision-based box pushing policy (from RL), a flip policy (from BC), and an MPC ball-on-plate balancing controller. Our experiments show that TAM significantly improves zero-shot real-robot execution compared to online system identification and RMA baselines and enables robust dynamic manipulation performance.

Abstract:
We consider the problem of sequential task continuation, where one robot acts in a workspace that has been affected by a previous robot. Current approaches in multi-robot cooperation rely heavily on co-training or co-optimization, assuming that the robots have access to collective interaction data during the policy training or planning process. However, as increasingly varied robotic embodiments and policies are deployed and operate in the same real-world environment, learning policies for cooperation becomes infeasible due to the combinatorial explosion of possible robot combinations that need to be accounted for. To address this challenge, this paper considers the problem of ad hoc sequential cooperation between robots that have never before operated together in a shared environment, and proposes PAVE (Policy Adaptation via Value Exchange) as a solution. We demonstrate that robots can explicitly communicate their environmental preferences through sharing their value functions, enabling coordination without the need for prior joint training. To ground this exchange and prevent overconfidence in out-of-distribution states, we further introduce a k-nearest neighbors (KNN)-based uncertainty filter. Our method yields an increase of up to 47.7% in task success rates across simulated and real-world experiments compared with purely selfish agents, demonstrating that value function transmission between agents is useful as a basis for ad hoc coordination.

Abstract:
3D manipulation policies derive actions from explicit scene geometry rather than color images alone, enabling them to reason over metric distances, object relations, and robot-scene reachability. This geometric interface is also a vulnerability: commodity RGB-D sensors return missing measurements, boundary artifacts, and plausible but wrong surfaces around transparent, reflective, thin, or textureless objects. We study this failure under realistic depth corruption and observe a sharp depth-noise cliff across modern 3D policies, even when they are trained with matched noisy depth. This residual gap motivates a spatial-encoding approach: corrupted depth becomes entangled with the token coordinates and geometry features consumed by the action expert. We propose Noise-Invariant Spatial Encoding (NISE), a 3D policy architecture that learns trustworthy geometry features even facing imperfect depth. NISE uses Geometric Cross-View Self-Supervision to shape shared policy features with calibrated multi-view depth, and Gaussian Marginalized 3D Attention to carry position uncertainty into deployed attention rather than treating every lifted token as exact. Across simulated benchmarks with sensor-realistic corruption and real-robot tasks involving transparent and reflective materials, NISE improves depth-noise robustness while preserving clean-depth performance, supporting 3D policies designed for commodity RGB-D deployment rather than clean-depth benchmarks alone.

Abstract:
We present Kamino, a GPU-based physics solver for massively parallel simulations of heterogeneous highly-coupled mechanical systems. Implemented in Python using NVIDIA Warp and integrated into the Newton framework, it enables the application of data-driven methods, such as large-scale reinforcement learning, to complex robotic systems that exhibit strongly coupled kinematic and dynamic constraints such as kinematic loops. The latter are often circumvented by practitioners; approximating the system topology as a kinematic tree and incorporating explicit loop-closure constraints or so-called mimic joints. Kamino aims at alleviating this burden by natively supporting these types of coupling. At its core lies a state-of-the-art constrained optimization algorithm that computes constraint forces by solving the constrained rigid multi-body forward dynamics transcribed as a nonlinear complementarity problem. This leads to high-fidelity simulations that can resolve contact dynamics without resorting to approximate models that simplify or convexify the problem. We demonstrate RL policy training on DR Legs, a biped with six nested kinematic loops, generating a feasible walking policy while simulating 4096 parallel environments on a single GPU.

Abstract:
For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on "Variational Automation" (VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL). GaP then generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines.

Abstract:
LLMs generate fluent plans for robots but routinely violate the syntactic and semantic constraints they must satisfy to execute, and existing remedies trade formal guarantees against plan quality: soft methods (affordance scoring, grounded decoding) give no guarantee, while symbolic planners (LLM+P) discard the LM's commonsense. We propose Meta-Ctrl, a constrained-decoding framework that guarantees the encoded constraints while preserving the base LM's plan quality. Meta-Ctrl introduces meta-tokens—a compact vocabulary of grounded actions—enforcing syntax at the token level and semantics (preconditions, goals, ordering) at the action level, an exact factorization that cuts the memory of constrained decoding from 107 TB to under 2 GB. With it, a small open-weight LM becomes competitive where it otherwise sits at the bottom of the leaderboard: on WAH-NL under the LoTa-Bench protocol it reaches the highest reported subgoal success rate, exceeding GPT-4's, with strong results across the Embodied Agent Interface (VirtualHome and BEHAVIOR). We further demonstrate it on a real tabletop robot, where every generated plan satisfies its preconditions and goals by construction.

Abstract:
Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.We evaluate our method in challenging humanoid balance tasks. Compared with the nominal reference policy and a learning-based safety filter baseline, residual reinforcement learning improves safety while still achieving competent performance.

Abstract:
We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional navigation methods that operate over simplified navigation abstractions, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO predicts 29-DoF joint-space actions for whole-body humanoid control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and reinforcement-learning-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO outperforms state-of-the-art vision-language-navigation and modular humanoid traversal baselines on long-horizon tasks requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, demonstrating robust language-guided traversal in cluttered real-world scenes without training on any real-world data.

Abstract:
While decoupled control schemes for legged mobile manipulators have shown robustness, learning holistic whole-body control policies for tracking global end-effector poses remains fragile against Out-of-Distribution (OOD) inputs induced by sensor noise or infeasible user commands. To guarantee safety under these perturbations without sacrificing task performance and continuity, we propose Competence Manifold Projection (CMP). Specifically, we utilize a Hierarchical Manifold Inclusion Framework that transforms the infinite-horizon safety constraint into a computationally efficient single-step manifold inclusion. To rigorously instantiate this competence manifold, we employ a Lower-bounded Safety Estimator that distinguishes unmastered intentions from the training distribution. We then introduce an Implicit Isomorphism Constraint (IIC) that aligns manifold geometry with safety probability, enabling efficient O(1) seamless defense against arbitrary OOD intentions. Experiments demonstrate that CMP achieves a 100% survival rate in a series of typical real-world OOD scenarios where baselines suffer catastrophic failure, incurring less than 10% tracking degradation. Notably, the system exhibits emergent "best-effort" generalization behaviors to progressively accomplish OOD goals by adhering to the competence boundaries.

Abstract:
Imitation learning for autonomous driving suffers from distribution shift between logged expert trajectories and self-induced deployment states. In Transformer-based planners, this shift can appear as excessive ego-state attention concentration, incomplete scene understanding, and unsafe mode selection under interaction uncertainty. This paper presents a robust imitation-learning planner based on a three-level robustness formulation. 1) Channel-level robustness: constrained ego-state attention to limit excessive concentration on a small subset of state channels. 2) Scene-level robustness: offline vision-language model (VLM) guidance to encourage the ego query to retain interaction-critical agent evidence and route-relevant map evidence. 3) Decision-level robustness: training-time multimodal supervision reshaped with a tail-risk score based on mean excess above Value at Risk. The scene-level VLM guidance and decision-level tail-risk supervision are used only during training, and the deployed planner incurs no additional inference-time overhead. On nuPlan test14, the proposed planner achieves the strongest performance among the evaluated learning-based methods and improves over CAR Planner, a strong constrained-attention imitation-learning baseline. On test14-hard, TRIAD Planner achieves NR-CLS and R-CLS of 73.39 and 67.12, compared with CAR Planner's reported 72.94 and 64.90, respectively. Ablation results show that combining all three-level components yields consistent performance across diverse scenarios, supporting the effectiveness of robust imitation learning for autonomous driving.

Abstract:
Vision-Language-Action models (VLAs) promise to ground language instructions in robot control, yet in practice often fail to faithfully follow language. When presented with instructions that lack strong scene-specific supervision, VLAs suffer from counterfactual failures: they act based on vision shortcuts induced by dataset biases, repeatedly executing well-learned behaviors and selecting objects frequently seen during training regardless of language intent. To systematically study it, we introduce LIBERO-CF, the first counterfactual benchmark for VLAs that evaluates language following capability by assigning alternative instructions under visually plausible LIBERO layouts. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. We propose Counterfactual Action Guidance (CAG), a simple yet effective dual-branch inference scheme that explicitly regularizes language conditioning in VLAs. CAG combines a standard VLA policy with a language-unconditioned Vision-Action (VA) module, enabling counterfactual comparison during action selection. This design reduces reliance on visual shortcuts, improves robustness on under-observed tasks, and requires neither additional demonstrations nor modifications to existing architectures or pretrained models. Extensive experiments demonstrate its plug-and-play integration across diverse VLAs and consistent improvements. For example, on LIBERO-CF, CAG improves pi_0.5 by 9.7% in language following accuracy and 3.6% in task success on under-observed tasks using a training-free strategy, with further gains of 15.5% and 8.5%, respectively, when paired with a VA model. In real-world evaluations, CAG reduces counterfactual failures of 9.4% and improves task success by 17.2% on average.

Abstract:
Robust and generalizable dexterous in-hand manipulation requires a policy to infer object pose, geometry, contact, and potential slip from partial and noisy observations. Although recent tactile and visuotactile RL methods achieve strong in-hand rotation in controlled settings, their robustness often degrades under pose shifts, force disturbances, and object variation. We propose WM-Craftnet, a world-model-conditioned framework that learns compact latent dynamics from proprioception, depth, tactile sensing, actions, and rewards. Rather than using the world model for latent imagination or policy optimization, WM-Craftnet uses the learned World Synesthesia Model (WSM) as recurrent task context for an asymmetric actor--critic policy. Importantly, WSM is trained to reconstruct clean depth targets from noisy depth inputs, providing a denoised geometric state for real-robot deployment. Ablations over WSM modality heads show how tactile and depth prediction shape the synesthetic state, with clean-depth supervision providing large gain, and a WSM pretrained on nine z-axis objects serves as a reusable prior for 49-object downstream policy learning. This context improves multi-object rotation, with quantitative and qualitative evidence for unseen-object and sim-to-real transfer, and qualitative recovery from challenging initial poses and external force disturbances. To benefit the research community, we will release the code upon paper acceptance.

Abstract:
Long-horizon robot mobile manipulation requires continual reasoning about localization, environment changes, and task progress, all of which are challenging to infer from image observations alone. In this paper, we show that conditioning a mobile manipulation policy on a spatiotemporal feature map improves reasoning over long horizons. The map represents the environment and the articulated robot body as neural points in a shared latent space and is updated online from egocentric observations and proprioceptive state. We update the environment neural points using object-level rigid tracking and the robot neural points using forward kinematics. We use our spatiotemporal environment and robot feature (SERF) map as a state input to a vision-language-action (VLA) model by extracting map tokens from multiple reference frames and spatial scales, providing the policy with both local and global context. We demonstrate SERF on BEHAVIOR-1K, a benchmark for long-horizon mobile manipulation in household environments. Experiments show that the SERF VLA policy outperforms image-only baselines, reaches subgoals faster by following more direct trajectories, improves robustness to scene-configuration shifts, and recovers from object-drop failures.

Abstract:
Robotic perception increasingly relies on heterogeneous sensor suites combining cameras with emerging modalities (event, SPAD, iToF/dToF, and thermal). Yet existing simulators idealize these sensors, hiding critical modality-specific failure modes that limit real-world deployment. We present VisionSIM, a unified, physically grounded simulation platform that elevates sensor realism to a first-class concern by modeling the raw physical measurement process for diverse conventional and emerging modalities. We expose fine-grained control over physical sensor parameters, reproducing failure modes overlooked by general-purpose simulators: pileup and glare for LiDAR; phase wrapping, saturation, and motion artifacts for iToF; refractory drops for event cameras; and history-dependent emission for thermal cameras. This enables principled cross-modality comparison on identical scenes and trajectories. We release the platform and a large indoor dataset with synchronized raw measurements and rich ground-truth annotations.

Abstract:
Humanoid parkour policies can traverse various terrains, but task completion may mask challenges of harsh landings, edge contacts, and unstable stance contacts. Humans naturally regulate foot–terrain interaction through tactile feedback, modulating contact compliance according to terrain stiffness. This highlights a key domain gap between humans and humanoid robots: the absence of rich tactile sensing in most humanoid systems. We address this problem with TactileStep, a deployable tactile learning framework that brings sole pressure sensing into humanoid locomotion control for softer touchdowns and more stable support. TactileStep aligns tactile simulation with the real pressure insole, allowing the policy to learn from the same contact features available on hardware. During training, we uses tactile and motion cues to recognize different foot-contact phases and apply phase-aware rewards that encourage safer landing and more stable stance. Evaluated in simulation and on a Unitree G1 humanoid across diverse terrains, TactileStep reduces peak touchdown force by up to 48.8% and peak A-weighted impact noise by up to 30.1dB over a strong perceptive baseline, while increasing stance contact area by up to 23.8%.

Abstract:
Long-horizon online visual mapping is a core capability for robot perception, requiring continuous camera-motion and scene-geometry estimation from visual streams under bounded memory and computation. Recent feed-forward 3D reconstruction models provide strong geometric priors, but their streaming variants often predict poses in a fixed coordinate system tied to the first frame or a persistent scene memory. This fixed-gauge design leads to train--test mismatch, attention bias toward early anchors, and accumulated drift on sequences much longer than those seen during training. We propose \emphAnchor3R, a streaming 3D reconstruction framework that treats feed-forward reconstruction as current-centric local measurement prediction rather than persistent global-gauge regression. At each time step, Anchor3R predicts window-relative poses and a local pointmap in the current-frame coordinate system, turning streaming reconstruction into relative-pose measurement generation. These measurements support online pose updates, while loop-closure reinsertion and motion averaging align the trajectory and transform local pointmaps into a coherent global reconstruction. % Anchor3R introduces an image-only cached pose-query Transformer, which reuses gauge-agnostic visual evidence while keeping gauge-dependent pose reasoning transient, and integrates overlapping window-relative predictions into a motion graph for online pose updates and loop-aware motion averaging. Experiments on indoor, outdoor, driving, and RGB-D benchmarks show that Anchor3R improves long-horizon pose accuracy and dense reconstruction quality over existing streaming baselines, while supporting bounded-memory online inference.

Abstract:
Scalable robot imitation learning relies on large-scale heterogeneous data from diverse robots or body-free data, making Cartesian end-effector actions a key interface for embodiment-agnostic policy learning. However, end-effector-only abstraction leaves Cartesian policies unaware of the deployed robot body, making them brittle under robot-specific constraints such as whole-body collision avoidance. To overcome this limitation, we present EmbodiSteer, a training-free framework that steers embodiment-agnostic visuomotor policies toward zero-shot, embodiment-aware deployment. EmbodiSteer keeps policy learning in Cartesian space while efficiently lifting inference-time diffusion sampling into the target robot’s joint space via forward kinematics and Jacobian-based updates. With whole-body collision-aware guidance over joint trajectories after each denoising step, the arm can be steered away from collisions while preserving learned end-effector behavior. Compared with Cartesian-only execution, EmbodiSteer reduces collision rate by 46.1% and improves task success rate by 28.5% across 9 simulated robots, and further achieves 90.0% collision rate reduction and 36.7% success rate increase on two physical robots in highly constrained scenarios.

Abstract:
We present CORD-SLS, a real-time control method for safe deformable object manipulation, with a focus on ropes and cloth. At its core is a GPU-parallel differentiable simulator with contact smoothing which enables efficient gradient-based planning through intermittent contact. To robustly satisfy constraints under model and sensing uncertainty, we develop a real-time, GPU-parallel output-feedback robust model predictive control (MPC) algorithm that plans with this simulator. We further show that the simulator accelerates model-based RL for training neural manipulation policies. To improve real-world robustness, we use conformal prediction to calibrate visual-feedback and perception-error bounds for MPC, producing reachable tubes that enable high-probability safe control. We evaluate CORD-SLS on high-dimensional, contact-rich rope and cloth manipulation tasks in simulation and hardware, including obstacle avoidance, routing, folding, and smoothing. Across settings, CORD-SLS achieves millisecond-speed planning, exceeding baselines in safety, speed, and task success.

Abstract:
Pretrained vision-language-action (VLA) policies provide strong language-conditioned manipulation knowledge, but they remain largely vision-driven and can struggle once manipulation enters contact states where the scene is occluded, depth is ambiguous, or small force errors push execution off the offline demonstration distribution. We present LIFT (Late Reactive Injection of Force for VLA Post-Training), a force-aware post-training framework that adds contact reactivity to a pretrained VLA policy while preserving its general manipulation knowledge. LIFT grafts a reactive action expert beside the original action expert, initializes it from pretrained action weights, and injects recent 6D end-effector force through causal force memory and zero-initialized cross attention, enabling actions to be refreshed during execution. To address the policy-dependent distribution shift of contact feedback, LIFT further couples reactive force injection with an online DAgger loop that trains on a mixture of offline task-alignment data and human-corrected online rollouts. Across towel folding, book insertion, and Hanoi ring placement, LIFT learns faster and reaches higher performance than vision-only post-training, while ablations show that reactive force memory and online corrective data are both important for robust contact-rich manipulation. Our code is publicly available at https://www.google.com/url?q=https://github.com/y-wng/lift&source=gmail-imap&ust=1792301747000000&usg=AOvVaw1EByExLgUUmVJkDya0AYgP.

Abstract:
Vision-based tactile sensors enable fine-grained perception for dexterous robotic manipulation, but current marker-based designs tightly couple surface texture and contact force signals within a single observation. Existing data-driven decoupling methods rely on serial mask-based inpainting, which suffers from mask-sizing dilemmas caused by physical indentation effects, error retention, and lack of real paired data for supervision. To address these challenges, we propose DeTac, a novel dual-branch generative framework that concurrently extracts clean shading and marker images from a single mixed tactile image. DeTac leverages full-image synthesis to naturally bypass mask dependency and introduces a cross-branch synergistic optimization that enables mutual supervision, effectively preventing error retention. Furthermore, we construct a strictly aligned, real-world paired tactile dataset using custom hardware to provide high-quality supervision. Extensive experiments demonstrate that DeTac achieves state-of-the-art decoupling accuracy, significantly facilitating downstream surface reconstruction and force estimation tasks while exhibiting strong generalization across sensors.

Abstract:
Flow-based vision-language-action (VLA) models generate action chunks for temporally coherent robot motion, but chunked control creates a fundamental closed-loop trade-off: long chunks provide smooth execution, whereas frequent replanning improves reactivity at the cost of action discontinuities. We introduce REACT, a rolling-denoising framework that makes flow-based VLAs more reactive while preserving long-horizon context. Instead of regenerating entire action chunks from scratch, REACT maintains a persistent action buffer with staggered flow timesteps. At each control step, the full horizon is denoised using the latest observation, the cleanest action block is executed, partially refined future blocks are shifted forward, and fresh noise is appended to the tail. As a result, each executed action block is refined across multiple recent observations before deployment. To support real-time control, we further introduce dual decoupling, which separates sensing, VLM encoding, DiT denoising, and action execution, enabling high-frequency observation updates and action streaming under practical compute constraints. Across the RoboTwin 2.0 simulation benchmark and real-world tasks spanning bimanual manipulation and dynamic control on multiple robot platforms, REACT improves task success and reduces reaction latency while producing smoother trajectories than frequent-replanning and asynchronous baselines.

Abstract:
Federated Reinforcement Learning (FRL) enables collaborative policy learning across distributed agents with heterogeneous environments. While recent methods based on variance reduction, divergence penalization, and momentum optimization improve FRL under heterogeneous settings, they still primarily synchronize policy or value-network parameters and do not explicitly address distributional mismatch among heterogeneous clients. Therefore, we propose FedGuide, a FRL framework that uses diffusion priors as behavior models to provide personalized data supported distributions for heterogeneous local policy learning. Instead of directly averaging local policies, FedGuide aggregates those diffusion priors through Optimal-Transport Mixture-of-Experts (OT-MoE), preserving heterogeneous behavior modes in distribution space. It further develops a Distribution Correction Estimation (DICE) value baseline to provide low-variance, return-aware guidance for local policy improvement. Experiments across heterogeneous environments show that FedGuide outperforms representative FRL methods in client average returns, final-round performance, and worst-round robustness, while maintaining stable learning under stronger heterogeneity.

Abstract:
Generalist manipulation policies are increasingly presented as foundation models for robotic control, but their real-world generalization remains difficult to diagnose. A policy may succeed on demonstrated tasks while still failing to execute fine-grained atomic skills or recombine learned skills in new task structures. We introduce ATOM-Bench, a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies. ATOM-Bench factorizes tabletop manipulation into motor atoms and instruction atoms, and contains 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. We collect 3,000 human demonstrations for atomic fine-tuning and will release both the demonstration data and evaluation rollout data to support reproducible real-world evaluation. Policies are fine-tuned on atomic tasks and evaluated on both atomic skill acquisition and held-out compositional tasks. We further introduce Atomic Score (AS) and Compositional Failure Share (CFS) to distinguish failures caused by weak atomic skills from failures caused by limited compositional reuse. Through 2,700 physical rollouts on five representative manipulation policies, we find that current policies can acquire simple instruction-grounding skills, but still struggle with fine-grained motor atoms, counting, and logical filtering. More importantly, strong atomic performance does not reliably transfer to held-out compositional tasks. ATOM-Bench provides a diagnostic testbed for studying whether failures arise from weak motor execution, poor instruction grounding, or limited compositional reuse.

Abstract:
Collaborative planning under operational constraints is essential for heterogeneous robot teams in large-scale missions. Unmanned Aerial Vehicles (UAVs) provide rapid area coverage but are limited by onboard energy, while Unmanned Ground Vehicles (UGVs) can serve as mobile recharging stations yet are constrained by road-network traversability. Effective collaboration must enforce operational constraints during execution while enabling parallel, coordinated actions. Masked Diffusion Models (MDMs) have shown promise through token unmasking but have two key limitations: entropy-guided scheduling assumes distributional alignment and reliance on differentiable constraints for diffusion-time optimization. We propose Masked Error-bounded Neuro-Symbolic Diffusion (MEND), which formulates collaborative planning as a conditional masked generative process to generate context-aware action sequences for UAV-UGV teams. MEND introduces a neuro-symbolic diffusion paradigm that constrains conditional masked diffusion policies at execution by reweighting the learned action distribution with a logic-induced admissibility indicator, thereby enforcing compliance with operational constraints. MEND introduces an error-bounded inference scheduling at execution that constructs the unmasking schedule by selecting positions with minimal predictive entropy over the constraint-aware action distribution, enhancing collaborative planning optimality under operational constraints. Extensive experimental results in simulation, on real-world maps, and with physical robot teams demonstrate that MEND advances UAV-UGV collaborative planning under operational constraints.

Abstract:
Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone. As a result, both human instructions and zero-shot language models can fail to reliably steer VLAs toward successful task execution. In this work, we propose a framework that interactively searches for language sequences that improve closed-loop VLA task performance, distills these sequences into a test-time language feedback policy (LFP), and learns an improvement head that predicts when language steering will improve performance. Crucially, our approach operates on arbitrary frozen pre-trained VLAs, requiring neither access to the original training distribution nor fine-tuning of the underlying model. We conformalize this improvement head to prevent harmful steering interventions, where the LFP decreases task performance relative to the original instruction on out-of-distribution scenarios. On seen environments, our conformalized LFP improves base VLA performance by 24.7% in simulation and 65.0% in hardware. Our conformalized LFP has strong harmlessness guarantees under visual and semantic perturbations and produces recovery behaviors not observed with open-loop prompting.

Abstract:
We address the problem of understanding when and why Vision-Language-Action models struggle with contact-rich manipulation tasks that require precise physical interaction. Prior work has primarily focused on addressing contact failures through force-augmented architectures and training-time regularizers, yet the root causes of these failures remain underexplored. We identify two distinct failure modes underlying this gap. Precision failures are rooted in a flow-matching policy training mismatch, and force failures arise from the distinctive structure of force signals. We address each failure mode with a targeted mechanism and combine them into FACT, which achieves 66% average success rate across five contact-rich tasks against 41% for the best prior baseline, in an evaluation spanning almost 2,500 real-world rollouts.

Abstract:
Robotic manipulation tasks exhibit varying memory requirements, ranging from Markovian tasks that require no memory to non-Markovian tasks that depend on historical information spanning single or multiple interaction trials. Surprisingly, simply extending observation histories of a visuomotor policy often leads to a significant performance drop due to distribution shift and overfitting. To address these issues, we propose Gated Memory Policy (GMP), a visuomotor policy that learns both when to recall memory and what to recall. To learn when to recall memory, GMP employs a learned memory gate mechanism that selectively activates history context only when necessary, improving robustness and reactivity. To learn what to recall efficiently, GMP introduces a lightweight cross-attention module that constructs effective latent memory representations. To further enhance robustness, GMP injects diffusion noise into historical actions, mitigating sensitivity to noisy or inaccurate histories during both training and inference. GMP achieves a 30.1% average success rate improvement over long-history baselines on our proposed non-Markovian benchmark MemMimic, a 26.6% improvement over VLA baselines on MIKASA-Robo benchmark, while maintaining competitive performance on Markovian tasks in RoboMimic. All code, data and in-the-wild deployment instructions will be released.

Abstract:
Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as \emphpolicy orchestration: runtime selection of a policy--goal pair that is likely to remain feasible while tracking the planner target. We develop \emphEPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy--goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision--robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Videos are available on the anonymous project website: \urlhttps://keen-naiad-620edc.netlify.app/.

Abstract:
Robotics foundation models carry broad manipulation priors from large-scale data, but specializing them to a new task remains the deployment bottleneck. This requires injecting new task-specific behavior from limited demonstrations without eroding their broad capabilities. We introduce Proxy Policy Steering (PPS), a novel adaptation method that addresses this challenge by training two lightweight proxy policies. A reference proxy models the frozen base's behavior on target-task observations, and a task proxy, initialized from the reference, captures how this behavior changes under task supervision. Their difference forms a calibrated velocity-space residual that steers the frozen base sampler at every denoising step. We identify the conditions under which this residual isolates the change induced by task supervision, and validate them empirically. On 8 real-world and 4 simulation manipulation tasks, PPS lifts the state-of-the-art \pi_0.5 base policy by 53% absolute success rate on average, with 0-to-1 gains on tasks the base never solves, while preserving the base's broad capabilities. PPS outperforms LoRA fine-tuning, from-scratch specialists, residual policies, and prior inference-time steering methods.

Abstract:
Tactile sensors are inherently heterogeneous: a model trained on one sensor cannot be directly used on another, which limits learning contact-rich manipulation policies from diverse tactile data at scale. To bridge this gap, we propose the Heterogeneous Tactile Transformer (HTT), a framework that learns shared tactile representations across heterogeneous sensors. HTT consists of sensor-specific encoders and a shared transformer trunk, and is pretrained with per-modality masked reconstruction together with cross-modal alignment between paired sensors. Pretraining uses our novel Heterogeneous Paired Tactile (HPT) dataset, containing 1.6M synchronized paired frames across four vision- and array-based tactile sensors. Across distinct tactile perception and real-world manipulation tasks, HTT is shown to learn transferable representations that adapt to new tasks and previously unseen sensors. Dataset, code, and model checkpoints will be released upon publication.

Abstract:
Humans routinely wield tools, swap grasps, and reposition objects within a single hand—seamlessly orchestrating contact transitions that span translation, re-orientation, and finger gaiting. Endowing robot dexterous hands with this level of in-hand dexterity through teleoperation requires precise control of object motion via dynamic hand–object contact, yet current teleoperation systems remain far from this capability. To bridge this gap, we take a major step towards human-level dexterous teleoperation by introducing TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution. The controller is trained on consecutive co-tracking subgoals derived from human reference motions, utilizing a hybrid reward that couples sparse subgoal objectives with dense tracking rewards to enable learning across diverse interaction modalities rather than frame-wise trajectory imitation. The entire pipeline requires only single-stage RL and, with random action masking and domain randomization, transfers zero-shot to the real robot. We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands, achieving a 75% average success rate where all baselines consistently fail. Furthermore, the collected demonstrations successfully train autonomous policies via behavioral cloning, marking a concrete step towards human-level dexterous teleoperation.

Abstract:
The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based VLAs for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, in part due to the scarcity of diverse training data and standardized evaluation, architectural constraints in current Vision-Language-Action (VLA) models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation addressing all of these limitations. We open-source a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mix-of-Transformer (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control, deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.

Abstract:
A flow-matching policy deployed in the field must often satisfy a constraint that was absent from its demonstrations, such as a collision-free path or a particular grasp orientation. Existing training-free samplers take one of two approaches. \emphGuidance steers the denoising trajectory with a constraint gradient at noisy intermediates: the gradient at a noisy sample need not point toward a clean feasible one. \emphProjection maps the finished sample onto the feasible set and executes it: it is efficient, but the result is an action the policy would never draw. We find that this dichotomy need not hold. Our key insight is that the projection is a cheap, clean, feasible signal, and the only mistake is executing it. We introduce Constraint-Satisfying Flow Policy Sampling \textttCS-Pi: we draw from the policy, project the clean sample to the nearest feasible point, and resample from the flow model biased toward that point. Iterating drives the samples toward constraint satisfaction with no noisy-state gradient and no auxiliary network; the method extends to arbitrary costs with a single tweak in the projection function. We evaluate on a 2D diagnostic, a 7-DoF shelf pick-and-place, and a zero-shot Franka deployment. In every setting, \textttCS-Pi stays closest to the policy while satisfying the constraint, and it attains the highest safe-success on a real-world obstacle avoidance task.

Abstract:
Real-time execution, enabled by asynchronous inference that ensures both smooth action trajectories and fast reactivity, is critical for realistic deployments of large-scale Vision-Language-Action models. However, recent work on real-time execution primarily focuses on variants of diffusion policies, even though it is more critical for autoregressive policies given their slower rollout speed in synchronous inference. In contrast, we demonstrate that autoregressive policies can achieve real-time execution by adjusting the tokenization horizon and applying constrained decoding, thereby guaranteeing strict latency bounds that enable multi-trajectory decoding to maximize performance. Across simulated and real-world environments, we find that the autoregressive policy consistently outperforms its equivalent-level flow-matching policy counterpart while achieving significantly improved task completion speeds from synchronous inference. Coupled with the inherent advantages of autoregressive policies, such as faster convergence and better generalizability in instruction-following, these results confirm that autoregressive policies can remain a competitive policy type supporting real-time execution.

Abstract:
Reliable performance evaluation is a central bottleneck for deploying robot-learning policies in real-world conditions. Real-world testing produces faithful evaluation but is costly and difficult to scale, whereas simulation-based testing is easy to scale but inevitably biased due to the sim-to-real gap. Neither source alone is sufficient for scalable and reliable policy evaluation. This motivates simulation-augmented policy evaluation, which combines limited real-world rollouts with abundant simulation proxies for sample-efficient performance estimation. Yet, existing methods focus on estimating performance averaged over initial conditions and deployment settings. By construction, such population-level averages collapse scenario-specific variation, providing limited information about when and where a policy can be safely deployed. To address this limitation, we propose SCAPE, a scenario-conditioned simulation-augmented policy evaluation framework that instead predicts scenario-conditioned real-world policy performance from limited paired sim-and-real evaluation samples and large-scale simulation-based testing rollouts. SCAPE corrects the sim-to-real bias in the simulation labels before using them to train the prediction model, and calibrates the prediction uncertainty through conformal prediction. We validate SCAPE on two embodied task domains, autonomous driving and quadruped velocity tracking. In sim-to-sim settings, it reduces the scenario-level performance prediction error by 4.9%/34.7% in driving and 14.5%/27.7% in quadruped velocity tracking relative to scene-conditioned neural baselines and aggregate statistical baselines on average. We also validate SCAPE for evaluating a velocity-tracking policy deployed on a physical Unitree Go2. In addition, SCAPE improves testing sample-efficiency, produces narrower calibrated prediction intervals, enhances generalization to out-of-distribution scenarios, and unlocks fine-grained deployment strategies.

Abstract:
Category-level in-hand manipulation of articulated objects is a formidable yet underexplored challenge for dexterous robotic hands. This difficulty stems from two core bottlenecks: first, controlling an object’s internal degrees of freedom is tightly coupled with maintaining grasp stability on a free-floating base; second, acquiring diverse object models and functional grasps at scale is highly labor-intensive, yet vital for generalization given the system’s sensitivity to initial configurations. In this work, we present our method, the first category-level articulated in-hand manipulation method that generalizes across object instances and diverse initial grasps. For initial configuration construction, we develop an automated pipeline that procedurally generates diverse articulated objects and synthesizes task-oriented functional grasps. For policy learning, we propose a robust two-stage training strategy that incorporates articulation physics randomization, reward curriculum, and latent representation distillation to handle complex contact and joint dynamics during deployment. Extensive experiments across four object categories demonstrate that our policy generalizes to unseen instances and varied configurations in simulation, and achieves zero-shot transfer to 12 real-world objects featuring diverse shapes and joint mechanics.

Abstract:
Executing reliable Humanoid-Object Interaction (HOI) tasks for humanoid robots is hindered by the lack of generalized control interfaces and robust closed-loop perception mechanisms. In this work, we introduce Perceptive Root-guided Humanoid-Object Interaction, Pro-HOI, a generalizable framework for robust humanoid loco-manipulation. First, we collect box-carrying motions that are suitable for real-world deployment and optimize penetration artifacts through a Signed Distance Field loss. Second, we propose a novel training framework that conditions the policy on a desired root-trajectory while utilizing reference motion exclusively as a reward. This design not only eliminates the need for intricate reward tuning but also establishes root trajectory as a universal interface for high-level planners, enabling simultaneous navigation and loco-manipulation. Furthermore, to ensure operational reliability, we incorporate a persistent object estimation module. By fusing real-time detection with Digital Twin, this module allows the robot to autonomously detect slippage and trigger re-grasping maneuvers. Empirical validation on a Unitree G1 robot demonstrates that Pro-HOI significantly outperforms baselines in generalization and robustness, achieving reliable long-horizon execution in complex real-world scenarios

Abstract:
Generative dynamics models enable planning in challenging robotic systems, but safe deployment requires reliably detecting policy-induced out-of-distribution (OOD) transitions. Existing methods typically treat the learned dynamics as fixed and attach post hoc support surrogates. We show that these surrogates can fail when the dynamics are locally insensitive to critical action choices: unsupported control actions may produce latent predictions that resemble demonstrated transitions, suppressing OOD signals despite large true predictive errors. To address this, we introduce support-conditioned control-sensitivity regularization, which promotes sensitive local response to control input changes in learned dynamics in high-support training regions. This preserves control-induced variation while limiting unstable extrapolation due to weak empirical support. Experiments in vision-based obstacle avoidance, manipulation, and real-robot navigation show improved OOD detection and safer closed-loop planning.

Abstract:
Mastering robot manipulation skills via reinforcement learning (RL) remains largely sample-inefficient. The most common RL algorithms rely on random action sampling to discover new strategies, resulting in agents that allocate most of their training budget to motions in free space, away from the contacts from which manipulation skills emerge. Existing intrinsic motivation methods based on model disagreement or epistemic uncertainty improve on isotropic noise, but they mostly drive the agent to remain curious about functionally irrelevant transitions, such as erratic motions and interactions. In this work, we argue that tactile feedback provides a natural signal for exploration, and introduce TacEx, a framework that incorporates touch into epistemic uncertainty-driven exploration by decomposing model uncertainty across sensory modalities and directing curiosity toward the tactile channel. By anchoring curiosity to the sense of touch, TacEx drives the robot to discover complex contact dynamics, learning to manipulate and grasp objects without prior knowledge or external rewards. The interaction-dense dataset collected through this tactile-driven curiosity yields strong zero-shot performance on downstream optimization tasks. We further use tactile-driven exploration to post-train vision-language-action (VLA) models. Although the VLAs are initially pre-trained without tactile feedback, post-training with \alg substantially improves downstream performance while remaining highly sample-efficient.

Abstract:
End-to-end autonomous driving models are typically trained on offline human-demonstration datasets that provide limited state coverage and often no closed-loop feedback, making them prone to compounding errors when deployed in closed-loop and brittle to long-tail agent interactions. To overcome these limitations, we propose an alternative strategy for training end-to-end driving models: large-scale self-play directly from pixels in simulation. While prior self-play approaches have shown promising transfer to real-world driving, they typically assume vectorized Bird’s-Eye-View (BEV) observations that are incompatible with end-to-end policies operating directly on sensor observations. To this end, we introduce \emphGigapixel, a high-throughput batched driving simulator with perspective rendering, enabling scalable self-play directly from pixel observations. Rather than targeting compute-costly photorealistic sensor simulation, Gigapixel renders a simplified bounding-box world that preserves essential scene structure while achieving throughput at 50k agent steps per second. Since direct pixel-space self-play RL is prohibitively sample-inefficient at end-to-end model scale, we propose \emphself-play DAgger training: we train pixel-based policies in self-play via on-policy distillation from a privileged RL teacher. To bridge the sim-to-real gap, we subsequently transfer the self-play trained policies to real-world sensor data through lightweight perception adaptation. Policies trained in Gigapixel and adapted to real-world sensor data achieve competitive performance on the HUGSIM and NAVSIM-v2 benchmarks without human trajectory supervision. Moreover, scaling self-play training yields proportional gains in policy performance, establishing self-play as a practical and scalable strategy for training end-to-end models.

Abstract:
End-to-end (E2E) autonomous driving models achieve strong planning performance but remain largely opaque, limiting observability and trust. We apply mechanistic interpretability to E2E driving by training Sparse Autoencoders (SAEs) on the internal activations of a simple, single-camera end-to-end path planning model. Unsupervised clustering of SAE latents reveals interpretable structure aligned with route intent, vehicle speed, and braking behavior, recovering driving intent with 98.3% accuracy without supervision. Through paired counterfactual image edits, we further show that specific SAE features are visually grounded and systematically respond to scene changes such as traffic light state and stop sign presence. Steering individual SAE latents produces predictable, monotonic changes in planner outputs including lateral displacement, curvature, and braking. Finally, we demonstrate a practical application of these representations: we build a lightweight adapter trained on SAE latents that reduces trajectory selection error by 3.44%, outperforming an equivalent adapter that omits SAE latents, confirming that the sparse representation captures useful information for interpretability.

Abstract:
Controlling an agent in an environment solely perceived through onboard vision is challenging: perception yields noisy, variable-length, and partially occluded context, while onboard compute is shared between perception and control, limiting the planning horizon executable in real time. We propose VIGOR-MPC, a modular student–teacher framework, agnostic to the perception pipeline, which distills a privileged, field-of-view-aware long-horizon MPC into a computationally lightweight short-horizon MPC whose cost parameters are determined online from vision. The teacher operates in simulation with full state knowledge and a multi-trajectory formulation that guarantees the agent can stay within its field of view. The student observes only the noisy, occluded context produced by a perception module and learns, via a recurrent mapping, to infer cost parameters that aim to recover the teacher's behavior despite missing information. We evaluate VIGOR-MPC on vision-based autonomous racing in simulation and on hardware. Using only onboard perception of the track, VIGOR-MPC generalizes to tracks unseen during training and approaches the lap times of the privileged long-horizon teacher while reducing per-step computation by over 90%.

Abstract:
Most vision-language-action (VLA) models map observations directly to actions without explicit intermediate planning, which limits performance on long-horizon tasks where early mistakes compound. We propose Coarse-to-Control, a plan-execute VLA that introduces planning natively in the action-token space. The key idea is to let the policy first predict a compact sequence of coarse action tokens that summarize the intended future trajectory, and then generate executable action tokens conditioned on this plan. Because both planning and execution share a unified discrete action vocabulary, the plan stays close to the control manifold and provides directly actionable guidance rather than an abstract hint that must be translated back to motor commands. Experiments on LIBERO, SimplerEnv-WidowX, and real-world manipulation tasks show that action-token planning consistently improves over direct action generation, with the largest gains on long-horizon multi-stage tasks.

Abstract:
Vision-Language-Action (VLA) agents face a combinatorial task space where semantic and physical requirements are recombined in novel ways. However, existing holistic policies typically entangle them in a single latent representation, limiting generalization to unseen compositions and compromising physical feasibility. To this end, this work proposes an energy-based compositional diffusion policy that models robot action trajectory generation with distinct semantic and physical energy terms. Specifically, the agent factorizes multimodal inputs into grounded semantic factors and models them as implicit semantic energies. In parallel, it formulates physical constraints, including motion smoothness and kinematic manipulability, as explicit differentiable energies. These energies are jointly combined to guide trajectory generation toward both task satisfaction and physical feasibility. Our agent shows consistent improvements in compositional generalization and physical validity in real-world experiments and across some public benchmarks. Comprehensive ablation studies further validate the effectiveness of each component. The code will be released. Comprehensive ablation studies further validate the effectiveness of each component. The code will be released.

Abstract:
Long-horizon manipulation remains challenging for vision-language-action (VLA) policies: real tasks are multi-step, progress-dependent, and brittle to compounding execution errors. We present LoHo-Manip, a modular framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM. The manager is decoupled from the executor and is invoked in a receding-horizon manner: given the current observation, it predicts a progress-aware remaining plan that combines (i) a subtask sequence with an explicit done + remaining split as lightweight language memory, and (ii) a visual trace—a compact 2D keypoint trajectory prompt specifying where to go and what to approach next. The executor VLA is adapted to condition on the rendered trace, thereby turning long-horizon decision-making into repeated local control by “following the trace.” Crucially, predicting the remaining plan at each step yields an implicit closed loop: failed steps persist in subsequent outputs, and traces update accordingly, enabling automatic continuation and replanning without hand-crafted recovery logic or brittle visual-history buffers. Extensive experiments spanning embodied planning, long-horizon reasoning, trajectory prediction, and end-to-end manipulation in simulation and on a real Franka robot demonstrate strong gains in long-horizon success, robustness, and out-of-distribution generalization.

Abstract:
Mobile manipulators deployed at home across many rooms and many visits should improve with experience: after failing to open a locked cabinet, or finding an object in a drawer, the robot should use that knowledge to avoid repeated mistakes and complete future tasks more efficiently. However, today's robots often treat each task as new. Currently, compact scene representations omit interaction-derived knowledge, raw video buffers quickly grow to become inefficient and difficult to query, and VLM planners reason at inference time without persistently updating what the robot knows. Memory for mobile manipulation must be spatially grounded and queryable, updatable from interaction, and fine-grained enough to surface visual context that a text summary would lose. We present MessyMem, a memory system that enables mobile manipulators to learn from doing by building a compact 3D scene graph and associating it with learned interaction-derived properties and keyframes for visual detail. A task planner can use this memory to avoid repeated failures, reuse object locations, and recover visual context for long-horizon decisions. We evaluate MessyMem across multiple tasks in simulation and on a real mobile manipulator, showing that persistent interaction-grounded memory significantly improves long-horizon task success over representative baselines.

Abstract:
Robot policies deployed in-the-wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common method to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although it is common to pick this buffer randomly from all prior experiences, our work shows that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences, Memory Anchors. We propose identifying Memory Anchors as areas of representation collapse between the set of old and new task data that conflict in their required actions, like manipulating a familiar object in a new manner. The old data present in this region regularizes the new task training and prevents destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Removing access to 10% of these Memory Anchors in the replay buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. We further demonstrate that enriching the replay buffer with Memory Anchors can decrease worst-case forgetting in LIBERO by 63% and lead to successful continual learning of highly-conflicting tasks on a real robot.

Abstract:
Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and single binary preferences collapse many competing notions of quality into one ambiguous signal. We introduce Freeform Preference Learning (FPL), a method for learning robot policies from freeform human preferences. Rather than asking annotators which of two trajectories is better overall, FPL lets them define natural-language preference axes, such as speed, safety, formality, or carefulness, and provide pairwise preferences along each axis. These annotations are used to learn a language-conditioned reward model that maps a trajectory and preference label to an axis-specific reward. We use this model to train a reward-conditioned policy that can optimize across the multiple human-specified dimensions. Across four real-world and two simulated long-horizon manipulation tasks, FPL improves over sparse-reward and single-preference baselines by 38%. Beyond improved performance, FPL learns dense progress signals without explicit subtask segmentation, shows compositionality of behavior not present in the data, and allows users to steer the policy towards different behaviors at test time without retraining.

Abstract:
Proactive robot assistance requires acting on user needs before explicit requests, but zero-shot vision-language models struggle in this setting: lacking knowledge of the user's personal routine, they treat every situation as general and uncertain, defaulting to inaction rather than risking an incorrect intervention. We observe that everyday routines are highly repetitive, so even a few interactions reveal stable user-specific patterns that can ground proactive decisions. Based on this, we propose a System 1--2 framework in which a VLM task predictor reads a \emphProcedural Assistance Memory (PAM) alongside the current observation to decide when and how to assist, and a VLA executor realizes the predicted task as low-level robot action. PAM is structured as a finite-state machine over activity phases, per-state assistance rules, and event-driven transitions; a reflection VLM revises PAM from each day's interaction trace without weight updates or retraining. We evaluate on a multi-day desk-assistance scenario with six pick-and-place tasks. Memory-free baselines remain conservative across all five evaluation days, yielding near-zero recall on proactive tasks. Our system improves consistently with accumulated experience, achieving the highest precision, recall, and F1 and reducing first-fire delay from approximately 5 seconds on Day 1 to 1 second by Day 5. Ablations further show that the FSM structure, rather than memory alone, is the primary driver of these gains.

Abstract:
Robot visuomotor policies are commonly formulated as autoregressive, diffusion-based, or more recently, flow matching models. Among them, Action-to-Action (A2A) flow matching improves inference efficiency by initializing generation from historical action priors rather than stochastic noise. However, strong dependence on historical motion priors, together with entangled global visual representations, can reduce robustness under spatial out-of-distribution (OOD) shifts and visual distractors. In this work, we propose SlotFlow, an object-centric flow matching policy for robust visuomotor manipulation. SlotFlow decouples scene observations into semantic (“what”) and geometric (“where“) representations to provide object-aware policy conditioning and object-centric spatial grounding. The semantic representation suppresses irrelevant background correlations, while the geometric representation improves adaptation to shifted object configurations under spatial perturbations. Extensive experiments demonstrate that SlotFlow significantly improves robustness under visual distractors and severe spatial perturbations while preserving the low-step inference efficiency advantages of A2A flow matching in both simulation and real-world robotic manipulation tasks.

Abstract:
Tremendous progress has been made toward zero-shot, 3D-object-prompted perception tasks, including detection, segmentation, and 6D pose estimation. However, prior methods still struggle to see widespread use in robotics due to complexities such as challenging materials, dense clutter, and runtime. To address these limitations, we introduce Z3P, the first competitive end-to-end transformer architecture that performs zero-shot detection, segmentation, and pose estimation from single/multi-view RGB or RGB-D data and a 3D-object prompt. While previous methods use bounding boxes as an intermediate step between multiple models, Z3P operates as a single network that maintains an internal representation through persistent object tokens, allowing for better runtime and improved performance in densely cluttered environments with challenging materials. Across five robotic benchmarks (IPD, XYZ-IBD, ITODD-MV, T-ROBI, and ROBI), Z3P establishes a new state-of-the-art by becoming the first zero-shot method to outperform all prior zero-shot and object-specific approaches on detection and pose estimation. Finally, we demonstrate Z3P's utility as a foundation model for 3D-object perception, generalizing to multiple demanding robotic applications, including transparent object bin picking, sorting, car charging, and electronics assembly.

Abstract:
Leg odometry relies on the assumption that the velocity of a foot in stance is zero with respect to the world, which makes accurate contact detection a critical subproblem. Many approaches classify contact using ground reaction force (GRF) sensors at the foot tip, yet these sensors are not available on all legged robots and remain unresponsive to disturbances such as slippage while the foot is still in contact. Approaches without GRF sensing instead rely on heuristic pseudo-labels, model-based foot-velocity assumptions, or supervised contact learning. In this study, we propose a self-supervised representation learning framework for contact detection that uses only standard proprioceptive sensing (joint encoders and IMU) and requires neither force sensors nor contact labels. A denoising autoencoder is trained on kinematic features, a two-component Gaussian Mixture Model is fitted in the learned latent space, and its posterior is used as a continuous stance belief that scales the zero-velocity measurement covariance of an error-state Kalman filter. Experiments in simulation and on real-world terrain show that our framework yields the lowest odometry error among supervised and probabilistic baselines, and that higher pseudo-label classification accuracy does not translate into better downstream odometry. Our code and dataset are publicly available.

Abstract:
We introduce Infinigen-Articulated, a toolkit for generating realistic, procedurally generated articulated assets for robotics simulation. We include procedural generators for 18 common articulated object categories along with high-level utilities for use creating custom articulated assets in Blender. We also provide an export pipeline to integrate the resulting assets along with their physical properties into common robotics simulators. Experiments demonstrate that assets sampled from these generators are effective for movable object segmentation, training generalizable reinforcement learning policies, and sim-to-real transfer of imitation learning policies.

Abstract:
Subjective user experience is important to human-robot interaction, but the outcomes users value, and how those preferences vary across individuals and contexts, are often unknown. While inverse learning approaches using human data can help identify user rewards, in many assistive settings the experimental costs of executing a control policy, measuring biomechanical or physiological outcomes, and collecting user feedback often limit the number of queries and optimization iterations. Here, we present Simultaneous Forward and Inverse Human-In-the-Loop Optimization (SFIHILO), which efficiently infers individual-specific reward functions from preferences over human outcomes and identifies a final control policy that maximizes the learned reward. SFIHILO bootstraps a forward model to predict user outcomes from control policies, uses this model for active querying to accelerate inverse reward learning, and then optimizes the final control policy without additional user trials. We score candidate policies by their expected reduction in uncertainty across both forward and inverse beliefs, targeting a regional preference boundary to robustly inform this simultaneous learning process. In simulation, we show that SFIHILO was effective across user heterogeneity, outcome dimensionalities, precision requirements, noise levels, and nonstationarity; compared with mutual information approaches, the proposed active querying strategy significantly improved sample efficiency in inverse learning while preserving forward model accuracy. This approach demonstrates the potential to infer latent human goals, enabling more transferable and effective human-robot interaction.

Abstract:
Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction–observation–action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.

Abstract:
Spatial understanding is essential for robots to perceive their surrounding environments and execute actions. Existing approaches typically encode 3D awareness into vision-language-action (VLA) models by directly aligning their representations with pre-trained 3D foundation models. However, we observe that such direct geometric feature alignment essentially exploits 3D foundation priors as a form of low-rank regularization in the feature space, resulting in severe rank collapse of the pre-trained VLA representations. In contrast, we shift the paradigm from enforcing point-wise equality to aligning pair-wise relationships. We propose a representation-alignment method that enables the injection of 3D spatial awareness without compromising the model's original expressivity. Extensive experiments on public benchmarks, our new spatial reasoning benchmark, and real-world manipulation scenarios demonstrate that our approach effectively incorporates 3D spatial representations into VLA reasoning, achieving robust performance and strong open-world generalization across diverse manipulation tasks.

Abstract:
When deployed in open-ended robotic environments, Vision--Language--Action (VLA) models need to continually acquire new skills, yet suffer from severe catastrophic forgetting. We observe that this degradation is related to the deterioration of cross-modal information structure, where dependencies among visual observations, language instructions, and actions progressively diffuse during continual adaptation. But existing continual learning methods fail to preserve such cross-modal information dependencies. Thus, we propose Info-VLA, an information-preserving continual learning framework that maintains cross-modal information structure through dual constraints. Replay Anchor Contrastive Learning constructs stable alignment anchors from a frozen teacher model, preserving cross-modal alignment in the representation space. Cross-Modal Mutual Information Maximization further preserves dependency structure between visual and language representations through mutual information constraints. By jointly preserving historical alignment and cross-modal dependency information, Info-VLA balances stability and plasticity during continual learning. Furthermore, experiments on the simulators and real-world show that Info-VLA significantly outperforms existing methods in both task retention and adaptation.

Abstract:
We study policy representation learning from unlabeled multi-policy behavioral data, where each episode is generated by a fixed but unobserved policy. We introduce Behavioral INR, a self-supervised generative model that adapts implicit neural representations from vision to behavior: instead of mapping coordinates to RGB values, it represents a policy as a state-action function mapping states to actions. An episode-level latent modulates this function through FiLM layers, yielding a generative prior over policies and enabling policy identity inference without labels. We also define policy-level out-of-distribution (OOD) shifts along state- and action-distribution axes, which arise when policies overlap but are not captured by standard agent- or environment-level OOD settings. Across synthetic GRF data, Seek-Avoid, MuJoCo, chess, Formula 1, and robotic manipulation, Behavioral INR most consistently improves policy identifiability in harder continuous state-action settings, especially when longer episodes, more policies, and OOD splits reduce marginal shortcuts. We will release code and checkpoints upon acceptance.

Abstract:
Vision-language-action (VLA) models bring natural language into closed-loop robot control, enabling robots to execute manipulation tasks directly from text instructions. The same interface gives text a recurring role in control because the prompt is reused at every replanning step, and each prompt-conditioned action changes the future observations on which the policy acts. Existing VLA attacks study adversarial prompts that elicit targeted low-level actions or make such actions persist across changing images. We identify a stronger trajectory-level failure mode: a prompt that still appears to specify the intended task but redirect the final physical outcome. We mathematically formalize this setting as command-preserving trajectory redirection, a prompt-only threat model in which the attacker chooses one prompt before the episode, all policy and environment components remain fixed, and the prompt must stay close to the benign instruction while omitting target words and correction language. To find such prompts, we introduce an on-policy prompt search method that uses rollouts to discover perturbations whose closed-loop behavior tracks a target task while satisfying the command-preserving constraints. Experiments in simulation and on hardware show that near-benign prompt perturbations can redirect VLA rollouts to attacker-specified targets. These results expose a trajectory-level vulnerability in VLA instruction grounding: text that appears to preserve the intended command can still give an adversary control over the robot's final physical outcome.

Abstract:
Multi-object grasping is a fundamental yet challenging problem, constrained by data collection costs, varying spatial layouts, and the need for broad generalization. To address these challenges, we propose MoDex, a data-efficient framework that provides the first unified solution for universal multi-object dexterous grasping across diverse spatial layouts. MoDex accomplishes this by formulating the task as a single-step decision problem of learning residual corrections to a reference trajectory, and introducing a spatially adaptive dual-strategy that performs single-shot grasping for proximal objects and sequential interaction for separated objects. Extensive evaluations show that MoDex significantly outperforms baselines in proximal configurations, achieving 64.17% success on unseen objects, and uniquely succeeds in separated layouts where prior methods struggle. Further experiments demonstrate its generalization across robot embodiments, scalability and zero-shot sim-to-real deployment through a vision-based policy trained solely on simulation data.

Abstract:
Humanoid robots are typically trained as solitary agents, leaving them unable to coordinate physically with human partners who naturally anticipate and adjust to one another. To bridge this gap, we present Social Humanoid, a hierarchical system that generates and executes socially responsive whole-body motion on a 29-DoF Unitree G1 conditioned on a human partner's motion. Our approach decomposes the problem into a high-level policy and a low-level policy. For the high-level policy, we introduce a shared-codebook VQ-VAE that tokenizes human and humanoid motion into a unified latent space, enabling cross-embodiment conditional generation via a Diffusion Forcing Transformer. The low-level policy uses a whole-body tracking controller for deployment on the Unitree G1. To train the system, we introduce a two-phase adaptive retargeting procedure that retargets one interaction partner to the robot while morphing the other partner to preserve contact dynamics, reachability, and relative body scale. We apply this procedure across five motion-capture datasets to produce a new Human-Humanoid Interaction (HHI) benchmark of 260 hours of paired human-humanoid interactions. Our results demonstrate that Social Humanoid produces coordinated, human-aligned behavior across diverse interaction types in simulation and the real world.

Abstract:
Recent advances in reinforcement learning (RL) have demonstrated impressive whole-body agility for humanoid robots, yet ensuring safety and satisfying constraints -- particularly those specified after training -- remains a challenge. Towards this goal, we present ConstrainedMimic, a control framework that leverages whole-body kinematics and dynamics for real-time constraint enforcement within RL tracking policies. By integrating principles from operational space control and control barrier functions (CBFs), we enable the satisfaction of arbitrary runtime constraints on both the kinematic reference motion and the underlying dynamics. In whole-body motion-tracking and teleoperation experiments on a (simulated) Unitree G1 with a learned policy, we demonstrate collision avoidance (both with the robot body and external obstacles), joint limits, and center of mass stability constraints. By remaining consistent with the current contact mode and tracking objectives, we minimally restrict the capabilities of the policy when constraints are active. Our method is fully differentiable, runs on CPU, GPU, and TPU, and can be deployed at up to 300-500 Hz. All software will be freely available upon publication.

Abstract:
Compliant whole-body controllers for humanoids achieve safety by uniformly relaxing stiffness across all directions, but this dissolves the control authority needed for contact-rich tasks, forcing a hard trade-off between safety and performance. We introduce functional compliance: a task-dependent compliant control scheme that preserves authority along task-relevant axes while staying compliant elsewhere, varying in both direction and time. We simultaneously learn a control policy using a reward shaped by compliant poses and a force estimator that can be used at deploy time to compose variable compliance profiles. We demonstrate the effectiveness of our approach on three contact-rich whole-body manipulation tasks on the Unitree G1. Our method matches state-of-the-art motion-tracking fidelity while improving task success on key insertion and book reorientation, where directional and time-varying compliance is required, and matching the strongest baseline on cabinet opening, where it is not. This suggests that task-specific variable compliance, not constant uniform relaxation, is the correct whole-body control primitive for humanoid manipulation.

Abstract:
High-quality demonstration data is becoming a central bottleneck for training general-purpose humanoid robots. While recent humanoid teleoperation systems have made substantial progress in retargeting human motion to robot motion, long-horizon loco-manipulation requires another capability: operators must maintain task-relevant spatial awareness over time, e.g., object locations, surrounding environments, the robot's pose. We call the extent of this awareness the operator's perceptual horizon. However, existing methods often shorten this: narrow views miss peripheral events, robot-mounted cameras become unstable during locomotion, and coupled head-view control makes looking around interfere with robot motion. We present SPOT, a Spatial Perception-Oriented VR Teleoperation system for collecting long-horizon humanoid demonstration data by providing extended perceptual horizon. SPOT combines a robot-mounted binocular fisheye camera, a wide-field stereoscopic display, viewpoint-decoupled free-looking, and visual stabilization to provide a robot-centric view that is wide, stable, and actively inspectable. Unlike conventional egocentric interfaces, SPOT decouples visual exploration from robot actuation: the egocentric stereo observation is rendered on a virtual hemisphere around the operator, so natural head rotations change where the operator looks within the wide-field view rather than commanding the robot head, camera, or torso. We evaluate SPOT on perception-critical humanoid data-collection tasks spanning drop recovery, peripheral retrieval, large-workspace bimanual manipulation, fine alignment, and dynamic interaction. SPOT improves efficiency, accuracy, recovery speed, and operator workload, demonstrating its effectiveness for user-friendly and scalable long-horizon humanoid data collection.

Abstract:
Control-space tree search sidesteps the steering requirement of geometric search methods and offers an asymptotic route to natively feasible kinodynamic trajectories. In practice, however, control-space search fails on the simplest of motion-planning instances for nonlinear systems such as quadrotors. This failure is rooted in poor geometric coverage induced when naive control-sampling measures are pushed forward through system dynamics. We resolve this problem by learning a lightweight sampler whose pushforward achieves low geometric dispersion in the local search space. The central idea is to utilize a well-dispersed reference map as a training scaffold, eliminating the need for an external expert. Our experiments then demonstrate that the learned sampler enables control-space search for quadrotor motion planning when naive sampling measures simply fail. Once enabled, control-space planning is shown to generate highly agile trajectory solutions in an obstacle-dense setting as compared to established trajectory optimization techniques. We also show that the geometric reach of the learned model dominates popular motion primitive libraries, spanning roughly thrice their volume. Arbitrarily low-dispersion primitives are thus made available for any state merely by scaling test-time compute.

Abstract:
World models are a promising path to zero-shot embodied control through planning. However, existing world model planners struggle on long-horizon, multi-stage tasks: prediction errors compound and naive search is exponential in the planning horizon. Hierarchy mitigates both by decomposing tasks into shorter, tractable subproblems; yet prior hierarchical approaches either amortize control into task-specific policies (hierarchical RL) or assume low-dimensional states and known dynamics (classical hierarchical MPC). We present Hierarchical Planning with Latent World Models (HWM), an architecture and planning paradigm for hierarchical model predictive control (MPC) directly on visual world models trained solely via next-latent prediction. HWM learns world models at multiple temporal scales within a shared latent space, so predictions from the long horizon model serve as subgoals for the short horizon model via latent matching—without task-specific rewards, skill learning, or hierarchical policies. To keep long-horizon search tractable, HWM learns an action encoder that compresses primitive action chunks into latent macro-actions. On real-world Franka manipulation, HWM solves pick-&-place from a single goal image at 70% success vs. 0% for single-level planning. Across simulated push manipulation and maze navigation, HWM consistently improves performance on long-horizon tasks while requiring up to 3× less planning compute.

Abstract:
Action-conditioned video world models predict future observations from an initial observation and an action signal. In robotics, this action signal becomes future video in two stages: a robot body and controller realize it as motion, and the scene responds through contact, occlusion, and object motion. Conditioning directly on the pre-realization signal asks the video model to learn robot-specific controller semantics, while conditioning on logged future states leaks interaction outcomes. We propose robot-factored world models, which target the middle signal: a deployment-available nominal trajectory produced by the robot's own controller and kinematics before observing scene interaction. Together with camera-aware static RGB/depth context, this rendered nominal geometry forms a shared visual world-model interface for fixed and dynamic viewpoints. The video model sees the action only as visible nominal geometry, and learns how objects, surfaces, and occlusions change around it. On DROID and RoboCasa-GR1, this rendered interface improves over Cartesian/vector action-conditioning baselines, while motion-source and depth ablations isolate the value of nominalization and 3D contact cues. We further probe the same rendered-prompt interface through counterfactual trajectory redirection, held-out embodiment composition, and human demonstration to robot video.

Abstract:
Human hand-object demonstrations provide dense reference motions for training dexterous manipulation RL policies through reference tracking. However, to use such demonstrations for robot policy learning, retargeting must preserve more than hand pose and maintain the object-relative interaction structure that defines task execution. Otherwise, contact and feasibility artifacts can propagate into downstream RL training. We introduce TopoRetarget, an interaction-centric retargeting framework that uses a single parameter set across diverse retargeting conditions while preserving task-relevant local hand-object topology and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning. Project website:https://toporetarget2026.github.io/TopoRetarget/

Abstract:
Supervised fine-tuning gives vision-language-action (VLA) policies strong manipulation skills in demonstration-like states, but deployment on long-horizon, contact-rich tasks often exposes compounding errors. Small action mistakes can move the robot off the demonstration manifold, where locally plausible action chunks may continue driving the system toward failure. We present TUTOR, a human-in-the-loop post-training method that uses intervention episodes to repair such policy-induced failures. Rather than treating an intervention only as a corrective demonstration, TUTOR also reuses the autonomous rollout before takeover as signed evidence about the policy's behavior. A hindsight keypoint partitions each rollout into recoverable chunks to preserve and failure-inducing chunks to reject, while the human takeover provides verified recovery chunks to imitate. TUTOR trains on these three signals with a negative-aware flow-matching objective. Failed rollout chunks are suppressed by a lagged-reference repulsion term tied to the policy snapshot that generated them, and recovery chunks are fit with standard flow-matching imitation. On a 14-DoF dual-arm tabletop platform with three long-horizon, contact-rich tasks, TUTOR improves autonomous success from 8.3% to 35.0% over supervised fine-tuning, under the same online-episode budget and takeover interface as the intervention baseline. Ablations show that rollout negatives and human recovery data provide complementary gains, while SFT preservation mainly stabilizes updates.

Abstract:
A hallmark of human dexterity is the cooperative use of fingers, where different fingers take on distinct yet coordinated roles to accomplish fine manipulation, such as capping a pen with the hand that holds it. We study this finger-level coordination through \emphin-hand assembly: mating two rigid objects within a single dexterous hand, with no second arm and no fixture. We present a reinforcement learning formulation to solve this problem in a unified framework, which is driven by a goal relative pose between the two parts. Finger coordination is shaped by a function-based auxiliary reward and regularized toward a single human reference pose, while domain randomization and a fusion of historical proprioception and object observation confer robustness to occlusion-induced estimation noise. The same recipe solves three different assembly tasks (Bottle, Syringe, and Marker). Trained purely in simulation, the policies transfer zero-shot to hardware with a single camera, demonstrating robustness to state-estimation errors caused by occlusion. Our experiments also reveal that in-hand assembly places demands on hand morphology, as a benchmark for modern robotic hand systems.

Abstract:
An ideal robot grasp is firm enough to securely handle an object, yet gentle enough to avoid damaging it. Achieving this balance requires knowledge of the object's material properties, such as its density, elasticity, and surface friction. These properties, however, are seldom precisely known a priori. In this work, we propose a visuotactile approach to estimating material properties in real time, during the process of grasping. Our method uses these estimated properties to determine the minimum grasp force required to handle the object. We contribute a new dataset of real-world objects (fruits and vegetables) with measured physical properties (shape, mass, elasticity, and friction), which we use to construct our force estimation model via simulations. We experimentally validate our approach to grasp force control using a robot with a parallel-jaw gripper. We demonstrate our system's ability to gently grasp a wide variety of objects, in each case adapting to their unique physical properties.

Abstract:
Human demonstrations are a scalable data source for learning robot manipulation policies. However, common human demonstrations, such as motion-capture trajectories and internet videos, do not capture the contact forces that are critical for forceful manipulation. In this paper, we introduce ForceBand a low-cost wrist-worn sEMG system for turning human muscle activity into force-enriched demonstrations. We first collect a 10-hour multimodal dataset containing egocentric video, sEMG, IMU, and fingertip forces across diverse actions and objects. Using this dataset, we train an EMG2Force model that predicts per-finger forces from sEMG and IMU signals after a short user-specific calibration. After calibration, users can collect target-task demonstrations using only the wristband and video; EMG2Force then labels these demonstrations with per-finger force traces, producing force-augmented demonstrations for robot policy learning. Experiments show that ForceBand recovers fine-grained fingertip interactions with over 50% lower force prediction error than vision-based baselines, and achieves an 87% success rate on pick, squeeze, and place tasks that require object specific force control across objects with diverse shapes, sizes, and weights.

Abstract:
Learning a single policy that reaches a goal with high geometric precision while interacting safely with nearby agents poses conflicting objectives. Precision favors commitment to a fixed geometric plan, whereas interaction requires immediate deviation when another agent intrudes, causing policies optimized for one objective to often fail at the other. We study this problem in the context of reactive autonomous parking, where multiple vehicles must reach assigned slots with sub-meter terminal accuracy while remaining responsive to neighboring vehicles throughout the maneuver. We propose CoPark, a self-play RL approach that learns reactive parking through multi-agent self-play, built on a residual-policy architecture. A precomputed offline plan provides a fixed action prior, while a residual head learns the reactive corrections. This decomposition separates geometric consistency from interaction. The residual policy learns behaviors under self-play, where data and scripting fall short, while the fixed prior holds the slot-frame geometry that pure policies struggle to reach reliably. The key design is a partner-threat-modulated, channel-asymmetric release of the prior. A continuous threat signal shifts authority of the longitudinal channel to the residual head to enable yielding, while the lateral channel remains anchored to the precomputed reference to preserve sub-meter slot alignment. A closed-loop refinement layer corrects residual terminal error from action-grid discretization. We train our policy on six parking lots and evaluate zero-shot on our new reactive-parking benchmark spanning Dragon Lake Parking (DLP) and DeepScenario Open 3D (DSC3D). CoPark achieves ~70-85% success with only 3-6% collision rate, substantially outperforming classical, imitation-learning, and large-scale RL baselines. Importantly, the results demonstrate emergent interaction behaviors such as reverse-yielding, mid-maneuver yielding, tight-corridor passing, and queuing.

Abstract:
Vision-language-action (VLA) models rely on discrete action tokenizers to bridge continuous robot control and autoregressive sequence modeling, yet existing tokenizers often trade off between compression, latency, and downstream performance. We revisit this design through the lens of neural audio codecs—convolutional encoder–decoder architectures with residual vector quantization that serve as the standard front end for audio foundation models. Motivated by their success, we introduce the Neural Action Codec (NAC), which treats short robot action trajectories as multi-channel 1D signals and compresses them using a multi-scale RVQGAN architecture. We observe that audio-specific mel-spectrogram objectives are ill-suited for kinematic signals; however, by replacing them with simple time-domain and non-mel spectral reconstruction losses, audio-codec-style models can autoencode actions with high fidelity without substantial architectural changes. NAC provides a compact, ordered token space via offset codebooks, enabling standard autoregressive policies to operate over short, structured sequences. Meanwhile, a Vocos-style decoder with an ISTFT head and adversarial discriminators recovers smooth, detailed trajectories. Across LIBERO-10, RoboMimic, and a suite of real-world manipulation tasks, NAC achieves lower reconstruction error and higher success rates than binning, FAST, and prior VQ-based tokenizers at comparable or better compression rates. These results demonstrate that repurposed neural audio codecs offer a strong, practical backbone for learned action tokenization in modern VLAs.

Abstract:
Vision-Language-Action (VLA) models leveraging the native autoregressive paradigm of Vision-Language Models (VLMs) have demonstrated superior instruction-following and training efficiency. Central to this paradigm is action tokenization, yet its design has primarily focused on reconstruction fidelity, failing to address its direct impact on VLA optimization. Consequently, the fundamental question of what makes for good action tokenizers remains unanswered. In this paper, we bridge this gap by establishing design principles specifically from the perspective of VLA optimization. We identify a set of best practices based on information-theoretic insights, including maximized temporal token overlap, minimized vocabulary redundancy, enhanced multimodal mutual information and token independence. Guided by these principles, we introduce ActionCodec, a high-performance action tokenizer that significantly enhances both training efficiency and VLA performance across diverse simulation and real-world benchmarks. Notably, on LIBERO, a SmolVLM2-2.2B fine-tuned with ActionCodec achieves a 95.5% success rate without any robotics pre-training. With advanced architectural enhancements, this reaches 97.4%, representing a new SOTA for VLA models without robotics pre-training. We believe our established design principles, alongside the released model, will provide a clear roadmap for the community to develop more effective action tokenizers. Code and checkpoints are available at https://anonymous.link.

Abstract:
We present Unified Motion Action (UMA) Model, an approach that uses 3D object motion trajectories as a shared interface to bridge visuomotor control and dynamics modeling. UMA treats object motion and robot actions as co-evolving variables under a masked generative objective, in which the mask pattern determines both the supervision regime during pretraining and the inference mode at deployment. Using hindsight-relabeled motion contexts and a contrastive objective that disentangles task intent from scene geometry, UMA enables multi-task pretraining across heterogeneous data sources without requiring manually annotated task instructions. At deployment, the same pretrained parameters support motion-conditioned visuomotor control, motion-based dynamics modeling, and task adaptation from few-shot demonstrations. Pretrained on publicly available robot demonstrations, human videos, and simulated data, UMA consistently outperforms state-of-the-art baselines specialized for each inference mode. Videos and additional details are available at https://uma-robot.github.io

Abstract:
Vision-language-action (VLA) models increasingly condition robot policies on history, depth, or 4D features to resolve ambiguity in long-horizon manipulation. However, more spatiotemporal evidence is not necessarily better: when the injected evidence is not motion-consistent, it can introduce geometric drift, fragmented temporal cues, and unstable action generation. This raises a simple question: should a VLA remember past frames, or remember the motion that connects them? We introduce MotionVLA, a motion-history interface that converts a short past-only video window into compact, time-continuous trajectory-field tokens. Instead of treating history as a sparse set of independently lifted frames, MotionVLA represents recent observations as physically coherent motion evidence. Current visual tokens query this history to retrieve task-relevant motion information, which is then recoupled into the VLA stream under trajectory-grounded supervision. Experiments across simulation benchmarks and preliminary real-robot rollouts show that MotionVLA improves long-horizon manipulation while producing smoother and more direct executions. These results suggest that effective VLA memory is not just about providing more 4D context, but about exposing motion-consistent evidence that is usable for control.

Abstract:
Vision-Language-Action (VLA) models are often pretrained via imitation learning, making post-training with reinforcement learning (RL) important once deployment moves beyond demonstration coverage. Existing VLA-RL paradigms expose a reliability--efficiency tension. Model-free refinement avoids simulator bias but is costly, static model-based refinement improves efficiency with stale imagination, and online-updated model-based refinement adapts the simulator without auditing generated rollouts. We propose Model-Evolving Reinforcement Learning (MERL), a trust-calibrated framework for VLA policy post-training with simulator-generated imagination. MERL couples policy optimization with an evolving simulator composed of a world model and a reward proxy head, and calibrates imagined rollouts through two-level trust rather than using them uniformly. Across training stages, MERL adjusts the imagined-data ratio and rollout horizon according to simulator reliability; within generated rollouts, it selects and weights reliable imagined chunks before policy optimization. This mechanism keeps refinement close to real-data-driven policy improvement when the simulator is unreliable, and increases the influence of imagination as reliability improves. Experiments on long-horizon perturbed manipulation tasks show that MERL improves final success rate and online learning efficiency over imitation learning, model-free RL, and static or online-updated model-based baselines.

Abstract:
Robots often observe information that determines a future action long before that action is executed. In a shell game, for example, a robot first sees which cup hides the ball, watches the cups move, and only later needs to choose the correct cup. The final observation alone is not enough for a decision: the correct action depends on an earlier event. We refer to this temporal gap as observation-action delay. It makes memory a policy-facing problem: a policy must keep similar histories distinct, retrieve the past event relevant to the current decision, and convert that recall into an action-ready state. We call these requirements separability, addressability, and prospectiveness. We introduce Chameleon, a ~60M visuomotor policy for control-indexed prospective memory. Chameleon writes embodied event memory, preserves separable histories, retrieves control-relevant traces, and trains the resulting working state to be prospective. We also introduce Camo-Dataset, a real-robot benchmark that isolates observation-action delay by making the decision scene visually ambiguous, so the correct action must be inferred from earlier observations. Chameleon improves decision/end-to-end success on Camo-Dataset from 22.5%/21.3% to 80.8%/71.3%. On public long-horizon memory benchmarks, it achieves 87.1% ± 0.8% on LIBERO-10, 97.3% ± 4.5% on MemoryBench, and 75.1% ± 1.4% on MIKASA-Robo, setting the state of the art for same-size models and exceeding multiple larger VLA baselines under the reported protocols. Probes and ablations show that Chameleon learns separable, addressable, and prospective memory, and that these properties drive its performance gains.

Abstract:
How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.

Abstract:
General-purpose robots require decision-making models that can generalize across diverse tasks and environments. Recent work has explored building robot foundation models by extending multimodal large language models (MLLMs) with action outputs, forming vision-language-action (VLA) systems. These approaches are motivated by the intuition that the large-scale pretraining of MLLMs on language and images can transfer effectively to robotic perception and instruction understanding. In this work, we explore an alternative paradigm: using large-scale video pretraining as the primary modality for robot foundation models. Unlike static images and language, videos capture spatio-temporal state-action dynamics in the physical world, making them naturally aligned with robotic behavior. To investigate this direction, we curate an internet-scale video dataset of human activities and task demonstrations and train, for the first time at foundation-model scale, an open video model for generative robotic planning. The model generates zero-shot video plans for novel scenes and tasks, which are further processed into executable robot actions. We evaluate task-level generalization through third-party selected tasks in the wild as well as real-robot experiments, demonstrating successful physical execution. Our results show strong instruction following, robust generalization, and real-world feasibility. We release both the model and dataset to support open and reproducible research in video-based robot learning. Our project website with video results is available in the supplementary materials.

Abstract:
Robot-mediated rehabilitation has the potential to deliver high-intensity, individualized therapy, but existing systems (which have not consistently improved outcomes) typically prescribe kinematic trajectories rather than explicitly targeting the neuromotor activations thought to support recovery. Prescribing motions that elicit desired activity is challenging because muscle activity is hysteretic and stochastic, noninvasive measurement methods (e.g., surface electromyography, or sEMG) are noisy and sensor-location-dependent, and interventional datasets are costly to collect, inhibiting model development. To address this gap, and ultimately enable activation-optimizing rehabilitation robot behavior, we present a personalizable transformer-based model mapping individuals’ muscle activations (as measured via sEMG) to prescribed contact force trajectories when interacting with an end-effector rehabilitation robot. The model uses a transformer encoder–decoder leveraging rotary positional embeddings (RoPE) to generate a latent representation of trajectory-relevant and user-specific neuromotor factors, and augments trajectory prediction with flow matching to produce a distribution over plausible, training-distribution-constrained prescriptions. Using data from 16 healthy participants completing 15 isometric trajectory tracking tasks, we evaluate generalization across held-out tasks and users, as well as the feasibility of user-specific fine-tuning within a therapy-session timescale. The model achieves mean trajectory prediction errors of <8% on held out tasks and users, with effectively instantaneous deterministic inference and sub-second sampling of trajectory distributions. User-specific fine-tuning further reduces error using only minutes of additional data collection and training. These results provide a proof of concept for data-driven, activation-targeted trajectory prescription in robot-mediated rehabilitation and establish a foundation for future validation in clinical populations.

Abstract:
Multi-robot imitation learning, particularly in settings where visuomotor policies are deployed in a communication-free, onboard decentralised style, represents an attractive paradigm. However, its realisation remains insufficiently understood, largely due to the difficulty of collecting high-quality collective demonstrations, since a single operator cannot control many robots simultaneously. Meanwhile, unlike coupled collaborative manipulation, many coordinated robotic tasks achieve system-wide efficiency primarily through minimising inter-robot interference. This structure motivates us to study whether data collected by a teleoperated single-robot can be leveraged for large-scale coordinated box-pushing as a testbed. We systematically investigate dataset design, data collection strategies, and lightweight policy architectures under the resource constraints typical of robot swarms. In particular, experiments with up to 40 robots highlight the difficulty of acquiring effective coordination solely through passive observation of other operating robots, posing open challenges for multi-robot research.

Abstract:
We introduce Task-Space Imitation Guidance for Efficient Reinforcement Learning (TIGER), a reward-construction and pretraining framework for sparse-reward tabletop robotic manipulation. TIGER treats an action-chunked imitation policy not as an executable controller or action prior, but as a local task-space progress estimator: predicted action chunks are converted, using controller-aware action-to-motion mapping, into short-horizon end-effector references, and the RL agent receives dense progress rewards toward these references while the sparse environment reward remains the dominant objective. During pretraining, TIGER uses imitation-guided look-ahead signals to relax conservative value penalties for actions predicted to make task-space progress, reducing off-manifold exploration during early online RL. Across simulation and real-robot experiments, TIGER improves early sample efficiency and reduces measured safety violations while matching or improving final success rates relative to prior RL and IL–RL baselines on the evaluated tasks.

Abstract:
Human-in-the-loop reinforcement learning (HIL-RL) improves sample efficiency in real-robot manipulation through online human intervention. However, successful trajectories may include suboptimal actions that deviate from the desired task-execution path and force human intervention. Existing HIL-RL methods typically apply the consistent credit assignment principle to all transitions, uniformly propagating discounted terminal rewards through suboptimal segments, ignoring the actual contribution of each transition to task success. This overestimates Q-values for critic learning and indirectly misguides actor updates toward suboptimal behavior patterns. To this end, we propose PACT, a Preference-calibrated Actor-Critic Training framework for HIL-RL. It leverages intervention-induced preferences to reassign credit on identified suboptimal segments and guide policy optimization for unbiased critic-actor learning. Specifically, we first design a progress model that learns from human demonstration and identifies suboptimal segments for credit correction. Then, from the human action and resampled policy action at the intervention state, we build preference pairs to define a counterfactual advantage that penalizes Bellman targets of the identified suboptimal segment, enabling directional credit calibration. Moreover, we directly align the policy with human corrective actions in the bounded mean space, providing an additional signal beyond critic-guided updates. Across five real-robot manipulation tasks, PACT improves the average success rate by 24.5% and achieves 1.3× faster convergence, improving both RL sample efficiency and performance. Code is available at https://anonymous.4open.science/r/HILRL-A1X-BC05.

Abstract:
Vision-language-action (VLA) policies are increasingly post-trained with rollout-level feedback, but a failed language-conditioned trajectory often hinges on a local decision that the terminal success label does not identify. World models make post-training scalable through imagined rollouts, yet treating the whole trajectory as the supervision unit still mixes object grounding, target selection, contact alignment, and recovery into one outcome. We introduce Decision-Centric Counterfactual Preference Optimization (DCCP), a world-model post-training method that converts imagined futures into localized action preferences. DCCP mines decision-sensitive states using progress curvature and action-token entropy, intervenes only on the current first action chunk under a shared visual-language prefix, and evaluates short counterfactual branches with a conditional progress scorer. High-margin branch differences become action-token preferences optimized by a local DPO-style loss, while a trajectory-level GRPO objective preserves full-task improvement. Across standard LIBERO, LIBERO-BDDL, and MimicGen-D0, DCCP improves average success from 75.6% with WMPO to 80.1%, with the largest gains on instruction-sensitive branch-selection tasks. On four RM65 real-robot tasks, DCCP improves success from 70.6% to 78.8% over WMPO and reduces semantic commitment errors from 13.1% to 7.5%. These results show that world models can provide not only more rollouts, but sharper credit for the action choices that determine downstream progress.

Abstract:
Pre-trained robot policies always require fine-tuning to adapt to new environments. Reinforcement Learning (RL) offers high performance potential because it improves action optimality rather than simply mimicking data. However, such potential depends heavily on reward quality. Sparse rewards lack process feedback, human-designed rewards are costly and biased, and semantic rewards from foundation models are often not control-centric. We propose Hindsight Reward Editing (HiRE), a training-free framework to break this reward bottleneck. HiRE bridges the broad knowledge of foundation representation models with physical control awareness, by contrasting successful and failed trajectories in hindsight. It calibrates foundation representation models with identifying ``trap states'' that are predicted as high-rewarding states yet eventually result in failure, and vice versa. HiRE explicitly penalizes these traps while boosting rewards for critical successful states. This approach can be flexibly compatible with any foundation representations and RL algorithms. Experiments show that HiRE significantly improves fine-tuning performance compared to other reward recipes with minimal computation and high efficiency. Qualitative results are at https://hire-project.github.io.

Abstract:
Real-world reinforcement learning (RL) systems still struggle with the demands of contact-rich industrial manipulation, including achieving micrometer-level precision, 99+% success rates, and human-level cycle times. Although off-policy algorithms can improve performance by leveraging demonstrations and interventions, a key bottleneck is the lack of an intuitive interface for collecting such guidance while complying with constraints of the physical system and the policy. We propose PAKT, a framework for kinesthetic teaching in real-world RL. As opposed to teleoperation approaches, PATK relies on kinesthetic guidance, which is widely used in industry. However, a critical weakness of kinesthetic guidance is the possibility for the operator to move the robot along trajectories (e.g., velocities, accelerations, jerk) that the robot and/or policy cannot physically reproduce. Using PAKT, operators guide the robot through admittance control which constrains the human input thus ensuring that collected trajectories remain feasible for the policy and the overall system. To support this teaching interface with an appropriate execution layer, PAKT adds a high-performance control stack that maps low-frequency RL actions to high-frequency torque commands. It consists of a reference generator and subsequent impedance controller, where the reference generator preserves the tracking performance of the impedance controller while improving contact handling and producing smoother policy actions. Across four difficult insertion and industrial assembly benchmarks, including a complex datacenter compute tray, PAKT reduces cycle time by 23%-48% and human intervention by 62%-86%. These cross-task results demonstrate that PAKT is a practical step toward deploying robot learning methodologies in production, supporting intuitive data collection, safe execution, and higher-throughput deployments. Project website: https://pakt-website.github.io/pakt-website

Abstract:
Language Models (LLMs) have emerged as powerful reasoning engines for embodied control. In particular, In-Context Learning (ICL) enables off-the-shelf, text-only LLMs to predict robot actions without any task-specific training while preserving their generalization capabilities. Applying ICL to bimanual manipulation remains challenging as the high-dimensional joint action space and tight inter-arm coordination constraints rapidly overwhelm standard context windows. To address this, we introduce BiCICLe (Bimanual Coordinated In-Context Learning), the first framework that enables standard LLMs to perform few-shot bimanual manipulation without fine-tuning. BiCICLe frames bimanual control as a multi-agent leader-follower problem, decoupling the action space into sequential, conditioned single-arm predictions. Evaluated on 13 tasks from the TWIN benchmark, BiCICLe achieves 70.5% average success rate, outperforming the best training-free baseline by 6.1 percentage points and surpassing most supervised methods. We also demonstrate superior real-world performance on 3 tasks without hardware-specific retraining. The code will be released upon acceptance.

Abstract:
SE(3)-equivariance provides a useful geometric inductive bias for visuomotor policies, encoding that actions covary with rigid transformations of the observation. However, existing SE(3)-equivariant visuomotor policies build this bias on a global equivariance assumption that treats the entire observation as governed by a single rigid transformation, which breaks down in real-world manipulation tasks where the task-relevant region changes across different objects. We address this by learning a phase-conditioned affordance field (PAF) over the observed point cloud, a per-point score that the SE(3)-equivariant policy uses in two complementary ways. Its score-weighted centroid serves as a learned, phase-conditioned anchor that canonicalizes the observation and the action around the active region, replacing the fixed gripper-frame reference, and the score itself is appended as a per-point feature. This introduces a local geometric inductive bias, while preserving the policy's global SE(3) equivariance. We validate on multiple MimicGen tasks under varying spatial randomization, and PAF consistently improves the success rate of the underlying policy. On multi-object real-robot tasks, PAF outperforms baselines and remains more robust under lighting variations.

Abstract:
We propose LCLA (Language-Conditioned Latent Alignment), a framework for vision-language navigation that decouples perception from control by aligning visual-language observations to the latent representation of a frozen privileged expert policy. A lightweight adapter maps frozen vision-language embeddings into the expert latent space using supervised latent alignment rather than end-to-end policy optimization, enabling expert navigation behavior to be reused across perception backbones and environmental variations. We evaluate LCLA across in-distribution simulation, zero-shot out-of-distribution environments, robustness perturbations, and direct real-world deployment on a TurtleBot platform, where it improves OOD success rate from 49.3% to 80.5% and reduces collision rates from 45.0% to 15.5% relative to direct behavior cloning. Without any real-world fine-tuning or calibration, LCLA attains 70.7% real-world success, compared to 60.9% for a navigation-pretrained baseline and 40.2% for direct behavior cloning, demonstrating robust and efficient sim2real transfer while remaining lightweight at inference time.

Abstract:
Recent embodied navigation methods leveraging Vision-Language Models (VLMs) have shown strong generalization. However, reliable path planning in complex environments remains challenging due to insufficient spatial awareness. The key challenge lies in learning spatial awareness that generalizes across diverse scenes and supports cross-task navigation reasoning. Motivated by this challenge, we propose SPAN-Nav, which advances generalized 3D spatial awareness for embodied navigation by estimating egocentric 3D occupancy from RGB video streams and using it as an intermediate spatial representation for navigation reasoning. SPAN-Nav represents 3D occupancy in a continuous latent space and projects it into a compact spatial token for VLM integration. This design provides navigation-relevant geometric cues while avoiding the inference overhead of dense volumetric representations. To align occupancy perception with navigation reasoning, SPAN-Nav adopts Two-Stage Spatial-to-Action Alignment (S2A-Align), a training strategy that first grounds navigation decisions on ground-truth occupancy and then adapts action reasoning to self-predicted spatial priors to reduce the training-inference gap. Based on a large-scale dataset with 4.2M occupancy annotations across diverse scenes and navigation tasks, SPAN-Nav achieves state-of-the-art performance on Vision-and-Language Navigation (VLN), urban navigation and point-goal navigation benchmarks, improving Success Rate (SR) by 4.6% / 5.3% on R2R-CE / RxR-CE, reducing MetaUrban collision cost by over 4×, and reaching 90.9% / 91.0% zero-shot SR on InternVLA-N1 System-1 point-goal navigation benchmark. Real-world experiments further demonstrate robust obstacle avoidance and reliable execution in cluttered environments.

Abstract:
Planning paths or trajectories for robot navigation requires understanding scene geometry and traversability. Classic approaches relied on accurate 3D maps to define occupancy based planning costs. Learning based alternatives predict planning costs either in terms of distance to goal or temporal distance between images. However, the former overfits to scene layout and the latter lacks geometric understanding. Moreover, most of these methods estimate image-level scalar costs, which are not sufficiently informative to effectively guide the robot. In this paper, we propose Plann3r as a 3D grounded method to predict pixel-level planning costs in terms of geodesic distances for any given set of images with an arbitrary subgoal pixel. We integrate Plann3r within a navigation pipeline, dubbed VGGT-Nav, in two unique ways: the offline mapping cum global planning phase iteratively uses Plann3r to generate subgoals and reference image global costmaps; and in the execution phase Plann3r performs simultaneous localization and local planning to generate planning costmaps to directly condition a learnt control policy. We show that the proposed Plann3r and VGGT-Nav largely outperform baselines respectively on the planning and navigation tasks on existing benchmark dataset. We further show real world demonstrations that highlight sim-to-real deployment. We will make the code publicly available: \urlhttps://plann3r.github.io/ (anonymized).

Abstract:
Successful autonomous robot navigation in off-road domains requires the ability to generate high-quality terrain costmaps that are able to both generalize well over a wide variety of terrains and rapidly adapt relative costs at test time to meet mission-specific needs. Existing approaches for costmap generation allow for either rapid test-time adaptation of relative costs (e.g., semantic segmentation methods) or generalization to new terrain types (e.g., representation learning methods), but not both. In this work, we present scaled preference conditioned all-terrain costmap generation (\textscspacer), a novel approach for generating terrain costmaps that leverages synthetic data during training in order to generalize well to new terrains, and allows for rapid test-time adaptation of relative costs by conditioning on a user-specified scaled preference context. Using large-scale aerial maps, we show that \textscspacer achieves the lowest regret across varied preferences in five of seven environments for global path planning and generates costmaps from the RELLIS dataset with up to 13% lower mean absolute error when compared to the ground-truth costmap than the next best methods.

Abstract:
Olfaction provides critical information about airborne chemical cues, yet it remains largely underexplored in embodied AI, with limited prior machine olfaction work focusing primarily on semantic odor perception. We introduce OdorSpaces, a physics-based simulation platform and benchmark for visual-olfactory embodied navigation. OdorSpaces augments Habitat-Sim with 3D odor transport and dispersion simulated by Fire Dynamics Simulator (FDS), enabling agents to navigate photorealistic indoor environments using both visual and olfactory observations. The benchmark includes two representative source-seeking scenarios: early-stage fire localization and invisible gas-leak localization. We develop an initial multimodal reinforcement learning policy that integrates visual observations and olfactory measurements over time. Experiments show that the policy substantially outperforms diverse baselines, establishing a strong reference for future research on visual-olfactory embodied navigation.

Abstract:
World-model-based planning is a promising direction for robot navigation because it enables explicit prediction of future observations under candidate actions. However, existing navigation world models typically rely on multi-step diffusion and autoregressive frame generation, which substantially increases model-side latency and complicates closed-loop deployment. In this work, we present a lightweight one-step world model for navigation built on a 3D U-Net with efficient spatial-temporal attention. The model predicts all future frames non-autoregressively in a single pass, while pretraining and random trajectory sampling improve performance in limited-data settings. We integrate this world model into a unified planning framework that supports image-goal, language-goal, and point-goal navigation through modality-specific scoring functions and anchor-based trajectory initialization. Extensive closed-loop experiments in both simulation and real-world environments show that the proposed approach improves generation efficiency and achieves strong navigation performance against state-of-the-art baselines.

Abstract:
Language provides a natural interface for humans to specify and reason about goals, but training language-conditioned robotic policies and reasoning typically requires expensive manual annotations. We propose Action Self-Distillation (ASD), a method for learning language-conditioned reasoning for robotic navigation without human-annotated language or reasoning labels. ASD uses an open-source VLM, which contains broad world knowledge and reasoning capabilities, as both a teacher network and a student policy. During training, the self-teacher receives privileged trajectory feedback and infers the underlying navigation goal together with goal-oriented reasoning. The student is then trained to condition on the inferred navigation goal, predict future ego-frame trajectory, and provide a description of the scene semantics, plan, and reasoning, from onboard observations alone. Across 13 challenging indoor and outdoor real-world navigation tasks, our method improves success rate over the strongest baseline by 30% in indoor settings, 30% on multi-stage tasks, and achieves 95% success rate on outdoor tasks, validating that privileged trajectory feedback can enable self-distillation for scalable generalist navigation policies without human annotations.

Abstract:
Safe navigation in dynamic environments requires robots to plan under obstacle predictions whose errors are uncertain, non-stationary, and can induce rare but safety-critical failures. Existing control-barrier-function safety filters can reject immediately unsafe controls, but they provide little guidance on when the current finite-horizon planning mode itself is becoming unsafe as prediction uncertainty evolves. We propose Conformal Event-Triggered Risk-Certified Replanning (\emphCERT-Replan), a framework that uses calibrated barrier risk as an early-warning signal for replanning. CERT-Replan calibrates horizon-indexed obstacle-prediction residuals online and uses the resulting uncertainty radii to evaluate dynamic-obstacle safety margins. A one-step safety filter protects the next applied control, while a horizon-level risk monitor evaluates the upper-tail CVaR of predicted barrier-violation losses along the current MPC rollout. When this risk exceeds an allocated budget, CERT-Replan rejects the current planning mode and selects a lower-risk alternative, such as a different speed profile, corridor, or homotopy class, rather than repeatedly correcting the same nominal plan. In a non-stationary benchmark, CERT-Replan achieves an 83.3% collision reduction relative to the safety-filter-only baseline, while reducing average safety-filter intervention by 32.4%. Hardware-in-the-loop experiments further demonstrate onboard feasibility.

Abstract:
Generalist visual-navigation policies tell a robot which direction to go but not how far. A separate lifting step is needed at deployment to convert their unitless trajectory into a metric path that a controller can track; existing scalar lifters apply uniform scaling that misses per-waypoint corrections, while in-policy approaches that inject monocular depth into the policy inherit the prediction errors of the depth model near obstacles and at close range. We present PathLift, a per-waypoint depth-conditioned adapter that lifts a pre-trained visual-navigation policy by consuming an explicit, sensor-grade depth source (RGB-D, stereo, or projected LiDAR) and fitting per-waypoint magnitude and lateral residuals to a canonicalized prior. PathLift consistently improves closed-loop navigation across diverse visual-navigation policies in simulation and on a real wheeled robot under both stereo and LiDAR depth. By turning the policy's unitless intent into a depth-feasible metric path in a single feed-forward pass, the adapter lifts real-world success without test-time optimization.

Abstract:
Generative imitation-learning policies predict dense action sequences that deployed systems cannot easily verify, repair, or procedurally generate supervision for. We present PolyFusion, a pipeline that replaces the dense action head with a piecewise Bernstein-polynomial representation parameterized by a small set of control points. A generative policy is used to sample control points conditioned on perceived scene geometry and affordance-predicted grasp poses. The properties of Bernstein polynomials enable us to: (i) procedurally generate synthetic training data from a single demonstration; (ii) analytically guarantee safety; and (iii) repair infeasible trajectories using a sequential convex projection that minimally adjusts control points while preserving task-dependent poses (e.g., start, grasp, and place). We show that PolyFusion results in a compact polynomial parameterization providing ~82-200x faster wall-clock time, uses ~129x fewer FLOPs per episode, and ~10x fewer gradient updates, while still maintaining performance comparable to current state-of-the-art methods.

Abstract:
Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules. This modularity exposes a large architectural design space, but current systems still rely on researcher intuition to choose where information is stored, how observations are processed, and how model calls are connected. Agent Architecture Search (AAS) automates such design for text-domain agents, but has not been systematically evaluated on perceptual embodied agents through simulator rollouts. We study this transfer. We introduce AgentCanvas, a typed-graph runtime that hosts embodied executors as editable node-and-wire programs with simulator-aware execution and episode-level logs, and KDLoop, a coding-agent search procedure that cycles through proposal, critique, experiment, and distillation, with triggered reflection after stalls. We evaluate three AAS variants across four embodied executors spanning navigation, embodied question answering, and language-conditioned manipulation. The resulting 3×4 matrix shows that architecture-level search can produce deployable and directional success-rate gains on embodied tasks, while one apparent high-scoring candidate is rejected as leak-bearing. At the same time, the experiments expose constraints that are muted in text-domain AAS: optimization signals can be masked by rollout noise, search can become trapped in local edit basins, and episode-level credit assignment only partially emerges even when detailed logs are available. These results characterize both the promise and the current limits of automated architecture search for embodied agents.

Abstract:
When AI interacts with the physical world --- as a robot or an assistive agent --- new safety challenges emerge beyond those of purely ``digital AI". In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, a hot cup of coffee should not be handed to a child, or that spilled liquid on the floor is a slip hazard? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational constraints. To probe multimodal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that makes safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations.

Abstract:
While asynchronous execution enables VLA models to operate in real-time environments by predicting the next chunk during execution, it inevitably introduces boundary misalignment with the committed action stream. Existing delay-aware solutions address this at the prohibitive cost of extensive retraining, architectural modifications, or fixed-prefix inpainting that inflates computation when low latency is vital. To address these problems, we propose FreeBar, a training-free real-time inference plugin that enables robust asynchronous execution with only marginal additional inference cost by casting chunk alignment in flow-based VLAs as barrier-guided action-space constraint. Specifically, FreeBar introduces a lightweight Control Barrier Function (CBF)-inspired consistency constraint between consecutive action chunks and solves the resulting correction in closed form, avoiding costly backpropagation through the VLA action head. By progressively tightening the consistency bound during generation, FreeBar aligns generated chunks with the committed execution stream while preserving early-stage generative flexibility and long-horizon adaptability. Across Kinetix, RoboCasa, and diverse real-world robotic platforms with multiple VLA backbones, FreeBar improves success rates over naive asynchronous execution by 10%--15% on average, with larger gains in highly dynamic scenarios. It also preserves real-time computational efficiency, decreasing inference latency per action chunk over 45% than RTC. Additionally, as a universal plugin, FreeBar remains robust across distinct latency environments, base policies, and hyperparameter settings.

Abstract:
Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output-distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM's pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language-Action Prediction), which prepends each numeric action sequence with a natural-language action description, causally conditioning precise action-token prediction on a language-action plan without modifying the backbone architecture. With single-epoch fine-tuning alone, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA-0) and improves robustness on LIBERO-PRO under language, object, and spatial perturbations. We release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM-to-VLA capability transfer.

Abstract:
Robot datasets are costly to collect and are usually tied to a specific embodiment, making it difficult to reuse demonstrations across robots with different geometry, appearance, and kinematics. We introduce UniLatent, a cross-embodiment transfer framework that avoids pixel-level image translation. UniLatent renders paired source-target robot observations in simulation and aligns a source visual encoder to a fixed target encoder, producing a shared latent space for policy learning. A policy trained on encoded source demonstrations can then be deployed on target observations through the target encoder. Across three simulation tasks and two target embodiments, UniLatent reaches up to 93% zero-shot transfer success (without target demonstrations) and improves the strongest pixel-translation baseline by over 30 percentage-point gain in mean target success. In real-world xArm-to-Franka few-shot transfer, UniLatent completes 27/30 target rollouts, consistently outperforming target-only training and pixel-space translation pretraining.

Abstract:
Robot policies are typically MLPs mapping observations to actions. Yet robot observations are physical variables, and many action-relevant cues arise not from individual variables but from their interactions; power, inertial effects, contact, slip, and compliance depend on products among observable signals. We introduce PRISM, a policy representation that makes polynomial interactions among observable physical variables explicit, learnable, and compact. Rather than listing all polynomial terms, PRISM uses a factorized polynomial module to expose higher-order interaction features efficiently. In reinforcement learning, it keeps the standard MLP backbone but applies a gradually activated element-wise polynomial function after it. In imitation learning, it replaces linear proprioceptive conditioning in Diffusion Policy with a polynomial layer trained end-to-end. Across humanoid locomotion and contact-rich manipulation, PRISM improves performance over standard MLP policies and larger MLPs with matched capacity, showing that interaction structure cannot be replaced by capacity alone. It also yields sensorless compliant behavior without force, wrench, tactile input, contact labels, or admittance control. These results suggest that polynomial representations should become a standard architectural choice for embodied motor control.

Abstract:
Recent robot manipulation benchmarks have enabled systematic evaluation of general-purpose robot policies across diverse objects, scenes, instructions, and embodiments. However, most existing benchmarks still assume that the robot is the only active agent and that task semantics are determined primarily by the instruction and object state. This assumption overlooks a common source of real-world variation: humans actively sharing the workspace. In such settings, humans may reveal an implicit goal, temporarily change object availability, complete part of the task, or constrain safe robot motion while working nearby. We introduce HARP-Bench, a benchmark for evaluating robot manipulation policies in workspaces shared with active humans. HARP-Bench treats human activity as part of the manipulation problem, rather than as background scene variation. It organizes human-present manipulation into four conditions: Cueing, Interruption, Contribution, and Co-presence. Each condition is instantiated as paired variants of everyday manipulation tasks, enabling controlled comparison between standard manipulation and human-present manipulation. HARP-Bench also includes an interactive data collection pipeline for gathering demonstrations in dynamic human-present settings. We evaluate a diverse set of policies on HARP-Bench, including VLA models, imitation-learning methods, and heuristic policies. Our results show that human-present task variations pose a significant challenge for current robot policies. We further show that training on demonstrations collected with HARP-Bench improves performance under these human-present conditions. Real-world experiments further confirm the validity of HARP-Bench, and our generated data can improve policy behavior when humans actively affect task execution.

Abstract:
Detecting and describing failure events during robot manipulation is critical for safe autonomous deployment, yet existing methods either reduce failures to per-frame binary scores or require the complete episode. We formulate robot failure detection as an online video temporal grounding problem, where the model processes streaming video and outputs both the temporal boundaries and a natural-language description of each failure event as it unfolds. Our framework operates chunk-by-chunk with a memory bank over previously observed frames. To bridge the domain gap between general video understanding and manipulation reasoning, we design an automated pipeline that converts robot trajectory datasets into temporal grounding annotations for pre-training, which yields a 73% relative improvement in temporal localization. On our evaluation benchmark, the full model outperforms the strongest online grounding baseline with an mIoU of 0.232 (versus 0.194) and produces failure descriptions with a CIDEr score of 0.854. We release the generated dataset and the human-annotated benchmark to support future research on real-time robot failure detection, description, and recovery.

Abstract:
Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints. They build an understanding of the physical world that helps them convert reasoning into actions and quickly adapt to new scenes, tasks, and rules. We refer to this capability as intuitive manipulation. Existing benchmarks fail to capture this integration: they evaluate physical reasoning in isolation from execution, or measure policy performance without requiring explicit reasoning. We introduce IMBench, a benchmark designed to evaluate intuitive manipulation as an integrated capability spanning perception, physical reasoning, action generation, and iterative execution. Our tasks require models to infer task-relevant physical structure and generate feasible action sequences under explicit constraints, including contact-rich manipulation, tool use, and multi-stage dependencies. We introduce a benchmark of 35 tasks, 14K filtered trajectories, and scalable tools for generating diverse scenarios. Experiments reveal a consistent gap: vision-language models show partial physical reasoning ability but fail to produce executable plans, while state-of-the-art vision-language-action models struggle to satisfy task constraints and generalize across scenarios. These results identify intuitive manipulation as a missing axis in current foundation models and generalist robot policies, and position IMBench as a step toward evaluating and enabling more integrated, adaptive physical intelligence.

Abstract:
A challenging bottleneck in manipulation research is reproducibility and comparability of methods in real-world environments. Since manipulation is an interactive task, it cannot be evaluated on a static dataset of observations. Benchmarking in simulation is a common remedy, but cannot cover the whole complexity of real-world tests. This paper proposes a new paradigm or stochastic challenges. Like the well-known YCB objects, it is based on reproducible real-world tasks and therefore allows system-level innovation and comparability with different platforms. Due to their stochastic nature, our challenges however mimc the riguour and surprise of hidden test sets. To do this, we show how different robots in different labs can be compared based on their success rate over different random task samples, if these samples come from the same distribution. We propose stochastic challenges as a general paradigm to robot benchmarking and implement this paradigm in a novel benchmark for mobile manipulation, on which we compare two robots tested in two different labs.

Abstract:
Inference-time steering adapts pre-trained generative robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this verification only with visual observations, vision alone is often insufficient for contact-rich manipulation, where success depends on both global task progress and subtle local interactions such as contact force. We introduce ViTaL, a visuo-tactile inference-time steering framework that formulates multimodal guidance as a bi-level optimization problem. At the high level, visual sampling-and-verification performs long-horizon mode selection, deciding what behavior the robot should execute. At the low level, tactile-guided diffusion editing refines the selected action sequence over a shorter horizon to satisfy local contact requirements. To support outcome-based steering, ViTaL learns a visuo-tactile latent world model and employs semantically aligned visual and tactile verifiers, including a novel text-conditioned tactile reward that scores predicted tactile futures directly in latent space. Across three real-world contact-rich manipulation tasks, ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive multimodal fusion by at least 20%. Website: corl2026vital.github.io.

Abstract:
Teaching humanoids loco-manipulation skills, such as carrying diverse objects, via visual imitation is a promising path toward generalist robots. However, acquiring high-quality, full-body human videos that span a wide variety of object interactions is difficult, leaving too few demonstrations to learn a generalist policy. We propose PRISM, a real-to-sim-to-real framework that overcomes this limitation by amplifying a handful of real videos into a large, diverse training set. PRISM first generates hundreds of diverse ``counterfactual'' human–object interaction videos via video-to-video (V2V) generation from a few exemplar real videos. Our contact-anchored real-to-sim pipeline then reconstructs both human and object motions, retargeting this imperfect video data into physically plausible trajectories. The intra-class variability across these counterfactual videos lets us train a single policy that generalizes to unseen objects within each category. We demonstrate the full pipeline by deploying this policy on a real robot without any real-world fine-tuning. Using only onboard depth observations, our humanoid picks up, carries, and drops objects—including boxes, barrels, bins, and balls---across novel instances, sizes, and initial configurations.

Abstract:
Current approaches for humanoid visuomotor whole-body manipulation, primarily relying on teleoperation or visual sim-to-real reinforcement learning, are hindered by hardware logistics and complex reward engineering. Consequently, demonstrated autonomous skills remain limited and are typically restricted to controlled environments. In this paper, we present the Humanoid Manipulation Interface (HuMI), a framework for learning whole-body manipulation tasks across various environments. HuMI enables humanoid-free data collection by capturing rich whole-body motion using portable hardware. This data drives a hierarchical learning pipeline that translates human motions into dexterous and feasible humanoid skills. Extensive experiments across five whole-body tasks, including kneeling, squatting, tossing, walking, and bimanual manipulation, demonstrate that HuMI achieves a 3x increase in data collection efficiency compared to teleoperation and attains a 70% success rate in unseen environments.

Abstract:
We present MotionDisco, a framework that discovers contact-rich, long-horizon humanoid loco-manipulation motions from scratch, without relying on teleoperation or motion retargeting from human demonstrations. This is challenging because the space of possible contact interactions grows combinatorially with the task horizon and the number of objects in the scene. MotionDisco enables rapid discovery of novel motions by coupling a large language model (LLM) guided evolutionary search over sequences of interactions with an efficient sequential kinodynamic trajectory optimizer and pruning strategy, enabling the rapid discovery of novel skills. Through extensive ablation studies, we show that our LLM-guided search discovers successful whole-body trajectories across several challenging long-horizon tasks. Finally, by training reinforcement learning tracking policies on the discovered trajectories, we transfer the motions to a real humanoid robot. This is the first work to discover and deploy long-horizon humanoid loco-manipulation skills entirely through automated evolutionary search.

Abstract:
Dynamic loco-manipulation requires legged robots to coordinate whole-body motion while maintaining stable physical interaction with grasped objects under uncertain external forces. While tactile sensing has been widely studied for robotic manipulation, its role in dynamic whole-body control remains largely unexplored. Existing works without tactile feedback commonly grasp firmly rather than regulate the grasp according to the interaction. We propose a tactile-augmented unified reinforcement learning framework that encodes tactile array observations from compliant grippers into a compact latent representation and joins it with proprioception for unified control of the legs, arm, and gripper. With effective grasp stability reward design, the policy learns to simultaneously track body velocity and end-effector trajectories, moderate grasp force, and prevent object slip under both gradual load changes and sudden release events. We deploy the policy zero-shot on a Unitree Go2 with an Interbotix WidowX 250 arm and tactile gripper, demonstrating dynamic tactile-informed loco-manipulation under varying external interactions, achieving a 47% reduction in grasping force and an object drop rate of less than 1%.

Abstract:
Brachiation enables primates to move across overhead supports when ground paths are blocked, suggesting a complementary locomotion mode for robots operating in cluttered or hazardous environments. Bringing this capability to high-DoF humanoid robots is difficult because the controller must discover a long-horizon release-swing-capture sequence, coordinate alternating contacts with whole-body momentum, and act without reliable measurements of segment-relative displacement or hook-contact state. We present SwingBot, a learning framework for continuous humanoid brachiation with passive wrist hooks. SwingBot makes the task trainable by organizing learning around the structure of brachiation: biomimetic keyframes make rare release-swing-capture transitions reachable during early exploration, and recurrent privileged-state estimation provides compact position and contact latents for deployment. Hardware experiments demonstrate continuous bar traversal and robustness to payload, external disturbances and different bar spacings, showing that this formulation offers a practical route to whole-body robotic brachiation.

Abstract:
Hierarchical pipelines for humanoid loco-manipulation typically pair a task-agnostic whole-body controller (WBC) with a downstream task policy. Such WBCs are often trained through manual reward shaping, which yields visibly machine-like motions; more consequentially, they are trained without the manipulated object in the scene. Object dynamics are instead approximated through randomized end-effector force disturbances, a proxy that fails to reproduce the distributed multi-contact wrench a real payload applies across the arms and torso, with the gap becoming critical near the platform's payload limit. In this work, we train the low-level (LL) controller itself as an object-aware motion prior: a single policy trained via motion imitation on retargeted motion-capture (MoCap) references that span the core motion primitives of the downstream task, with the manipulated object physically present in the scene. This single stage inherits natural human-like coordination from the references and absorbs the realistic contact wrench at its source, while exposing the same compact task-space command interface used by command-tracking WBCs. A reference-free high-level (HL) policy is then trained on top, with its action channel anchored to the empirical envelope of the low-level training data so that it generalizes the priors to a long-horizon multi-stage task while structurally preventing queries that would push the low level off-distribution. We instantiate the framework on a Unitree G1 for bimanual box pick-and-place: the same deployed policy generalizes across object masses and dimensions, initial and goal poses, and reactively recovers from intentional drops mid-carry.

Abstract:
Keypoint tracking alone is insufficient for object interaction tasks such as sitting on a chair, wiping a board, or pushing furniture, where the robot can reach the correct pose without making meaningful physical contact with the object. We present ContactMimic, a learning framework that tracks explicit part-level binary contact commands alongside keypoint trajectories. ContactMimic is made possible through the use of contact-following rewards and a trajectory augmentation scheme aimed at breaking the correlations between keypoint trajectories and contact labels. The resulting policy successfully decouples contact behavior from keypoint geometry, and achieves precise physical contact as well as contact-controllability, producing or suppressing contact during deployment as desired. Simulation experiments across 10 diverse human-object interaction motions confirm that ContactMimic exhibits contact controllability that enables it to complete manipulation tasks without task-specific rewards, while also outperforming keypoint-only trackers on contact-relevant tasks. Ablations confirm the necessity of the proposed trajectory augmentation scheme, and sim-to-real deployment validates contact controllability in the real world across 5 different motions.

Abstract:
World Action Models (WAMs) improve robot policies by predicting future visual consequences, but inference-time video prediction increases latency. Recent efficient WAMs avoid this cost by using the visual branch only during training and deploying an action-only policy. However, predicted futures remain an auxiliary loss and do not explicitly guide policy improvement. We propose WAM-STEP, an efficient post-training framework that turns the training-only visual branch of a WAM into an internal imagination module for self-improvement. WAM-STEP first calibrates the visual branch on mixed-quality policy rollouts to model policy-induced outcomes, using an action-conditioned preference loss to improve action-following future prediction. It then freezes the calibrated visual branch and optimizes the action branch using reward-scored imagined futures, with a policy-anchor baseline to stabilize group-relative updates. This enables efficient consequence-to-action feedback without introducing a separate world-model simulator or adding video generation at inference. On RoboCasa365, WAM-STEP improves the base policy from 20.7% to 24.0% average success. On two real-world tasks, WAM-STEP achieves 62.5% average success, outperforming the base policy by 20 points. These results show that training-time visual consequences can be internalized into stronger action-only robot policies.

Abstract:
Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce CAM, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that CAM outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions.

Abstract:
Vision-Language-Action (VLA) models are promising for generalist robot manipulation but remain brittle in out-of-distribution (OOD) settings, especially with limited real-robot data. To resolve the generalization bottleneck, we introduce a hierarchical Vision-Language-Action framework VISTA that leverages the generalization of large-scale pre-trained world model for robust and generalizable VIsual Subgoal TAsk decomposition (VISTA). Our hierarchical framework VISTA consists of a world model as the high-level planner and a VLA as the low-level executor. The high-level world model first divides manipulation tasks into subtask sequences with goal images, and the low-level policy follows the textual and visual guidance to generate action sequences. Compared to raw textual instructions, these synthesized goal images provide visually and physically grounded details for low-level policies, making it feasible to generalize across unseen objects and novel scenarios. We evaluate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios, and the performance of the same-structured VLA in novel scenarios could boost from 14% to 69% with the guidance generated by the world model. Results demonstrate that our method outperforms previous baselines with a clear margin, particularly in out-of-distribution scenarios.

Abstract:
World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7px reprojection error on 90% of episodes for external cameras). We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.

Abstract:
World models that capture how actions induce physical change enable scalable robot learning without reliance on embodiment-specific action labels. Pixel-space video models provide broad visual priors but expend model capacity on dense appearance reconstruction, while direct action models require embodiment-specific labels that hinder scalability. We present \mu_0, a scalable world model based on 3D traces. Rather than predicting dense pixels or directly modeling actions, \mu_0 forecasts smooth 3D trajectories for salient interaction points such as objects, tools, hands, and contact regions, yielding a compact, embodiment-agnostic motion interface. To enable training from diverse video sources, our TraceExtract system automatically extracts 3D supervision by selecting keypoints, constructing globally aligned traces, and associating motion segments with hierarchical language captions. This TraceExtract supervision pretrains \mu_0 by combining a pretrained vision-language backbone with a modular trace expert, which represents each query via B-spline control points and predicts future trace. Experiments show that \mu_0 outperforms baselines in both 2D and 3D trace prediction, including trace prediction models, and tokenized VLM methods. Because \mu_0 is frozen and reusable, it can be paired with action experts for downstream robot embodiments. Despite action-free pretraining, the resulting trace-conditioned policies achieve performance competitive with VLA models pretrained with action supervision, such as \pi_0. These results establish 3D traces as a scalable and transferable representation for cross-embodiment manipulation.

Abstract:
Human videos are a scalable source of supervision for robot manipulation, as they are abundant and naturally capture rich object interactions. However, transferring human demonstrations to robots remains challenging due to embodiment mismatch, scene variation, and robot-specific feasibility constraints. We present Human2Any, a framework for learning reusable object-centric interaction priors from human videos without requiring real-world robot demonstrations in the target task contexts. Human2Any represents manipulation through object--object interaction motion, capturing task-relevant scene changes while abstracting away embodiment-specific details. It composes learned interaction priors with robot-side feasibility reasoning and motion planning, allowing the same human-derived knowledge to adapt to different embodiments, scene geometries, and task contexts. We validate Human2Any across diverse manipulation settings, including real-world experiments on a Franka tabletop setup and an RBY-1 humanoid mobile robot, demonstrating robust interaction-centric manipulation without real-world robot training data.

Abstract:
Open-vocabulary mobile manipulation (OVMM) requires a robot to carry out a natural-language instruction on a mobile platform, and the success of learned manipulation policies hinges on placing the base within the policy's training distribution. We focus on policy mobilization, the problem of predicting an SE(2) base pose from which a fixed pre-trained manipulation policy will succeed, given a single ego-centric RGB-D frame and a language instruction. Prior work estimates a per-task, instruction-agnostic success likelihood over base poses, which limits generalization across tasks and language-based disambiguation. To address these limitations, we propose MoFlo (Mobilization Flow), a multi-task model that learns a language- and observation-conditioned transport from the robot's current base pose to a policy-compatible endpoint, trained by conditional flow matching with x_1 prediction. On five RoboCasa kitchen tasks, MoFlo achieves an 80% mean success rate, outperforming prior policy-mobilization baselines, and further extends to language-based disambiguation among multiple fixtures of the same category. In real-world experiments on a mobile manipulator, MoFlo places the base for successful manipulation from a single ego-centric observation, including cases that require selecting a language-specified target in an ambiguous scene, outperforming a representative baseline.

Abstract:
Learning-based manipulation requires supervision that is both semantically meaningful and physically executable, but current data pipelines often provide only one of these properties. Human demonstrations capture intent but are costly to collect and constrained by the human--robot embodiment gap, while simulation can scale data generation but often under-specifies functional behavior. We present IM-ENGINE, a simulator-grounded pipeline that uses image editing as an intermediate representation for embodied data generation. Given a rendered scene with known geometry, depth, segmentation, and camera parameters, IM-ENGINE edits the image to inject task-relevant semantics, recovers explicit 3D state using simulator priors and an unchanged anchor object, refines the state in physics, and converts it into robot-executable supervision. We instantiate the pipeline for dexterous grasp synthesis and goal-state generation. For grasping, IM-ENGINE generates a human grasp in image space, recovers the hand-object interaction, retargets it to a robot hand, and refines it into physically validated robot grasps. For goal generation, it edits a rendered scene into a desired outcome, recovers the target-object pose, and refines it into physically valid, semantically meaningful goals and trajectories. This combination of generative semantic priors and simulator grounding enables scalable task-relevant supervision for robot learning.

Abstract:
Generative BC policies model expert action distributions well, but they do not explicitly reason about what scene a candidate action would produce, so they cannot tell whether it reaches a task-relevant state. We address this by coupling action generation with an action-conditioned dynamics model that predicts the future observations induced by actions. However, naively placing such a dynamics model inside a generative policy creates a circular dependency. Future observations are needed to guide action generation, yet those observations can only be predicted from the action samples being generated. We introduce Dynamics-Aligned Flow Matching Policy (DAP), where a flow matching policy and an action-conditioned flow matching dynamics model co-sample actions and future observations through flow extrapolation. At each step the two flows correct each other, and the samples converge to a self-consistent action and future pair on which the policy and dynamics model agree. Across four simulated tasks and three real-world tasks, DAP achieves the highest average success rate, with relative improvements of 47.0% in simulation and 34.0% in the real world over the strongest baseline in each setting. Moreover, when the future-observation estimate is corrupted at inference time, DAP recovers a self-consistent action-future pair through iterative refinement, whereas video-conditioned and post-hoc dynamics baselines, which use a fixed future surrogate, degrade or fail.

Abstract:
Equivariant architectures improve sample efficiency and generalization in robot learning, but they require the symmetry group to be specified in advance. Existing symmetry-discovery methods focus on static datasets, autonomous dynamical systems, or input--output mappings without separate action representations, and to our knowledge none directly addresses controlled transition datasets \(s_i,a_i,s_i')\, where actions transform under their own representation and data distribution introduced by the behaviour policy need not be symmetric. We introduce \emphTransition-Manifold Tangency (TMT), a geometric framework for discovering symmetries from transition data. Valid transitions form a submanifold of state--action--next-state space, and a dynamical symmetry must move along it: every infinitesimal generator is tangent to this manifold. The estimation target -- the local normal frame -- depends on data geometry but not on the density along the manifold, which yields robustness to nonlinear dynamics and to asymmetric behaviour policies. We provide a model-free local-PCA estimator (TMT-KNN) and a denoiser-based estimator (TMT-NN), and extend both to discrete symmetry discovery via a manifold-consistency test. On continuous benchmarks TMT-NN recovers the ground-truth generator subspace where Jacobian-based and distribution-matching baselines fail under nonlinear dynamics and policy bias respectively. On discrete benchmarks, TMT-NN's candidates approach the denoiser noise floor in absolute manifold distance while a ratio-based metric favours the near-identity candidates of distribution-matching baselines.

Abstract:
Observing objects grasped by a robot hand is challenging due to severe visual occlusions. The manipulator itself obscures large portions of the object, while a fixed camera provides viewpoint-constrained observations. Although these hidden surfaces can be exposed by in-hand manipulation, most existing approaches rely on predefined or open-loop reorientation strategies that do not explicitly target unobserved regions. To address this limitation, we propose AURORA, an active 3D reconstruction framework that leverages in-hand manipulation for object inspection. At the core of AURORA is Ray-GPIS, a novel informative planner that estimates reconstruction uncertainty along candidate viewing rays. By identifying directions that maximize uncertainty-novelty score, Ray-GPIS generates next-best-view targets, which are executed via an axis-conditioned in-hand rotation policy. The resulting visual observations are integrated through a visual fusion pipeline for incremental 3D reconstruction. Extensive real-world experiments on objects with diverse geometries demonstrate that AURORA outperforms baselines, achieving higher surface completeness and faster reconstruction. The project webpage is available at https://aurorahand.github.io/

Abstract:
Imitation learning for contact-rich manipulation requires high-quality expert data that is expensive to obtain. This makes learning a spatial robust policy under limited demonstrations a key issue. To address this, we propose VISTA, an equivariant visuotactile diffusion policy for data-efficient contact-rich imitation learning. VISTA projects vision and tactile into spherical tokens, injects tactile contact cues into visual spherical directions through permutation-equivariant spherical fusion, and rotates the fused harmonic representation using the end-effector orientation. The resulting representation conditions an equivariant diffusion policy to predict spatially consistent actions. Extensive experiments in both simulation and real-world robotic settings show that VISTA substantially improves data efficiency and spatial robustness over strong visuotactile imitation learning baselines.

Abstract:
Dexterous robot hands can receive useful motion intent from teleoperation or learned policies, but successful execution also depends on contact decisions that these commands rarely specify. We introduce DexFLEX, a contact-aware foundation controller that turns upstream fingertip-motion drafts into contact-consistent joint commands. Instead of treating a draft as a trajectory to copy, DexFLEX treats it as evidence about intent: from the current tactile-proprioceptive state, it proposes feasible short-horizon motion chunks, predicts their contact consequences, and selects the candidate that best follows the command while preserving future contact stability. At inference time, a simple Draft-Dream-Select procedure combines pure-prior proposals for recovery with draft-seeded proposals for responsiveness, then decodes the selected motion into executable joint-space targets. Across degraded-command simulation, real-world shared control, and visuomotor policy learning, the same trained controller improves robustness without retraining, increasing real-world teleoperation success from 29.2% to 78.3% and reaching 46.7% policy-learning success, 3.5 times direct joint-action prediction, and 1.56 times prior-only correction. Videos are available at dex-flex.github.io.

Abstract:
World models promise a paradigm shift in robotics, where an agent learns the physics of its environment once and then acquires behaviors efficiently. Yet the learned dynamics models at their core are typically morphology locked. In legged locomotion, a dynamics model trained on an ANYmal-D quadruped fails on a Unitree Go1 because it overfits to one robot's embodiment rather than capturing the locomotion dynamics shared across robots, so even a small change in actuator dynamics or limb length forces retraining from scratch. However, if we formalize a robot's unique physical traits into a morphology specification, a controller for a family of robots can utilize this blueprint in two ways. It can feed the specification to a model-free policy, or it can feed the specification to a learned dynamics model and extract the policy in imagination. We argue for the second route and introduce the Quadrupedal World Model (QWM), which conditions a single generative dynamics model on scale-invariant physical features and trains policies entirely inside it, through a physical morphology encoder, an adaptive reward normalizer, and morphology conditioning in the latent dynamics. Holding the morphology information identical, a model-free policy matches QWM on the training cohort but degrades on unseen morphologies, while QWM transfers zero-shot with no fine-tuning, adaptation, or warm-up in such cases. To our knowledge, this is the first world model to demonstrate zero-shot cross-embodiment transfer within the quadrupedal family.

Abstract:
Humans seamlessly navigate complex environments—such as winding trails, uneven stairs, and sparse stepping stones—over miles of continuous walking without losing their balance. Getting humanoid robots to achieve this long-horizon reliability requires overcoming compounding errors between perception and action. Current perceptive locomotion methods often fail during kilometer-scale deployments because they rely on lossy dense terrain representations that smooth over action-critical details, and myopic imitation objectives that lack trajectory-level credit assignment. In this paper, we present SOLO (Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion), an end-to-end framework designed to conquer these compounding errors using only a single, noisy chest-mounted depth camera and proprioception. SOLO achieves this through two key innovations: a Query Reconstructor (QR) that explicitly retrieves precise, per-cell spatial evidence to preserve sharp terrain edges, and Trajectory-Aware MSE (TA-MSE) Distillation. By injecting next-state teacher disagreement directly into the reinforcement learning reward channel, TA-MSE unlocks explicit trajectory-level credit assignment, naturally penalizing actions that drive future state divergence. We demonstrate SOLO zero-shot on a real humanoid robot, achieving a continuous 1.5-kilometer outdoor traversal across naturally occurring stairs, slopes, and unstructured terrain. In rigorous stress tests, our system cuts terrain estimation error by up to 4× and raises traversal success on extreme sparse footholds from 3% to 96%, unlocking robust, deployable humanoid autonomy in the wild.

Abstract:
Driving Vision-Language-Action Models (Driving VLAs) aim to use language to improve end-to-end planning, but the language-action gap limits this promise. We propose DriveMA, a Driving VLA framework built on verifiable meta-actions, which summarize future ego motion into compact language-domain intentions and can be both derived from expert trajectories and verified against generated trajectories through rule-based mapping. DriveMA exploits this verifiability with action-centric supervised training and a data-efficient turn-level credit-assignment reinforcement learning framework, explicitly aligning high-level decisions with low-level trajectory planning through dense rewards and precise credit assignment. DriveMA achieves a new state of the art on the Waymo End-to-End Driving Challenge already with a 2B model, reaching an RFS of 8.060, while its 4B version further improves the state of the art to 8.079; it also obtains competitive closed-loop planning performance on NAVSIM. These results show that even a simple meta-action interface can achieve state-of-the-art planning when made verifiable and optimized for language-action alignment.

Abstract:
Autonomously navigating off-road terrain at high speed demands accurate control, which often relies on dynamics models that capture complex interactions between the robot and terrain. In off-road regimes, terrain interaction is a major component of the vehicle's dynamics. Prior work attempts to capture this effect by training learned models of the vehicle's dynamics, conditioned on its perception of the local environment. However, said conditioning is often overly-simplistic, ignoring the spatial structure of the dynamics problem. Our approach exploits this spatial structure; at each step, we query the BEV features around the predicted state, conditioning the next prediction on current geometric and semantic cues. We additionally provide a benchmarking approach to extensively evaluate dynamics models on challenging datasets collected across all seasons, decreasing the worst-case prediction accuracy by 23.8%, compared to physics-based and learned baselines. We further evaluate on hardware on high-speed (\geq10m/s) and geometrically challenging courses with a 34.9% reduction in maximum cross-track error.

Abstract:
We present GeomVLA, a 3D motion-aware VLA framework that unifies semantic perception, latent future-motion reasoning, and action generation within a shared robot-centric 3D coordinate frame. Our approach lifts pretrained VLM features into spatially grounded 3D scene tokens while preserving pretrained visual-language priors. We further introduce an Actionable 3D Trajectory Field, a task-conditioned latent motion representation that predicts how task-relevant scene points are expected to evolve directly in 3D space. Rather than serving as an explicit open-loop plan, the predicted trajectory field acts as a latent geometric reasoning process that conditions a 3D flow-based action policy through geometry-aware attention. GeomVLA achieves state-of-the-art performance on CALVIN and LIBERO, competitive performance on RoboTwin2.0, and consistently outperforms strong baselines in real-world manipulation settings. Extensive ablations demonstrate that future-motion reasoning alone is insufficient: the primary gains arise from maintaining geometric consistency between scene representation, motion prediction, and robot actions throughout the entire perception-to-action pipeline.

Abstract:
Vision-Language-Action (VLA) models have shown strong potential for language-conditioned robotic manipulation, but their reliance on 2D visual backbones limits precise reasoning about depth, spatial relations, and contact-rich actions. We propose SAGE-3D, a semantic-aware 3D VLA framework that aligns geometric structure with task semantics for instruction-conditioned action prediction. Motivated by representation analysis, our key insight is that effective 3D VLA policies require both transferable geometric cues and instruction-relevant semantic grounding: geometry should not be injected as an isolated modality, but organized around objects, relations, and affordances relevant to the task. To realize this, SAGE-3D introduces a 3D Semantic-Aware Feature Generator (SAFG), which integrates RGB semantics, depth information, and task instructions to produce 3D Reasoning Tokens. These tokens are injected into the action head through a FiLM-based geometry-conditioned decoder, allowing semantic-aware spatial context to directly modulate action generation. Extensive experiments in simulation and real-world manipulation show that SAGE-3D improves overall benchmark performance and training efficiency, with the largest gains on spatially demanding tasks. Notably, SAGE-3D reaches 100% success on LIBERO-Spatial, establishing a new state-of-the-art result and, to our knowledge, the first result to saturate this benchmark. Anonymous code is available at https://anonymous.4open.science/status/SAGE3D.

Abstract:
Discrete Vision-Language-Action (VLA) models typically formulate action generation as next-token prediction over discretized action spaces, conditioning each token autoregressively on prior context. While effective, this paradigm incurs high inference latency and largely ignores the temporal structure inherent in action trajectories. Recent efforts introduce parallel decoding to improve efficiency, enabling faster inference, but lack explicit mechanisms for modeling token dependencies. We introduce TBD-VLA, a discrete token-based VLA framework that incorporates block diffusion to enable temporal action generation. We partition action sequences into temporal blocks and perform masked discrete diffusion within each block, while maintaining autoregressive generation across blocks. This design unifies temporal autoregression and parallel action decoding, achieving both strong temporal coherence and improved inference speed. In addition, the explicit temporal modeling enables asynchronous execution of action chunks (e.g., Real-Time Chunking) via temporal in-painting. TBD-VLA significantly outperforms prior VLA approaches in both simulation and real-world manipulation tasks, offering a scalable path toward fast, temporally aware, discrete VLA models.

Abstract:
Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions. While latent action models reduce the action execution gap by learning action abstractions, they still rely on visual features. Thus, misaligned human and robot visual representations can lead to inconsistencies in policy inputs and induce domain-dependent latent actions, hindering effective co-training with human videos. To address this, we propose HARP, a human-robot aligned representation learning framework for more effective VLA pretraining from human videos. Specifically, HARP uses limited paired human-robot demonstrations as cross-embodiment bridges and abundant unpaired human and robot videos as a scalable dynamics supervision data source. It trains a robot-adapted visual encoder and a latent action model with manipulation-centric auxiliary cues and a source-relative pair-discriminative alignment loss, which adapts robot representations toward human semantics while preserving pair-level discrimination. The learned aligned vision encoder and latent action model provide a unified vision and action representation for VLA-style policy learning, where human and robot videos provide vision-language-to-latent-action supervision and a lightweight robot action head grounds latent actions into executable commands. Experiments on feature visualization, simulation, and realworld manipulation show improved human-robot alignment and downstream policy performance, achieving 4.481 average length on CALVIN ABC\rightarrowD and a 7.1% realworld success rate gain over the strongest baseline.

Abstract:
Action-supervised fine-tuning of VLA policies fits demonstrations effectively but constrains only the directions that change predicted actions, leaving visual structure consistent across action-equivalent states free to collapse. We formalize this as residual visual collapse along local action fibers and propose FIBERTUNE, a training-time objective that preserves teacher-structured visual residuals without adding inference-time overhead. FIBERTUNE uses an online action probe to estimate action-predictive feature directions, filters them from intermediate visual-token representations, and aligns the resulting probe-filtered residuals to a frozen visual teacher while regularizing their effective rank. Under identical training conditions, FIBERTUNE improves over task-loss-only fine-tuning in every one of six controlled simulation settings spanning two benchmarks and two architectures (π0.5 and OpenVLA-OFT), as well as on physical SO-101 pick-place; representative gains include +10.7 pp SR(5) on long-horizon CALVIN ABC→D and physical SO-101 task success rising from 72.7% to 78.1%. Residual diagnostics show that these gains coincide with increased probe-filtered residual teacher alignment and effective rank, consistent with the action-fiber motivation.

Abstract:
Latent action models (LAMs) infer pseudo-action labels from visual transitions, providing scalable supervision from internet-scale video for embodied AI. Existing LAMs rely primarily on visual reconstruction, which pushes the latent to encode any cue predictive of the next frame, including background appearance and future shortcuts. The resulting latents entangle motion with nuisance content and their norms decouple from motion magnitude, weakening cross-environment transfer and providing poor supervision for downstream policies. We argue that these failures stem from the absence of structural priors on the latent space, and propose to endow the latent space with an explicit algebraic structure. We introduce the Additively Compositional Latent Action Model (AC-LAM), which imposes a structural constraint on the latent action space, requiring latent actions to compose additively over time. Such additivity is naturally satisfied by rigid-body motion, yet violated by many nuisance signals. We show theoretically that this single constraint induces identity, inverse, and cycle-consistency structure in the latent space, and discourages additively-separable nuisance components such as scene-constant and goal-only offsets. Empirically, we develop a diagnostic suite that probes these algebraic properties and shows that AC-LAM learns more physically meaningful, displacement-calibrated latents with markedly less scene-identity leakage than prior LAMs. The same latents translate into consistent gains on downstream policy learning across both simulated and real-robot tabletop manipulation under in- and out-of-distribution conditions.

Abstract:
Robots under autonomous operation may require decisions based on evidence that is no longer visible. We study delayed-evidence tasks, where an early cue disappears before a later decision point, so visually similar observations can require different actions. In these settings, the current observation is not a sufficient state for control. We introduce TRAjectory-routed Causal Evidence (TRACE), a memory framework for visuomotor imitation policies. TRACE stores task-relevant visual and robot-state evidence, such as object identity, target choice, or route-dependent state, in a fixed-size latent memory that remains bounded over long episodes. Instead of indexing memory by raw time or manually provided task labels, TRACE uses path signatures: compact, order-sensitive features of the executed robot-state trajectory. These signatures do not store the visual cue itself; rather, they provide trajectory-conditioned keys for writing and retrieving the evidence stored when the cue was visible. When the robot later reaches an ambiguous observation, the policy conditions on TRACE memory to recover the missing context and choose the correct branch. TRACE attaches through lightweight adapters to policies, without changing the policy backbone, action head, or imitation objective. Across real-world long-horizon manipulation tasks with visually ambiguous branch points, TRACE improves branch selection and task success over alternative baselines, including short-history and recurrent memory.

Abstract:
Complex robotic manipulation tasks frequently require a long-term memory of past events and actions. As conditioning on full histories renders policies prone to spurious correlations and degrades performance, many approaches to policy memory involve compressing historical information through expensive VLM queries in-the-loop to process only task-salient information. In this paper, we propose an alternative approach in which computationally intensive VLM queries are made during train-time to learn a lightweight latent memory that can be efficiently queried at deployment time. Our representation, which we call the workspace token, is trained by (1) using a VLM to identify current and historical information necessary for completing a task, then (2) distilling these into the workspace token using a set-reconstruction decoder loss. In both simulation and hardware, we show that the workspace token can be used as a drop-in replacement for observations during policy deployment, enabling solving memory-intensive tasks without the need for VLM reasoning in-the-loop. Interestingly, we found that workspace tokens are not only more lightweight but also lead to better policy performance.

Abstract:
Multi-robot collaboration allows robots to efficiently take on a wide range of tasks, from moving a couch through a doorway to assembling structures on a construction site. Achieving such coordination in mobile multi-robot settings, however, remains challenging: centralized methods conditioned on the combined observations of a team scale poorly with team size, and decentralized methods that train one policy per robot often require explicit alignment procedures or information sharing at inference time to overcome partial observability. Our key insight is that the visuomotor priors of pretrained vision-language-action (VLA) models should enable reactive, decentralized collaboration from each robot's local observations alone, without these inference-time assumptions. We propose CHORUS, a framework that adapts a single VLA backbone to control diverse, multi-robot teams. At inference time, each robot runs an independent copy of CHORUS, conditioned only on its own observations and a robot-identifying prompt. In real-world experiments including mobile tape measurement, library book handovers, and laundry basket lifting, CHORUS achieves a 64% point improvement over decentralized, from-scratch models, improves reactivity to teammate behavior by 40% points, and outperforms centralized baselines. Together, these results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration, without per-robot policies or inter-robot communication at inference.

Abstract:
When humans hand each other objects, they incorporate both geometric and semantic information into this process. For example, passing a knife with the handle towards the recipient, rather than the blade, is both more ergonomic and safer. Recent state-of-the-art methods for task-oriented robot-human handovers have progressed from modeling object geometry to incorporating object affordances. However, they often forgo predicting the explicit, task-specific hand poses a human selects to utilize an object. Since many objects support multiple interaction modalities, e.g., a claw hammer used to strike or pull nails, this variability must be modeled to achieve robust task-oriented handovers. To tackle this, we propose a novel approach, GENESIS-Handover (GENErative HypotheSIS), which leverages VLM image generation to produce a variety of task-specific hand-object interaction hypotheses. These hypotheses are matched in real time to the observed human hand pose, enabling inference of the most suitable handover configuration proposal. By leveraging VLMs as priors of plausible human-object interactions, the method produces task-conditioned handover strategies for previously unseen object-task pairs. We evaluate the standalone interaction proposal module before deploying the full system on a mobile manipulator. In a user study with 12 participants across five task-object pairs, 83.3% perceived our method to have better task understanding than the previous state of the art.

Abstract:
Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real-world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators. Specifically, UniIntervene first performs future-conditioned action-value estimation, predicting the latent consequence of the current action and evaluating its induced value, which provides a more stable progress signal. Building on this, a temporal value-risk critic aggregates recent value dynamics and triggers intervention when the estimated value exhibits sustained stagnation or degradation. When intervention is required, UniIntervene retrieves a high-value recovery target from a memory of past intervention episodes and produces executable corrective actions through a goal-conditioned recovery policy. In this way, UniIntervene turns intervention from passive human correction into a value-aware recovery process for efficient real-world RL. Extensive experiments on diverse real-world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.

Abstract:
The ability to efficiently and reliably learn new tasks has been a foundational challenge in robotics. Vision-Language-Action (VLA) models have demonstrated strong generalization across diverse manipulation tasks, yet pretrained policies consistently fall short of the reliability required for real-world deployment. Reinforcement learning (RL) fine-tuning offers a promising path to bridge this gap, but existing approaches either train from scratch without fully leveraging pretrained priors, or fine-tune VLAs without achieving the sample efficiency and success rates that practical deployment demands. We present EXPO-FT, a system for stable, sample-efficient RL finetuning of pretrained VLA policies that closes this gap. Our system solves a suite of challenging manipulation tasks, including routing string lights and inserting the plug to light it up, striking a pool ball into a pocket, and inserting a flower into a wine bottle, each requiring combinations of high precision, dynamic actions, and robustness to varied initial states. Our system achieves perfect task performance (30/30 successes) across all evaluated tasks within an average of 19.1 minutes of online robot data, outperforming both prior RL-from-scratch and VLA finetuning approaches. We release an open-source codebase with the aim of facilitating broader adoption of RL finetuning of VLA models in robotics.

Abstract:
As robots are deployed in the real world, they will inevitably make mistakes. Human interventions provide valuable feedback for improving their behavior, enabling them to adapt online to new scenarios. However, studying human-in-the-loop robot learning is difficult: real-world experiments are slow and expensive, so most algorithm development relies on simulated human models. We show that common human modeling assumptions do not match how real people intervene in practice. Through an analysis of intervention data from human participants, we find that humans tend to intervene when agent progress stalls, and that their corrective actions are often suboptimal. Motivated by these findings, we introduce a progress-based model of human intervention that achieves higher rank correlation with real-user algorithm rankings, enabling realistic algorithm benchmarking and development in simulation. Driven by these insights, we identify shortcomings in prior methods for learning from interventions and propose Q-OIL, a method that introduces optimistic bonuses on intervention transitions in reinforcement learning, encouraging the agent to revisit and learn from informative intervention states. We show that with the right design choices, including decoupled critics to avoid over-optimism and imitation-learning regularization, our proposed method learns more efficiently than competitive baselines, achieving 10-50% higher success in simulation and on real-world robotic tasks. By grounding both evaluation and learning in realistic human behavior, this work provides a practical path toward scalable robot learning from interventions.

Abstract:
Behavior cloning for robot manipulation is based on expert demonstration datasets. However, for tasks that require dynamic stability, precise contact timing, or dexterous coordination, human teleoperators may find it hard or even impossible to collect data. We study this infeasible-demonstration regime and propose GLIDE, a framework that infers task-specific failure points and converts them into executable guardrails for data collection and policy deployment. Given a task description, GLIDE writes guardrails that use robot state to filter teleoperation and policy commands, constrain failure-prone actions, and iteratively improve from closed-loop dataset feedback. Across Tomato plate transfer, Marker handover & stand, and Wine serving tasks, GLIDE improves data collection efficiency over raw VR teleoperation and manually designed guardrails. After refinement, GLIDE raises data-collection efficiency from 0-10% to 70-90% across the three tasks. During policy execution, guarded deployment raises Tomato plate transfer policy success from 0% to 60-70%, enabling efficient learning on mixed-quality demonstrations. These results show that GLIDE guardrails can support policy learning when direct demonstrations are unreliable.

Abstract:
Whole-body mobile manipulation requires coordinating mobile base and manipulator under shifting viewpoints, posing challenges in geometric perception and action generation. Current policies either rely on 2D features or sparse 3D representations that lack dense spatial structure, and typically encode arm and base within one action vector that ignores their distinct control demands. Moreover, existing dense fusion strategies risk corrupting pretrained representations under noisy depth while incurring heavy computational overhead. We present GeoHAT, an end-to-end diffusion-based framework built on a simple principle: geometry should be injected only where reliable and attended to only where needed. GeoHAT employs a lightweight Fourier spatial encoder that maps dense per-pixel 3D coordinates into geometric tokens without an additional 3D vision backbone. These tokens are then selectively injected into vision foundation model features through per-token gated fusion modulated by depth validity, preserving the semantic prior while enriching spatial understanding. For action generation, a Hybrid Whole-Body Action Decoder decomposes arm and base into distinct subspaces and lets each action modality attend to its task-relevant visual context through sparse cross-attention, while causal temporal modeling captures intra-timestep coordination and inter-timestep dependencies. Experiments on the ManiSkill-HAB simulation benchmark demonstrate that GeoHAT achieves a 79.3% mean success rate, surpassing the strongest baseline by 23.7%. Furthermore, real-world experiments on diverse tasks also confirm consistent improvements over all baselines.

Abstract:
Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes while adaptively planning, executing, monitoring, and recovering from failures. We address these demands with a closed agent loop in which a VLM orchestrates heterogeneous robot capabilities as interruptible tools. Unlike in virtual AI agents, the timing of decisions, actions and tool calls is important in a physical world that does not pause for reasoning. We refer to this setting as Physical Orchestration, and propose VoLoAgent, a VLM that plans, monitors, and recovers by treating a VLA/WAM as an interruptible tool it steers mid-rollout, complemented by vision models and action primitives. To evaluate these long-horizon capabilities, we introduce RoboVoLo, a high-fidelity benchmark for open-vocabulary long-horizon manipulation across common sense, memory/state tracking, complex references, and world knowledge, with both task-level success and failure-mode diagnostics. Experiments show VoLoAgent substantially outperforms single VLA/VLM or tool-based systems, with validation on real-robot experiments.

Abstract:
Long-horizon, contact-rich complex manipulation tasks, such as seating a GPU into a PCIe slot, demand both millimeter high precision and out-of-the-box generalization to new tasks. Existing paradigms struggle to satisfy both: classical pipelines use brittle, task-specific interfaces to achieve high-precision control but require costly pipeline redesigns to adapt to new tasks, whereas monolithic end-to-end policies provide better generalization but lack high precision on complex, out-of-distribution tasks unless retrained with new data. Both paradigms share an implicit assumption: once a manipulation capability is acquired, it must be deployed as a rigid pipeline or monolithic whole, rather than being freely decomposed and recomposed. In this paper, we show that complex manipulation capabilities can emerge naturally from the composition of simple, independent behaviors. Rather than deploying a monolithic policy or a rigid pipeline, we propose \ourshort, a framework orchestrating foundation models and diverse sensing modalities into multiple composable core behaviors: a semantic behavior extracting spatial constraints via foundation models; a predictive behavior forecasting trajectories by tracking keypoints in imagined videos; and a reactive behavior enabling compliant corrections via high-frequency tactile reflexes. A single composition operator continuously fuses these outputs into final control actions. We demonstrate \ourshort on 8 real-world tasks spanning everyday manipulation and precision assembly, with the strongest gains in contact-rich assembly and object transfer, and show robust recovery from manual perturbations during execution.

Abstract:
Real-robot demonstrations are prohibitively expensive, while simulation data and real-world human demonstrations are both scalable but each leaves a distinct gap—simulation suffers from a sim-to-real visual gap, and human data suffers from a human-to-robot embodiment gap. In this work, we identify a natural yet underexplored complementarity between these sources: simulation contributes robot-valid actions absent in human data, while human data provides real-world observations that simulation struggles to render. Building on this insight, we present SimHum, a co-training recipe that extracts kinematic priors from simulation and visual priors from human observations, then fine-tunes on a small real-robot dataset. SimHum exhibits strong data-efficient and scene-generalizable capabilities: in a controlled data-collection study, SimHum outperforms the best single-source pre-training baseline by up to 40%; and across four bimanual tabletop tasks, with only 80 real-robot episodes, it achieves 62.5% success on held-out OOD scenes—a 7.1× improvement over the Real only baseline.

Abstract:
Contrastive Reinforcement Learning (CRL) has seen recent success in a wide variety of goal-conditioned robotics tasks by learning structured representations of the dynamics. However, despite its success in locomotion and simpler control domains, CRL often struggles in interaction-rich manipulation. We argue that a key source of this difficulty is object-centric interaction, such as contact or grasping, that induces distinct changes in the underlying dynamic modes. In this work, we formulate manipulation dynamics as a piecewise-smooth Markov process and show that interaction-induced mode changes create piecewise nonlinear reachability structures that are difficult for standard CRL energy functions to represent and plan over. Based on this analysis, we introduce Interaction-weighted Resampling (IWR). IWR performs interaction-aware resampling around phases before, during, and after interactions, encouraging the learned representation to preserve the mode boundaries that determine future reachability to capture multi-modal and piecewise nonlinear reachability. Across interaction-centric environments, including 2D dynamic control, robotic manipulation, and robot air hockey, IWR improves both sample efficiency and overall performance over prior CRL methods, with 19.8% average improvement in simulation. Finally, using a sim-to-real pipeline with policies trained by IWR, we demonstrate the first real-world goal-conditioned robot air hockey agent capable of hitting goals, improving success from 25% to 60%.

Abstract:
Robotic rearrangement tasks require planning sequences of actions that fulfill high-level specifications while also being geometrically feasible. Symbolic abstractions make the combinatorial structure of rearrangement tasks tractable, but they usually do not fully capture the geometric constraints that determine which actions a motion policy can realize. This is due to the complexity of expressing the underlying geometric feasibility constraints in the symbolic model. We propose REAP (Rearrangement with Elicited Action Preconditions), a framework in which a relational graph neural network heuristic is learned over a simple symbolic abstraction. A motion policy serves as a supervisor that elicits the hidden geometric preconditions of the problem during training. Deployed with greedy best-first search, the learned heuristic generalizes out-of-distribution along object count, plan length, and object arrangement. Compared to classical heuristic search and TAMP baselines on the same instances, REAP requires approximately an order of magnitude fewer motion policy queries because the heuristic anticipates which low-level actions are geometrically feasible. We show that the same architecture, training recipe, and search procedure adapt to various robotic rearrangement domains.

Abstract:
Evaluation practice in social robot navigation rests largely on geometric metrics that treat pedestrians as independent obstacles, assuming social compliance is well-approximated by spatial separation from individuals. As robot navigation transitions to complex urban scenarios, the limits of this assumption become apparent: many pedestrians move within social formations rather than as independent individuals, so a trajectory that keeps safe distance from every individual may still bisect a queue or cut through a gathering's interactional core. To make this gap measurable, we present TopoUrbanBench, an evaluation framework for urban robot navigation. First, we define social structure as an intention-driven graph at generation time and measure trajectory intersection with this graph through three group-structural compliance metrics, one per archetype. Second, built on CARLA, we procedurally generate pedestrian formations anchored to urban semantic features under three archetypes---stationary gatherings, goal-anchored queues, and frame-mobile cohesive walking groups. Third, we apply this framework to evaluate vision foundation models and online RL agents, diagnosing where geometric proficiency and structural compliance diverge. Evaluations on a real-world social navigation dataset highlight the persistent challenges of group-structural compliance, exposing the limitations of current models in physical environments. The full generation pipeline is released to enable extension beyond our predefined scenarios.

Abstract:
Real-world environments are messy. Paths are often blocked by everyday objects like chairs and toys, requiring interactive navigation (the ability to alter the environment to clear a path) rather than standard navigation alone. Existing methods rely on extensive simulation-based learning or per-scenario instructions specifying which objects to avoid or manipulate. They struggle to generalize across novel objects, diverse environments, and varied contexts such as a cart of fragile glassware that should be avoided while an empty one can be pushed. We formalize this problem as open-world interactive navigation and present MessyNav, a zero-shot framework based on vision-language models (VLMs). At its core, MessyNav performs explicit VLM-based affordance reasoning before task planning to decide whether to push, pick up, or avoid an object and produce a high-level plan. Deployed on a mobile manipulator across multiple real-world indoor and outdoor environments, MessyNav achieves substantially higher success rates than existing baselines without any task-specific training.

Abstract:
Object navigation requires a robot to search for an unobserved target in an unknown environment by deciding where to explore next under partial observability. Effective search resembles human-like exploration: selectively probing visually promising frontiers while relying on spatial memory to avoid redundant revisits. We propose IntentNav, a spatial-visual imitation framework that learns human-like ObjectNav policies from human demonstrations. To infer high-level search intent from low-level human actions, we introduce Frontier-based Human-Intent Labeling, which looks ahead in human demonstrations and labels the frontier that best explains the demonstrator's future search direction. We construct a spatial-visual candidate space, where BEV memory tracks explored regions, unexplored frontiers, and trajectory history, while egocentric visual memory provides semantic cues for each candidate. A VLM policy is trained to select among these grounded candidates, using Intent-Aligned Objective to encourage consistent and human-like exploration. IntentNav achieves state-of-the-art performance on the MP3D, HM3D-v1 and HM3D-v2 ObjectNav benchmarks. The proposed candidate-level navigation interface transfers zero-shot to wheeled, quadruped, and humanoid robots without further VLM fine-tuning. \hrefhttps://anonymous.4open.science/w/IntentNav/Project page.

Abstract:
Safe formation navigation requires a robot team to reach task goals while avoiding collisions and maintaining a desired formation geometry. In this multi-robot cooperative task, task progress, collision avoidance, and formation maintenance often impose competing requirements on the team. Rather than treating these requirements as equally weighted objectives, we view them as prioritized constraints, where safety-critical constraints should take precedence over recoverable coordination errors such as temporary formation deformation. We formulate this setting as prioritized multi-constraint coordination and introduce a CTDE-based priority-conditioned actor--critic method. During execution, each agent uses a decentralized actor conditioned on local observations and two online budgets that describe the current safety and formation regimes. During training, a multi-head centralized critic shares a common representation and predicts navigation return, safety violation, and formation degradation through three value heads. These estimates are used to update the online budgets and compute a hierarchical advantage for policy optimization. This design enables the team to avoid collisions, temporarily deform the formation when safety requires it, recover the desired structure after local conflicts, and continue goal-directed navigation. We evaluate the method in VMAS-based formation-navigation scenarios with static and dynamic obstacles, and further demonstrate the learned policy on a fleet of omnidirectional AGVs.

Abstract:
Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. This task is inherently non-trivial: viewpoint changes are driven by latent perceptual intentions, and a valid motion may operate at different semantic granularity, from entering a room to looking around a corner, inspecting a visible object, or revealing an occluded detail. To model this structure, we mine multi-intention camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across experiments and downstream robotic tasks, we show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.

Abstract:
Registering two captures of the same indoor space taken at different times underpins persistent spatial memory for robots and AR systems, yet the realistic version of this task is egocentric and its most scalable form is RGB-only. Head-mounted cameras yield blurry, fast-moving, partially overlapping views from which dense geometry is hard to recover. Classical registration leans on exactly the clean point clouds this setting lacks, while learned scene-graph methods require a pre-built or annotated graph and a trained matcher that we find brittle under egocentric data. We take a different route, using a pretrained vision-language model as the source of both scene understanding and cross-scan matching. Our method, PROSE (Prompted Scene rEgistration), lifts each RGB sequence into an object-level 3D scene graph using off-the-shelf foundation models for geometry, segmentation, and language, then prompts the same VLM to match object instances across the two RGB sequences. To make this matching tractable and reliable, we leverage object heights as a prior and verify each proposed match with a paired same/different query, then solve for the rigid transform by hypothesizing a candidate per matched object and selecting the one with the strongest geometric consensus. PROSE adds no learned parameters and requires no depth sensor, training, or annotated graph. On the egocentric Aria Digital Twin and Aria Everyday Activities benchmarks, it outperforms both geometric and learned scene-graph baselines in registration accuracy, on ground-truth and RGB-reconstructed point clouds alike, and the scene graph it produces transfers directly to downstream tasks.

Abstract:
Absolute pose regression (APR) provides an attractive route to real-time camera relocalization through a single feed-forward pass. However, the image-to-pose formulation hides the perspective and scene geometries that make localization well constrained, limiting its accuracy. To address this mismatch, we reformulate APR as a geometry-aware regression pipeline. The query image is first lifted into a scene pointmap, providing the downstream regressor with a geometric representation rather than appearance-only features. We then introduce a Gram-based pose proxy inspired by classical absolute pose estimation, where the network predicts a structured Gram matrix that jointly encodes perspective constraints and scene geometry. During training, we further propose geometry-guided pointmap augmentation, which constructs additional samples at novel viewpoints directly in the pointmap space. Compared with novel-view-synthesis-enhanced APR, it provides more geometry-focused samples without costly rendering or training an additional neural scene representation. Indoor and outdoor experiments show that our method substantially improves accuracy while retaining efficient feed-forward inference, achieving a favorable accuracy-efficiency trade-off.

Abstract:
Floorplans provide compact and widely available geometric maps for indoor localization, but existing high-performing floorplan-based methods still convert them into dense scene-specific offline databases, tying accuracy, storage, and runtime to the sampling resolution of the discretized pose space. We present FreeLoc, an online RGB-based floorplan localization framework that treats the floorplan as a directly queryable geometric map. FreeLoc introduces an efficient online geometric querying and diffusion-aided refinement scheme, which retrieves plausible pose anchors through on-the-fly floorplan ray querying and refines them into accurate continuous pose estimates. For sequential localization, FreeLoc develops an online likelihood construction strategy that bridges single-frame localization and probabilistic temporal fusion by constructing likelihoods from coarse-sampled candidates and refined pose hypotheses, enabling histogram-filter-based temporal fusion without offline databases. Experiments demonstrate real-time online inference and state-of-the-art performance in both single-frame and sequential localization, while real-world results validate practical deployability in indoor robotic localization scenarios.

Abstract:
Recovering the relative 6-DoF pose between two image groups underlies cross-sequence relocalization and multi-camera rig odometry. Each group carries known intra-group geometry from visual odometry or rig calibration, and pretrained multi-view backbones already fuse such geometry into visual features. Yet current models treat all views as an unstructured set, leaving cross-group reasoning as the missing piece. We introduce \ours, which keeps the foundation model entirely frozen and adds three lightweight trainable modules to bridge the two groups: a perceiver resampler, a cross-group bridge with merged self-attention, and a multi-frame pose head. The trainable footprint totals about 32M parameters, under 6% of the full model, and is supervised only by relative poses. Across four datasets that span indoor and outdoor simulation, real-world cross-season capture, and zero-shot sim-to-real transfer, \ours attains state-of-the-art accuracy on both tasks, while every baseline is retrained with its full original supervision. Code is available at \urlhttps://anonymous.4open.science/r/G2G/.

Abstract:
Existing SLAM systems lack modeling of the functional relations required for fine-grained robotic interaction. Functional 3D scene graphs can represent relations between objects and interaction elements, but existing methods rely on offline reconstruction, making them inadequate for real-time interaction in real-world exploration. To address this limitation, we propose Functional-SLAM, the first framework that continuously and recursively maintains a functional scene graph as an online SLAM state. The framework combines anchor-keyframe geometry with functional-context constraints for persistent node maintenance, accumulates multi-frame evidence through temporal relations to commit stable functional edges, and supplements visual loop-closure candidates with functional topology in scenes with repetitive appearance or degraded texture. Experiments show that Functional-SLAM efficiently constructs stable functional maps online, substantially improving runtime over offline methods while maintaining highly competitive accuracy. Compared with peer SLAM systems, it further improves pose estimation accuracy through functional-topology-assisted loop closure.

Abstract:
Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.

Abstract:
LLM-based robot planners are fluent but cannot guarantee that their plans are executable or safe. Classical PDDL planners can guarantee these properties, but only after the problem is fully specified, and they make poor use of an LLM's ability to read context and repair plans. This paper presents a neuro-symbolic framework with three parts. All LLM calls use a locally-hosted open-weight model, so the pipeline can be deployed on-robot with no cloud dependency. First, an \emphoffline procedure that mines a single global Signal Temporal Logic (STL) constraint on mobility from demonstration data. The procedure recovers codified rules (e.g., stopping at red lights, mined from nuPlan driving logs) or population preferences (e.g., social-navigation comfort, mined from SCAND teleoperation), depending on what the demonstrations encode. Because the demonstrations are a one-class signal, we generate the missing negatives with counterfactual perturbations and an LLM violation generator and then fit the constraint by evolutionary search. We use the mined constraint to shield a vision-language driving policy on Bench2Drive and two discrete-action navigation policies on HA-VLN-CE. Second, an \emphevolutionary PDDL planner: an LLM proposes and repairs plans, programmatic validators decide which ones survive, and the validated portion of the plan grows over iterations. We test the planner on the open-world ALFWorld Text benchmark, where it beats strong baselines and stays robust when the goal vocabulary does not match the action-model vocabulary. Third, a \emphconstrained execution loop: the planner's plan is compiled into waypoints, the waypoints are checked against the mined constraint, and the planner re-plans on a violation. We illustrate the full pipeline in Gazebo.

Abstract:
Physical control tasks in the natural world, such as driving or object manipulation, frequently exhibit dramatic variations in sensory and compute complexity over time. Correspondingly, a natural resource-efficient choice for robot control is to dynamically switch between control modes with varying resource allocations. However, such “mode-switching controllers” (MSCs) have historically required laborious, expert-driven design and synthesis for each new task. Driven by these design difficulties, modern robotic control architectures often fall back to a wasteful “monolithic” one-size-fits-all structure, where resource allocation is permanently anchored to the hardest, most resource-intensive task phases. To facilitate the design of performant yet efficient MSCs, we propose LLM-Guided synthesis of Efficient Mode-Switching Control Architectures (LEMCA). LEMCA represents MSC designs as interpretable programs to be iteratively refined in an evolutionary loop. To evaluate design fitness, we propose MSC-compatible extensions of automated controller synthesis approaches, such as reinforcement learning in simulation. LEMCA then leverages the semantic priors, reasoning, and coding capabilities of Large Language Models (LLMs) to iteratively edit controller modes, their corresponding sensory-compute resource allocations, and mode transitions. Our experiments across diverse control benchmarks show that LEMCA consistently discovers strategies that surpass the Pareto frontier of monolithic designs by reclaiming wasted resources during “easy” task phases. LEMCA thus presents an automated, low-effort path to synthesize resource-efficient MSC designs.

Abstract:
Unlike chatbots, physical AI must act while the world keeps evolving. Therefore, the inter-chunk pause of synchronous executors are fatal for dynamic tasks regardless of how fast the inference is. Asynchronous execution---thinking while acting---is therefore a structural requirement, and real-time chunking (RTC) makes it viable by recasting chunk transitions as inpainting: freezing committed actions and consistently generating the remainder. However, RTC with flow-matching policy is structurally suboptimal: its inpainting comes from inference-time corrections rather than the base policy, yielding little pre-training benefit, specific fine-tuning, heuristic guidance, and extra computation that inflates the latency. In this work, we observe that discrete diffusion policies, which generate actions by iteratively unmasking, are natural asynchronous executors that resolve all limitations at once: they are fine-tuning free since inpainting is their native operation, while early stopping further provides adaptive guidance and reduces inference cost. We propose DiscreteRTC, which replaces external corrections with native unmasking, and show on dynamic simulated benchmarks and real-world dynamic manipulation tasks that it achieves higher success rates than continuous RTC and other baselines. In summary, DiscreteRTC is simpler to implement with 0 lines of additional code to enable async inpainting, faster at inference with only ~0.7x computation compared with generating actions from scratch, and better at execution with 65% higher success rate in real-world hockey defend task compared with flow-matching RTC, and 30% higher compared with training-time flow-matching RTC. Code and visualizations are provided in the supplements.

Abstract:
Asynchronous inference has emerged as a prevalent paradigm in robotic manipulation, achieving significant progress in ensuring trajectory smoothness and efficiency. However, a systemic challenge remains unresolved, as inherent latency causes generated actions to inevitably lag behind the real-time environment. This issue is particularly exacerbated in dynamic scenarios, where such temporal misalignment severely compromises the policy's ability to interpret and react to rapidly evolving surroundings. In this paper, we propose a novel framework that leverages predicted object flow to synthesize future observations, incorporating a flow-based contrastive learning objective to align the visual feature representations of predicted observations with ground-truth future states. Empowered by this anticipated visual context, our asynchronous policy gains the capacity for proactive planning and motion, enabling it to explicitly compensate for latency and robustly execute manipulation tasks involving actively moving objects. Experimental results demonstrate that our approach significantly enhances responsiveness and success rates in complex dynamic manipulation tasks.

Abstract:
Vision-language-action (VLA) models inherit a shared synchronous clock from vision-language pretraining, processing every input at one rate. This is misaligned with physical interaction, where a high-frequency modality changes at hundreds of hertz, vision evolves more slowly, and language stays constant across an episode. A synchronous VLA oversamples slow modalities, undersamples fast ones, and caps action generation at the lowest effective frequency. We hypothesize that decoupling temporal processing per modality, letting each update and retain information at its own sensor rate, yields stronger representations and more robust control. We present \glsdavla, which maintains per-modality latent buffers refreshed at sensor rates and read continuously by the action head, integrating new high-frequency modalities through gated cross-attention that leaves the pretrained backbone intact. Across seven contact-rich real-world manipulation tasks, \glsdavla more than doubles the average success rate of the strongest synchronous baseline (95.2% vs.\ 40.95%) while sustaining smooth, reactive 100\,Hz control.

Abstract:
Deploying robot foundation models at scale is the next step towards realizing the potential of general-purpose robots. However, Vision-Language-Action (VLA) and other foundation models are computationally demanding, and on-device compute is constrained by power and space. In this paper, we introduce the problem of serving a robot policy to multiple robots from a remote GPU and formulate it as a scheduling problem. We build Armory, a serving system validated on fleets of both simulated and real robots. Our experiments show that naive scheduling heuristics perform well when all robots are the same, but fall short when robots consume action chunks at different rates, uncovering a mismatch between conventional batching methods and the closed-loop requirements of robot policy execution. To address this, we propose a scheduling algorithm that accounts for this heterogeneity and improves overall system throughput by up to 18% in real-world experiments. Additional details are available at https://actionchunkscheduling.github.io/.

Abstract:
Why do pretrained diffusion or flow-matching policies fail when the same task is performed under ambiguous instructions or slightly complex scenarios? Such failures rarely reflect missing motor skills; instead, they exposea limitation of imitation learning under train–test shifts, where action generation is tightly coupled to training-specific semantic and spatial context. To address this, we propose Vision–Language Steering (VLS), a training-free framework for inference-time adaptation of frozen generative robot policies. VLS treats adaptation as an inference-time control problem, steering the sampling process in response to out-of-distribution observation–language inputs without modifying policy parameters. By leveraging vision–language models to synthesize programmatic, trajectory-differentiable reward functions over grounded 3D keypoints, VLS guides denoising toward action trajectories satisfying test-time spatial and task requirements. In simulation and real-world evaluations, VLS outperforms prior steering methods, with up to 9.6×gain over the base policy on CALVIN, +13 pts over the same frozen π0.5 on LIBERO-PRO, and +4.3 pts on average over the strongest VLM-verifier baseline on LIBERO-10-R. Real-world deployment further demonstrates robust adaptation under spatial and semantic shifts.

Abstract:
We present Token Steering (TS), a method for dynamically steering trajectories generated by an autoregressive vision-language-action (VLA) model through direct intervention in the action-token space. TS injects low-dimensional user inputs into the model's native action-token representation, allowing users to influence trajectory generation without modifying the underlying vision-language model (VLM) architecture. Because TS operates entirely at inference time, it requires no additional training or finetuning. User inputs guide rather than override the pretrained policy, allowing users to influence robot actions while preserving the dexterity, smoothness, and task priors learned by the VLA. We evaluate TS on two household manipulation tasks---drawer closing after object placement and state-aware object swapping---and improve success rates from 10.0% to 72.5% and from 14.3% to 93.8%, respectively. By enabling lightweight, intuitive steering over robot foundation models, our interface has the potential to improve human-robot interaction in consumer environments and broaden accessibility for individuals with limited physical control.

Abstract:
Generative policies have achieved strong performance in imitation learning tasks, but their success is often constrained by the limited availability of expert demonstrations. Although preference-based fine-tuning can leverage suboptimal rollouts to improve performance, existing approaches heavily rely on expensive human annotations. In this work, we introduce Progress-Guided Preference Generation (PGPG), an unsupervised preference alignment framework that eliminates the need for human-in-the-loop supervision. Our method leverages ordered coverage alignment to measure sequential progress relative to a small set of expert demonstrations, enabling the automatic extraction of pseudo-preference labels from autonomously collected rollouts. Furthermore, we introduce key enhancements to improve the reliability of these progress-guided preferences, enabling effective fine-tuning of pretrained generative policies without requiring manually designed reward functions. Extensive experiments across a suite of challenging long-horizon manipulation tasks demonstrate that PGPG outperforms standard supervised fine-tuning baselines. Moreover, our conceptually simple framework is broadly applicable across multiple classes of generative policies, demonstrating strong versatility when integrated with both diffusion and flow-matching policies.

Abstract:
Vision-language-action models (VLAs) are promising general-purpose robot policies, but adapting them to new tasks typically requires costly task-specific teleoperation data. As an alternative, we study one-shot demo-conditioned VLAs, where a robot policy is conditioned on a single demonstration video of an unseen task. We find that existing end-to-end approaches often struggle when successful execution requires precisely localizing small target regions. To address this limitation, we propose SeeTraceAct, a demo-conditioned VLA framework that encourages precise spatial grounding through visibility-aware prediction of future end-effector traces. To enable reproducible evaluation with cross-embodiment demonstrations, we introduce and release RoboCasa-DC, a demo-conditioned extension of RoboCasa with episode-paired humanoid videos. Experiments on RoboCasa-DC and a real-world benchmark, where a Franka Panda arm is conditioned on human demonstrations, show that SeeTraceAct outperforms baselines, achieving the best success rate across all four RoboCasa-DC settings and improving real-world average success by 12.5 percentage points.

Abstract:
Trajectory demonstrations show robots how to behave but not why, leaving them to guess which features of the task actually matter. Incorrect guesses produce misaligned internal representations and lead to incorrect robot behaviors. Because task features are often semantically defined, we argue that natural language is better suited for specifying features rather than relying on demonstrations. Although existing representation editing methods have proposed using language to specify new features, only allowing additive edits produces feature sets that are often over-specified or too narrowly scoped, especially in sequential editing tasks. We propose the Flexible Feature Editing Pipeline (FLEX), which uses language models to iteratively refine a robot's internal representation of task features from natural language corrections via three complementary operations: Add missing features, Generalize existing specific features, and Mask irrelevant features. We evaluate FLEX on 2D navigation scenarios, 7-DoF Franka tabletop manipulation tasks, and a real-world study using naturalistic human language corrections and real-robot demonstrations. Comparisons against single-operation baselines show that all three operations are necessary in interactive scenarios, and that \ours achieves consistently higher normalized reward compared to other feature editing approaches while maintaining compact, transferable representations.

Abstract:
Generalist policies trained on diverse datasets can learn a wide range of skills. In order to solve or improve on challenging novel tasks, we need a way to identify and invoke the appropriate actions from the policy's rich behavioral prior, especially when directly commanding the policy fails to do so. We focus on flow matching generalists and propose Flow Reversal Steering (FRS): a method that takes suboptimal but "reasonable" actions, finds their latent noises, and maps them to nearby generalist behavioral modes. Across extensive simulated and real-world manipulation experiments, we show FRS in several settings that require converting suboptimal actions into better ones in the generalist's distribution. First, FRS can convert coarse semantic behavioral guidance from humans or vision-language models (VLMs) into corresponding effective robot actions to improve zero-shot control. These performance gains can be distilled back into the policy via latent noise behavioral cloning, showing up to 95% absolute task success rate boost in under a minute of training. Finally, FRS also enables policy improvement via bootstrapping reinforcement learning with semantic prior knowledge, improving on several tasks that standard reinforcement learning fails to improve on.

Abstract:
Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes. Prior data-driven approaches, particularly imitation learning, have shown some promise in narrowly defined settings but typically require thousands of demonstrations for specific tasks and cable types, limiting scalability and generalization. We introduce a sim-to-real reinforcement learning (RL) framework for multi-stage cable routing that leverages GPU-parallelized simulation to approximate linear deformable behaviors. Training across thousands of parallel simulations enables the learned policies to generalize across diverse cable geometries and deformation patterns. To bridge the sim-to-real gap, we propose a novel deployment strategy that combines a Simulation In the Loop (SILO) execution framework, localized RL policies, and robust cable state estimation. On real-world cable routing tasks, our approach achieves higher success rates and ～2× reduction in cycle times compared to prior state-of-the-art learning methods. To our knowledge, this is the first successful sim-to-real transfer of RL policies for multi-stage cable routing. Videos and additional visualizations are available at https://silo-cable-routing.github.io/

Abstract:
Deformable object manipulation (DOM) is challenging due to high-dimensional, partially observable states that evolve through long-horizon, topology-changing interactions with multiple valid manipulation modes. We introduce Instant-Fold, an in-context imitation learning framework for DOM. Given a single human demonstration, our policy infers and executes diverse manipulation modes directly from the demonstration—including variations in spatial execution and ordering—without requiring gradient updates. Our approach first learns deformation-aware visual representations via temporal contrastive pretraining, after which a flow-matching transformer policy conditioned on the demonstration predicts actions to execute the intended manipulation mode. Trained entirely in simulation, Instant-Fold generalizes across diverse folding modes and transfers zero-shot to real-world settings without additional data collection or finetuning. Videos are available at https://instant-fold.github.io.

Abstract:
Human video datasets have emerged as a compelling way to alternative to expensive real-robot data, leveraging visual diversity present in human videos. To bridge the inherent human-to-robot embodiment gap, existing methods either constrain video sources to narrow, task-aligned environments or focus solely on visual representation transfer while leaving the underlying action trajectories unutilized. In this work, we focus on robotized video as a means to further capitalize on human data, examining its effectiveness as a scalable source for policy pretraining. To this end, we propose a robotization pipeline capable of handling heterogeneous human videos, effectively accommodating different levels of initial annotation by inferring missing intermediate signals. Building on this pipeline, we construct the HuRo Dataset, a large scale collection containing about 630K robotized episodes and 142M processed frames converted from a wide variety of human video sources. Across four real-world downstream manipulation tasks, we show performance scales with the amount of robotized pretraining data. Policies pretrained on our dataset surpass strong baseline foundation models that were pretrained solely on large scale, purely robotic data. These results suggest that robotized human videos can serve as a practical and scalable source of supervision for robot policy learning.

Abstract:
Robot manipulation requires policies that are both accurate and efficient, as robot control must respond to changing observations under tight latency constraints. Recent diffusion and flow policies are promising, but they often treat conditions as auxiliary signals rather than jointly evolving them with action trajectories. We find that this limitation can be effectively mitigated by a simple yet effective unified condition-action modeling design that represents conditions and actions in a shared token space, allowing a compact model to achieve high performance while improving both inference speed and accuracy. Therefore, we propose UCA-Flow, a unified condition-action modeling framework for accurate one-step action generation. Our method unifies observation conditions, timestep conditions, interval conditions, and action tokens into a single sequence, and processes them with a Unified Condition-Action Transformer for joint condition-action representation learning. As a result, condition representations are dynamically reconstructed according to the current generation stage, highlighting information most relevant for action refinement. Furthermore, we introduce an improved dual-pass supervision scheme over u and v for stronger optimization of unified condition-action modeling. UCA-Flow improves the average success rate by 9.3 percentage points over the strongest baseline, while achieving 45.6× and 33.4× speedups over DP3 and Simple DP3, and remaining 4.3× and 2.3× faster than one-step FlowPolicy and MP1, respectively.

Abstract:
Diffusion-based visuomotor policies operating directly in raw action spaces conflate scene comprehension with trajectory generation within a single denoising process. The resulting velocity field must simultaneously encode scene information and generate precise trajectories, increasing learning complexity and limiting performance on tasks demanding precise temporal coordination across multiple arms. To simplify this joint learning problem, we introduce Latent Diffusion Policy (LDP), a two-stage framework performing flow matching in a deliberately shaped latent space. By absorbing scene understanding into an observation-conditioned CVAE encoder, LDP concentrates the conditional distribution of each observation. Consequently, the flow model avoids implicitly resolving scene-dependent structures; instead, it generates within a pre-concentrated distribution featuring a smoother velocity field, simplifying learning from limited demonstrations. Furthermore, to capture temporal dependencies among latent tokens, LDP trains with per-token diffusion forcing and employs staircase inference sampling to resolve the resulting distributional mismatch. We also propose reconstruction FID (rFID) as a lightweight proxy predicting downstream task success solely from latent space statistics. On coordination-intensive tasks from RoboTwin 2.0, LDP outperforms DP3 by a substantial margin and transfers effectively to real-world bimanual deployments.

Abstract:
Visuomotor manipulation policies face a persistent trade-off: image-only policies inherit the semantic priors of large-scale 2D pretraining but are brittle to lighting, texture, and viewpoint shift, whereas 3D-only policies are robust to such factors yet are often trained from scratch. Some recent multi-modal policies combine both streams but typically process 3D through a from-scratch encoder, preserving this asymmetry. We present UniFlow, a unified 2D+3D co-training recipe for manipulation in which 3D observations are represented as image-shaped pointmaps so that pretrained 2D backbones can be used to encode both modalities, and RGB and pointmap streams are fused by a unified diffusion transformer with per-modality modulation. On average across thirteen dexterous manipulation tasks from the challenging LBM simulation benchmark, UniFlow outperforms a strong RGB-only baseline in a distribution-shifted setting. UniFlow also outperforms the generalist VLA \pi_0.5 when averaged over five RoboTwin bimanual tasks as well as four real-world YAM tasks.

Abstract:
Diffusion models have become a powerful tool for generative modeling in robotics, with diffusion policies excelling at modeling multimodal action-trajectory distributions. However, when demonstrations are limited, standard sampling often reproduces dominant behaviors while neglecting valid but rare modes, limiting the discovery of novel solutions. Existing approaches, such as guidance methods or combining reinforcement learning (RL) with diffusion, either push samples into infeasible regions or struggle to escape local minima, failing to systematically uncover diverse behaviors. To address these challenges, we propose a framework that combines Feynman–Kac correctors with a novel guiding potential that systematically guides diffusion policy samples towards promising yet underrepresented samples. These trajectories are refined using sampling-based trajectory optimization and reincorporated into the training set to retrain the diffusion policy. Our method effectively mines and repairs novel trajectories, enabling the systematic discovery of diverse and executable behaviors. We demonstrate the effectiveness of our framework across a range of manipulation environments, consistently discovering new behaviors.

Abstract:
Imitation learning has achieved impressive results in robotic manipulation, yet most existing approaches assume clean backgrounds and lack explicit mechanisms for obstacle-aware motion generation. Extending such policies to cluttered, real-world scenes with unstructured obstacles remains a key generalization challenge. We present ObstaDiff, a decomposed diffusion-policy framework with a lightweight obstacle-aware visual encoder. ObstaDiff extracts a structured target--obstacle--background representation, enabling the downstream alignment policy to generate end-effector trajectories toward a target-centered bottleneck pose while reasoning about surrounding obstacles. We evaluate ObstaDiff on 61 real-robot greenhouse trials spanning target-pose, obstacle-layout, and target-appearance generalization. ObstaDiff achieves 75.41% average task success and 8.20% average obstacle collision rate, outperforming representative imitation-learning baselines and improving generalization in cluttered agricultural scenes.

Abstract:
Reliable real-time failure detection is critical for deploying learned robotic policies in real-world environments. However, explicitly enumerating failures is intractable in open-world settings, making failure detection from nominal demonstrations alone a necessary alternative. Existing approaches face key limitations: purely visual methods are highly sensitive to benign background variations, while kinematic monitors lack awareness of the surrounding environment. Crucially, both often fail to capture relational errors involving object identity, spatial arrangement, or task progression, such as grasping the wrong object or violating interaction structure. In this work, we introduce Robot-WATCHDOG, a self-supervised failure detection framework based on an object-centric representation of dynamic scenes as spatio-temporal graphs over tracked labeled objects and their interactions. Within this representation, the framework deploys two complementary methods to adapt across data regimes. For data-rich settings, we propose GnnT, a predictive graph transformer that identifies failures as deviations from learned object-centric spatio-temporal dynamics. To address data scarcity, we introduce Trajectory Projection and Correlation (TPC), a non-parametric method that detects failures through geometric consistency and changes in object-relation dynamics in extreme few-shot scenarios. We evaluate the framework on a public benchmark and a newly collected dataset featuring relationally challenging manipulation tasks. Results show that modeling object-centric spatial and temporal interactions improves failure detection over existing baselines and enables reliable detection of failure modes that are difficult to capture using visual or kinematic monitoring alone.

Abstract:
Although the advancement of vision-language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensive evaluation methodology is important to advance the integration of VLMs in robotic navigation and manipulation. However, current benchmarks lack comprehensive methods to evaluate diverse robotic tasks, and evaluation metrics remain relatively constrained, making it difficult to assess the embodied capabilities of VLMs in a thorough and fine-grained manner. To address the issues, we propose RMMBench, an evaluation benchmark that requires robots to understand language instructions and perform long-horizon tasks in continuous spaces. RMMBench seamlessly integrates high- and low-level embodied tasks into a unified framework, constructing a ``navigation-manipulation'' task suite comprising 70 core use cases that range from localized manipulation to long-horizon composite navigation. The results reveal that the existing leading VLMs still face major challenges in spatial localization accuracy when coping with mobile manipulation tasks. The findings highlight the necessity of enhancing robots' spatial perception capabilities during long-period interactions to bridge this gap, and emphasize the potential for further research in this field.

Abstract:
Due to the high cost of hardware experiments, robot policies are predominantly evaluated by comparing their empirical performance on a limited number of hardware trials. However, finite-sample statistical uncertainty can lead to erroneous conclusions, especially when ranking multiple policies using a fixed batch of evaluation data. We propose \algname: a \emphsequential method for policy ranking that (i) \emphadaptively allocates trials to hard-to-separate policies, (ii) provides (partial) rankings at \emphany stopping time while bounding the probability of an incorrect ranking, and (iii) optionally \emphexploits prior information to prioritize hardware evaluations more efficiently using a small set of held-out hardware evaluations or rollouts in a world model. We achieve these properties through novel theoretical contributions that combine sequential and multiple-hypothesis testing. Experiments in simulation and real-world policy evaluation benchmarks show that \algname requires up to 50% fewer hardware evaluations compared to existing statistically rigorous approaches, while guarding against the incorrect rankings produced by heuristic methods used by state-of-the-art benchmarks.

Abstract:
Generalizing dexterous grasping across diverse robot hands remains a fundamental challenge due to differences in geometry, topology, and kinematics. Inspired by recent advances in scaling up robot data, we show that scaling both the diversity and quantity of hand embodiments significantly improves generalization to unseen morphologies. We present XDex, a unified cross-embodiment grasp synthesis framework trained on the largest and most diverse multi-hand grasp dataset to date, spanning 1,000 heterogeneous robotic hands and 7k object scenes. XDex couples an embodiment encoder that jointly captures geometric and topological features with a three-way disentangled representation of object geometry, hand embodiment, and grasping pose. It outperforms prior methods in grasp quality, consistency, and diversity, and generalizes to unseen hands in real-world settings.

Abstract:
Dexterous in-hand manipulation is commonly formulated around explicit object pose estimation, yet reliable pose tracking is challenging under severe hand-object occlusions and often requires object-specific supervision. This dependence creates a bottleneck between perception and control that can limit robustness and scalability. We present an end-to-end visuomotor framework for dexterous in-hand reorientation that operates directly from RGB observations and proprioception without intermediate pose estimation. Our approach jointly learns visual representations and control policies through behavior distillation from privileged-state teachers and auxiliary reconstruction objectives. Furthermore, we introduce image-goal-conditioned reorientation, allowing desired object configurations to be specified directly through images rather than canonical pose representations. We evaluate our approach across a diverse set of object geometries and instances in simulation, where it consistently outperforms pose-based methods and policies built on pretrained visual encoders. Finally, we demonstrate direct sim-to-real transfer of a single image-goal-conditioned policy across five real-world objects, showing that accurate dexterous reorientation can be achieved without explicit pose estimation. We will release the code and training setup upon acceptance to facilitate reproducibility.

Abstract:
Multi-finger dexterous manipulation relies on stable hand-object interactions, yet these interactions are partially observable in practice. Visual observations are often occluded by the hand, tactile sensors introduce hardware-specific modalities and calibration burdens, and existing policies rarely model how these cues evolve under actions, making them brittle under contact uncertainty. To address these, we present ProxiDex, a dynamics-guided proximity policy framework that treats hand-object proximity as an interaction state for dexterous manipulation. ProxiDex reconstructs interaction point clouds and converts geometric distances into proximity cues, forming a hardware-agnostic contact representation that provides immersive feedback during VR teleoperation. Built on this representation, ProxiDex learns action-conditioned proximity dynamics with a coupled forward–inverse design: future observation latents are predicted from actions, while proximity variations are decoded from latent changes. Leveraging these dynamics, ProxiDex adaptively reweights proximity tokens across manipulation phases and uses dynamics-consistency supervision to guide policy inference, stabilizing action generation under unreliable visual feedback. Simulation and real-world experiments demonstrate improved success rates and robustness over representative baselines across standard, unseen objects, and perturbation scenarios.

Abstract:
Tactile sensors provide critical information for contact-rich manipulation, yet tactile representations and policies remain tightly coupled to each specific sensor, limiting transferability across robots and hardware platforms. We propose TACTX, a framework for learning a transferable tactile representation across sensors spanning three fundamentally different transduction modalities: resistive, magnetic, and vision-based. TACTX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data. Such paired interactions provide a natural alignment signal across modalities, and the encoders are jointly trained across all sensor pairs, inducing a consistent latent space for all sensor types. Our experiments show that TACTX aligns tactile representations across sensors while preserving object-level contact information, as evidenced by sensor-identity prediction and object classification in the learned latent space. We evaluate TACTX on four contact-rich manipulation tasks—pick-and-place, plug insertion, board wiping, and object reorientation—and show that policies trained with one sensor transfer zero-shot to physically distinct sensors through the shared latent. This improves the average success rate from 27.5% for vision-only transfer to 45.9%, providing a step toward sensor-agnostic tactile manipulation.

Abstract:
Touch sensing is essential for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose a novel architecture using modality-specific convolutional stems and transformer-based fusion that effectively fuses information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multi-sensor representation then conditions a flow-matching policy for downstream task solving. Experimental results across five contact-rich manipulation tasks demonstrate the effectiveness of multi-resolution tactile features in imitation learning. MiTaS achieves an average success rate of 80 %, while vision-only (31 %) and visual-tactile (54 %) baselines cannot solve the task reliably. Co-training a visuo-tactile model with multi-tactile data boosts performance by over 10 % in certain tasks, without having access to the Evetac sensor during policy evaluation. A detailed sensor-reading and attention analysis reveals the importance of different sensors throughout task execution, validating our multi-resolution tactile sensing approach.

Abstract:
Integrating Vision-Language-Action Model (VLA) with force awareness is not merely a matter of encoding wrench measurements as an additional modality, but of determining the optimal timing and strength when and how physical interaction should influence contact-conditioned robot action generation. Existing force-aware VLAs typically treat force-to-action conditioning as a constant channel, leaving its decision relevance to be inferred implicitly during action prediction. We argue that force-informed action generation requires an intermediate force-relevance mechanism inside the VLA, where visual-language context estimates the task-dependent relevance of physical feedback before force is injected into the action pathway. FR-VLA addresses this gap with a VLM-based implicit force reasoning module that derives a vision-language-grounded force-admission signal from learnable force-query tokens and uses it to modulate an external tokenized force pathway for action generation. Real-world experiments on two robotic platforms and five contact-rich tasks show a 20.8% average success rate improvement over the strongest baseline, with visualizations confirming phase-dependent force admission dynamics.

Abstract:
Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands.

Abstract:
Humanoid soccer is a challenging testbed for dynamic whole-body control, requiring robots to coordinate balance, locomotion, object interaction, and skill switching over long horizons. Existing humanoid sports methods often rely on task-specific multi-stage pipelines, making it difficult to jointly learn and compose multiple object-interactive skills within a single deployable policy. To address this, we present SkillX, a unified reinforcement learning framework that learns and composes multiple atomic soccer skills through a single command-conditioned policy. SkillX integrates three core designs: skill-specific adversarial motion priors, skill-specific critics, and an object-aware temporal encoder, enabling the robot to execute atomic skills and transition among them such as dribbling, trapping, and shooting. Experiments in simulation and on a real Noetix E1 humanoid demonstrate robust multi-skill execution, long-horizon skill composition, and successful sim-to-real deployment.

Abstract:
Imitation learning is a promising approach for training humanoid robots to both walk and manipulate, but it requires a large number of demonstrations, which are time-intensive and difficult to collect via teleoperation. Existing data-generation algorithms can automatically synthesize demonstrations for manipulators, but they are ineffective on humanoids because their high-dimensional composite action spaces involve arms, legs, and torsos. We present HumanoidMimicGen, an algorithm for generating humanoid legged loco-manipulation data. Our algorithm adapts contact-rich whole-body skills from a handful of source demonstrations to new states, generalizing across changes in object pose. By interleaving these single- and dual-arm skills with whole-body locomotion and manipulation planning, the method generates stable, collision-free data across diverse scenes and layouts. To evaluate our approach, we introduce a new simulated loco-manipulation benchmark containing nine diverse tasks that test humanoid loco-manipulation capabilities. There, we demonstrate that HumanoidMimicGen automatically generates large datasets for imitation learning and enables a systematic study of how data generation and policy learning decisions impact model performance. We show that whole-body visuomotor policies co-trained with HumanoidMimicGen data outperforms those trained only on real-world data by 20%.

Abstract:
Humanoid loco-manipulation requires long-horizon coordination between locomotion and object interaction, as well as robust adaptation to diverse objects and noisy real-world observations. Recent approaches have shown promising results by learning loco-manipulation policies from reference motions or task-specific rewards, but they often struggle to generalize across objects. We propose \textscEvolve, a framework for learning egocentric vision-based humanoid loco-manipulation through simulation-based demonstration editing and hierarchical visuomotor policy training. Given a single successful demonstration, our method adapts it to diverse objects by applying structured transformations to robot keypoint trajectories. These edited trajectories are executed by a task-agnostic low-level whole-body controller that tracks future target poses of sparse body keypoints. We then train a high-level visuomotor policy from the successful edited rollouts to enable real-world deployment. In simulation, our method generalizes across substantial object variations in size, shape, pose, and initialization, outperforming representative reference-based and reference-free baselines. We further deploy the learned policy zero-shot on real-world tasks. These results suggest that combining demonstration editing with hierarchical visuomotor control is a scalable strategy for transferring humanoid loco-manipulation skills from a single demonstration to diverse real-world objects.

Abstract:
Whole-body humanoid manipulation of bulky, deformable, and shared-load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly. Demonstration sources provide complementary modalities with inherent trade-offs: human demonstrations capture natural contact forces but not robot-executable actions, while teleoperation directly records robot actions but with less natural force regulation. This paper presents WT-UMI, a wearable whole-body tactile interface worn by human operators or mounted on humanoids, providing accurate observations of tactile images, contact forces, and end-effector poses across both human demonstration and humanoid teleoperation modes. We introduce a force-conditioned target-pose correction module that converts measured human poses into contact-aware robot targets by learning corrections from a small amount of teleoperation data. To leverage the natural force interaction in human data, we propose a force-supervised planner that predicts end-effector pose chunks and contact-force trajectories. The predicted contact force serves as the reference for a tactile-based admittance controller. Across five contact-rich tasks spanning deformable objects, bulky rigid objects, and human--humanoid collaboration, WT-UMI improves success rate and reduces contact-position tracking error over four policy baselines. Our project page is available at https://wt-umi.github.io/WTUMI/.

Abstract:
Building a low-latency humanoid teleoperation system is essential for collecting diverse reactive and dynamic demonstrations. However, end-to-end latencies in existing approaches consistently hover around 200ms, roughly equal to human reaction time, that severely limits responsiveness and prevents tasks requiring rapid feedback. We identify position-only PD control as the root cause of this latency floor and propose ExtremControl, a low-latency whole-body control framework that: (1) incorporates scaled velocity feedforward in low-level control, reducing tracking delay by approximately 70ms with a provably optimal feedforward ratio; (2) operates directly on SE(3) poses of selected rigid links, primarily humanoid extremities, to avoid joint-space retargeting in the control loop. To minimize retargeting time, we derive a closed-form Cartesian-space mapping that uses a single anthropomorphic calibration step to translate human motion into humanoid link targets, accommodating operators with varying body proportions. Furthermore, we provide a theoretical framework for selecting control hyperparameters, and validate the effectiveness of velocity feedforward control in the reinforcement learning for locomotion setting through experiments in both simulation and the real world. Building on ExtremControl, we implement a teleoperation system supporting both optical motion capture and VR-based tracking, achieving end-to-end latency as low as 50ms --- well beneath the threshold at which system delay adds a full human reaction cycle to the control loop. This enables a qualitatively new class of reactive behaviors, including ping-pong ball balancing, juggling, and real-time return, that are beyond the capability of existing teleoperation systems.

Abstract:
Legged manipulators combine the mobility of legged platforms with the manipulation capability of robotic arms. However, arm-induced Center-of-Mass shifts and dynamic disturbances make the system more prone to instability under actuator failures, potentially leading to falls, task failures, or safety risks. Existing fault-tolerant control methods mainly focus on locomotion alone, leaving the coupled problem of whole-body stability and arm reachability in fault-tolerant loco-manipulation largely unaddressed. To bridge this gap, we propose FT-WBC, a fault-tolerant loco-manipulation framework for robust whole-body control of legged manipulators under actuator failures. FT-WBC adopts a decoupled upper- and lower-body policy architecture and introduces two key modules: a Fault Estimator (FE) and a Posture Adaptation Module (PAM). The FE predicts faulty joints from lower-body proprioceptive histories, while the PAM uses this fault information to adapt the base posture plan generated by the arm policy, converting potentially unstable posture requests into safe and executable base posture commands. Through this fault-aware posture adaptation mechanism, FT-WBC synthesizes compensatory gaits under actuator failures and preserves as much arm workspace as possible while maintaining whole-body stability. Simulation and real-world experiments show that FT-WBC significantly improves survival rate and workspace under weakening or locked failures, and transfers zero-shot to a real legged manipulator in the real world.

Abstract:
Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing. Prior geometric control approaches extend differential flatness through Hopf fibration-based attitude representations to support bidirectional thrust, but struggle with actuator saturation and motor reversal delay during inversions, requiring heuristic thrust posture scheduling and waypoint tuning. We propose a learning-based framework that modulates a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes. Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions. In JAX-based simulation, the proposed method achieves the lowest position deviation and settling time across all evaluated baselines, reducing position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline. Hardware experiments demonstrate successful inversion across multiple yaw configurations with position RMSE below 0.35m, and compatibility with downstream trajectory generation and control through circular flight in both regimes. Additionally, we provide an open-source implementation of the proposed framework.

Abstract:
Threading a quadrotor through a moving narrow gap is a demanding test of agile flight, requiring centimeter-level spatial alignment and precise temporal execution. We address this challenge in a purely state-free setting, where a control policy operates directly on onboard visual and inertial measurements to output collective thrust and body-rate commands. The task is difficult because success lies on a thin feasible manifold: small errors in timing, position, attitude, or visual observability cause collision. Prior planning and learning methods rely on privileged state, teachers, planners, or distillation; learning dynamic narrow-gap traversal directly, from scratch, remains open. We introduce a differentiable simulation framework that trains visual-inertial, observation-to-command policies entirely from random initialization. By backpropagating through differentiable quadrotor dynamics and smooth task costs, our approach constitutes the first method to learn dynamic narrow-gap traversal without relying on a teacher, planner, or distillation. We validate our approach in both synthetic and real-world experiments. In controlled benchmarks, we demonstrate that pathwise gradients significantly outperform score-function gradients for precision-critical tasks, yielding a 99.3% success rate in simulation after only a few minutes of training on a single GPU. Furthermore, the policy transfers directly to a real quadrotor, successfully crossing a dynamic narrow gap from varied initial conditions using exclusively onboard sensing, achieving an 80% success rate over 10 physical trials.

Abstract:
UAV target following in cluttered environments is inherently predictive: current-state followers can lag behind turns, choose blocked corridors, or trade tracking for unsafe near-horizon motion. We propose a future-aware flow planning framework for state-informed UAV target following. Predicted target futures guide clean UAV trajectory generation as horizon-aligned residual signals, while risk-scored executable-prefix repair is embedded inside the sampling loop. On fixed ID/OOD receding-horizon benchmarks, the planner improves the intended safety--tracking trade-off rather than dominating every metric: it matches zero measured ID collision rate with the highest ID safe-tracking time, and gives the lowest OOD macro collision rate and final tracking error among the displayed methods, while Future-MPC remains smoother and stronger on some thresholded OOD success metrics under its hand-designed objective. Ablations show that future adaptation improves candidate generation before safety repair, and simulator-facing stress tests probe interface, sensing, and controller-execution effects. These results support horizon-aligned future adaptation and embedded prefix repair as complementary ingredients for safe UAV target following under the tested simulation conditions.

Abstract:
World models provide predictive priors that are essential for vision-language-action (VLA) policies in long-horizon and dynamic manipulation. However, existing world-model-based VLA methods are often optimized with reconstruction-oriented objectives, which may prioritize low-level visual fidelity over task-relevant state evolution for control. We propose PLaW-VLA, a predictive latent world model for VLA policy learning. Rather than reconstructing future observations in pixel space, PLaW-VLA predicts future world states in a pretrained latent space, providing compact, semantically grounded, and dynamics-aware context for action generation. The model adopts a Mixture-of-Transformers (MoT) architecture with shared causal self-attention, unifying environment understanding, latent world-state prediction, and action generation within a single sequence-modeling framework. Experiments in simulation and on real robots show that PLaW-VLA achieves state-of-the-art performance, especially on long-horizon, interaction-rich, and dynamically perturbed tasks, demonstrating the effectiveness of latent world-state prediction as a control-aligned objective for generalist robot policy learning.

Abstract:
Collecting contact-rich robot experiences at scale remains a major bottleneck for generalizable manipulation. Beyond data quantity, robot learning also requires diverse experiences across embodiments, viewpoints, and scenes. Human egocentric videos provide abundant physical interactions, but each video captures only a narrow slice of experience under a single body, camera trajectory, and environment. We propose AnyWorld, a cross-embodiment world modeling framework that expands a single human interaction into diverse robot-native rollouts without paired human-robot demonstrations. Our model factorizes an interaction into action, camera, and embodiment: action controls capture the motion structure, camera controls specify viewpoint evolution, and the target embodiment context defines the acting body and its interaction geometry. This formulation enables independent recomposition of embodiment, viewpoint, and scene factors, allowing a single model to generate many robot-domain experiences while preserving the underlying dynamics and object interactions. We train the model with large-scale human interaction pretraining followed by mixed-embodiment fine-tuning. Experiments show that our model supports controllable recomposition across embodiments, viewpoints, and scenes, and we further demonstrate that the generated data can improve manipulation performance on the RoboCasa GR1 tabletop benchmark and a real IRON humanoid robot.

Abstract:
Vision-Language-Action (VLA) models have shown substantial progress in enabling robots to act with instructions and visual observations, yet most existing policies implicitly assume that the initial observation is completely actionable. This assumption often breaks down in realistic scenes, especially wrist-view settings, where the target object may be partially visible or fully occluded. In such cases, manipulation failures may arise not from insufficient manipulation skill, but from the lack of a task-ready observation. To address this problem, we propose Seek-before-Act, a plug-in active approach framework that enables a robot to acquire right observations before downstream manipulation. Given only a language instruction and an initial wrist RGB image, Seek-before-Act uses task-conditioned visual foresight to imagine target-revealing view sequence, lifts them into an planning-oriented 3D representation, and plans a collision-aware approach motion toward a viewpoint that improves target visibility, reachability, and downstream manipulability. A semantic verification module further checks whether the reached view is ready for manipulation and triggers closed-loop replanning when necessary. Experiments in simulation and real-world setups show that Seek-before-Act substantially improves downstream manipulation success under occlusion, clutter, limited viewpoints, visual noise, and moderate scene changes, while remaining compatible with existing VLA policies and model-based planners.

Abstract:
The generalization of vision-language-action (VLA) models heavily relies on diverse training data. However, acquiring large-scale robot manipulation data across varied object appearances is costly and labor-intensive. To address this, we introduce Embodied Manipulation Media Adaptation (EMMA), a framework for augmenting VLA policies that combines a generative data engine with an effective training pipeline. We introduce DreamTransfer, a diffusion Transformer-based architecture for generating multi-view consistent and geometrically plausible embodied manipulation videos. It supports text-driven video editing on foreground, background and lighting while preserving 3D structure and geometric fidelity. We also utilize a hybrid training set of real and generated data and propose AdaMix to enhance the training process. AdaMix is a training strategy that adaptively weights samples according to policy performance to emphasize challenging samples. Evaluations show DreamTransfer outperforms previous video generation models in multi-view consistency, geometric fidelity, and text-conditioning precision. We conduct over 1800 trials across simulation and real robot platforms. In real-world robotic tasks with zero-shot visual settings, our framework achieves a relative performance increase of over 92% compared to training with real data alone, and improves by an additional 17% with AdaMix.

Abstract:
Action-conditioned video generative models promise to serve as general-purpose world models for robotics, leveraging internet-scale data to simulate complex physical interactions. While these models excel at visual realism, they often struggle with controllability--reliably simulating diverse future outcomes from the same state under different action sequences. In this work, we identify a structural cause for this failure: world models trained on successful, goal-directed trajectories tend to ignore actions entirely, collapsing into success-biased video players that hallucinate desirable outcomes regardless of the input action. We characterize this lack of "persistence of excitation" as a collapse in empowerment. To remedy this, we propose an Action Influence Maximization (AIM) objective and Batch Online Reinforcement Learning algorithm that encourages controllability by biasing the training distribution toward state-action pairs that maximize mutual information gain. We evaluate our method extensively for training world models on robotic interaction data on simulated manipulation and navigation tasks. Across our experiments, we find that AIM significantly improves the controllability (and hence, fidelity) of world models in comparison to naively training on expert data. Most notably, world models trained with AIM can outperform the controllability of state-of-the-art world models trained with 85x more data for challenging robotic manipulation tasks.

Abstract:
Learning robot manipulation policies from human video demonstrations constitutes a promising avenue for scalable robot learning. However, comparing different human-to-robot (H2R) transfer methods remains challenging, as existing approaches are evaluated under different settings, including differing task suites, scene layouts, object instances, and amounts of robot supervision. To address this challenge, we present H2RBench, a Real2Sim benchmark for evaluating H2R transfer methods. H2RBench provides a standardized protocol built on real human video demonstrations and simulated robot demonstrations, and includes four manipulation tasks spanning diverse interaction requirements. We evaluate multiple representative H2R transfer methods that leverage task-specific human demonstrations and limited robot data, each adopting a different strategy for bridging the embodiment gap. Using H2RBench, we systematically characterize how each method scales with the amount of human demonstrations, revealing that methods differ substantially in their ability to leverage additional human data. We further show that simulation performance is broadly predictive of real-world robot performance, with an overall Pearson correlation of r=0.89, Spearman correlation of \rho=0.85 and Mean Maximum Rank Violation (MMRV) of 0.06 across method--task configurations. These results establish H2RBench as a practical and scalable benchmark for comparative H2R evaluation prior to real-world deployment.

Abstract:
Scaling robot learning is bottlenecked not only by data volume, but also by whether the data targets the policy's failures. Existing robot-free handheld interfaces enable portable in-the-wild data collection, yet remain open-loop: users record demonstrations without feedback from the current policy. Interactive methods such as DAgger provide such feedback by collecting corrective on-policy data, but require deploying imperfect policies on physical robots, making them costly, unsafe, and difficult to scale. We present RoboPocket, a smartphone-based system for robot-free policy iteration. By combining on-device sensing, display, and communication with cloud-side policy inference, RoboPocket provides real-time collection guidance and visualizes the policy's predicted trajectory in augmented reality as AR Visual Foresight. This allows users to identify policy weaknesses and collect targeted corrective data without deploying a physical robot. An asynchronous finetuning pipeline continuously incorporates new corrections and updates the policy within minutes, realizing a DAgger-inspired robot-free correction loop. Across diverse real-world manipulation tasks, we validate RoboPocket as a scalable data engine and show that online corrective data collection achieves an average 2× sample-efficiency gain over blindly scaling offline demonstrations.

Abstract:
We present the stochastic decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end within a few hours on a single NVIDIA RTX 4080 GPU. SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments and substantially reducing compute and memory overhead. On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards. Finally, to support future research, we introduce a suite of realistic visual robotics benchmarks spanning dexterous manipulation, challenging locomotion. Furthermore, we demonstrate effective sim-to-real transfer on physical hardware

Abstract:
Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks, but their imitation-learning-based policies remain brittle in precise physical interactions due to compounding execution errors; Can a reinforcement learning policy trained purely in simulation improve the robustness of real-world VLAs zero-shot? Residual RL, which learns a corrective policy on top of a frozen VLA, offers a natural framework, but existing approaches face a fundamental sim-to-real dilemma: privileged-state methods require lossy distillation for deployment; image-based methods suffer from the visual domain gap; and real-world RL is costly and unsafe. We propose an object-centric residual RL framework that refines VLA actions using object poses, enabling a compact observation space that transfers consistently between simulation and reality. To align the two domains, we additionally replay the same teleoperation demonstrations in simulation to train a sim counterpart of the real-world VLA. The residual RL policy is trained only in simulation with pose noise injection and dropout, and transfers zero-shot to the real. Across five manipulation tasks on a real Franka robot, our method improves success from 42% to 76% zero-shot, and the improved rollouts can be further reused to retrain the base VLA for self-improvement without additional teleoperation.

Abstract:
In-context imitation learning (ICIL) enables robots to learn new tasks from a small number of demonstrations by conditioning a pre-trained policy on task-specific examples, without retraining at test time. Despite this promise, training generalizable and scalable in-context imitation policies remains an open challenge. We present SynthICL, a scalable framework that trains ICIL policies entirely from RGB-only synthetic data. Specifically, we build a data generation pipeline to produce high-fidelity ICIL data and train a flow-matching transformer policy on the resulting dataset. SynthICL avoids the need for depth sensing, precise camera calibration, and real-world training data in prior approaches, offering a simpler and more scalable alternative. We further incorporate subgoal prediction by training the model to predict the next subgoal images, enabling more precise and visually grounded control. Evaluated on 16 unseen real-world manipulation tasks, SynthICL achieves an average success rate of 79% with only one demonstration provided at test time and outperforms prior methods. Project page: https://synth-icl.github.io

Abstract:
Generating articulated 3D objects, from everyday tools like pliers and eyeglasses to robotic arms and quadrupeds, requires modeling both geometric diversity and kinematic constraints, a challenge that existing point cloud generators ignore and prior articulated methods address by assuming predefined kinematic skeletons. We propose ArticFlow, a flow matching framework that learns a controllable velocity field from noise to target point sets under explicit action control. ArticFlow couples (i) a latent flow that transports noise to a shape-prior code and (ii) a point flow that transports points conditioned on the action and the shape prior. On PartNet-Mobility and MuJoCo Menagerie, a single model synthesizes novel shapes via latent interpolation and produces kinematically valid deformations under unseen actions, with consistently lower reconstruction error than per-instance baselines. We further verify that generated point clouds yield functional URDFs in a robot simulator, closing the loop from skeleton-free generation to simulation-ready assets.

Abstract:
Imitation learning policies in robotics are trained on datasets collected through costly real-world teleoperation. Often, the data-collection process is unguided, relying on the teleoperator's intuition, resulting in data that contributes little to overall policy performance. Recent work has shown that curating such unguided datasets can substantially improve policy performance, but these approaches operate by filtering existing datasets after collection and therefore do not prevent the acquisition of uninformative data. In this work, we propose a novel Guided Data Collection (GDC) method that actively reasons about which data should be collected to maximize downstream policy performance. We introduce an iterative framework that alternates between 1) task proposal and data collection and 2) data evaluation to close the loop. For task proposal, we leverage semantic priors from vision–language models to identify relevant environments, object configurations, and goals for data collection. However, since semantic relevance does not imply policy improvement, we perform data evaluation. By leveraging predictive data attribution we approximate how the collected trajectories influence policy performance on a target deployment setting. This signal is then used as feedback to guide subsequent data collection, closing the data collection loop. We evaluate our approach in simulation and real robot settings, demonstrating that guided data collection substantially improves sample efficiency and deployment performance compared to passive data collection strategies, multiple baselines and experienced roboticists.

Abstract:
Occlusion poses a significant challenge to robotic grasping in cluttered scenes, especially when the target object is completely invisible. Under such full-occlusion conditions, the key difficulty shifts from grasp pose estimation alone to actively discovering the hidden target and ensuring a feasible retreat path. Existing methods mainly focus on partially visible targets, leaving target discovery and safe retreat under full occlusion insufficiently explored. To address this gap, we propose MIND-Grasp, a multi-view grasping framework inspired by human-like reasoning in occluded scenes. Given a language instruction and current observations, MIND-Grasp reconstructs an amodal 3D target model and the corresponding target-centric occlusion point cloud. Based on this occlusion cloud, MIND-Grasp performs NBA decision-making: it searches for observable corridors and updates the occlusion cloud in a closed loop, while executing grasp-remove actions only when critical occlusions must be removed. Once the target becomes visible and its geometry is reconstructed, the system further selects grasp poses that satisfy feasible retreat constraints. We validate the effectiveness of imagination-based point cloud reconstruction on the LINEMOD Occlusion and GraspNet-1Billion datasets, and extensive real-world occluded grasping experiments demonstrate that MIND-Grasp can handle different levels of occlusion, enabling robust target discovery, occlusion removal, and safe grasp execution.

Abstract:
Dexterous manipulation of deformable objects demands continuous fingertip-level regulation of pressure, friction, and incipient slip. We study one of the most challenging cases: dexterous cable tracing, feeding a cable through the hand with repeated pinch-and-curl motions of the thumb and index finger. We introduce Touch2Trace, a tactile-driven imitation-learning system for this task, and provide, to our knowledge, the first systematic real-world characterization of encoder pretraining, control rate, temporal context, and spatial resolution each shape policy performance. The winning learning recipe combines a tactile encoder pretrained for a custom 32×32 piezoresistive sensor (TacV5) via self-supervised learning with a lightweight transformer policy trained on teleoperated demonstrations via behavior cloning, deployed at 60 Hz on a Tesollo DG-5F hand. Tactile feedback without vision or explicit cable-state estimation significantly improves tracing performance versus a proprioception-only baseline: from 0.2 cm to 20.1 cm mean distance and 0% to 93% success rate, with zero-shot transfer to unseen cables and routing conditions. The results quantify the influence of key parameters in tactile-driven systems for reliable dexterous deformable object manipulation.

Abstract:
Robot manipulation often fails in the final millimeters: a policy may recognize the right object yet miss the pose offsets, boundaries, or pre-contact alignments needed for action. We argue that such failures arise when semantic invariance suppresses correspondence cues for closed-loop control, or when these cues are not exposed to the policy in a usable form. Modern visual encoders provide strong semantic abstractions, but contact-rich manipulation requires correspondence sensitivity: discriminative feature responses to action-relevant changes in pose, boundary, and contact geometry. Diffusion features provide a strong prior for dense correspondence, but direct use is impractical due to stochasticity, latency, and representation drift. We introduce Robot-DIFT, a deterministic diffusion-derived backbone for real-time control. Through Manifold Distillation, Robot-DIFT converts a noise-conditioned diffusion Teacher into a clean-input, single-pass Student while preserving the teacher's feature manifold. A Spatial-Semantic Feature Pyramid Network (S2-FPN) fuses coarse-to-fine Student decoder features into visual tokens that expose semantic context and fine contact detail to the policy. Across RoboCasa, LIBERO-10, and real robots, Robot-DIFT outperforms vision-language, self-supervised, geometry-oriented, and diffusion baselines on contact-sensitive tasks. Controlled backbone/readout swaps show that S2-FPN unlocks, rather than replaces, the diffusion correspondence prior.

Abstract:
Contact-rich manipulation requires robots to regulate both motion and interaction forces, yet achieving adaptive compliance remains a fundamental challenge. Learning from real-world data is costly and risky, while simulation-based approaches struggle with the sim-to-real gap in contact dynamics; existing sim-to-real methods either require real-world adaptation or sacrifice adaptive compliance by relying on isotropic compliant controllers. Our key insight is that force regulation decomposes into a time-varying but simulation-transferable directional component and a dynamics-sensitive but manually tunable magnitude component. We instantiate this directional component as two policy outputs, a task frame and per-axis control gains, predicted by a visuomotor policy adapted from a pre-trained VLA model and trained via imitation learning on automatically generated simulation demonstrations. At deployment, an admittance controller integrates these predictions with human-specified stiffness and target wrench values to realize adaptive compliance. Our approach achieves adaptive compliance using only simulation data and can benefit from large-scale VLA pre-training. Extensive real-world experiments on four contact-rich tasks, microwave opening, peg-in-hole insertion, whiteboard wiping, and door opening, demonstrate strong task success rates and robustness to external disturbances.

Abstract:
Enabling robots to adapt across related contact-rich assembly tasks remains challenging. Policies trained in simulation with diverse geometries and task parameters often degrade under real execution due to contact-model mismatch, residual alignment error, friction variation, and force-response differences. We introduce FACA, a task-residual factorized contact world model for parameterized axial assembly. FACA separates known nominal task descriptors from hidden execution-time contact residuals. The task descriptor defines the nominal assembly prior, including clearance, part scale, insertion depth, and task type, while recent force/contact history, proprioception, and action history update a latent contact belief over residual contact state. This belief conditions task-dependent latent dynamics, value estimation, and action selection, and future rollout proceeds from the belief state without assuming access to future measured force. We evaluate FACA on a compact but controlled axial assembly family rather than claiming full AutoMate benchmark coverage. The experiments are organized along two axes: generalization across known task parameters and adaptation to hidden residual/contact variations that are not provided as privileged inputs. When target-domain data are available, sparse autonomous rollouts or human correction snippets are used only as transition data to calibrate a lightweight residual dynamics adapter while keeping the base task-conditioned model and policy fixed. The experiments test whether FACA can better handle continuous task-parameter changes and difficult-to-encode contact residuals under the same low-level control interface.

Abstract:
Generalizing learned locomotion policies across quadrupedal robots with different morphologies remains a challenge. Policies trained on a single robot often fail when deployed on embodiments with different mass distribu- tions, kinematics, joint limits, or actuation constraints, forcing per-robot retrain- ing. Prior works have approached this primarily either by scaling via training across real or generated embodiments or using large architectures producing trans- ferable policies which are both storage and compute heavy. We argue that both of these approaches circumvent a key failure mode in actor-critic learning: a shared value function tends to average incompatible value targets across embodi- ments, yielding miscalibrated advantages and fixing that helps transfer and reduce the compute cost. We present MorFiC, a reinforcement learning approach for zero-shot cross-morphology locomotion which fixes this issue using multiplica- tive critic conditioning on a morphology latent. Trained with a single source robot with morphology randomization in simulation, MorFiC achieves zero-shot trans- fer to total seven robots and reaching forward velocity competitive or surpassing scaling-based and morphology-conditioned PPO baselines for examples 1.98 m/s on AlienGo, where as additive PPO baselines remain below 0.65 m/s. MorFiC achieves this using single training robot within 2 hours of training time compared to tens or hundreds of hours required for scaling baselines training on multiple embodiments. We diagnose critic calibration via three metric: Explained vari- ance, advantage sign-flip rate and policy gradient cosine, confirming our critic conditioning as determinant for policy update direction. Finally, we demonstrate zero-shot deployment on Unitree Go1 and Go2 robots without fine-tuning.

Abstract:
Learning-based quadrupedal locomotion typically relies on complex reward formulations that entangle task specification, operational limits, gait preference, and terrain adaptation within a single optimization objective. We instead treat these functions through distinct mechanisms: rewards for task specification, constraints for operational limits, energy minimization for gait preference, and exteroceptive perception for adapting energy use to terrain difficulty. We show that these components jointly enable efficient, terrain-adaptive locomotion, and that removing each component exposes a distinct failure mode. Our formulation removes explicit gait priors (including air-time, contact-count, and foot-clearance targets) in favor of emergent behavior. Compared to a conventional complex-reward baseline, our formulation achieves comparable terrain traversal while reducing cost of transport by 56% and operational-limit violations by 96%. The resulting policies transfer zero-shot to a physical Unitree Go2 using LiDAR-based elevation mapping. Project website with videos: https://tinyurl.com/locomposition.

Abstract:
Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.

Abstract:
Humanoid locomotion requires control policies that remain stable under imperfect sensing while capturing long-term temporal dependencies for consistent and adaptive movement. Conventional end-to-end policies based on multi-layer perceptrons lack explicit temporal memory and often struggle to recover from sensor dropouts, whereas Transformer-based policies introduce substantial computational overhead that limits their use in high-frequency real-time control. To address these challenges, we propose RoboDreamer, a framework built upon Mamba's selective state-space mechanism for efficient long-horizon temporal modeling. Through randomized temporal masking during training, RoboDreamer learns to recover missing observations and predict future states, which supports robust control under partial sensory corruption. Leveraging this capability, RoboDreamer further enables action chunking, generating multi-step actions within a single inference pass to improve control smoothness. Experiments show that RoboDreamer outperforms MLP and Transformer baselines in robustness and efficiency, maintaining stable locomotion under masked observations while supporting real-time deployment for high-frequency humanoid control.

Abstract:
Humanoids deployed in human-centered environments must handle force-interactive tasks, where external contacts introduce unexpected disturbances that disrupt locomotion accuracy and stability. Existing learning-based approaches rely on broad domain randomization, task-specific force objectives, or learning-based force estimators from motion history, each of which compromises accuracy, task transferability, or out-of-distribution (OOD) robustness. We present Analytical Disturbance-Aware Policy Training (ADAPT), a framework that equips humanoid policies with a physically grounded disturbance observer. The core of ADAPT is an analytical whole-body disturbance observer that estimates residual force/torque online with the accessible robot dynamics, without requiring force/torque sensors. Fed directly into the policy, the estimated disturbances give the humanoid an explicit, physics-derived sense of external force/torque that can generalize across diverse unseen scenes. Experiments on a Unitree G1 humanoid show that ADAPT achieves accurate disturbance prediction and stronger robustness than a proprioception-only baseline under torso perturbations, standing pushes, and asymmetric hand payloads, with improved velocity tracking even on OOD disturbances. Moreover, ADAPT enables penalizing inferred disturbances at lower-body joints to encourage lighter locomotion.

Abstract:
We introduce \emph\lambda-Reachability, a scalable approach to Hamilton--Jacobi safety analysis for high-dimensional robotic systems. Unlike prior discounted formulations that rely on fixed one-step Bellman updates, \lambda-Reachability employs a \emphstochastic multi-step estimator of the safety value, using a geometrically distributed rollout horizon together with a randomly absorbed terminal. Conceptually analogous to TD(\lambda), \lambda-Reachability interpolates between local self-consistency updates and long-horizon max-over-trajectory safety targets via an interpretable horizon-control parameter. Unlike TD(\lambda), where the terminal value is always incorporated in learning targets, the terminal safety value in \lambda-Reachability is only used at a probability controlled by parameter \delta. We formally show that for \delta<1, the update induces a contraction mapping that allows temporal-difference learning; as \lambda \to 1, the estimator recovers the undiscounted reachability objective. We apply \lambda-Reachability to high-dimensional safety learning problems with both simulated and real humanoid robots under balance and collision avoidance constraints. Experimental results demonstrate that \lambda-Reachability significantly improves both safe-set boundary classification and safety margin estimation compared to single-step temporal-difference baselines.

Abstract:
Multi-quadruped coordination has attracted increasing attention due to its enhanced payload capacity, broader contact coverage, and improved adaptability to challenging tasks. Existing methods for multi-quadruped manipulation typically focus on predefined or closed task families, often relying on multi-agent reinforcement learning (MARL) to train task-specific coordination policies. However, such methods struggle in open-ended continual learning settings, where tasks arrive sequentially and robots are expected to acquire new coordination skills while reusing previously learned ones without catastrophic forgetting. To address this challenge, we propose Conquer, a semantic skill-library framework that formulates continual multi-quadruped coordination as a retrieve-adapt-update process. First, to accommodate varying team sizes across tasks, we design a team-structured Self-Allies-Goal (SAG) backbone that supports variable-cardinality robot teams by explicitly modeling each robot’s own state, teammate context, and task goal. For each incoming task, Conquer constructs a task-level semantic descriptor from pre-execution information and retrieves a relevant skill from the library for adaptation. After successful execution, Conquer updates the skill library by extracting trajectory-level semantic descriptors and organizing them according to semantic distance, thereby enabling continual skill accumulation and cross-task knowledge transfer. Simulation experiments show that Conquer achieves a final average success rate of 95.6 %, demonstrating strong forward transfer and negligible catastrophic forgetting. Real-world rollouts on Unitree Go2 teams further validate the deployment feasibility of Conquer for practical multi-quadruped coordination.

Abstract:
Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene features a policy relies on to make decisions. Existing platforms, however, exhibit a sim-to-real visual gap that corrupts policy perception, undermining their ability to assess a policy’s closed-loop decision-making. To this end, we propose DreamStream, a generative, closed-loop simulator that achieves policy-oriented fidelity using a simulator-grounded autoregressive video model. Our video model is distilled from a large pretrained video model via traffic-scenario guidance, varying visual appearance while preserving policy-relevant features such as scenario layout and the temporal consistency of dynamic objects. We further observe that perceptual metrics like FID misrank how well these features are preserved. To tackle this, we introduce FD\pi^k , a new multi-representation metric that measures the sim-to-real gap as the Fréchet distance over scene-context features from public E2E policies. Under FD\pi^k, DreamStream improves over the strongest prior closed-loop simulator by 1.6× on nuScenes and 4.7× on NAVSIM, and induces the least perturbation to policy observability. Based on DreamStream, we construct Navhard-CL benchmark, which turns non-reactive real-world benchmark NAVSIM into interactive testing environments with adversarial driving behaviors and weather variations. This benchmark exposes many failure modes of driving policies that prior closed-loop benchmarks overlook. Codebase will be made available.

Abstract:
Autonomous driving requires safe and efficient planning in complex, open-world environments. Although recent Vision/Video-Action models learn action-relevant representations directly from visual observations, trajectory planning is fundamentally constrained by scene geometry, such as drivable space, obstacles, dynamic agents, and their spatial relationships. In this work, we propose GeoWorldAD, a unified video geometry world-action model that provides planning with explicit geometric priors over both present observations and future scene evolution. Built upon a streaming video geometry foundation model, GeoWorldAD extracts ego-aligned, multi-scale geometry tokens from current observations to provide fine-grained spatial constraints. A Q-Former-style geometry world model then learns latent future tokens to model short-horizon geometric evolution, supervised by future depth prediction and conditioned on ego states. Compared with pixel-level future prediction, geometry prediction offers a compact, action-relevant abstraction directly aligned with planning constraints. Finally, a geometry-oriented action model iteratively aggregates present and future geometry tokens to refine trajectory proposals, enabling safer and more efficient planning. Extensive experiments on NavSim v1 and v2 demonstrate that GeoWorldAD achieves state-of-the-art closed-loop performance with improved driving safety.

Abstract:
Vision-Language-Action (VLA) models promise to bring end-to-end reasoning to autonomous driving, but their computational cost is orders of magnitude too high for real-time control. The core challenge is that VLA inference is not a single bottleneck but a pipeline of four stages (visual encoding, language model prefilling, reasoning-token decoding, and flow-matching action prediction), each dominated by a different source of inefficiency: spatial redundancy in overlapping video frames, autoregressive serialization of reasoning chains, and over-computation in iterative denoising. Addressing any one stage in isolation leaves the others untouched. We propose FlashDrive, an algorithm-system co-design framework that targets all four stages simultaneously. A key insight is that each bottleneck admits a distinct, lightweight algorithmic shortcut: temporal overlap enables streaming KV-cache reuse across frames; the low entropy of domain-specific reasoning tokens makes them amenable to diffusion-based speculative decoding; and the non-uniform structure of the denoising velocity field, sharp at the endpoints but nearly constant in the middle, permits adaptive step caching that concentrates compute where it matters. Layered on top of system-level CUDA Graph compilation and kernel fusion, these techniques compound multiplicatively. Applied to Alpamayo1.5-10B with W4A8 quantization, \method reduces end-to-end latency from 716~ms to 159~ms (4.5x) with negligible accuracy loss (~0.07m on minADE6@6.4s and ADE@6.4s even improves), bringing VLA-driven autonomous driving to real-time viability on consumer-level GPUs.

Abstract:
Vision-language-action (VLA) models have become an important paradigm for robotic manipulation by unifying visual perception, language understanding, and action generation. Since reliable manipulation requires sufficient observation of objects, end-effectors, and their spatial relationships, multi-view inputs are widely used to provide complementary spatial information. However, our experiments show that existing VLA models do not fully exploit such multi-view complementarity. The policy tends to persistently rely on the wrist view, lacking the ability to dynamically adjust global-local view dependency across manipulation stages, which limits stage adaptability and robustness. To address this issue, we propose FocusVLA, a stage-aware multi-view routing framework. FocusVLA predicts future interaction intent from historical actions and uses it to guide dynamic view routing during action generation. By adaptively coordinating global and local observations across different task stages, FocusVLA improves multi-view utilization. On RoboTwin 2.0, FocusVLA improves the average success rate over the strongest baseline π0.5 by 10.08%. In real-world evaluation, FocusVLA improves the success rate from 71.7% to 85.0%, achieving a 13.3% gain over π0.5 . Moreover, the intent prediction and dynamic routing modules of FocusVLA can be integrated into existing VLA frameworks without modifying their core architectures.

Abstract:
Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing codes that preserve motion geometry but provide only weak semantic supervision to the backbone. We therefore formulate action tokenization not as mere compression, but as semantic interface learning between multimodal reasoning and executable control. To this end, we introduce X-Tokenizer, a lightweight encoder--Semantic Residual Quantization (SRQ)--decoder architecture that provides a shared action interface across diverse robotic arm embodiments. Its key component, SRQ, imposes an asymmetric structure on residual vector quantization: the first level is trained with Masked Action Modeling (MAM) to form a discrete action language that captures coarse motion intent, while deeper levels remain reconstruction-oriented residuals that preserve fine-grained details. To further align action tokens with multimodal semantics, X-Tokenizer is pretrained with contrastive alignment to the representation space of a pretrained foundation model and with next-frame vision-language feature prediction. Pretrained on 2.4M trajectories (2.0B action frames), a single frozen X-Tokenizer plugs into a mixed discrete-continuous VLA as a representation-shaping supervision signal. X-Tokenizer achieves top real-world aggregate and strong RoboTwin 2.0 simulation results. Outperforming FAST in multimodal grounding (+13.5%) and long-horizon tasks (+8.25), it shows that action tokenizers serve as semantic interfaces for VLA pretraining beyond mere action compression. [Project page](https://anonymous.4open.science/w/anonymizationsWEB-3D07/)

Abstract:
Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To address this, existing approaches adopt Chain-of-Thought (CoT) reasoning to enable subgoal decomposition and spatial anticipation. However, those methods lack a unified architecture for effective cross-modal reasoning and fail to explicitly include inverse reasoning ability based on the target state. We argue that manipulation planning naturally decomposes into prediction, anticipating the next visual state, and inverse dynamics, inferring the actions to reach it. Bridging both requires a unified autoregressive architecture that interleaves textual and visual reasoning in a single generation process. We propose ThinkingVLA, a generative model that realizes this decomposition within a unified Mixture-of-Transformers architecture. ThinkingVLA consists of a forward CoT that identifies the immediate subgoal and guides the visual forecasting; the predicted image then serves as the target state, grounding an inverse CoT that reasons about spatial relationships and action intent based on the predicted image; and the final action is generated conditioned on this full reasoning context. Extensive experiments on simulation and real-world benchmarks demonstrate that ThinkingVLA consistently outperforms state-of-the-art baselines, with particularly large gains on long-horizon manipulation tasks.

Abstract:
Robot imitation learning is limited by the scarcity of action-labeled data, while action-free demonstrations such as videos are abundant. Recent latent-action methods learn action-relevant representations from observation transitions, but they do not optimize latent actions for downstream control, leading to latent representations that are difficult for policies to model and difficult to accurately decode into continuous robot actions. This paper introduces ELA, an end-to-end framework for learning from video demonstrations via latent actions jointly optimized for policy learning and action decoding. ELA jointly trains a latent action model, latent action policy, and action decoder, enabling policies to leverage abundant video data with only a handful of action-labeled robot demonstrations. Experiments on simulated and real-world manipulation tasks show that ELA consistently outperforms prior latent action baselines. Moreover, it integrates seamlessly with existing imitation learning methods through lightweight 4M-parameter action decoders, requiring no changes to policy architectures or training objectives. Additional materials can be found on: https://end2end-latent-action.github.io/

Abstract:
Vision-language-action (VLA) models typically inject proprioception only as a late conditioning signal, preventing robot state from grounding instruction understanding or directing visual attention. We introduce ThinkProprio, which discretizes proprioception into VLM-vocabulary tokens and uses them jointly with the instruction to gate visual patches before VLM computation, steering the model toward action-relevant evidence while discarding redundant tokens early. We find that proprioception added as a passive conditioning signal leaves performance essentially unchanged; its value emerges when token-form state acts as an active query that, with the instruction, selects which visual patches the VLM processes. Systematic ablations show that VLM-vocabulary tokens outperform learned projectors as the state encoding, and that retaining only about 12% of the visual tokens surpasses on CALVIN ABC\toD. Across CALVIN, LIBERO, and real-world manipulation, ThinkProprio reduces end-to-end inference latency while improving the matched full-token baseline. Code: \urlhttps://anonymous.4open.science/r/wyrngg2.

Abstract:
Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot’s state within the scene, frequently underperforming even vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from kinematic velocity, providing look-ahead visual context. Across 67 tasks, GeoProp improves success rates by 9.4% in simulation and 11.2% in the real world, while adding only 3% to the parameter count. These results demonstrate that explicitly resolving state–vision misalignment is a simple yet high-impact inductive bias for generalist embodied policies.

Abstract:
Memory remains a critical bottleneck for long-horizon robotic manipulation, as standard Vision-Language-Action (VLA) policies often fail when task-relevant cues become occluded or unobservable over time. While existing memory-augmented methods utilize historical context, they either suffer from severe information bottlenecks, incur high latency via decoupled dual systems, or rely on unselective buffers that accumulate massive visual redundancies. To address these limitations, we introduce EventVLA, an end-to-end framework founded on the concept of sparse visual evidence memory that comprises two core components: foundational visual anchors to retain initial and short-term contexts, and a dynamic Keyframe Evidence Memory (KEM) module. Specifically, KEM directly predicts future keyframe probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events. This foresight-driven mechanism empowers the policy to dynamically evaluate the future causal utility of current observations, preserving transient visual evidence before it becomes unobservable. Furthermore, we propose RoboTwin-MeM, a diagnostic benchmark specifically designed to evaluate non-Markovian manipulation tasks with interactive visual evidence. Extensive evaluations show that across 17 memory-requiring simulation tasks and 4 real-world bimanual tasks, EventVLA achieves an average success rate improvement of +40% over state-of-the-art memory-augmented VLAs.

Abstract:
An always-on robot faces an endless stream that never resets: instructions arrive and lapse, the scene changes, and its own past actions reshape what it must reason about. Today's action models are built for the opposite: a fixed instruction, no mid-task intervention, single-step reasoning. In an open-ended world a robot must watch a live stream for far-future cues, recall its own far-past actions, and act on them under dual-arm concurrency. We present ARMS (Always-on Robot in Multi-modal Streams), a deliberately simple streaming policy: a single pretrained \pi_0.5 backbone augmented by three lightweight modules that turn live perception, embodied states, and the robot's own past actions into context the backbone reads before it acts. The modules update this context asynchronously, so watching and recalling never block acting and the two arms act at once. Rather than inventing new mechanisms, ARMS integrates these learned context providers with an agent-causal self-history that logs which arm did what, and when. To supervise them without extra annotation, we build ARMS Dataset, whose staged construction script itself labels every module from real dual-arm teleoperation. Trained on it, ARMS reaches 45% on the combined task against 28% for the strongest VLA and streaming baselines, and ablations confirm the memory module, the embodied-state head, and asynchronous concurrency are each necessary.

Abstract:
World Action Models (WAMs) extend robot policy learning by incorporating future prediction as an additional training objective, encouraging the policy to encode task-relevant temporal structure in its representations. Current WAMs often rely on large-scale generative architectures that incur high training costs and inference latency, making them difficult to deploy as efficient closed-loop policies. We propose Light-WAM, a lightweight World Action Model for efficient robot manipulation. Specifically, it is built with a compact video backbone and performs future-video supervision in a downsampled latent space, reducing the cost of video co-training while retaining its benefits for representation learning. For action prediction, Light-WAM introduces the StateFusionActionExpert, which reads adapted states from multiple backbone layers, fuses them through learned-query pooling, and directly predicts action chunks in a single forward pass. This design provides an efficient interface between video backbone representations and robot actions, avoiding the need for heavy generative action experts. Experiments demonstrate that Light-WAM maintains strong performance on LIBERO and achieves usable multi-task performance on RoboTwin 2.0, while using only 0.44B trainable parameters. It also achieves 72.03ms inference latency with 4.1GiB peak GPU memory and improved training throughput.

Abstract:
Human-robot collaboration (HRC) combines the complementary strengths of humans and robots to improve task efficiency. However, many existing collaborative systems rely on hand-engineered pipelines, limiting their scalability and flexibility for new tasks. In this work, we show that models trained end-to-end with imitation learning, specifically vision-language-action (VLA) models, can support collaborative manipulation, and characterize the key factors affecting their real-world performance. We evaluate two state-of-the-art models and identify a failure mode of action-chunking policies in implicit HRC, where demonstration action leakage (i.e., action chunks crossing latent task transitions) can cause premature assistive behavior. We find that this issue increases with longer execution horizons and occurs in real-world collaborative VLA systems, such as when a robot attempts to hand over a tool before the person is ready. We propose an inference-time steering method to mitigate these erroneous assistive actions while preserving policy performance. Finally, through a 16-participant user study on a long-horizon collaborative assembly task, we show that steering enables a longer execution horizon while mitigating premature assistance, leading to faster collaboration and fewer failures compared to a shorter-horizon policy.

Abstract:
Robots are increasingly expected to provide personalized services in everyday environments. To do so, they must ground natural-language commands such as “Where is my backpack?” or “Find my bottle” and execute them by reasoning about object instances, people, locations, and ownership. This is challenging because ownership is rarely labeled explicitly and must be inferred from long-term, behavioral evidence of human-object interactions. To address this, we present COOL, a novel robotic framework for autonomously learning object ownership from everyday observations and maintaining a long-term spatial memory of its environment. To keep its memory current, COOL uses an agent-based curiosity-driven data collection strategy that guides the robot toward the most promising locations to gain information and refreshes stale observations. Offline experiments, ablation studies, and real-world evaluations show that COOL can infer ownership relations from real-world interactions and use this knowledge for ownership-conditioned navigation and task execution.

Abstract:
Post-training with real-world experience is critical for reliable deployment of robot foundation models, but heterogeneous data from demonstrations, human corrections, and autonomous rollouts are difficult to convert into stable policy updates. We present PIVOT, an iterative real-world post-training system built on Trust-Region Flow Matching Optimization (TRFO). TRFO provides a unified reinforcement-learning objective for heterogeneous deployment experience, using signed advantages to attract high-value transitions and repel low-value ones within a trust region while preserving the flow-matching training form. At each iteration, we first collect on-robot data using the current policy, estimate transition-level values via distributional TD learning, and then apply TRFO to update the policy. The updated policy is subsequently redeployed on the robot to gather the next round of experience, forming an iterative post-training procedure. On a suite of precise, contact-rich bimanual manipulation tasks, PIVOT yields consistent, monotonic gains over successive iterations, substantially improving success rates and throughput across all evaluated tasks.

Abstract:
General-purpose robot learning requires dense, instruction-conditioned feedback that can distinguish meaningful task progress from stalled, failed, or partially completed behavior. Yet obtaining such feedback at scale remains difficult, since existing approaches often rely on manual progress annotations, task-specific demonstrations, or reward models trained on curated robot datasets. We introduce TOPReward, a training-free progress reward method that probes pretrained Video-Language Models (VLMs) through their internal token probabilities rather than asking them to generate numerical progress values. Given a video prefix and a language instruction, TOPReward measures the model's likelihood that the instructed task has been completed, converting latent video-language understanding into a dense reward signal without task-specific reward-model training or manually annotated progress labels. We evaluate TOPReward on ManiRewardBench, our real-world manipulation benchmark spanning over 130 distinct tasks and four robot platforms, as well as on Open X-Embodiment datasets. Across these settings, TOPReward substantially outperforms prior training-free VLM reward methods on open-source models and is competitive with trained reward-model baselines on progress-estimation metrics, while requiring no reward-model training. Additional analyses show that the reward is sensitive to the specified instruction and is not explained by prefix position alone. Finally, TOPReward supports downstream applications including success detection and offline reward-weighted behavior cloning.

Abstract:
The strong performance of large vision–language models (VLMs) trained with reinforcement learning (RL) has motivated similar approaches for fine-tuning vision–language–action (VLA) models in robotics. Many recent works fine-tune VLAs directly in the real world to avoid addressing the sim-to-real gap. While real-world RL circumvents sim-to-real issues, it inherently limits the generality of the resulting VLA, as scaling scene and object diversity in the physical world is prohibitively difficult. This leads to the paradoxical outcome of transforming a broadly pretrained model into an overfitted, scene-specific policy. Training in simulation can instead provide access to diverse scenes, but designing those scenes is also costly. In this work, we show that VLAs can be RL fine-tuned without sacrificing generality and with reduced labor by leveraging 3D world generative models. Using these models together with a language-driven scene designer, we generate hundreds of diverse interactive scenes containing unique objects and backgrounds, enabling scalable and highly parallel policy learning. Starting from a pretrained imitation baseline, our approach increases simulation success from 9.7% to 79.8% while achieving a 1.25× speedup in task completion time. We further demonstrate successful sim-to-real transfer enabled by the quality of the generated scenes together with domain randomization, improving real-world success from 21.7% to 75% and achieving a 1.13× speedup. Finally, we further highlight the benefits of leveraging the effectively unlimited data from 3D world generative models through an ablation study showing that increasing scene diversity directly improves zero-shot generalization.

Abstract:
Reinforcement learning (RL) has become a powerful paradigm for robot learning, particularly in sim-to-real settings, but its broader adoption remains limited by the engineering pipeline surrounding the algorithms. Building tasks, shaping rewards, and tuning hyperparameters require substantial expert effort, making RL workflows costly and difficult to scale. We introduce HARBOR, an agentic framework that frames robot RL automation as a harness-engineering problem: given a simulator codebase and a task specification, it automates the workflow from environment setup to policy training in simulation. HARBOR decomposes such high-level objectives into bounded stages executed by specialized agents through standardized commands, persistent artifacts, executable gates, and reusable knowledge, and scales iteration via decentralized parallel trials and experience learning across runs. We evaluate HARBOR across 6 benchmarks and 16 tasks in total, spanning manipulation, locomotion, and bimanual dexterous control. We demonstrate that HARBOR automates the simulation RL workflow end-to-end, designs rewards and tunes algorithms to match or improve over default configurations, and reduces engineering effort at practical token and wall-clock cost; the resulting policies can also be deployed on real robots.

Abstract:
We introduce ABC, a fully open-source stack for manipulation with behavior cloning. At its core is ABC-Data: the largest open-source, bimanual teleoperation dataset to date, featuring 3,500 hours of data spanning over 130K episodes across 195 diverse tasks. Furthermore, we open-source our accessible hardware setup, training infrastructure, and simulation pipeline. We also release 280 hours of sim-teleop data and provide a co-training recipe that produces correlated simulation and real-world evaluation. This allows researchers to effectively evaluate design choices without deploying on physical robots. We explore various training recipes and compare common architectural choices for Diffusion Transformers (DiT) and Vision-Language-Action (VLA) models, grounding our findings in real-world evaluations. The resulting policies successfully execute dexterous tasks such as box folding and extracting credit cards from wallets. By providing a reproducible toolkit, we aim to place researchers on an equal footing, establishing the necessary foundation to learn the ABCs of Behavior Cloning together as a community.

Abstract:
Mobile manipulation is a fundamental capability for general-purpose robotic agents, requiring both coordinated control of the mobile base and manipulator and robust perception under dynamically changing viewpoints. However, existing approaches face two key challenges: strong coupling between base and arm actions complicates control optimization, and perceptual attention is often poorly allocated as viewpoints shift during mobile manipulation. We propose InCoM, an intent-driven perception and structured coordination framework for mobile manipulation. InCoM infers latent motion intent to dynamically reweight multi-scale perceptual features, enabling stage-adaptive allocation of perceptual attention. To support robust cross-modal perception, InCoM further incorporates a geometric-semantic structured alignment mechanism that enhances multimodal correspondence. On the control side, we design a decoupled coordinated flow matching action decoder that explicitly models coordinated base-arm action generation, alleviating optimization difficulties caused by control coupling. Experimental results demonstrate that InCoM significantly outperforms state-of-the-art methods, achieving success rate gains of 28.2%, 26.1%, and 23.6% across three ManiSkill-HAB scenarios without privileged information. Furthermore, its effectiveness is consistently validated in real-world mobile manipulation tasks, where InCoM maintains a superior success rate over existing baselines.

Abstract:
High-quality, long-horizon demonstrations remain a major bottleneck for embodied AI, especially for wheeled mobile manipulators that require tight coordination between planar locomotion and dexterous arm control. Existing teleoperation systems and wearable interfaces often suffer from cognitive decoupling, limited scalability, and inconsistent action representations across data collection modes. We present SuperSuit, a bimodal data acquisition framework that unifies robot-in-the-loop teleoperation and active human demonstration through a shared kinematic interface. Both modalities produce structurally identical joint-space trajectories, enabling direct data mixing without modifying downstream policies. For locomotion, SuperSuit maps natural human stepping to continuous planar base velocities, eliminating discrete command switches. For manipulation, it employs a strictly isomorphic wearable arm in both modes, while policy actions are formulated in a shift-invariant delta-joint representation to mitigate calibration offsets and structural compliance without inverse kinematics. Real-world experiments on long-horizon mobile manipulation tasks show 2.6× higher demonstration throughput in active mode than a teleoperation baseline, comparable policy performance when substituting teleoperation data with active demonstrations at a fixed dataset size, and monotonic performance improvement as active data volume increases. These results indicate that consistent kinematic representations across collection modalities enable scalable data acquisition for long-horizon mobile manipulation.

Abstract:
Tool use greatly expands robot capabilities in unstructured environments. Despite significant progress, most prior work assumes a pre-specified tool and struggles to ground affordances in a scene with multiple open-category objects. We present GROW^2 (GROunding Which and Where), a method for open-world affordance grounding. Our key insight is that object parts naturally represent both affordance semantics and geometric structure, allowing us to decompose affordance grounding into two levels. At the semantic level, GROW^2 leverages the reasoning capabilities of vision-language models (VLMs) to select an appropriate tool and identify task-relevant parts. At the geometric level, GROW^2 exploits the perception capabilities of vision foundation models to ground the selected parts into 3D affordance regions. GROW^2 outperforms state-of-the-art baselines on both existing affordance prediction benchmarks and our newly constructed benchmark, which introduces the additional challenge of tool selection. Moreover, in both simulation and real-world robot tool-use experiments, GROW^2 enables robots to achieve higher task success rates than the baselines.

Abstract:
Open-vocabulary 3D affordance detection requires localizing interaction regions on point clouds given novel affordance descriptions. Recent methods extend multimodal large language models (MLLMs) with special output tokens that are decoded into segmentation masks. However, these tokens are produced through autoregressive generation, which models sequential dependencies rather than spatial neighborhood relations, leaving them semantically rich but spatially impoverished for 3D localization. We propose Voxel-enhanced Affordance detection (VoxAfford), which bypasses this bottleneck by injecting multi-scale geometric features from a frozen pre-trained 3D VQVAE encoder into the output tokens after generation. Each output token uses its affordance semantics as a query to retrieve relevant geometric patterns from its paired voxel scale via cross-attention, with a learned compatibility gate controlling the injection strength. The enhanced tokens are then aggregated into a spatially-aware affordance prompt through semantic-conditioned attention and propagated alongside per-point features to generate the final mask. Experiments on open-vocabulary affordance detection tasks show that VoxAfford achieves state-of-the-art performance with an approximately 10% absolute improvement in instance-level mIoU, and real robot experiments confirm zero-shot transfer to novel objects.

Abstract:
Solving complex long-horizon robotic tasks requires joint reasoning over abstract task structure and low-level physical interaction. While combining Vision-Language Models (VLMs) and video generation models offers a promising path for zero-shot planning, their individual tendencies to hallucinate physics or violate geometric consistency often compound over time, preventing reliable real-world execution. We introduce NovaPlan, a hierarchical framework that enables robust, zero-shot long-horizon manipulation by systematically proposing, verifying, and repairing visual plans. At the high level, a VLM planner decomposes tasks and filters out dynamically inconsistent futures by verifying multiple candidate video rollouts. To translate these imagined futures into reliable physical actions, NovaPlan utilizes a hybrid geometric representation that adaptively switches between object-centric flow and human hand flow. Finally, NovaPlan closes the loop by continuously monitoring execution to verify outcomes and synthesize local, non-prehensile corrective behaviors, such as fingertip poking, when failures occur. Across diverse multi-stage tasks, NovaPlan substantially outperforms prior zero-shot systems, achieving complex assembly and dexterous error recovery entirely without task-specific training or demonstrations. Please visit our project website for additional results: [https://nova-plan.github.io/](https://nova-plan.github.io/).

Abstract:
Bimanual manipulation requires policies that coordinate two arms while adapting their functional roles to scene geometry, object configuration, and task context. Learning such scene-conditioned role adaptation remains challenging, as demonstrations may contain uneven role distributions that limit generalization to underrepresented arm--role configurations. In addition, many bimanual policies predict actions in fixed left- and right-arm action spaces. While this provides a natural parameterization for robot control, it does not explicitly specify how behaviors should transform when functional roles are exchanged across arms. Across different scene initializations, the two arms may follow a similar coordination pattern, but the role-specific behavior assigned to each arm should change with the scene. Therefore, we propose BiRoAD, a Bimanual Role-Adaptive Decomposition framework for learning shared and role-adaptive representations in bimanual policies. Given bimanual trajectory or action-token features, BiRoAD decomposes these features into swap--symmetric and swap--antisymmetric components: the former captures coordination structure invariant to arm exchange, and the latter captures role-specific distinctions that vary consistently with functional role assignment. The two components are then recomposed as residual updates to the original paired arm representations, allowing BiRoAD to serve as a modular feature transformation without changing the policy inputs, imitation-learning objective, or requiring manually defined role labels. Across multiple bimanual manipulation tasks with balanced and imbalanced role distributions, BiRoAD improves robustness across role configurations over corresponding base policies, with notable gains on underrepresented role configurations.

Abstract:
Vision-Language Models (VLMs) enable robots to follow open-language instructions. However, dense VLM embeddings have shown to be noisy and lack spatial consistency. This is problematic for robotic applications, which require simultaneous reasoning over semantics and 3D space. We examine spatial structure across recent VLMs and propose ReSiReg, a feature reconstruction method that uses spatially consistent VLM intermediates to improve dense language-grounded retrieval. ReSiReg clusters intermediates into visual prototypes, derives their language descriptors, and reconstructs each patch as a soft mixture of prototype-level language embeddings. We evaluate quantitatively on OVSS and 3D mapping across backbones, and qualitatively in real-world manipulation scenes. Quantitative results show improved dense retrieval; manipulation scenes show more spatially consistent target activations. We further provide a compact 25M dense VLM for robotic applications, substantially smaller than and competitive with ViT-B baselines. Code and model weights will be released.

Abstract:
Visual navigation requires generating smooth and collision-free trajectories under complex geometric and physical constraints. Existing reactive policies that directly map observations to actions lack anticipatory reasoning, limiting their ability to proactively avoid obstacles. While visual imagination offers predictive foresight, conventional modular approaches separate scene prediction from policy learning, often leading to error accumulation and inefficient inference. To address these limitations, we propose WAM-Nav, a Latent World-Action Model for embodied visual navigation that jointly learns action generation and latent visual foresight, enabling more robust and foresighted navigation decisions without compromising inference efficiency. Specifically, WAM-Nav utilizes a shared Diffusion Transformer for asymmetric joint diffusion to concurrently generate long-horizon actions and short-horizon visual foresight, reducing the inference latency and visual error accumulation inherent in multi-step autoregressive rollouts. To further encourage smooth and consistent trajectory generation, we introduce a dual-stream contextual conditioning mechanism that integrates episode-level ego-motion history with sequential visual observations. Combined with a unified goal alignment module that preserves balanced representations across goal types, WAM-Nav naturally supports Image-Goal, Point-Goal, and No-Goal exploration within a single policy. Extensive experiments on the challenging ClutterScenes and InternScenes benchmarks demonstrate strong generalization of WAM-Nav, particularly on Image-Goal and Point-Goal navigation, where it improves success rates by 15.7% and 3.3%, respectively. Real-world deployment further validates effective zero-shot sim-to-real transfer, achieving an average 85% task success rate across diverse indoor and outdoor environments.

Abstract:
Visual navigation has made substantial progress in learning goal-reaching policies from egocentric observations. Most existing approaches primarily focus on geometric traversability, however, they do not explicitly account for the social constraints induced by human-populated environments where collision-free space can still be socially occupied, such as the space between walking partners or within conversational formations. To address this, we present GRACE, a group-aware framework for socially compliant visual navigation. The key insight of GRACE is to explicitly model social relations and group interaction spaces, allowing the robot to avoid paths that are physically feasible but socially undesirable. Specifically, our social perception module builds a visual pedestrian--robot graph from RGB observations and uses soft F-formation targets to supervise group-aware attention. This social representation guides a visual trajectory generator, which we further finetune using group-relative reward-weighted flow matching to preserve personal and group spaces. Real-world experiments on a Unitree Go2 demonstrate that GRACE significantly reduces group-space intrusions and collision events compared with representative baselines, while maintaining competitive navigation efficiency.

Abstract:
Reinforcement learning enables mobile robots to make fast and scalable navigation decisions in constrained human-shared environments. However, directly extending single-robot approaches to multi-robot settings often fails to ensure safety and efficiency, especially under occlusions and out-of-distribution (OOD) scenarios, where individual robots may overlook potential risks or make inaccurate predictions. Leveraging cooperation effectively for both prediction and navigation remains a major challenge. We propose CoopUQ, a multi-robot social navigation framework that bridges uncertainty-aware cooperative occupancy forecasting and robust decision making. CoopUQ predicts future occupancy grid maps through cross-robot spatiotemporal feature fusion and further leverages forecasts and their uncertainty to guide policy via constrained multi-agent reinforcement learning, where the cost penalizes navigation through regions predicted to be occupied or highly uncertain. This design enables the policy to adapt its risk sensitivity under distribution shifts. Experimental results across multiple Unity3D-based simulation environments with both in-distribution and OOD settings, including diverse layouts, pedestrian densities, and rushing behavior, demonstrate improved prediction performance and safety over representative baselines. Real-robot experiments further demonstrate its robustness across various scenes.

Abstract:
Robots deployed in delivery, campus, and emergency-response settings often need to navigate from buildings to streets within a single continuous episode. Existing benchmarks usually evaluate indoor and outdoor navigation separately, and many abstract away robot execution, leaving exit finding, boundary traversal, adaptation, and kinodynamic failures underexplored. We introduce Navverse, a physics-enabled benchmark for indoor-to-outdoor embodied navigation. Navverse contains 100 indoor scenes, 50 urban outdoor scenes, and 50 indoor-to-outdoor scenes, and supports Object Navigation, Vision-and-Language Navigation, and Place Navigation tasks, where agents search for semantic points of interest such as restaurants or banks. Agents are evaluated through executable robot interfaces using task-success, path-efficiency, and safety metrics. Zero-shot experiments with RL, VLA, and modular baselines show that current agents remain far from solving cross-context navigation: end-to-end VLAs obtain the highest zero-shot success, while modular method provides the strongest safety profile. PlaceNav further reveals a clear drop from outdoor to indoor-to-outdoor scenes, indicating that adaptation remains major bottlenecks.

Abstract:
Object navigation in unseen indoor environments requires agents to perform semantic search under partial observability. Vision-language models (VLMs) provide strong semantic-spatial priors for this task, but how to interface them with robot navigation remains challenging: dense VLM inference is expensive, while abstracting environments into symbolic memories often separates high-level reasoning from the raw visual evidence that supports it. We propose we propose PIGEON (Point of Interest Guided Exploration for Object Navigation), a VLM-driven framework that formulates object navigation as raw-observation-grounded sparse decision problem. PIGEON introduces Points of Interest (PoIs) as sparse visual decision units that couple geometrically executable waypoints with raw egocentric observations. Rather than using VLMs as dense controllers or restricting them to frontier ranking, PIGEON enables VLMs to select among task-critical PoIs, including exploration frontiers, suspected target objects, traversable stairs, and floor-level summaries, while low-level planners execute continuous motion between them. This PoI interface further makes high-level navigation decisions verifiable, allowing us to develop an RLVR pipeline that improves local VLMs without manual Chain-of-Thought annotations. Extensive experiments on Habitat ObjectNav benchmarks show that PIGEON achieves state-of-the-art zero-shot performance, scales consistently with foundation model capacity, and transfers to Active Embodied Question Answering with only prompt modifications. Real-world deployments on physical robots further demonstrate its robustness and efficiency.

Abstract:
Safe autonomous navigation requires not only geometric obstacle avoidance but also semantic awareness. Different object categories demand different safety clearances, and the set of relevant categories may expand as the robot explores novel environments. Vision-language models (VLMs) offer rich semantic representations for building maps of unknown scenes, yet their confidence scores are poorly calibrated, making raw predictions unreliable for safety-critical planning. Conformal prediction (CP) can provide coverage guarantees, but existing methods assume a fixed label set and stationary data, limiting their use in open-world settings. We propose Conformal Semantic Mapping for Open-World Safe Robot Navigation (CoSMO), a semantic safety mapping framework that transforms VLM embeddings into calibrated coverage sets and safety-aware obstacle dilations over a dynamically expanding hazard vocabulary, even under distribution shift. Experiments on the Matterport3D (MP3D) dataset show that CoSMO reduces semantic collisions by 61.6% relative to a geometry-only baseline, while maintaining formal coverage guarantees even under delayed expert feedback. Code will be released upon publication.

Abstract:
Multi-robot control in cluttered environments is a challenging problem that involves complex physical constraints, including robot–robot collisions, robot–obstacle collisions, and unreachable motions. Successful planning in such settings requires joint optimization over high-level task planning and low-level motion planning, as violations of physical constraints may arise from failures at either level. However, jointly optimizing task and motion planning is difficult due to the complex parameterization of low-level motion trajectories and the ambiguity of credit assignment across the two planning levels. In this paper, we propose a hybrid multi-robot control framework that jointly optimizes task and motion planning. To enable effective parameterization of low-level planning, we introduce waypoints, a simple yet expressive representation for motion trajectories. To address the credit assignment challenge, we adopt a curriculum-based training strategy with a modified RLVR algorithm that propagates motion feasibility feedback from the motion planner to the task planner. Experiments on BoxNet3D-Obs, a challenging multi-robot benchmark with dense obstacles and up to nine robots, show that our approach consistently improves task success over motion-agnostic and VLA-based baselines.

Abstract:
Vine robots, a class of soft, growing robots, are suitable for navigating complex and confined environments due to their compliant bodies and self-supporting growth mechanism. However, hysteresis, tether interactions, and deformations make them difficult to predict and model, which in turn limits the effectiveness of conventional planning and control approaches. In this work, we present a data-driven, vision-based control framework for the first autonomous vine robot system. Our system integrates 19 cameras distributed along the robot's body to provide comprehensive feedback of both the robot state and the surrounding environment. Using this rich whole-body vision feedback, we train an end-to-end visuomotor policy from demonstrations for closed-loop autonomous control in complex environments. The policy efficiently aggregates information from distributed sensing while maintaining robustness to inaccurate robot states and actuation. Experimental results demonstrate that the learned policy enables robust navigation and manipulation in challenging scenarios, including steering through branched structures, climbing up slopes, traversing unsupported terrain, reaching objects precisely, and maneuvering through confined spaces and obstacles.

Abstract:
Multi-camera perception is a load-bearing pillar of indoor robot manipulation, yet current feed-forward 3D foundation models still provide only centimeter-level camera poses on cluttered manipulation rigs. In our RH20T setting, off-the-shelf COLMAP and RGB-D ICP can degrade rather than reliably improve WorldMirror-v2 initializations. We present latent score-based pose refinement (LSPR), a foundation-init-conditioned calibration method that repurposes diffusion from global pose generation into local residual refinement on the \mathrmSE(3) tangent space. LSPR learns the score field of the residual pose-error distribution induced by a strong initializer: ground-truth rigs are corrupted by tangent-space perturbations covering the initializer's small-noise regime, and inference follows this field from the foundation initialization rather than sampling poses from noise. To make this local refinement reliable, LSPR combines score-level ensembling, hard-tier sample mining, and a confidence gate that falls back to the initializer under distribution shift. On held-out RH20T manipulation scenes, LSPR improves WorldMirror-v2 initializations from centimeter-level to sub-centimeter, sub-degree \mathrmSim(3)-aligned accuracy, and these calibration gains transfer to downstream multi-view depth fusion.

Abstract:
Collaborative perception enhances perceptual accuracy and range through multi-agent information sharing. To address spatial misalignment caused by pose error and communication latency in collaborative perception, existing methods enhance robustness by extending adaptive local feature fusion regions. However, computational bottleneck of basic operators prevents local windows from being extended to the global scope, making it difficult to cover complex and diverse perception scenarios. Accordingly, this paper proposes CoLA, a robust collaborative perception scheme which achieves global spatial semantic matching and alignment across agents with linear computational complexity. CoLA integrates Global Structure Rectification Module for global structure modeling and inter-agent denoising, and Ego-Anchored Gate Fusion module for semantic alignment and fusion. Furthermore, we design a patch-level structural consistency loss to suppress severe position jitter. Extensive experiments on both real-world and simulated datasets demonstrate that CoLA outperforms state-of-the-art methods under various noisy conditions.

Abstract:
Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.

Abstract:
Recent progress in vision foundation models (VFMs) has made automated open-vocabulary 3D bounding box annotation increasingly feasible by jointly leveraging LiDAR point clouds and images. However, partial observation remains a major challenge in sparse driving scenes, where visible LiDAR points often cover only a small fraction of the object's surface and provide insufficient cues for estimating its full spatial extent. As a result, existing methods often produce mislocalized boxes or leave objects unlabeled. To address this limitation, we propose AmodalBox, an automated annotation framework that leverages image-generated 3D assets as amodal geometric priors for partially observed objects. By establishing correspondences between input images and rendered views of 3D assets, AmodalBox robustly aligns canonical amodal geometry with partially observed world-space LiDAR point clouds. To improve temporal consistency and reduce computational cost, AmodalBox propagates anchor-based alignments for dynamic instances and aggregates multi-frame evidence for static objects. 3D object detectors trained on annotations from AmodalBox improves performance over strong baselines by 14.1 mAP on the nuScenes dataset and by 12.45 mAP at IoU 0.5 for the vehicle class on the Waymo Open Dataset (WOD).

Abstract:
Neural inertial odometry increasingly uses networks as learned measurements inside filtering pipelines. Such measurements should transform consistently under arbitrary IMU mounting conventions: their mean must transform as a vector, and their covariance must transform congruently as a second-order tensor. We present GINIO, a geometric \(\mathrmSO(3)\)-equivariant interface for neural inertial odometry under arbitrary rotations of the IMU measurement frame. Given calibrated IMU windows, our framework predicts a motion measurement and uncertainty obeying these tensorial laws. To support efficient sensor-frame learning, we introduce Last-Frame Alignment (LFA), a deterministic preprocessing step that is provably equivalent to world-frame training for \(\mathrmSO(3)\)-equivariant predictors. The connected estimator tracks sensor-local states such as IMU bias, separating nuisance estimation from the geometric law enforced by the learned measurement. We instantiate the same interface in filter-connected NIO, AirIO-style recurrent aerial prediction, EqNIO-style full-\(\mathrmSO(3)\) canonicalization, and ResNet-style temporal backbones. On TLIO, GINIO achieves 2.018 m ID/\(\mathrmSO(3)\) ATE while EqNIO degrades to 76.389 m, using \(11.6×\) fewer FLOPs. On NanoBench, our AirIO-style instantiation improves ATE from 5.579 m to 1.430 m without external attitude input, and our ResNet-style instantiation reaches 0.581 m ATE versus 0.645 m for ResNet1D. On Fetch, GINIO reduces unseen physical-remount ATE from 8.15 m to 0.50 m without retraining.

Abstract:
Achieving dexterous robotic manipulation in the real world relies heavily on human supervision and algorithmic engineering, which is a central bottleneck in the pursuit of general physical intelligence. Although emerging coding agents can generate code to automate algorithm search, their successes remain largely confined to digital environments. We conjecture that the missing abstraction to automate robotics research is a repeatable feedback loop for real-world policy improvement: reset the scene, execute a policy, verify the outcome, and refine the next iteration. To bridge this gap, we introduce ENPIRE, a harness framework for coding agents that instantiates this physical feedback routine through four core modules: an Environment module (EN) for automatic reset and verification; a Policy Improvement module (PI) that launches policy refinement; a Rollout module (R) to evaluates policies with one or multiple physical robots operating in parallel; and an Evolution module (E) in which coding agents analyze logs, consult literature, and improve training infrastructure and algorithm code to address failure modes. This closed-loop system transforms real-world robot learning into a controllable optimization procedure that agents can manage, thus minimizing human effort while allowing fair ablations across training recipes and agent variants. Powered by ENPIRE, frontier coding agents can autonomously develop policies that achieve a 99% success rate on challenging dexterous manipulation tasks in the real world, including PushT, organizing pins into a pin box, and using a cutter to cut a zip tie. Coding agents can improve policies under various PI regimes, such as heuristic learning, tool calling, behavior cloning, and offline or online RL. Moreover, ENPIRE can be significantly accelerated on a robot fleet, and we propose two metrics, Mean Robot Utilization (MRU) and Mean Token Utilization (MTU) to measure the efficiency of multi-agent physical autoresearch. We also include simulation results in RoboCasa. Our findings suggest a practical and scalable path toward autonomously advancing robotics in the real world, and code will be released soon.

Abstract:
Control barrier functions (CBFs) have become one of the most popular tools for encoding and enforcing state constraints in safety-critical robotics. Standard CBF approaches are inherently myopic in nature as they enforce safety only at the current time step. Consequently, the system can be driven toward the boundary of the safe set where no feasible safe control exists at a future timestep. Model predictive control (MPC) based approaches address this by enforcing state constraints over a receding horizon. However, such approaches require solving a constrained optimization problem at every step, which is computationally expensive for real-time deployment. We propose BarrierFormer, a barrier-supervised transformer framework that addresses these limitations by encoding rollout-level CBF constraints in learning a safe policy. A causal transformer encodes observation-action history, outputs a residual correction to a nominal controller through the action head, and autoregressively generates a predictive rollout through the dynamics head. A barrier critic operating on local observations evaluates CBF constraint violations along this rollout, and a safety teacher computes barrier-consistent actions satisfying these constraints as direct supervision targets for the control policy. During inference, the policy maps observation-action history to control actions without any online optimization, enabling real-time usage of the proposed history-based predictive safety enforcement. Evaluations across linear and nonlinear dynamical systems for safe goal-directed navigation show that BarrierFormer outperforms reinforcement learning (RL)-based, diffusion-based, MPC-based, and transformer-based baselines in safety rate, success rate, and inference latency.

Abstract:
Motion planning in unstructured environments is a bottleneck for robot deployment. Current visuomotor policies predominantly rely on external cameras, introducing perceptual blind spots and fragility due to occlusions, clutter, and calibration constraints. To overcome these limitations, we propose ESCAPE, an online egocentric sensing framework for collision-free trajectory generation. ESCAPE aggregates sparse measurements from distributed depth sensors, converting them into a structured, obstacle-aware point cloud representation. Our policy leverages a flow matching-based generative framework, and trains on a high-quality dataset comprising over 100K safety-critical trajectories. With joint optimization of goal-reaching and collision objectives, it produces safe trajectories directly from these egocentric inputs. Crucially, ESCAPE is trained entirely in simulation and transfers zero-shot to physical hardware without any real-world fine-tuning. Extensive simulation and real-world evaluations demonstrate that ESCAPE achieves stable motion planning using only egocentric perception, outperforming external-view and voxelized-planning baselines across both simulation and real-world evaluations. Our website: https://escape-policy.github.io/.

Abstract:
Generalist robot policies often represent actions as end-effector target poses in task space, which facilitates policy transfer across robots performing the same task. However, a task-space pose that is feasible for one morphology may be infeasible for another due to joint limits, workspace differences, or self-collision constraints. Existing pose reachability methods often learn monolithic fields over the full SE(3) pose space, which spreads expensive inverse-kinematics supervision over a high-dimensional domain and leaves near-boundary regions poorly sampled in cross-morphology settings. We propose Cross-Morphology Reachability Distance Fields (CMRDF), a morphology-conditioned task-space reachability representation that lowers the complexity of SE(3) reachability learning by decomposing full-pose feasibility into a global position-reachability field and a local, position-conditioned orientation-reachability field. This hierarchy replaces a monolithic field over all position-orientation combinations with a dense Position-SDF over the 3D workspace and a conditional Orientation-SDF over local orientation feasibility. To align supervision with this hierarchical representation, CMRDF uses dense forward-kinematics coverage and distance-to-boundary targets for the Position-SDF, and applies inverse-kinematics boundary mining only at representative position anchors for the Orientation-SDF. The resulting differentiable distance fields provide physically meaningful margins for reachability queries, near-boundary modeling, and gradient-based repair of infeasible poses. With only 0.51x the full-pose inverse-kinematics supervision budget, CMRDF achieves 85.53% SE(3) accuracy, improving by 6.45 percentage points over the strongest evaluated 1.00x non-CMRDF baseline, while also supporting local pose repair and zero-shot transfer under structured morphology shifts.

Abstract:
Model Predictive Control (MPC) provides a structured and constraint-aware mechanism for decision-making, but its reliance on optimization-friendly analytical dynamics models limits its use in tasks with contacts and other hard-to-model state dependencies. Model-free reinforcement learning avoids explicit modeling assumptions but typically requires large amounts of interaction data. We present a learning-based MPC framework that combines the data efficiency and structure of local model-based planning with learned components that compensate for incomplete dynamics and finite-horizon myopia. The method augments a nominal analytical model with a residual dynamics network that learns missing state-dependent effects from data and combines the resulting planner with a learned action-value critic that injects long-horizon MDP structure into the local iLQR optimization. To make this practical at reinforcement-learning scale, we develop a GPU-accelerated batched iLQR solver that evaluates learned dynamics and critic networks inside the optimal-control loop and solves thousands of trajectory-optimization problems in parallel. The complete system is integrated into a robotics simulator, enabling scalable model-based reinforcement learning under incomplete dynamics. Experiments on biased and incompletely modeled control tasks show that the approach improves closed-loop control performance while preserving the model-based structure needed for efficient constrained trajectory optimization.

Abstract:
Achieving complex, long-horizon robot behavior remains challenging because successful execution depends on skills grounded in a robot's physical affordances. Existing training-task distributions are often handcrafted or sampled from large language models (LLMs), relying on semantic plausibility rather than embodiment-specific feasibility. This mismatch can produce curricula that are too difficult, redundant, or blind to non-obvious but learnable behaviors. We propose Generative Flow on Symbolic Abstractions (FOSA), a framework for discovering embodiment-specific capabilities by learning a grounded task generator. FOSA parameterizes an abstract policy as a Generative Flow Network that constructs symbolic task skeletons. Each skeleton is compiled into a simulation environment, where low-level policy-learning performance provides the reward signal for the abstract policy. Across heterogeneous robot platforms, FOSA discovers diverse and executable tasks near the robot's empirical capability frontier. After distilling the discovered task policies into a single language-conditioned policy, FOSA improves performance on unseen complex and long-horizon tasks.

Abstract:
We introduce flow control of vision-language-action (VLA) models, a simple and effective way to steer VLA actions in real-time through generic inputs, such as a keyboard. This method can be used out-of-the-box and does not require retraining or fine-tuning VLAs. It enables relatively crude user inputs to steer a VLA to align with user intent. The VLA transforms these inputs into action samples drawn from the VLA expert action distribution learned during training, so that the generated actions are high quality (conformity to the action expert distribution) and high fidelity (reflecting the user's intent). We demonstrate that flow control has many desirable properties: (1) flow control accurately and responsively steers robot actions with user inputs, (2) it is robust to suboptimal user inputs, (3) it enables users to steer VLAs to achieve significantly higher success rates and faster task completion, and (4) fine-tuning a VLA on flow control trajectories improves the autonomous policy. Together, these results provide a simple and intuitive way for users to help steer VLA actions, increasing task performance.

Abstract:
Robotic imitation learning requires policies that capture multimodal action distributions, run at real-time control rates, and effectively exploit available sensors. Existing generative approaches, including diffusion, flow matching, and IMLE, satisfy these requirements only partially. We present PRISM, a single-pass multisensory policy built on a calibrated batch-global RS-IMLE objective. We show that the RS-IMLE rejection mechanism, originally designed for few-shot image synthesis, degenerates at policy-scale batch sizes, and introduce the calibration needed to restore effective rejection. PRISM couples a temporal multisensory encoder for RGB, depth, tactile, audio, and proprioceptive inputs with a linear-attention Performer generator trained under this objective. On real-world loco-manipulation with a Unitree Go2 and 7-DoF arm, and tabletop manipulation with a UR5, PRISM improves success over diffusion policies by 10–25 percentage points while sustaining 30–50 Hz closed-loop control. On language-conditioned CALVIN ABCD, PRISM improves success by approximately 31 percentage points over diffusion and approximately 24 percentage points over flow matching, while reducing trajectory jerk by 20×–70×. Project page with videos: https://anonymous227111-png.github.io/

Abstract:
Learning-based robotic models have recently been widely used to improve robot generalization and task success rates. Previous works apply efficient-inference techniques such as quantization, pruning, and asynchronous inference to reduce the per-step cost and inter-action latency. However, unlike traditional static ML tasks, embodied tasks involve repeated interaction with the environment, and task-level performance is determined not only by per-inference cost, but also by closed-loop effects unique to embodied execution, which remain insufficiently characterized in current efficient-inference studies. In this work, we propose TISED (\underlineTask-level \underlineInference \underlineSpeedup \underlineEffect \underlineDecomposition), an analytical framework that unifies diverse inference-optimization techniques and decompose their effects on static and dynamic tasks, and uncovers some paradoxical effects on task-level performance: (1) on static tasks, optimization sometimes can lengthen end-to-end completion time even as per-step latency drops; (2) on dynamic tasks, moderate lightweighting can raise task success rate even above the baseline; and (3) the monotonicity and sweet-spot location of both effects can shift with hardware configuration. Together, our findings provide a new perspective on adapting AI inference-optimization techniques to the embodied tasks.

Abstract:
A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.

Abstract:
Predicting how deformable objects evolve under robotic manipulation is a longstanding challenge. Existing approaches typically rely on per-object optimization to fit material parameters, which can be slow and cannot generalize, while end-to-end learned alternatives extrapolate poorly and often violate basic physical structure. We present PhysCoRe, a physics-corrected residual world model that couples a differentiable Material Point Method (MPM) simulator with two feed-forward neural networks. A material refinement module, Material from Motion (MfM), infers per-particle elasticity from visual observations, grounding the simulator in object-specific physics. A residual correction module, Residual from Dynamics (RfD), learns the discrepancy and predicts corrections to the simulator's internal dynamics, absorbing systematic biases that the analytical model cannot capture. This design also supports online material identification on novel objects. MfM adapts from limited interactions, and its predictive uncertainty steers further exploration toward the regions where its estimate is least confident. Experiments on real deformable-object manipulation sequences show that PhysCoRe outperforms state-of-the-art baselines in prediction accuracy, and that its predicted confidence forms a reliable distribution across the object's geometry, providing a natural signal for future uncertainty-guided exploration.

Abstract:
Mobile dexterous manipulation requires long-horizon coordination of a mobile base, dual arms, and multi-fingered hands, making policy learning highly data-intensive. While human demonstrations offer scalable and diverse interaction data, directly using them for robot learning is challenging due to the embodiment gap. We present DexRoam, a full-stack system for learning mobile dexterous manipulation from tracker-free egocentric human demonstrations. DexRoam uses a consumer VR headset and a head-mounted stereo camera to capture whole-body motion and first-person observations in everyday environments. To transfer these demonstrations, DexRoam introduces a novel whole-body retargeting pipeline that maps human motion into robot-executable base, arm, head, and dexterous hand commands. We train a vision-language-action policy on retargeted human demonstrations and teleoperated robot data, and evaluate how different human-to-robot data ratios affect policy performance under co-training and pretraining-finetuning strategies. Across four real-world tasks, DexRoam improves average success from 33.8% to 71.3% over robot-only training, with consistent gains across reaching, grasping, and mobile coordination. Our analysis further shows that human demonstrations do not replace robot data but substantially improve its data efficiency. We will open-source the full system.

Abstract:
Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (Retry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

Abstract:
Modern embodied agents achieve impressive success rates, yet their actual instruction-following ability is far weaker than these numbers suggest. We trace this illusion to a structural property we term low \emphscene entropy: when a visual scene admits only one valid task, language becomes redundant and a policy can score highly while barely using it. We introduce RoboFollow, a diagnostic benchmark with three principles. (1) High Scene Entropy: each training scene supports up to 16 kinematically distinct task branches, making vision alone insufficient and forcing reliance on language. (2) Hierarchical Diagnostic Protocol: a four-level protocol (L0--L3) progressively perturbs visual layout and semantics, probing whether equivalent instructions yield consistent behavior and distinct ones yield discriminable behavior across spatial relations, attributes, trajectory constraints, and logic. (3) Confound-Controlled Diagnosis: we simplify interaction objects, restrict actions to the trained repertoire and report stage-wise Intent and Execution scores, isolating comprehension from motor execution. Extensive evaluation shows that state-of-the-art VLA and VA models can get high performance in L0, but follow instructions poorly under minimal L1--L3 perturbations. Our small-scale real-robot experiments further reproduce this trend. Representative mitigations, including stronger VLM backbones, QA co-training, LangForce, and Classifier-Free Guidance, all fail to close this gap. RoboFollow exposes genuine instruction following as a critical, overlooked bottleneck. Dataset and code will be released.

Abstract:
Vision-language-action models (VLAs) achieve state-of-the-art performance on many robotic manipulation tasks, yet they can still behave unpredictably in out-of-distribution scenarios. Runtime failure detection is therefore essential for the safe real-world deployment of VLAs. However, existing task failure detectors require computationally expensive action sampling, are based on architectural assumptions that limit their applicability to VLAs, or need access to failure rollouts. We propose VLA-FAIL, a lightweight and broadly applicable failure detection framework for VLAs that combines two novel failure detectors with minimal overhead, without requiring failure data. The first, last-layer Mahalanobis distance (LLMD), detects out-of-distribution states by measuring token-wise deviations in last-layer features relative to the training data. The second, action chunk consistency (ACC), exploits the temporal overlap induced by receding-horizon control and detects failures when consecutive action chunks become inconsistent. To capture the trade-off between detection accuracy and detection latency, we introduce AUCPDT, a threshold-independent metric that jointly evaluates precision, recall, and detection time. Through extensive real-world and simulation experiments, we demonstrate that LLMD and ACC capture complementary failure modes whose combination enables reliable and early failure detection across diverse tasks, frequently outperforming significantly more expensive baseline methods.

Abstract:
Real-world physical evaluation of generalist robot policies remains a major bottleneck: standard protocols require manual scene resets, costly human grading, and extrapolation from small fixed task suites, yielding low throughput and coarse policy diagnostics. We present ScaleEvals, a reset-free evaluation protocol that autonomously probes generalist policies through continuous play without per-rollout human intervention. ScaleEvals uses a VLM-based pipeline in which an AutoPrompter continuously generates grounded tasks and rubrics from an evolving scene, while an AutoGrader assesses task progress from multimodal observations. Each prompt is annotated with skill tags capturing action and language factors, enabling performance analysis across heterogeneous task streams. We normalize comparisons by instruction-derived difficulty and aggregate progress scores via a soft-win Bradley–Terry model with tag-stratified ANOVA, producing global policy rankings and interpretable capability diagnostics. Across five generalist policies and 1,750 real-world trajectories, ScaleEvals achieves a 3x increase in rollouts per hour per station while AutoGrader maintains 90.7% agreement with human graders. Reset-free performance rankings align with manual evaluation baselines with 0.92 (p < 0.05) Pearson correlation, while exposing capability differences that aggregate success rates obscure. These results suggest that real-world robot evaluation can scale as an autonomous, continuously running system rather than a sequence of reset-required benchmark trials.

Abstract:
Realizing true end-to-end biological laboratory automation requires robotic systems that can support two essential capabilities: directly manipulating liquids, and preparing and clearing the programmable automated liquid-handler platforms through which liquids are ultimately processed. However, existing laboratory benchmarks such as AutoBio and LabUtopia focus on rigid-body instrument operations or chemistry procedures with limited fluid modeling fidelity. We introduce LiquidLab, a robotic benchmark to jointly evaluate physically grounded liquid manipulation and automated liquid-handler operation for biological laboratory automation, comprising sixteen tasks across direct liquid manipulation, platform operations, and long-horizon wet-lab protocols. We further propose a parallel SPH fluid simulation pipeline built on Genesis that enables large-scale parallel liquid simulation on a single GPU, and ground task success in particle-level volumetric criteria rather than pose proxies. Benchmarking four representative imitation-learning and vision-language-action policies reveals that current models remain far from reliable, with failures concentrated in millimeter-scale labware staging and free-surface fluid control, exposing a clear gap between current robotic capabilities and the demands of fully autonomous biology.

Abstract:
Vision-language-action (VLA) models adapted through supervised fine-tuning (SFT) inherit a structural asymmetry: expert demonstrations teach the policy where success behavior lies, but provide no signal about where it ceases to be reliable. We argue that robust VLA adaptation should therefore be viewed not as further demonstration fitting, but as Failure-Boundary Learning---the problem of Discovering, Localizing, and Shaping the boundary between recoverable deviations and task failure. To instantiate this view, we propose DLS: built on a real-grounded behavioral prior from few real demonstrations and simulated co-training, DLS discovers failure boundaries at scale through on-policy digital twin rollouts. Rather than reducing each rollout to a binary label, semantic progress localization uses privileged simulator states to assign progress-aware signals that capture where the failure boundary is crossed, not merely whether. These signals drive directional boundary shaping in the flow dynamics---reinforcing success-producing denoising directions and suppressing failure-producing ones, without action likelihoods or auxiliary critics. Across real-robot manipulation tasks, DLS improves robustness over SFT and online RL baselines, especially under randomized initial states and unseen visual conditions.

Abstract:
An often overlooked factor of robot manipulation performance is the embodiment of the robot itself. Motivated by this problem, we study motion-conditioned robot co-design, where the goal is to generate complete robot designs that track target end-effector trajectories (from human demonstrations) while optimizing user-defined rewards. We introduce Transformer Transformer, a diffusion transformer trained on RoboTokens, a unified tokenization of robot embodiments, states, and actions. The same architecture can be used across embodiment spaces (e.g., wheeled bimanual, quadrupeds, humanoids) and use cases (embodiment generation, cross embodiment controller). Rather than overfitting to one reward function, Transformer Transformer is a dynamics model, whose reward-agnostic state and action predictions can be converted into reward-specific value predictions. These value predictions are used to steer embodiment diffusion towards high value robot designs, through a procedure we call Dynamics Self-Guidance. Experiments across multiple design spaces show zero-shot optimization of unseen rewards and trajectories, improving performance and runtime over the evolutionary baseline. Finally, we fabricated an optimized ALOHA design, which reduced tracking error by over 70% compared to the original design.

Abstract:
Multi-hand semantic grasp generation aims to produce feasible, semantically appropriate grasp poses for different robotic hands from natural language instructions. Although the task is highly valuable, due to the lack of multi-hand grasp datasets with fine-grained contact description between robotic hands and objects, it is still a long-standing, difficult task. In this paper, we present Multi-GraspSet, the first large-scale multi-hand grasp dataset with automatically annotated contacts. Based on Multi-GraspSet, we propose Multi-GraspLLM, a unified language-guided grasp generation framework, which leverages large language models (LLM) to handle variable-length sequences, generating grasp poses for diverse robotic hands in a single unified architecture. Multi-GraspLLM first aligns the encoded point cloud features and text features into a unified semantic space. It then employs a flow-matching process that progressively transforms a noise distribution into a target grasp pose conditioned on the fused semantics, using a learnable cross-hand latent space that enables genuine knowledge transfer and hand-specific decoders that produce joint angles for each robotic hand. The experimental results demonstrate that our approach significantly outperforms existing methods in both real-world experiments and simulator. Furthermore, our Multi-GraspLLM, trained on multiple robotic hands, exhibits exceptional adaptability. For an unseen robotic hand, fine-tuning with LoRA using only 20% of the new hand data is sufficient to match the performance of a model trained from scratch on the entire new hand dataset.

Abstract:
Current dexterous manipulation systems commonly rely on external or wrist-mounted cameras for visual feedback. However such sensing setups are prone to occlusion and can limit performance on tasks requiring close-range, multi-view perception. In this work, we design a vision-enhanced fingertip module that brings clear visual perception directly to the fingertips and present FingerViP, a learning system that leverages fingertip visual perception for dexterous manipulation. Specifically, the fingertip module integrates miniature-camera sensing within compact-footprint hardware and is installed on each finger of a multi-fingered hand, which substantially improves visual perception by providing comprehensive, multi-view observations of objects and surrounding environment. Building on the integrated fingertip modules, we develop a diffusion-based whole-body visuomotor policy conditioned on a third-view camera and multi-view fingertip vision, enabling the robot to effectively learn complex manipulation skills directly from human demonstrations. To improve view-proprioception alignment and contact awareness in the policy observation representation, each fingertip visual feature is augmented with its corresponding camera-pose encoding and per-finger joint-current encoding. We validate the effectiveness of multi-view fingertip vision and demonstrate the robustness and adaptability of FingerViP across a range of challenging real-world tasks, including retrieving objects behind an occluding curtain, achieving a 56.6% relative improvement in average success rate over the baseline using an external third-view camera and a wrist-mounted camera. All hardware designs and code will be fully open-sourced.

Abstract:
Object handover between two dexterous hands is a fundamental yet underexplored skill for bimanual robots. Unlike single-hand grasping, dexterous handover requires anticipatory coordination: the giver must leave clearance for the receiver while both hands manage finger-level contacts and avoid interference. We address this challenge with DexHandover, a generative ``Shape-to-Motion'' framework that maps an object's geometry directly to a complete bimanual handover motion sequence. DexHandover combines: (i) an optimization-based pipeline for generating large-scale synthetic dataset of physically validated bimanual handover poses; (ii) a two-stage generative model that predicts handover poses conditioned on object geometry; and (iii) a reinforcement learning strategy that synthesizes physically plausible giver-to-receiver motion sequences anchored at the generated handover pose. We validate the pipeline in simulation and retarget the synthesized trajectories to robot hands, demonstrating reliable robot-to-robot dexterous handover from object geometry.

Abstract:
Vision-Language-Action (VLA) models have shown remarkable generalization by mapping web-scale knowledge to robotic control, yet they remain blind to physical contact. Consequently, they struggle with contact-rich manipulation tasks that require reasoning about force, texture, and slip. While some approaches incorporate low-dimensional tactile signals, they fail to capture the high-resolution dynamics essential for such interactions. To address this limitation, we introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future. Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs coupled with wrist-camera local vision and third-person macro vision. To reconcile these multi-scale sensory streams, we first train a unified policy with a Hierarchical Spatial Alignment (HSA) loss that aligns tactile tokens with their spatial counterparts in the wrist and third-person views. To further deepen the model’s understanding of fine-grained contact dynamics, we finetune the system with a tactile world model that predicts future tactile signals. To mitigate tactile data scarcity and the wear-prone nature of tactile sensors, we construct a hybrid large-scale dataset sourced from both high-fidelity digital twin and real-world experiments. By anticipating upcoming tactile states, DreamTacVLA acquires a rich model of contact physics and conditions its actions on both real observations and imagined consequences. Across contact-rich manipulation tasks, it outperforms state-of-the-art VLA baselines, achieving up to 95% success, highlighting the importance of understanding physical contact for robust, touch-aware robotic agents.

Abstract:
Force signals provide critical interaction cues for contact-rich robotic manipulation. However, existing methods mostly use force as an additional observation modality, without fully exploiting its role in modeling future interaction dynamics or guiding execution-time feedback correction. In this paper, we propose FAWAM, a force-aware world action model that incorporates force information at three levels: perception, prediction, and closed-loop execution. FAWAM first encodes historical 6-axis force/torque signals to modulate action generation, then jointly predicts future actions and end-effector wrenches to explicitly model contact evolution. It further introduces a residual correction module that uses the predicted wrench trajectory as an execution-time reference to refine actions online based on real-time force feedback. Real-world experiments across multiple contact-rich tasks show that FAWAM improves the average success rate by 36.25% over vision-only baselines and 21.25% over existing force-aware baselines, demonstrating the effectiveness of our force-aware framework for robust contact-rich manipulation.

Abstract:
Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches.

Abstract:
Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation, but visual observations alone often miss the local contact state required for contact-rich tasks. We present TacCoRL, a scalable framework that injects Tactile feedback into VLA policies and improves them through sim-real Co-training and simulation-based reinforcement learning (RL), without requiring large-scale tactile pretraining or extensive real-world contact exploration. The key idea is not only adding touch as an input, but learning how contact readings should modulate action responses in near-failure states that are rare in demonstrations and risky to collect on hardware. We use a real-aligned simulator as a closed-loop training environment for contact interaction. Mixed simulated and real trajectories first warm-start tactile-conditioned actions in the pretrained policy. Reinforcement learning with verifiable task rewards then optimizes the policy using simulated contact rollouts. It reinforces tactile-conditioned actions that lead to task completion, while a supervised objective on real trajectories keeps the refined policy anchored to deployment visual, tactile, and action distributions. The resulting policy transfers directly to the real robot without privileged simulation state or online real-world RL. Across four bimanual contact-rich tasks, the final visuo-tactile policyachieves an average success rate of 72.5%, compared to baseline of 50.0%.

Abstract:
Tactile dexterous manipulation is essential to human manipulation, yet learning effective tactile enabled robot policies remains a challenge. While recent work relies on imitation learning, obtaining high quality demonstrations for multi-fingered hands via robot tele-operation or kinesthetic teaching is prohibitive. Alternatively, with reinforcement learning (RL) we can learn skills in simulation, but the lack of fast and realistic tactile simulation remains a core issue. To bridge this gap, we introduce sim-to-real Privileged Tactile Latent Distillation (\ourmethod). Our method anchors sim-to-real transfer on privileged sensors and learns a tactile encoder that maps real-world touch into the latent state of the privileged simulation policy, bypassing tactile simulation entirely. We demonstrate that \ourmethod improves over proprioceptive manipulation policies trained in simulation significantly by incorporating tactile sensing. On the benchmark in-hand rotation task, \ourmethod achieves a 182% improvement. \ourmethod also enables the challenging task of tactile in-hand reorientation where we see a 57% improvement in number of goals reached.

Abstract:
Humans excel at contact-rich manipulation by combining visual context, fingertip tactile feedback, and force-sensitive wrist interaction. While robots can access analogous signals, multimodal Diffusion Policies often fuse them as direct conditioning features, without preserving distinct roles or pairwise relations. We present ViTaForce, a cross-modal representation learning framework that learns three paired encoders from RGB vision (scene and hand-eye), dual side tactile pressure maps (T), and high-rate end-effector wrench windows (F). Each encoder is anchored to a distinct physical question: the V–T encoder learns where contact occurs on the fingers (hand-eye vision), the V–F encoder learns when contact will spike from scene approach cues, and the T–F encoder learns what wrench components reflect genuine contact rather than motion-induced bias (tactile-gated force). Validated on assembly tasks such as nut-bolt insertion, these representations enable more efficient policy learning from multimodal feedback and improve accuracy and robustness under external disturbances.

Abstract:
Recently, world-action models (WAMs) have emerged as a powerful framework for robot manipulation. By jointly modeling action generation and future observation prediction, WAMs can act and anticipate the outcomes of the actions. However, existing WAMs typically rely on frozen pretrained encoders trained with reconstruction, which may not produce representations well suited for robot control. In this paper, we tackle the problem of learning physically-grounded, control-relevant representations for WAMs. We propose MDF++, a visuo-tactile world-action model that learns latent representations of multisensory observations end-to-end with a diffusion backbone. While naive joint training can lead to feature collapse, MDF++ introduces a multi-task learning strategy that leverages cross-modal correlations to learn task-relevant representations. We evaluate MDF++ on a suite of contact-rich manipulation tasks, including real-world bolt screwing and nut mating, and show that it outperforms WAMs built on frozen pretrained representations by up to 50%.

Abstract:
Humanoid foundation models are advancing faster than we can evaluate them. While real-world testing is expensive and difficult to reproduce, existing simulation benchmarks focus primarily on table-top or wheeled robots. A scalable and reproducible benchmark for whole-body humanoid loco-manipulation remains an open problem. To this end, we present SIMPLE, a unified simulation testbed for humanoid policy learning and evaluation. SIMPLE couples the accurate contact-rich dynamics of MuJoCo with the photorealistic rendering of IsaacSim. It provides a large-scale environment comprising 60 diverse whole-body tasks, 50 indoor scenes, and over 1,000 object assets. To facilitate scalable data collection, the framework integrates two data generation pipelines: automated trajectory generation via motion planning and a low-latency VR teleoperation interface. We further integrate and benchmark mainstream humanoid policies at scale in SIMPLE, including lightweight imitation networks, large vision-language-action (VLA) models, and recent world action models (WAMs). Our experiments reveal a strong correlation between policy performance in simulation and the real world. Furthermore, we demonstrate that policies trained on data collected in SIMPLE can be transferred zero-shot to physical humanoid robots under similar settings, providing a robust and reproducible foundation for humanoid robotics research.

Abstract:
Humanoid loco-manipulation in unstructured environments is often hindered by egocentric occlusions and the complexity of high-DOF whole-body coordination. To bridge this gap, we present a framework HuG, for Learning Humanoid Whole-Body Affordance-Aware Grasping with Active Perception. Our system enables a robot to autonomously discover and acquire objects through a hierarchical perception-action coordination approach. First, an active perception module manages the robot's viewpoint to resolve visual ambiguity by maintaining a recursive spatial belief of the target. Second, an affordance-aware alignment strategy leverages learned reachability priors to coordinate whole-body posture, ensuring both kinematic feasibility and dynamic stability for optimal manipulation. Finally, the acquisition is performed by a robust whole-body grasping policy distilled entirely from large-scale synthetic data. We demonstrate zero-shot sim-to-real transfer on a humanoid, showing that the coordination of perception and action significantly enhances the reliability and generalization of object acquisition across diverse spatial configurations.

Abstract:
Scaling humanoid whole-body control toward general-purpose deployment requires large human motion corpora and training experience shared across robot bodies. Existing methods usually train one policy per robot, leaving motion experience isolated across embodiments. We introduce X-WBC, a cross-embodiment whole-body control foundation framework for diverse humanoid robots. The framework decouples control into relatively embodiment-independent motion representation and embodiment-dependent physical execution. Human motion serves as a natural carrier of motion semantics, learned from multi-robot rollouts through a shared motion backbone, while lightweight robot-specific modules condition execution on each robot's proprioception, dynamics, and action space. To unify heterogeneous control sources, it aligns full human motion, robot reference motion, and sparse VR motion into human-centered command tokens. A causal Transformer with historical context models temporal structure, enabling shared motion understanding across robots. We evaluate X-WBC in simulation on nine humanoid robots with ablations, and deploy it on multiple real robots with substantially different body sizes. Results show that the learned motion representation transfers across robots while maintaining robust execution. These findings support heterogeneous humanoids as joint data sources for WBC training and provide a real-world deployable foundation model for cross-embodiment humanoid whole-body control.

Abstract:
Learning a general whole-body controller for humanoid robots remains challenging due to the diversity of motion distributions, the difficulty of fast adaptation, and the need for robust balance in high-dynamic scenarios. Existing approaches often require task-specific training or suffer from performance degradation when adapting to new motions. In this paper, we present FAST, a general humanoid whole-body control framework that enables \underlineFast \underlineAdaptation and \underlineStable Motion \underlineTracking. FAST introduces Parseval-Guided Residual Policy Adaptation, which learns a lightweight delta action policy under orthogonality and KL constraints, enabling efficient adaptation to out-of-distribution motions while mitigating catastrophic forgetting. To further improve physical robustness, we propose Center-of-Mass-Aware Control, which incorporates CoM-related observations and objectives to enhance balance when tracking challenging reference motions. Extensive experiments in simulation and real-world deployment demonstrate that FAST consistently outperforms state-of-the-art baselines in robustness, adaptation efficiency, and generalization. See supplementary materials for video demos.

Abstract:
While current Behavior Foundation Models (BFMs) provide robust control priors for humanoids, they only execute pre-defined reference motions. As a result, they are vulnerable to environmental shifts and incapable of reactive whole-body coordination. Naively cascading them with generative motion planners fails to achieve true reactivity, as inevitable tracking discrepancies induce fatal cumulative exposure bias. To bridge this gap, we propose ReactiveBFM, a real-time closed-loop planning-control framework. At its core, we effectively mitigate exposure bias via a scheduled prefix sampling curriculum, forcing the generative planner to actively learn error-recovery behaviors from imperfect physical states rather than ground-truth trajectories. Systematically, to reconcile the severe latency mismatch between auto-regressive planning and high-frequency tracking, we introduce an asynchronous replanning mechanism. Combined with trajectory chunking to temporally ensemble spatial references, our system guarantees spatio-temporally fluid execution without physical jitter. Deployed on the Unitree G1 humanoid, ReactiveBFM demonstrates unprecedented physical agility across a vast repertoire of text-conditioned closed-loop motions. Notably, ReactiveBFM achieves zero-shot moving target reaching, showcasing intricate whole-body coordination and on-the-fly replanning. In sim-to-sim benchmarking under severe perturbations, ReactiveBFM achieves a 93.1% success rate, significantly outperforming cascaded open-loop baselines by 28.6%.

Abstract:
Flapping-Wing Micro Aerial Vehicles (FWMAVs) provide exceptional maneuverability and aerodynamic efficiency but pose significant challenges for onboard control due to nonlinear dynamics and stringent Size, Weight, and Power (SWaP) constraints, as exemplified by a butterfly-inspired robot less than 30 gram. To this end, we present a hierarchical neuromorphic control framework that enables fully onboard, closed-loop flight on a widely available, resource-constrained ESP32 microcontroller with a unit cost of approximately 5. Specifically, our method deploys two lightweight Spiking Neural Networks (SNNs) onboard: one for state estimation from raw sensory feedback and another for control via modulation of a Central Pattern Generator (CPG) for wing actuation. Trained by imitation learning, the system achieves stable pitch and heading angle tracking during untethered real-world flight. Experimental results further reveal that the SNN-based controller reduces latency by 36% (1059us to 680us) and power by 18% (0.033W to 0.027W) for inference compared to the conventional Artificial Neural Network (ANN) baseline, demonstrating the viability of spike-based computation without specialized hardware. To the best of our knowledge, this work constitutes the first demonstration of fully onboard neuromorphic control for autonomous flight of a FWMAV, highlighting the potential of SNNs to enable energy-efficient autonomy under stringent SWaP constraints. Visual abstract: http://bit.ly/4nI8ECY Code: https://anonymous.4open.science/r/Espikify-76E3/

Abstract:
We present HyperDrone, an extremely fast quadrotor simulator that is focused on high-performance batch rendering and is more than 20x faster than existing state-of-the-art batch quadrotor simulators. We design a ray-tracing engine that can take full advantage of modern ray-tracing hardware (ray-tracing cores) and render RGB and depth observations. HyperDrone supports configurable levels of fidelity, including reflections, realistic lighting, motion blur, anti-aliasing, and Physically Based Rendering (PBR). Even in the highest-fidelity settings, HyperDrone achieves millions of Frames Per Second (FPS) on consumer-grade GPUs, while simulating realistic 3D scenes. We demonstrate the merits of HyperDrone by training a vision-based yaw-control policy purely in simulation and transferring it to a real small-scale quadrotor (Crazyflie) with a camera, running completely onboard.

Abstract:
Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present \tau_0-World Model (\tau_0-WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, \tau_0-WM provides two complementary interfaces. First, a video action model jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state. Second, an action-conditioned video simulator rolls out candidate action chunks into multi-view futures with dense task-progress scores. The model is trained on approximately 27,300 hours of real-robot teleoperation, UMI-style interaction, egocentric human videos, and rollout or failure trajectories using modality-specific supervision masks. At inference time, \tau_0-WM uses test-time computation to sample action candidates, rank them with re-denoising consistency, and invoke simulator-based rectification for low-quality candidates. On challenging long-horizon and fine-grained robotic manipulation tasks, this procedure improves average performance over relevant baselines.

Abstract:
World Action Models (WAMs) present a promising paradigm for robotic control via video prediction. However, current WAMs suffer from fundamental spatial bottlenecks: standard text inputs introduce referential ambiguity in cluttered scenes, while unstructured RGB predictions lack semantic grounding and remain biased by task-irrelevant backgrounds. To overcome these limitations, we introduce MaskWAM, an object-centric world-action model. By jointly integrating masks as both explicit inputs and predictions via a unified Mixture of Transformers (MoT), MaskWAM unlocks robust policy generalization. This design provides two key benefits: (1) predicting future masks yields object-centric semantic supervision that suppresses visual noise, significantly enhancing even standard text-conditioned WAMs; and (2) coupling this predictive supervision with first-frame visual prompts, such as target object masks, establishes a precise spatial anchor that substantially reduces language ambiguity. Crucially, as WAMs are inherently vision-driven architectures, direct mask conditioning yields substantially stronger guidance than text alone, establishing a precise and robust paradigm for manipulating unseen objects. Evaluations on LIBERO, RoboTwin, and real-world tasks demonstrate that MaskWAM significantly outperforms baselines in both language-clear and language-ambiguous tasks.

Abstract:
Contact-rich manipulation failures are often detected only after the robot has committed to contact. This is especially limiting in wrist-camera setups: close gripper--object views help observe contact, but a poor approach may already push, miss, slip, or disturb the object before conventional detectors react. We introduce \emphContactGuard, a pre-contact execution monitor for chunked visuomotor policies. Given the policy's planned action chunk, ContactGuard predicts its short-horizon consequence in latent visual space and aborts if the predicted future latent indicates likely failure. Its latent world model is trained from unlabelled robot trajectories to predict compact multi-view visual embeddings under planned actions, avoiding pixel-level video prediction. A lightweight failure probe is then trained from a small labelled set of pre-contact clips. At deployment, ContactGuard anchors prediction before an imminent contact event, rolls the model forward under the policy's own actions, and verifies the predicted post-contact latent. Across real-world contact-rich manipulation tasks, ContactGuard predicts failure more accurately than direct and corrupted-action ablations, and transfers to live robot rollouts as a pre-contact abort signal without modifying the underlying policy.

Abstract:
Action-conditioned video models ("world models") are emerging as a promising foundation for data-driven robot simulators that learn world dynamics directly from data. Yet, the role of training data remains poorly understood in shaping their accuracy and reliability. While existing approaches typically train on human demonstrations or task-specific policy rollouts, they exhibit persistent hallucinations in modeling contact-rich manipulation due to systematic biases and insufficient coverage of the training distribution. To address this gap, we present PlayWorld, a new data paradigm for training robot world models from autonomous play. Rather than relying on manual teleoperation or narrow policy rollouts, PlayWorld enables robots to autonomously explore their environments, capturing a broad spectrum of object dynamics and behavioral modes. The resulting world models can simulate diverse policy behaviors with high physical fidelity. We show that PlayWorld enables fine-grained failure prediction and policy evaluation, achieving up to 40% improvements over human-collected data baselines. Using the learned world model for reinforcement learning yields policies that achieve 65% higher real-world success rates upon deployment. To our knowledge, PlayWorld is the first work to demonstrate autonomous play as an effective training paradigm for video world models.

Abstract:
Imitation learning has emerged as a powerful paradigm for learning visuomotor policies, but its generalisation and stability are limited by the scale and quality of demonstration data needed. A promising direction is to leverage more abundant but heterogeneous data sources, which differ in action space and often lack action labels altogether. Existing co-training approaches that combine heterogeneous data sources rely on heuristic and hand-engineered alignment techniques. In contrast, we argue that action representations should be grounded in prediction: actions that produce the same effect on the environment should share the same representation, regardless of their sources. To this end, we instantiate this principle by using a grounded latent-action world model (GLAM), a pair of generative models with a shared latent action space across data sources that is grounded by predicting future observations consistently across sources. This latent action space is used to train downstream behavioural cloning (BC) policies which map observations to latent actions and decode them back to robot actions, providing a paradigm for learning from heterogeneous data. Empirically, we demonstrate that GLAM successfully learns an aligned latent action space that facilitates action transfer across data sources with and without action labels. Across five manipulation tasks in simulation and in the real world, GLAM-aligned policies significantly outperform BC baselines and prior latent-action methods, achieving an average of +48% improvement in task success rate with the same data-scarce setting.

Abstract:
Robots performing long-horizon visual manipulation observe high-dimensional images, but successful plans depend on action-relevant facts: what can be done now and what changes afterward. A useful planning representation should discard irrelevant visual details while preserving action applicability and effects. Classical task planners exploit this structure through symbolic operators with preconditions and effects, but obtaining such representations from raw visual experience remains challenging. We study a visual task-planning setting in which a robot receives only image transitions: the current image, executed high-level action, and the resulting image. At test time, given a start image and a goal image, the robot must produce a sequence of high-level actions that reaches the goal. To address this problem, we introduce STRIPS-WM, a framework for learning image-grounded STRIPS-style world models directly from visual transitions. STRIPS-WM first induces a finite abstract transition graph from images, then learns latent binary predicates and one grounded propositional operator per action label. The learned operators form a symbolic action model with sparse preconditions and add/delete effects. Finally, the learned predicates are distilled into a visual encoder, enabling classical planning directly from novel start and goal images. Experiments on visual rearrangement tasks show that STRIPS-WM improves image-to-plan success over the tested visual rollout, latent graph-search and latent-symbolic baselines.

Abstract:
Direct transfer from human demonstration to learnable robot action is a crucial step towards scalable whole-body mobile manipulation. While human data scales better than mobile teleoperation, it requires overcoming significant embodiment gaps. Existing retargeting methods yield imprecise or inconsistent solutions, causing action multi-modality that prevents supervised policies from reliably converging. We present Whole-body-Aware Retargeting from human Pose (WARP), an offline pipeline that explicitly models embodiment differences to extract precise, unique whole-body actions. WARP leverages a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for exact end-effector tracking while preserving whole-body structural intent. Paired with lazy mobile-base control, it extracts accurate, consistent robot trajectories. Evaluations show WARP provides highly reliable data for open-loop real-world replay. To our knowledge, WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations, eliminating the need for human-in-the-loop teleoperation action data. More details on https://warp-retarget.github.io/

Abstract:
Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand-crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand-specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration-free retargeting method for intuitive dexterous teleoperation across human-like dexterous hands. AnyDexRT combines self-supervised fingertip correspondence learning with few-shot human guidance to anchor the mapping in task-relevant regions, and further refines pinch-related poses using a contact classifier. Experiments on diverse dexterous hands and real-world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods.

Abstract:
Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion into kinematic references, then train policies via reinforcement learning (RL) to track them. But how does this recipe transfer to dexterous manipulation? The answer is not obvious, as manipulation involves complex, contact-rich dynamics, and affords less margin for tracking error. We present \textscRegrind, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration. \textscRegrind retargets human hand-object motion into a reference that preserves hand-object spatial and contact relationships, trains a residual RL policy in simulation to track object-centric keypoints along that reference, and transfers zero-shot to hardware with careful system identification. The resulting policies produce fluid, human-like behavior on two different multi-fingered hands across contact-rich tool-use tasks, including operating scissors and turning a screwdriver. Through systematic hardware experiments, we identify and analyze the key factors that govern sim-to-real transfer in dexterous manipulation, offering practical guidance for retargeting-based learning in contact-rich settings.

Abstract:
Vision--Language--Action (VLA) fine-tuning supplies dense per-frame visual--action supervision but leaves implicit which part of the scene the demonstrator treated as task-relevant at each timestep---what we call the instruction's \emphtemporal grounding. We introduce \empheye-tracker-supervised gaze prompting to supply this signal: an HMD eye tracker records the operator's fixation during VR teleoperation, a lightweight predictor (～50M parameters) learns to estimate fixation from RGB and language, and the prediction is rendered as a crosshair on the image input of an unchanged \pi_0 policy---with no eye tracker or architectural modification at deployment. Across six real-world bimanual manipulation tasks, gaze prompting improves over standard fine-tuning on every task, raising mean success rate from 26.3% to 56.0%. Two isolations explain the gains: (i)~with the gaze signal fixed, the input-pixel route carries two signatures of temporal grounding to the policy---instruction-conditional attention and per-frame stage awareness over long horizons---that an auxiliary attention loss on the same gaze does not; and (ii)~under the same route, real human gaze outperforms detection and random cues, with the gap explained by a third signature, anticipation, that engineered cues structurally cannot supply.

Abstract:
Simulation offers a scalable and low-cost way to enrich vision-language-action (VLA) training, reducing reliance on expensive real-robot demonstrations. However, most sim-real co-training methods rely on supervised fine-tuning (SFT), which treats simulation as a static source of demonstrations and does not exploit large-scale closed-loop interaction. Consequently, real-world gains and generalization are often limited. In this paper, we propose an RL-based sim-real Co-training (RL-Co) framework that leverages interactive simulation while preserving real-world capabilities. Our method follows a generic two-stage design: we first warm-start the policy with SFT on a mixture of real and simulated demonstrations, then fine-tune it with reinforcement learning in simulation while adding an auxiliary supervised loss on real-world data to anchor the policy and mitigate catastrophic forgetting. We evaluate our framework on four real-world tabletop manipulation tasks using two representative VLA architectures, OpenVLA and \pi_0.5, and observe consistent improvements over real-only fine-tuning and SFT-based co-training, including +24% real-world success on OpenVLA and +20% on \pi_0.5. Beyond higher success rates, RL co-training yields stronger generalization to unseen task variations and substantially improved real-world data efficiency, providing a practical and scalable pathway for leveraging simulation to enhance real-robot deployment.

Abstract:
Diffusion and flow policies have recently demonstrated remarkable performance in robotic applications by accurately capturing multimodal robot trajectory distributions, especially in the context of vision language action (VLA) models. However, high quality policy performance also requires fast inference and high quality demonstrations, which are often hard to get. Lack of these leads to suboptimal policy behaviors and failure under distribution shifts. In this work we address the problem of fine-tuning and accelerating suboptimal flow-based policies using the robot's experience through RL post-training. We introduce Optimal Transport Q-Learning (OTQL), a new methodology for finetuning flow policies using advantage weighted conditional optimal transport flow matching. OTQL can finetune and accelerate flows with an interaction budget of 50-60 episodes while avoiding computationally expensive distillation in simulation and real-world robot tasks. Our results show that OTQL post-trains flow policies using the robot's own experience, increasing average success percentage of single-task policies from 36% to 86% and of a pre-trained VLA from 38% to 76% while reducing the number of inference steps per action generation by 70%.

Abstract:
Recent advancements in foundational models, such as large language models and world models, have greatly enhanced the capabilities of robotics, enabling robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation to generate accurate action-video pairs while maintaining real-world consistency. Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data to generate diverse, large-scale training datasets that cover a broader spectrum of real-world scenarios. We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments. Through extensive experiments, we demonstrate that our method significantly reduces the sim2real domain gap, resulting in higher success rates in real-world policy model training. Our approach offers a scalable solution for generating robust training data and bridging the gap between simulated and real-world robotics.

Abstract:
This work introduces Spatial Annotations from Robot Demonstrations with Reliability Calibration (SPARC), a risk-aware framework that automatically labels robot demonstrations with structured spatial annotations and assigns each annotation a reliability score. Structured spatial annotations, such as bounding boxes, object trajectories, and manipulation phase labels, benefit a broad range of robotics applications, from training grounded robot policies and embodied foundation models to motion planning and hierarchical task composition. Existing automated pipelines generate such annotations at scale but provide no reliable quality signal: detector confidence is poorly calibrated for annotation correctness, forcing a choice between accepting noisy labels or discarding useful samples. In contrast, SPARC leverages the spatio-temporal structure inherent to robot tasks to generate a reliability signal, reducing noisy labels while retaining more useful samples. We further introduce Interaction-Aware Bench (IA-Bench), a benchmark that measures model accuracy in grounding the locations of interacted objects in robot demonstrations. On 1.7k human-annotated demonstrations spanning diverse embodiments and scenarios, SPARC significantly outperforms detection-only baselines in object localization accuracy while retaining three times more samples at high-precision operating points. Our experiments demonstrate that models finetuned on our annotations achieve state-of-the-art results on object-grounding and pointing benchmarks among similarly sized models, while remaining competitive on broader spatial-reasoning suites without any manually verified or annotated training data. Furthermore, policies trained on SPARC-generated annotations significantly outperform baselines in cluttered, visually ambiguous real-world scenes. Code, data, and models are publicly available at [https://sparc-labeling.github.io](https://sparc-labeling.github.io)

Abstract:
We present RoboLineage, an agent-native data lifecycle governance system for robot policy iteration. Modern robot policies are improved through repeated data collection, review, retraining, evaluation, and release decisions, but the evidence connecting these steps is often scattered across local tools, scripts, and expert memory. RoboLineage makes this lifecycle explicit. It represents rollouts, reviews, dataset decisions, training runs, policy metadata, evaluations, deployment recommendations, and next-collection plans as typed lineage artifacts. Agents interpret embodied rollout evidence, adapt accepted data to existing training stacks, maintain data health, and summarize cross-iteration state under explicit artifact boundaries. In real-robot manipulation workflows, RoboLineage makes routine policy iteration faster and more auditable while maintaining downstream policy performance. We will open-source RoboLineage as a lightweight lifecycle layer for different robot embodiments and training families.

Abstract:
Tendon drives paired with soft muscle actuation enable faster and safer robots while potentially accelerating skill acquisition. Still, these systems are rarely used in practice due to inherent nonlinearities, friction, and hysteresis, which complicate modeling and control. So far, these challenges have hindered policy transfer from simulation to real systems. To bridge this gap, we propose a sim-to-real pipeline that learns a neural network model of this complex actuation and leverages established rigid body simulation for the arm dynamics and interactions with the environment. Our method, called Generalized Actuator Network (GenAN), enables actuation model identification across a wide range of robots by learning directly from joint position trajectories rather than requiring torque sensors. Using GenAN on PAMY2, a tendon-driven robot powered by pneumatic artificial muscles, we successfully deploy dynamic but precise goal-reaching, ball-in-a-cup, and table tennis policies, trained entirely in simulation. To the best of our knowledge, this result constitutes the first successful sim-to-real transfer for a four-degrees-of-freedom muscle-actuated robot arm.

Abstract:
Exploiting symmetry in robot policy learning requires knowing how a symmetry group operates on observations and actions, an assumption that often breaks in partially observable MDPs (POMDPs) with pixel observations, where this operation is nonlinear and difficult to specify by hand. Existing approaches sidestep this challenge using environment-specific preprocessing, limiting portability across tasks and observation modalities. We introduce MIRROR, a framework that discovers how a symmetry group operates jointly on observations and actions in POMDPs directly from offline interaction data, while explicitly enforcing the group axioms. On four Robosuite tabletop tasks, MIRROR attains the lowest aggregate error on three of four symmetry-quality metrics, including a 61% reduction in compatibility error over LaLiGAN, the strongest GAN baseline on that metric. The discovered operations instantiate an equivariant behavior cloning (BC) policy that outperforms vanilla BC on all four tasks, with per-task gains up to 9.66%, and the same framework transfers zero-shot to a real Kinova GEN3 arm under a different observation modality. Project page: https://pomdp-mirror.github.io/review/.

Abstract:
Learning generalist policies capable of accomplishing a plethora of everyday tasks remains an open challenge in dexterous manipulation. In particular, collecting large-scale manipulation data via real-world teleoperation is expensive and difficult to scale. While learning in simulation provides a feasible alternative, designing multiple task-specific environments and rewards for training is similarly challenging. We propose Dex4D, a framework that instead leverages simulation for learning task-agnostic dexterous skills that can be flexibly recomposed to perform diverse real-world manipulation tasks. Specifically, Dex4D learns a domain-agnostic 3D point track conditioned policy capable of manipulating any object to any desired pose. We train this 'Anypose-to-Anypose' policy in simulation across thousands of objects with diverse pose configurations, covering a broad space of robot-object interactions that can be composed at test time. At deployment, this policy can be zero-shot transferred to real-world tasks without finetuning, simply by prompting it with desired object-centric point tracks extracted from generated videos. During execution, Dex4D uses online point tracking for closed-loop perception and control. Extensive experiments in simulation and on real robots show that our method enables zero-shot deployment for diverse dexterous manipulation tasks and yields consistent improvements over prior baselines. Furthermore, we demonstrate strong generalization to novel objects, scene layouts, backgrounds, and trajectories, highlighting the robustness and scalability of the proposed framework.

Abstract:
Proactive dynamic grasping requires robots to predict the future state at which physical contact will occur, rather than reacting to the currently observed object pose. But what future must the robot predict, and at what level of abstraction? We argue that contact-relevant rigid-body state in SE(3) is a compact task-sufficient abstraction for this problem: it directly determines where and when the robot should establish contact, without requiring visually complete future scenes. This insight leads to a lightweight object-centric forecast-then-select framework that predicts future rigid-body trajectories from short pose histories and uses a reachability-aware interception selector to choose feasible contact states under kinematic and timing constraints. Trained with randomized physics simulation over primitive shapes, the forecaster transfers to real objects once their metric geometry and SE(3) state histories are recovered. Experiments across simulation and real-world dynamic grasping tasks show that our framework improves the average success rate from \(24.5%\) to \(62.0%\) in simulation, and from \(15.3%\) to \(67.3%\) in real-world catching, conveyor, and turntable interception. The SE(3) forecaster runs in \(15\,\mathrmms\), orders of magnitude faster than dense visual future prediction, enabling real-time contact decision making. Our results suggest that effective proactive grasping does not require imagining the entire future scene. Instead, predicting the compact physical future that determines contact provides an effective and practical abstraction for real-time dynamic grasping.

Abstract:
Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural language provides an intuitive way for humans to communicate such preferences. Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than language-conditioned policies.

Abstract:
Contact-rich manipulation, requiring robots to regulate not only motion but also how they yield to external forces, has emerged as the next frontier for Vision-Language-Action (VLA) models. However, existing VLAs output purely kinematic commands, degrading performance on real-world contact-rich tasks. In this paper, we introduce CompVLA, a unified VLA framework that jointly predicts motion and stiffness matrix from RGB and language inputs. Our approach augments the conventional architecture with a dedicated Compliance Expert, which outputs time-varying stiffness and virtual displacement profiles executed via geometric impedance control. We demonstrate that CompVLA achieves the highest average success rate across diverse contact-rich tasks, outperforming both vanilla and compliance-aware VLA baselines, with ablations confirming each component is essential.

Abstract:
Robot manipulation often depends on sensory data beyond vision, especially in contact-rich tasks where force, tactile, or audio feedback reveals interaction states not directly visible from images. Yet such modalities are hardware- and task-specific, and large multisensory datasets remain scarce, making it impractical to pretrain policies with every sensor they may encounter. We study multi-sensory continual learning: adapting a pretrained robot policy to new tasks with newly introduced modalities while preserving performance under the original sensor suite. We propose MuSe, which integrates limited multisensory data into pretrained vision-action policies using token-level multi-stage fusion, multisensory future prediction, and episodic recall on pretraining data. We instantiate MuSe by adding force-torque sensing to a pretrained vision-only policy and evaluate it on real-world manipulation tasks. Experiments show that MuSe not only performs well on contact-rich finetuning tasks but also yields performance gains on pretraining tasks - suggesting that a modest multisensory dataset can enhance general model capability beyond the finetuning distribution.

Abstract:
Humanoid perceptive locomotion has made significant progress and shows great promise, yet achieving robust multi-directional locomotion on complex terrains remains underexplored. To tackle this challenge, we propose RPL, a two-stage training framework that enables multi-directional locomotion on challenging terrains, and remains robust with payloads. RPL first trains terrain-specific expert policies with privileged height map observations to master decoupled locomotion and manipulation skills across different terrains, and then distills them into a transformer policy that leverages multiple depth cameras to cover a wide range of views. During distillation, we introduce two techniques to robustify multi-directional locomotion, depth feature scaling based on velocity commands and random side masking, which are critical for asymmetric depth observations and unseen widths of terrains. For scalable depth distillation, we develop an efficient multi-depth system that ray-casts against both dynamic robot meshes and static terrain meshes in massively parallel environments, achieving a 5x speedup over the depth rendering pipelines in existing simulators while modeling realistic sensor latency, noise, and dropout. Extensive real-world experiments demonstrate robust multi-directional locomotion with payloads (2kg) across challenging terrains, including 20° slopes, staircases with different step lengths (22cm, 25cm, 30cm), and 25cmx25cm stepping stones separated by 60cm gaps.

Abstract:
Achieving highly dynamic parkour with humanoid robots on complex terrains unseen during the pre-training stage remains a challenge in robotics. Although general locomotion policies demonstrate capabilities across broad terrain distributions, they often struggle with arbitrary and highly challenging environments. To overcome this limitation, we propose a real-to-sim-to-real framework that leverages rapid test-time training (TTT) on novel environments, significantly enhancing the robot's capability to traverse extremely difficult terrains. We adopt a two-stage end-to-end learning paradigm: a policy is first pre-trained on diverse procedurally generated terrains, followed by rapid fine-tuning on high-fidelity meshes reconstructed from real-world captures. Specifically, we develop an efficient, feed-forward, and high-fidelity geometry reconstruction pipeline using RGB-D inputs, ensuring both speed and quality during test-time training. We demonstrate that TTT-Parkour empowers humanoid robots to master complex obstacles, including wedges, stakes, boxes, trapezoids, and narrow beams. The entire pipeline, including capture, reconstruction, and test-time training, takes less than 10 minutes on most tested terrains. Extensive experiments show that the policy after test-time training exhibits robust zero-shot sim-to-real transfer capability.

Abstract:
Humanoid motion control has witnessed significant breakthroughs in recent years, with deep reinforcement learning (RL) emerging as a primary catalyst for achieving complex, human-like behaviors. However, the high dimensionality and intricate dynamics of humanoid robots make manual motion design impractical, leading to a heavy reliance on expensive motion capture (MoCap) data. These datasets are not only costly to acquire but also frequently lack the necessary geometric context of the surrounding physical environment. Consequently, existing motion synthesis frameworks often suffer from a decoupling of motion and scene, resulting in physical inconsistencies such as contact slippage or mesh penetration during terrain-aware tasks. In this work, we present MeshMimic, an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video. By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects. We introduce an optimization algorithm based on kinematic consistency to extract high-quality motion data from noisy visual reconstructions, alongside a contact-invariant retargeting method that transfers human-environment interaction features to the humanoid agent. Experimental results demonstrate that MeshMimic achieves robust, highly dynamic performance across diverse and challenging terrains. Our approach proves that a low-cost pipeline utilizing only consumer-grade monocular sensors can facilitate the training of complex physical interactions, offering a scalable path toward the autonomous evolution of humanoid robots in unstructured environments.

Abstract:
Vision-based perceptive locomotion has enabled legged robots to achieve agile and terrain-aware locomotion across complex environments. However, the reliability of such systems heavily depends on visual observations, which are inherently susceptible to occlusion, reflective surfaces, and illumination variations, often resulting in catastrophic locomotion failures under degraded perception conditions. In this paper, we present PROOF (Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion), a perceptive locomotion framework that enables robust locomotion under severe visual disturbances. Our method adopts a two-stage teacher–student learning paradigm that explicitly models perception reliability: the teacher learns both perception-driven and proprioceptive fallback behaviors, while the student predicts exteroceptive features with uncertainty and suppresses them under out-of-distribution inputs. We evaluate our method on a Unitree Go2 quadruped equipped with a RealSense D435 depth camera in real-world settings. Under normal visual conditions, our method enables the robot to successfully perform high-dynamic parkour maneuvers, including traversing large gaps, elevated platforms, and narrow beam terrains. Under severe visual degradation, such as heavy occlusion and strong illumination changes the robot continues to maintain stable and robust locomotion performance.

Abstract:
Humanoid locomotion across complex terrain demands forward-looking exteroception to anticipate obstacles, yet this signal is unreliable in real-world deployment, failing partially and intermittently. Existing perceptive policies often assume that depth observations remain clean and in-distribution, while recent attempts to unify perceptive and blind control typically route or switch between separate sub-policies, leaving recoverable information in partially corrupted depth unexploited. We instead propose CAP, a single-stage humanoid locomotion policy that recovers this signal with a perceptive world-model encoder trained as a learned denoiser to reconstruct clean depth from a corrupted input, together with a co-active proprioceptive variational encoder that supplies depth-free body-state information. A coupled training recipe pairs a depth-noise curriculum on the world-model input with world-model feature dropout on the policy-facing latent, exposing the policy to failures across the entire perception-quality spectrum. In simulation, CAP matches or improves upon perceptive baselines when depth remains informative, and degrades more smoothly than a binary-switching baseline as perception worsens. On the Unitree G1, controlled trials and indoor-outdoor deployments demonstrate perception-robust locomotion under intermittent occlusion, real-sensor corruption, and outdoor depth artifacts.

Abstract:
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging. Unlike fixed-base or wheeled manipulators, humanoids must coordinate locomotion, posture, gaze, and dual-arm manipulation over a high-dimensional whole-body action space, where inaccurate base or waist motions can easily induce unstable and aggressive upper-body compensation. We propose HAF-VLA, a hierarchical action-flow framework for humanoid VLA control. Built upon a pretrained flow-matching VLA, HAF-VLA inherits the vision-language prefix encoder and action expert weights, and adapts them to humanoid whole-body control with learnable stage embeddings and cross-stage KV-cache conditioning. Instead of generating the full action chunk in a single pass, HAF-VLA progressively denoises a shared whole-body action space: first locomotion and head commands, then waist refinement, and finally full-body manipulation. To preserve structural dependencies across stages, we introduce a cross-stage KV cache conditioning mechanism, enabling subsequent action generation stages to refine and augment their outputs based on the action representations from preceding stages. Experiment results across six real-world tasks demonstrate that our framework significantly enhances the performance of existing models in humanoid loco-manipulation tasks, demonstrating an effective pathway for adapting strong mobile-manipulation VLAs to humanoid whole-body control.

Abstract:
Despite impressive agility in locomotion, humanoid forceful manipulation remains limited by the lack of scalable, easy to train compliant force control in reinforcement learning (RL). We introduce CHIP (\underlineCompliant \underlineHumanoid natural control through h\underlineIndsight \underlinePerturbation), a plug-and-play framework that bridges this gap. Our core insight is that a demonstrated motion can be reinterpreted, in hindsight, as a compliant response to an external perturbation. By adjusting only sparse motion goals in response to perturbations while leaving the dense reference trajectories untouched, CHIP allows compliant interaction behaviors to emerge naturally without degrading motion tracking rewards. A single CHIP policy successfully executes forceful interaction tasks (e.g., wiping, cart pushing, door opening) alongside highly agile behaviors (e.g., dancing, running), and seamlessly extends to multi-agent cooperative transport.

Abstract:
Reinforcement learning (RL) for locomotion frequently converges to locally optimal but undeployable behaviors, such as vibrating limbs or scooting on the torso, that maximize return without producing a usable gait. We present MPC-Injection, a low-overhead method that steers RL toward a designer-preferred gait by inserting transitions into the replay buffer from a model predictive controller solving the same Markov decision process. Unlike reward shaping, MPC-Injection does not require redesigning the task reward, and unlike adversarial imitation learning, it adds no discriminator, no kinematic retargeting, and no auxiliary objective. Instead, the controller’s preferred behavior is transferred to the policy purely through the replay state distribution. On a 2D walker in simulation and with sim-to-real evaluation on a Go2 quadruped, we show that MPC-Injection drives the policy into the controller’s behavior basin using a one to two-term task reward, producing gaits qualitatively comparable to those of reward shaping with twenty-one tuned terms and of adversarial motion priors without their discriminator and retargeting overhead. We further analyze how the injected transitions bias actor-critic updates toward controller-visited states, allowing the policy to learn behaviors that pure RL may fail to reach under simple reward functions.

Abstract:
Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in closed-loop (CL) deployment. In this paper, we unveil the root causes of this systemic failure and propose a practical remedy. Specifically, we demonstrate that OL policies suffer from Observational Domain Shift and Objective Mismatch. We show that while the former is largely recoverable with adaptation techniques, the latter creates a structural inability to model complex reactive behaviors, which forms the primary OL-CL gap. We find that a wide range of OL policies learn a biased Q-value estimator that neglects both the reactive nature of CL simulations and the temporal awareness needed to reduce compounding errors. To this end, we propose a Test-Time Adaptation (TTA) framework that calibrates observational shift, reduces state-action biases, and enforces temporal consistency. Extensive experiments show that TTA effectively mitigates planning biases and yields superior scaling dynamics than its baseline counterparts. Furthermore, our analysis highlights the existence of blind spots in standard OL evaluation protocols that fail to capture the realities of closed-loop deployment.

Abstract:
High-speed off-road autonomy requires precise closed-loop control for a target vehicle while remaining robust across changing terrains. Recent forward kino-dynamic prediction foundation models suggest a promising path: start from a generalist model and specialize it to the target platform. However, effective specialization remains challenging: it often requires substantial real-world data, and models adapted to one setting can still overfit to specific terrains or driving regimes. We present OptCar (Optimum Car), a recipe for bridging the gap from generalist to specialist FKD models that preserves cross-terrain generalization and optimizes performance for a specific vehicle. OptCar fine-tunes a generalist model using limited real-world data together with targeted synthetic rollouts from environment-specific system identification. Additionally, OptCar introduces a history-conditioned dynamics adaptation module that encodes recent state-action observations into a dynamics context token. In closed-loop model predictive control (MPC) experiments across three terrains and an out-of-distribution cart-pulling task, we report the 6 m/s regime, the highest speed we evaluate and the setting in which slip dominates tracking error. On vegetation and dirt, the most slip-diverse terrain, OptCar cuts 6 m/s trajectory tracking error by roughly 55% relative to a fine-tuned AnyCar baseline, and it stays the most accurate even when an unseen cart payload changes the dynamics. With only 5 minutes of real data per terrain, OptCar is competitive with a specialist trained on 30 minutes of real road data on road, and substantially outperforms it once the terrain changes.

Abstract:
Large scale representation learning has produced public foundation models that provide strong general purpose visual features.However, their pretraining objectives are not designed for robot representation learning, so the resulting embeddings tend to emphasize global semantics rather than the temporally sensitive representations required for robot control. Training robot models from scratch is also undesirable due to the limited scale and high collection cost of robotics data, as well as the substantial computational burden of pre-training. In this paper, we propose SPARK (Simple Post-training for Adapting pRetrained Knowledge), a post training method that adapts foundation models to robot control. SPARK is built on two principles: dynamics-aware abstraction, which encourages the encoder to derive compact features that preserve information essential for understanding temporal changes and action-relevant scene structure, and knowledge preservation, which aligns patch-level representations with those of the original foundation model to retain useful pretrained semantics.These objectives yield compact visual state representations that remain semantically meaningful while preserving useful prior knowledge. Experiments across multiple robotics benchmarks show that SPARK consistently improves success rates and generalization over vanilla foundation models, and further demonstrate that these gains transfer to real-world robot manipulation.

Abstract:
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. Our method allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework defined by a dual-level flow supervision design. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot’s Tool-Center-Point (TCP) trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.

Abstract:
Generalization remains a central bottleneck for vision-language-action (VLA) models: under distractors, appearance shifts, and semantically similar tasks, the policy must often infer local execution details from coarse instructions while also deciding which parts of the image matter for control. We present S2 (See Less, Specify More), a framework for improving VLA generalization by training the executor under a cleaner interface. Specify More preserves the original instruction as a stable high-level goal while relabeling each trajectory into refined trajectory- and subtask-level language that disambiguates the current execution mode. Unlike native attention, See Less imposes an explicit visual evidence budget, training the executor to act from task-sufficient evidence rather than unconstrained visual context, without any region or mask annotation. This interface lets the executor follow detailed guidance without relying on distracting visual patches or resolving avoidable ambiguity on its own, and it remains compatible with off-the-shelf VLM planners through in-context learning. Across our main evaluation settings, S2 improves overall generalization metrics by changing the executor's learning problem: coarse instructions induce avoidable supervision aliasing, goal-preserving local guidance outperforms instruction replacement in our main ablations, and explicit evidence budgeting reduces dependence on broad visual context beyond efficiency considerations. Across eight real-robot tasks on TX-G2 (an AgiBot G2-compatible variant) and HSR, S2 raises mean subtask success from 54.2% to 79.0% over pi0.5. Together, these results suggest that VLA generalization improves when the executor is trained to act from informative local guidance and task-sufficient visual evidence, rather than recovering both from weak supervision.

Abstract:
Focusing on spatially localized, control-relevant visual cues has been shown to improve data efficiency in visuomotor policy learning by reducing the need to model task-irrelevant visual variation. Existing methods often impose this focus through input preprocessing, such as cropping control- or object-centric regions in RGB images or point-clouds. However, it remains underexplored whether such localized features can be exposed directly from commonly used convolutional neural network (CNN) encoded features. In this paper, we show that intermediate CNN features preserve useful localized visual context, but existing pooling methods fail to aggregate it effectively. We introduce FocusPool, an attention pooling module that selectively aggregates intermediate visual features according to their relevance to the robot's current proprioceptive context. The resulting pooled representation captures task-progressive, control-relevant local information and is used directly for policy learning. Across simulation and real-world experiments, FocusPool improves success rates over pooling and explicit local focus methods by 36.2% and 41.2%, with training only 5.8% of encoder parameters.

Abstract:
Vision-language-action (VLA) models have shown great promise as generalist policies for a large range of relatively simple tasks. However, they demonstrate limited performance on more complex tasks, such as those requiring complex spatial or semantic understanding, manipulation in clutter, or precise manipulation. We propose OmniGuide, a flexible framework that improves VLA performance on such tasks by leveraging versatile sources of guidance, such as 3D foundation models, semantic-reasoning VLMs, and human pose models. We show how to naturally express many kinds of guidance as differentiable energy functions with task-specific attractors and repellers located in 3D space, that influence the sampling of VLA actions. In this way, OmniGuide enables complementary task-relevant guidance to improve a VLA model's performance on challenging tasks. Extensive experiments in both simulation and real-world environments, across diverse sources of guidance like collision safety and human demonstration, prove that OmniGuide enhances the performance of state-of-the-art generalist policies (e.g., \pi_0.5, GR00T N1.6) significantly in terms of both success and safety rates. Critically, our unified framework matches or surpasses the performance of prior methods designed to incorporate specific sources of guidance into VLA policies.

Abstract:
Humans understand and interact with the real world by relying on diverse physical feedback beyond visual perception. Motivated by this, recent approaches attempt to incorporate physical sensory signals into Vision-Language-Action models (VLAs). However, they typically focus on a single type of physical signal, failing to capture the heterogeneous and complementary nature of real-world interactions. In this paper, we propose MoSS, a modular sensory stream framework that adapts VLAs to leverage multiple sensory signals for action prediction. Specifically, we introduce decoupled modality streams that integrate heterogeneous physical signals into the action stream via joint cross-modal self-attention. To enable stable incorporation of new modalities, we adopt a two-stage training scheme that freezes pretrained VLA parameters in the early stage. Furthermore, to better capture contact interaction dynamics, we incorporate an auxiliary task that predicts future physical signals. Through extensive real-world experiments, we demonstrate that MoSS successfully augments VLAs to leverage diverse physical signals (i.e., tactile, force, and torque), integrating multiple signals to achieve synergistic performance gains. The \text\hrefhttps:\//corl-moss.github.io/project page is available.

Abstract:
Similar to the natural capabilities of humans to sequentially learn new tasks, robots with Vision-Language-Action (VLA) model should possess lifelong learning ability to learn a new task when deployed for open-world environment. However, most recently-proposed lifelong learning models aim to effectively learn the current task (plasticity) or maintain high accuracy on the previous tasks (stability), the plasticity-stability trade-off remains largely unsolved in robot manipulation models. To address this fundamental challenge, we propose a cache-efficient lifelong Vision-Language-Action learning framework for robotic manipulation (i.e., LIFELONGVLA), which alleviates plasticity-stability trade-off with a dual-timescale adaptation mechanism while achieving low-cost robotic deployment with a cache-efficient replay strategy. More concretely, we propose a dual-timescale LoRA gating module to decompose VLA adaptation into two lightweight pathways: a short-term adapter for plasticity and a long-term adapter for stable consolidation. These pathways are integrated via a task-aware gate, which could enable explicit control of the plasticity–stability trade-off. In the skill replay phase, a cache-efficient stochastic replay strategy is proposed to preserve more balanced retention signals without full-trajectory storage. To the end, experiments show that LIFELONGVLA outperforms existing baselines, demonstrating efficient skill expansion, robust retention of learned manipulation behaviors, and less reliance on retraining reliance for real-world deployment on xArm robot.

Abstract:
Large language models (LLMs) and vision-language models (VLMs) have significantly advanced zero-shot task planning for embodied agents. However, most LLM- and VLM-driven methods struggle to generate safe high-level actions due to a lack of physical risk awareness, particularly under partial observability, where hazards lie outside the immediate field of view. To address this challenge, we propose a novel safe task-planning framework, SafeMem, which constructs and maintains a long-term semantic graph memory of the open and dynamic environment. Based on egocentric observations, the proposed framework incrementally accumulates knowledge about surrounding objects and their relationships with a graph. Then, an LLM-based risk predictor evaluates candidate actions using the graph memory, triggering a conservatism-modulated replanning loop with explanations for detected hazards. Extensive experiments on the IS-Bench benchmark and a real-world robot platform demonstrate that the SafeMem framework substantially improves safe success rates compared to state-of-the-art VLM-driven task planners.

Abstract:
Video-world-model policies learn action-relevant representations by predicting future observations. However, they condition on only a short observation window, which renders long-horizon manipulation non-Markovian when the correct action depends on earlier events that are no longer visible. We present MemoryVAM, an episodic memory mechanism for video-world-model policies. We employ a Recap-Cue (RC) module, in which a Perceiver-based Recap Compressor maps per-frame CLIP embeddings into compact memory tokens, and a lightweight Cue Gate estimates task completion from memory and language. These tokens are injected into both the video backbone and the action decoder, aligning policy imagination with episode progress and conditioning actions on history. Our model trains the memory module with video prediction, a delta-reconstruction auxiliary loss, and episode-boundary supervision, requiring no per-frame progress labels. The same mechanism applies to UNet and Diffusion Transformer (DiT) backbones by changing only the cross-attention injection interface. On LIBERO-Mem, our model improves average success from 5% to 42.5%. On real robots, it achieves 78.3% success on counting tasks, 80.0% on spatial recall, and 75.0% on sequential tracking.

Abstract:
Safe human--robot collaboration requires more than visual description: a monitor must determine whether the robot body is safely separated, already colliding with the scene or a person, or about to collide. We call this capability \emphcollision grounding: binding visual observations to robot body geometry, camera viewpoint, scene layout, human proximity, and temporal motion in order to infer present and imminent contact. We introduce TouchSafeBench, a physics-grounded benchmark for evaluating collision grounding in vision-language models (VLMs). Built in Habitat~3.0, TouchSafeBench contains 2,940 simulated indoor co-presence episodes across social navigation and social rearrangement, with synchronized multi-view RGB-D observations, top-down trajectory maps, calibrated camera metadata, and simulator-derived contact labels. We study two deployment-facing tasks: classifying the current safety state and warning about imminent collision before contact. Across three frontier or robotics-oriented VLMs and nine visual representations, current models remain far from reliable: the best average Macro-F1 stays below 50%, explicit depth is not automatically transformed into robot-body collision evidence, and robot--scene contact is consistently harder than human-contact risk. TouchSafeBench reveals a central limitation of embodied VLMs: visual fluency does not imply physical accountability. Reliable robot safety monitors will need representations that explicitly bind viewpoint, robot morphology, metric geometry, and future collision. We will release the benchmark upon acceptance.

Abstract:
Fielding socially competent robots requires joint reasoning over spatial and social information. While current models excel at either spatial reasoning (e.g., modeling physical dynamics) or social reasoning (e.g., parsing text-based interactions), few natively unify both. Moreover, data scarcity in Human-Robot Interaction (HRI) makes training large models from scratch challenging. To bridge this gap, we introduce STARS: SpatioTemporal Autoencoded Representations for Social-interaction, a method for learning event-level relational representations from HRI datasets. Rather than training task-specific models, STARS models fixed-duration interaction scenarios as spatiotemporal graphs. A self-supervised message-passing graph neural network autoencoder then compresses these physical dynamics into a compact latent space for each node and edge. This injects strong relational inductive bias to naturally capture the underlying social situation. We evaluate STARS' representations on the SEAN Together dataset, which provides VR-based navigation data annotated with subjective human perceptions, and the SocialNav-SUB benchmark for social scene understanding. Linear probing demonstrates that STARS achieves highly competitive performance on perception prediction and pedestrian action classification tasks. Specifically, STARS exhibits excellent data efficiency, achieving an F_1-score of 0.635 on predicting perceived competence using just 1% of labeled data, whereas baselines require up to ten times as much. Furthermore, qualitative analysis of the learned latent space reveals that STARS naturally encodes high-level social semantics, demonstrating cohesive clustering of distinct human action intents without explicit task supervision.

Abstract:
AI copilots can substantially boost human performance through shared control, but excessive assistance can induce over-reliance and skill atrophy. This paper studies how an embodied AI agent can act as a coach that accelerates human motor-skill development. We argue that effective coaching requires strategic scaffolding and stepping back that are aligned with the learner's capability, allowing productive failures that drive learning. We formalize the interactive AI coaching process as a non-cooperative dynamic game in which the learner optimizes task performance while the coach targets the learner's independent competence. Building on this formalism, we develop a reinforcement learning framework combining adaptive shared control with probabilistic models of the coach's causal influence on skill evolution, enabling tractable training of coaching policies. A comprehensive user study (N=33) on first-person-view drone racing shows significant gains in human learning outcomes over state-of-the-art AI coaching baselines.

Abstract:
Robot learning has made rapid progress in control, manipulation, and navigation, yet the affective capabilities of companion robots remain largely unexplored. Watching movies together with humans provides a natural and emotionally rich setting for studying robot affective perception and response. It requires robots to understand viewer-level emotion and remain robust to first-person-view (FPV) screen observations. In RoboAffect, we introduce EgoScreen-Emotion (ESE), the first benchmark dataset for robot affective perception from shared viewing. ESE contains 224 movie trailers and 28,667 egocentric keyframes captured under 18 systematic viewing conditions (camera height, viewing distance, and horizontal angle), annotated by 5 raters with multi-label confidence scores. Inter-annotator agreement reaches Fleiss’ κ=0.34 overall and κ=0.72 on high-confidence frames. We demonstrate a significant FPV domain gap: Macro-F1 drops from 28.06 to 14.64 under Raw→FPV transfer, corresponding to a 47.8% relative decline, motivating FPV-matched training for reliable affective perception on robots. Building on ESE, we develop a multimodal long-context framework integrating 3-frame visuals, synchronized audio, and narrative summaries, trained with confidence-aware aggregation and label smoothing to handle affective ambiguity. Our full model achieves Macro-F1 32.74, reaching 84.3% of the human leave-one-out reference (38.82) and outperforming all zero-shot baselines including Gemini-2.5-Flash (23.50). We further validate the pipeline on a physical quadruped robot, demonstrating end-to-end affective perception and expression under real FPV conditions. Dataset, code, and model weights will be released for reproducibility.

Abstract:
Gaze plays a crucial role in human communication, conveying intent, emotion, and vitality through subtle eye and head movements. However, realistic gaze behaviors in humanoid robots are often constrained by task-specific controllers, such as object tracking systems, resulting in rigid, unnatural motion that lacks human-like expressiveness. In this work, we propose a VLM-guided framework for a facial robot that decouples high-level cognitive reasoning from low-level motor control, with a action head learned directly from human motion data. A Vision-Language Model interprets the scene and designates gaze targets that condition our action head, the Realistic Gaze Transformer (RGT). Built on a Transformer-VAE, RGT is first pre-trained on human motion sequences in a self-supervised manner to learn a compact latent representation of full-head dynamics, then fine-tuned autoregressively for stable long-horizon rollouts under closed-loop deployment. At inference, conditioned on a brief motion history and the VLM-directed eyeball trajectory, RGT generates real-time 11-DoF commands synthesizing fluid eye–head coordination, eyelid motion, and natural blinking. By separating motion realism from gaze targeting, our framework preserves task-directed gaze while enhancing naturalness, yielding robot gaze that is both purposeful and imbued with the subtle vitality of human eye contact.

Abstract:
Reward learning typically assumes the human's input covers everything that matters for the task. In practice, humans can only attend to so much at once, so their teaching naturally attends to some features---or aspects of the task---better than others. Prior work has shown how the robot can identify which features were poorly supervised directly from its teaching data. But the current solution is to ask the human to try again, putting the burden on someone who already struggled to convey those features. We propose shifting this burden to the robot. Our key insight is that the same information that tells the robot what it doesn't know can help it ask better questions about it. Preference queries offer a low-effort channel for resolving this uncertainty, but only if the robot knows where to direct them and the human knows what to attend to when answering. We act on this insight in two ways: we build an informed prior that targets preference queries at the features that need resolving, and we accompany each query with an explanation that focuses the human's attention on exactly those features. In simulation, the informed prior beats the standard active preference learning baseline by up to 30.8% in the low-query regime, where reducing human effort matters most. In a user study, the informed prior and explanations each significantly improve reward recovery and amplify each other.

Abstract:
Generalist robot policies increasingly benefit from large-scale pretraining, but offline data alone is insufficient for robust real-world deployment. Deployed robots encounter distribution shifts, long-tail failures, task variations, and human correction opportunities that fixed demonstration datasets cannot fully capture. We present Learning While Deploying (LWD), a fleet-scale offline-to-online reinforcement learning framework for continual post-training of generalist Vision-Language-Action (VLA) policies. Starting from a pretrained VLA policy, LWD closes the loop between deployment, shared physical experience, policy improvement, and redeployment by using autonomous rollouts and human interventions collected across a robot fleet. To stabilize learning from heterogeneous, sparse-reward fleet data, LWD combines Distributional Implicit Value Learning (DIVL) for robust value estimation with Q-learning via Adjoint Matching (QAM) for policy extraction in flow-based VLA action generators. We validate LWD on a fleet of 16 dual-arm robots across eight real-world manipulation tasks, including semantic grocery restocking and 3--5 minute long-horizon tasks. A single generalist policy improves as fleet experience accumulates, reaching an average success rate of 95%, with the largest gains on long-horizon tasks.

Abstract:
A central challenge in offline reinforcement learning is ensuring that value updates remain grounded in the support of the dataset. Querying a learned critic at out-of-distribution actions can lead to overestimated values, which, unlike in the online setting, cannot be corrected through further interaction. A prominent strategy is to fit the value function using only in-distribution data while approximating an upper expectile of action values to induce optimism. We retain this in-distribution principle but reconsider how optimism is obtained. Instead of estimating an expectile through an asymmetric regression loss, we use a categorical critic representation, where an optimistic target can be obtained simply by selecting an upper quantile of the learned value distribution. Crucially, this distributional perspective allows optimism to be calibrated according to evidence, with the quantile level adapted on a per-state basis. States with well-supported value distributions are treated more optimistically, whereas states with weaker evidence are assigned more conservative value estimates. We call our method Distributional Implicit Q-Learning for offline-to-online reinforcement learning (DIQL). Across four representative robot task suites from OGBench, DIQL outperforms state-of-the-art offline RL methods. On Robomimic tasks, we further show that DIQL naturally extends to robot offline-to-online fine-tuning.

Abstract:
Reasoning in language allows foundation models to spend more test-time compute on hard problems, such as those requiring decomposition, constraint tracking, and prediction of future consequences. Whether this mechanism can improve robotic manipulation remains unclear, where long-horizon tasks require tracking partial progress, reasoning about object relations, recovering from mistakes, and steering noisy low-level policies. In this paper, we study whether VLMs can be trained to reason directly in natural language to guide low-level manipulation policies. We introduce \mathcalR^3, a simple post-training recipe that turns off-the-shelf VLMs into robotic reasoners: it first mid-trains a VLM on expert-generated reasoning traces to initialize the desired reasoning style, then improves the reasoner with single-step rubric-based RL from offline action data. Unlike prior robotic reasoning methods that mostly use structured traces as auxiliary supervision, \mathcalR^3 trains free-form language reasoning to produce test-time guidance for action. We instantiate \mathcalR^3 on Language Table, a controlled testbed for studying robotic reasoning, and find that it improves exploration and generalization across unseen tasks. These results suggest that free-form language reasoning can function as a test-time compute mechanism for steering low-level policies.

Abstract:
Future planetary rovers will traverse steep, deformable terrain where slip, sinkage, and terrain deformation can threaten mobility. Although simulation can capture some wheel-soil effects, unmodeled deformable-terrain phenomena create sim-to-real gaps requiring deployment-time adaptation. Yet, rover operations leave little room for extensive trial-and-error. We introduce TERRA (Targeted Exploration for Reward-Relevant Adaptation), an online adaptation framework that efficiently seeks deployment data exposing task-relevant model errors and corrects them. TERRA adapts a simulation-trained world model with a reward-weighted prediction loss and trains a loss predictor to estimate where future performance-relevant errors are likely to occur, using this signal to guide exploration during MPC. In simulation, TERRA reaches approximately 90% slope-climbing success within five trials under severe out-of-distribution pretraining conditions, outperforming exploration baselines in reward, completion time, and energy to first success. On the ERNEST rover, TERRA recovers coordinated active-suspension locomotion on a deformable Mars-simulant slope within three to five real-world trials.

Abstract:
Open-world mobile manipulation (OWMM) requires a vision-language model (VLM) agent to do more than recognize a target: after visual contact, the robot must reacquire, approach, and stop at a pose from which pick or place can succeed. We argue that this failure is partly a supervision-interface mismatch. Synthetic teachers often succeed through privileged low-level execution traces that are later converted into VLM labels, while the deployed student must act from egocentric images, compact memory, and high-level skill calls. We introduce AIDA-OWMM, an agent-interface data alignment framework that collects active target-acquisition demonstrations directly in the deployed VLM agent interface and annotates them with object-centric spatial reasoning targets. On the OWMM benchmark, AIDA-OWMM raises full-task success from 7.65% for an exact OWMM-style legacy baseline to 71.76%, with the largest gains appearing after object retrieval, in pick readiness, object pickup, goal search, and final task completion. Cross-benchmark EmbodiedBench navigation results further suggest that the same supervision format transfers beyond the original mobile-manipulation benchmark. Code will be publicly released.

Abstract:
Mobile manipulators must adapt not only to objects, but also to the spatial context in which they appear: where to position the base, how to manipulate, and where to look while navigating surrounding scene geometry. To address this, we present COWBOY, a scalable sim-to-real framework for learning such contextual whole-body manipulation. On whole-body dexterous grasping, COWBOY transfers zero-shot to hundreds of objects and diverse in-the-wild scenes, spanning cluttered tabletops, shelves, drawers, and other unstructured settings. Rather than learning scene adaptation directly with RL in simulation, a vectorized environment-aware whole-body controller (WBC) handles reaching in thousands of scenes by coordinating base-arm motion, collision avoidance, and active vision. Then, local RL experts are trained in local scenes for object interaction and can be reused across thousands of full-scene layouts. We distill these composed WBC--RL teachers into a single vision policy that controls the mobile base, arm, dexterous hand, and actuated camera from point-cloud observations. The resulting policy exhibits contextual whole-body behavior, automatically adapting base placement, camera viewpoint, arm trajectory, and grasp strategy to observed scene and object geometry. Videos are available at \urlcowboy-manip.github.io

Abstract:
Object-centric manipulation policies improve generalization by modeling object motion instead of directly predicting robot actions. However, existing methods are often limited by representations which are either too simplistic to capture interaction dynamics or too dense to learn efficiently. We observe that many relational manipulation tasks are primarily governed by short interaction phases where the relative motion between task-relevant objects is tightly constrained. Based on this observation, we propose Foci Policy, an interaction-centric framework that achieves a two-fold abstraction: (1) temporally, by automatically extracting compact interaction segments from demonstrations; (2) spatially, by representing skills as relative SE(3) motion between task-relevant objects, yielding invariance to scene configurations and robot embodiment. Experiments on RLBench, COLOSSEUM, and real-world tasks show that Foci Policy achieves strong performance with less training data than prior object-centric and action-centric policies. These results suggest that modeling object-object interactions provides a simple and efficient inductive bias for relational manipulation.

Abstract:
Social navigation requires the robot to reason and respond in complex real-world environments. While recent works attempt to incorporate human-level intelligence into robot planning using large Vision-Language Models (VLMs), end-to-end frameworks often create an unpredictable black-box, and existing instruction-following methods are not designed for full autonomy. To bridge this gap, we present G2-Nav, a novel framework that grounds abstract social reasoning and guards safe real-world deployment. Instead of asking the VLM for direct planning decisions, G2-Nav translates its semantic reasoning into a vision-language costmap with reliability and interpretability. The VLM evaluates traversable regions and social agents from open-set perception, mapping social context into the costmap. To improve real-world robustness, the VLM performs semantic verification on upstream tracking, and we introduce a high-frequency safety check to guard against system latency prior to trajectory generation. We demonstrate through real-world experiments that G2-Nav delivers safe, efficient, and socially compliant autonomous navigation in unstructured environments. Code will be made publicly available.

Abstract:
Reliable trajectory planning under partial observability depends not only on computing a feasible geometric path, but also on whether the robot receives informative observations while executing that trajectory. Existing approaches usually keep the environment fixed and adapt the robot through belief-space planning, active localization, or added sensing, often incurring costly uncertainty propagation and brittle behavior in observation-poor regions. We flip this perspective and address the largely open problem of \emphtask-aware environment augmentation: given a mapped environment, a planned task trajectory, and a small budget of visual fiducial markers, where should the environment be augmented so that the planned trajectory can be executed reliably under uncertainty? Our key observation is that useful marker layouts are defined by the localization support they provide along the task trajectory: a small number of well-timed observations can be sufficient to prevent uncertainty from accumulating in regions where state-estimation error would otherwise compromise control. Building on this observation, we present SCoDA, Shielded Conditional Diffusion for Environment Augmentation. SCoDA learns a conditional distribution over high-performing fiducial layouts from data, using the environment, planned trajectory, disturbance context, and desired execution profile as conditioning. Its shielded sampler reasons over where along the planned execution pose corrections should occur, and steers this distribution toward task-relevant, finite-budget augmentations. Across simulated benchmarks and hardware deployments, we show that SCoDA improves trajectory execution reliability and completion time over strong baselines.

Abstract:
Autonomous navigation in real-world environments often requires long-horizon operation under partial observability and evolving goals. Existing mapless navigation approaches predominantly assume fixed goals throughout an episode, limiting applicability to practical scenarios where goals change online during execution. We study online chained-goal navigation, where goals evolve through existing-goal redirection or new-goal emergence without explicit transition signals. Under this setting, we find navigation may require either continued exploration or strategic revisitation, creating conflicting memory requirements. We propose Global-Local Spatio-Temporal Belief Learning (GLST), a memory-decomposition framework that separates latent beliefs into global-local spatial and long-short temporal representations for long-horizon reasoning. We further introduce goal-conditioned rewards to balance exploration and exploitation under evolving goals. Extensive experiments in simulation across diverse terrains demonstrate improved chained-goal navigation performance over baselines. Real-robot experiments using the Spot quadruped further validate our memory decomposition framework. We open-source our code for benchmarking.

Abstract:
Vision-Language-Action (VLA) models are gaining attention in autonomous driving, inheriting commonsense priors from large language models and widening the navigation interface from a fixed command set to free-form natural-language instructions. This open-ended interface, however, exposes a new vulnerability surface: an inattentive driver may issue instructions that are unsafe given the current scene, and a malicious driver may issue dangerous ones by design. We first construct a suite of benchmarks that evaluate VLA models along three axes: (i) safe navigation following, (ii) navigation-scene conflict awareness, and (iii) robustness to adversarial instructions, revealing that existing policies are deficient on both safe following and conflict awareness. Guided by these observations, we propose SafeDriveVLA, which decouples conflict reasoning from action generation through an interpretable conflict prediction before the action generation, together with navigation-conditioned world model dreaming that exposes whether the instructed maneuver is feasible before the policy commits. SafeDriveVLA reaches state-of-the-art 83.64 DS and 59.82% SR on Bench2Drive, lifts closed-loop language following by 48.7%, and reduces collisions under unsafe instructions from 66 to 45 compared to prior state-of-the-art.

Abstract:
Feature matching is a fundamental component of robotic perception systems, enabling visual localization, mapping, SLAM, and autonomous navigation. Existing semi-dense matching methods formulate correspondence estimation as discrete confidence prediction over feature grids, inherently producing grid-aligned correspondences that require increasingly high-resolution feature maps for accurate localization, resulting in substantial computational and memory overhead that limits real-time deployment on embedded robotic platforms. We propose CoReMatch, a recurrent coordinate refinement framework for efficient semi-dense feature matching. CoReMatch reformulates semi-dense matching as iterative coordinate-space optimization and progressively refines correspondences through recurrent hidden-state propagation. At the coarse level, a Deep Aggregated Transformer performs efficient global correspondence refinement through recurrent coordinate updates. At the fine level, a lightweight local refinement module predicts sub-pixel coordinate corrections within compact feature windows for accurate geometric refinement. Extensive experiments on indoor and outdoor benchmarks demonstrate that CoReMatch achieves superior accuracy--efficiency trade-offs compared with state-of-the-art semi-dense matchers. In particular, CoReMatch achieves higher accuracy on MegaDepth while delivering over 10x faster inference than Efficient LoFTR on embedded edge devices. These results suggest that recurrent coordinate refinement provides an efficient and scalable formulation for real-time robotic perception.

Abstract:
As the number of autonomous robots continues to grow, safety becomes increasingly important. Control barrier functions (CBFs) provide an elegant, theoretically grounded solution, but existing design methods often fall short in effectiveness, scalability, interpretability, or conservativeness. In this paper, we propose \emphVertexCBF, a framework for learning neural CBFs in a scalable, systematic, and explainable way. We approximate the stationary Hamilton--Jacobi value function with a neural network trained by combining physics-informed and sparsely supervised learning. By exploiting control-affine dynamics and a convex polytope control set---under which the Hamiltonian is maximized at the control vertices---we efficiently generate supervision points via GPU-parallel vertex-restricted tree search, while a residual architecture guarantees that the learned CBF is never larger than the specified constraint function. We evaluate the method on 15 systems and compare it against relevant baselines, showing that it reliably recovers large safe sets where the baselines are conservative or fail outright. In addition, we perform a hardware experiment in which a mobile robot safely avoids pedestrians using a neural CBF trained with our method.

Abstract:
Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible, violating reachability, collision-avoidance, or closed-loop executability requirements. This embodiment gap limits zero-shot deployment across robots, even when the task-space behavior itself is transferable. We propose an inference-time optimization framework that couples the behavior generation to physical feasibility by formulating diffusion guidance as a constrained optimization problem. Our key insight is to replace the sampling perturbation in the backward process with an optimized correction, allowing hard constraints or soft penalties to be imposed during sampling without the need to retrain the diffusion model, while keeping samples close to the learned prior. We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints. Across settings and robot embodiments, optimization-guided denoising matches the feasibility of projection- and gradient-guidance baselines while better preserving grasp quality, and improving controller-level executability and task success, with task success improving by up to 20pp. on dexterous grasping and 23pp. on visuomotor manipulation over the best baseline.

Abstract:
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24× lower wall-clock latency than pixel-space WAMs.

Abstract:
Vision-Language-Action (VLA) models are promising for end-to-end robotic manipulation, but their real-world deployment is bottlenecked by inference latency, especially the iterative action denoising with flow-matching. Existing remedies only address a single side: efficient-inference methods make each forward pass cheaper but leave the decoding loop intact, while asynchronous methods hide latency behind execution at the cost of a prediction-execution mismatch that grows with the lookahead. We trace both failure modes to the same structural choice: each action chunk is decoded in isolation. We introduce FlashVLA, a streaming action decoder that decodes chunks jointly. FlashVLA maintains a buffer of action chunks at staggered noise levels and advances them under chunk-wise causal attention, so each forward pass emits one executable chunk and every higher-noise chunk attends to the lower-noise chunks already in flight. Both costs collapse under this single design: per-step action-decoding latency drops by up to 20x, and asynchronous continuity is recovered without an explicit future-state predictor. Across LIBERO, RoboTwin2.0, and real-world tasks, \method matches or improves \pi_0.5 task success under one-step asynchronous delay, achieves up to 1.83x lower end-to-end per-action latency, and sustains 30Hz smooth control on a consumer-grade GPU.

Abstract:
Action chunking enables robot policies to produce temporally coherent behavior, but multi-step generation in flow-based policies introduces latency that conflicts with real-time control. During asynchronous execution, the robot continues executing the previous chunk while the next one is generated, so even small delays can create mismatches at chunk boundaries. Existing methods address this mismatch by steering the generation process toward the executed prefix. We instead show that prefix consistency can be achieved by selecting an appropriate initial noise before generation begins, allowing the unmodified flow ODE to produce a coherent next chunk. This reframes asynchronous inference as a noise selection problem rather than a trajectory steering problem. We introduce PAINT, a training-free method that finds this noise via backward Euler inversion and constructs the final chunk through a repainting rule. In summary, PAINT requires no gradients, retraining, or policy modification; yet it improves execution consistency and task performance across 12 simulated benchmarks and 6 real-world manipulation tasks spanning single-arm, bimanual, and humanoid embodiments.

Abstract:
Diffusion Policy achieved strong performance in robot control, but its iterative denoising process hinders deployment on resource-constrained edge devices. To overcome this problem, we propose Agile Diffusion Policy (AgileDP), which introduces integer-based quantization of diffusion policies. We address fundamental challenges in quantizing diffusion policies: (1) drastic activation distribution shifts across denoising steps and (2) large relative quantization error at low-magnitude regions of the action sequence. First, diffusion models exhibit wide variation in activation distributions across denoising timesteps, rendering fixed quantization grids ineffective. We propose an implicit parametric activation quantization scheme utilizing a lightweight neural network to project timestep information, dynamically adapting the quantization scale without extra training overhead. Second, we observe that low-magnitude regions of the action sequence consistently exhibit large relative quantization error. We propose a magnitude-aware error reweighted distillation that targets these large-error regions to recover action accuracy. Extensive evaluations of 7 tasks in 3 benchmarks show robust performance even with 4-bit precision, comparable to the full-precision model. Furthermore, by achieving up to 8.5x speedup and 3.2x memory savings, this work validates practical deployment on robotic on-board devices.

Abstract:
In policy learning, stitching and compositional generalization refer to the extent to which the policy is able to piece together sub-trajectories of data it is trained on to generate new and diverse behaviours. While stitching is a strength of offline reinforcement learning, recent generative behavioural cloning (BC) methods have also shown proficiency at stitching. Yet, the main factors behind this are poorly understood, hindering the development of new algorithms that can reliably stitch by design. Focusing on diffusion planners trained purely via generative BC, we find two properties are key enablers of composition: shift equivariance and local receptive fields. We use these properties to explain data augmentation, data scaling, and inference choices in existing generative BC methods based on diffusion planning. We use our findings to develop Eq-Net, a new architecture for diffusion planners that enables compositionality while being trained and evaluated like a vanilla diffusion planner. We show that Eq-Net both produces much more diverse trajectories than existing diffusion backbones and can be trained on only short sequences while still generating consistent long-horizon plans.

Abstract:
Vision-Language-Action (VLA) models that couple pretrained Vision-Language Models (VLMs) with continuous action experts have achieved strong manipulation performance, yet generalization to out-of-distribution (OOD) language instructions remains poor. A known challenge is the structural imbalance in VLA data, where language is far less diverse than visual and action content, making policies prone to visual shortcuts. While discrete-action methods mitigate this through vision-language co-training, continuous action experts lack such protection: they start from random initialization and learn entirely from imbalanced data, producing noisy gradients that corrupt the VLM and fail to exploit its language capability. We address this from a Bayesian perspective, factorizing the policy into a language-agnostic Vision-Action (VA) prior and a language-conditioned VLA likelihood, and propose APT, a two-stage training method emphasizing Action expert Pretraining. In Stage 1, the action expert is pretrained as a VA prior on vision-action pairs from a frozen VLM, bypassing the language imbalance. In Stage 2, language tokens are injected through a gated fusion mechanism that integrates VLM features while preserving the learned visuomotor prior. APT applies to mainstream VLA architectures, including the \pi and GR00T-style architectures. Comprehensive experiments validate that APT achieves consistent gains on unseen instructions and compositional tasks.

Abstract:
Human-collected real-world robotic manipulation demonstrations often contain mixed-quality trajectories, including hesitant, indirect, or inefficient action segments. While diffusion- and flow-based policies can model such multimodal data, standard behavior cloning (BC) assigns equal likelihood to all demonstrated actions and may therefore reproduce suboptimal modes. Offline RL value estimates offer a natural signal for biasing policies toward higher-value behavioral modes, but directly fine-tuning the generative policy or using critic gradients for policy steering can amplify critic errors and induce prior drift. We propose Advantage-Weighted Latent Steering (AWLS), an offline post-training method for improving frozen one-step generative manipulation policies without updating their decoders. AWLS learns a residual noise shaper, an observation-conditioned residual correction to the policy's latent noise, and trains it with an advantage-weighted action reconstruction objective. The advantages are detached, scale-normalized, and used only as supervised weights on demonstrated actions, avoiding critic-gradient policy updates while preserving fast single-step inference. We further introduce a rollout-free decoder-sensitivity criterion for selecting pretrained checkpoints that remain responsive to latent-noise perturbations. Experiments on simulated and real-world robotic manipulation tasks with clean and mixed-quality demonstrations show that AWLS improves over the frozen base policy while limiting drift from the pretrained behavior prior.

Abstract:
Vision-Language-Action (VLA) models have shown promise for robotic manipulation, yet most existing policies operate reactively by directly regressing actions from current observations, without explicitly modeling future dynamics. This limits their ability to generalize under out-of-distribution perturbations. To address this issue, we propose ELAN4D, an embodiment-centric, 4D-aware training framework that enhances VLA policies with future robot keypoint tracks as predictive spatio-temporal supervision. Using only forward kinematics from proprioceptive states, we derive 3D displacement tracks of robot keypoints, such as joints and the end-effector, with negligible preprocess cost. These tracks provide metric and compact supervision without requiring external trackers or reconstruction. A plug-and-play auxiliary branch with a lightweight track decoder injects this 4D signal into the action expert while preserving the pretrained vision-language backbone through gradient isolation. The track decoder is discarded during inference, leaving the base policy interface unchanged. Extensive experiments on LIBERO, LIBERO-Plus, RoboTwin2.0 and real-world manipulation tasks demonstrate that ELAN4D consistently improves over strong VLA baselines, achieving the best overall performance and substantial gains under out-of-distribution perturbations, including camera, background, and layout shifts. These results highlight the effectiveness of embodiment-centric 4D supervision for building more robust and generalizable manipulation policies.

Abstract:
Vision-Language-Action (VLA) models have emerged as a promising approach for general-purpose robot manipulation. However, little research has mechanistically explored when and why they generalize across objects, scenes, and instructions. To probe internal representations, we train Sparse Autoencoders (SAEs) on the VLA's hidden-layer activations. SAEs learn sparse dictionaries over model activations, often revealing features that correspond to interpretable directions in the model’s representation space. We identify SAE features corresponding to motion primitives and semantic concepts, including features that are general across episodes and causally steerable. We propose a metric to categorize features as general transferable primitives or episode-specific memorizations, offering a promising glimpse towards VLA generalization. We validate these findings through steering experiments on both the LIBERO simulation benchmark and on real-world DROID hardware. We find that amplifying general and semantic features induces behaviors consistent with their meanings, whereas ablating them destroys model performance. Furthermore, we demonstrate steering as a way to control behavior in unpromptable directions. Together, these results provide mechanistic evidence that VLAs can learn reusable internal features linking perception, language, and action across tasks and scenes.

Abstract:
Generalist robot manipulation requires a single vision-language-action (VLA) policy to execute diverse skills across scenes and instructions. However, current multi-skill VLA policies often leave two aspects of an instruction insufficiently modeled: which primitive skill to execute and where/how to contact and move objects. We propose MAST-VLA (Multi-skill Adaptation with Skill-conditioned experts and Trajectory-affordance priors), a dual-conditioning framework for robot manipulation. MAST-VLA introduces skill-conditioned LoRA-MoE adapters in the LLM backbone, where skill-query tokens and routing biases guide low-rank expert specialization, and converts a predicted affordance heatmap into a patch-level trajectory-affordance spatial prior for the action head. On the CALVIN ABC-D long-horizon multi-skill cross-scene generalization setting, MAST-VLA achieves state-of-the-art performance, reaching an Avg. Len of 4.65 over five-subtask evaluation sequences. Real-robot multi-skill sequence experiments further validate its deployability under real perception and control noise.

Abstract:
Current Vision--Language--Action (VLA) models often optimize for semantic grounding, whereas executable manipulation requires geometry-aware spatial alignment and dynamic affordance selection. We introduce GeoAlign, a state-guided spatial alignment architecture for VLA policy learning. GeoAlign post-trains an RGB geometry branch with robot-domain RGB-D supervision, yielding RGB-derived Geometry-Enhanced Post-Trained (GEP) features for policy rollout. The robot's proprioceptive state queries the GEP feature grid, producing compact, phase-dependent geometry tokens for action prediction. GeoAlign achieves 99.0% on LIBERO,85.3% across three SimplerEnv-Fractal tasks, and 78.8% on eight geometry-critical real-world ALOHA tasks, with ablations confirming the value of geometry post-training and proprioceptive-state-guided querying.

Abstract:
Many robotic tasks are unforgiving; a single mistake in a dynamic throw can lead to unacceptable delays or unrecoverable failure. We introduce Wiggle and Go!, a two-stage framework for zero-shot rope manipulation: a brief, safe wiggle action is observed to predict descriptive rope parameters, which then conditions a trajectory optimizer for zero-shot goal-conditioned execution. Unlike prior dynamic rope manipulation methods that require large real-world datasets or iterative real-world refinement, our identification module is task-agnostic, supporting diverse manipulation policies without retraining. We achieve a 3.55 cm average accuracy on 3D target striking in real using rope system parameters in comparison to 15.29 cm for uninformed baselines, and over 50 % success on multi-objective lobbing and draping tasks. Predicted parameters transfer to unseen motions with 0.95 Pearson correlation between simulated and real rope dynamics, indicating that the identification module generalizes across the task corpus.

Abstract:
Dexterous manipulation is limited by the cost of collecting large-scale robot demonstrations. Egocentric human videos offer a scalable source of diverse manipulation behaviors, but directly using them for robot learning requires bridging two gaps: the visual gap between human and robot observations, and the action gap between human motion and robot-executable action. We propose EgoEngine, a scalable framework for transforming egocentric human manipulation videos into high-fidelity robot data. Given an egocentric RGB video, EgoEngine produces: (i) a high-fidelity robot observation video replacing human with robot while preserving scene context and temporal alignment, and (ii) a task-aligned, executable robot action trajectory under feasibility constraints. Experiments in simulation and on real robots show that EgoEngine enables scalable conversion of human videos into robot data and, to our knowledge, demonstrates the first zero-shot visuomotor dexterous policy learning from egocentric human videos without real-robot demonstrations. Project website: https://egoengine.github.io.

Abstract:
Vision-language-action (VLA) models benefit from large-scale and diverse embodied data, yet collecting scalable robot trajectories remains costly and labor-intensive. Egocentric human videos offer a complementary source of rich real-world interactions. However, jointly training on human and robot data remains challenging due to differences in embodiments, action spaces, temporal dynamics, and supervision fidelity. We introduce ACE-Ego, a VLA pretraining framework for heterogeneous embodied data. ACE-Ego bridges human and robot trajectories through unified camera-space actions, embodiment-aware morphology tokens, temporally aligned action chunking, and a reliability-aware objective for mixed-quality supervision. We instantiate on 4.53K hours of robot and simulation data together with 1.48K hours of pseudo-labeled egocentric human videos. Experiments show that incorporating high-quality human supervision consistently improves both pretraining and supervised fine-tuning. ACE-Ego achieves state-of-the-art performance on RoboCasa GR1 TableTop and RoboTwin~2.0, while also demonstrating strong transfer to real-world bimanual manipulation.

Abstract:
Long-horizon precise manipulation produces mixed-quality data for post-training vision-language-action (VLA) models: even successful demonstrations and intervention-assisted rollouts may contain detours, hesitations, failed attempts, and imperfect recoveries. The key challenge is to identify which action chunks actually advance the task, rather than uniformly cloning or discarding imperfect trajectories. We propose Graph-Reweighted Behavior Cloning (GR-BC), an offline iterative post-training framework that builds a task-conditioned goal-reachability graph over demonstrations and iteratively collected rollouts, including both successful and failed episodes with human interventions. Reverse graph search computes minimal remaining steps to task success within the dataset support, and each action chunk is weighted by how much its execution reduces this potential. This consequence-based weighting reuses recoverable segments from failed trajectories, leverages interventions as recovery paths, and down-weights locally regressive chunks even inside successful trajectories. New rollouts are incorporated by appending them to the graph and rerunning search, without training a reward model, action-conditioned critic, or progress predictor. Across four precise manipulation tasks and three VLA backbones, GR-BC improves single-round supervised fine-tuning and reaches 91% success after a few offline iterations, without trajectory comparisons or progress labels.

Abstract:
Human-collected robot manipulation data often mix successful behavior with retries, pauses, recoveries, and failures. Advantage-Conditioned Behavior Cloning (AC-BC) can exploit such mixed-quality data, but only if the conditioning signal is both accurately constructed from offline trajectories and effectively incorporated into the policy's action-vector-field prediction. Existing pipelines can weaken this signal by thresholding estimator-derived advantage scores into noisy conditions, and by routing conditions through prompts that weakly affect the action vector field. We introduce ALIGN, a condition-aligned AC-BC framework for vision-language-action policies. ALIGN directly predicts categorical pairwise conditions in the policy-facing label space and injects them into a DiT action expert through AdaRMS modulation fused with flow time. We evaluate ALIGN on deliberately mixed real-robot datasets for ARX X5 bowl stacking and Franka key hanging, each combining standard demonstrations with retry-heavy trajectories. ALIGN improves condition fidelity, strengthens conditional vector-field separability, and achieves 90.0% and 56.7% closed-loop success, compared with 56.7% and 33.3% for the strongest AC-BC baseline. These results indicate that aligning conditions from estimation to action prediction is an effective way to extract robust manipulation policies from data in which success, recovery and error coexist.

Abstract:
Robots must generate trajectories that remain faithful to learned expert behavior while satisfying safety constraints and task-specific objectives specified only at inference time. We formulate constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling, with the learned demonstration distribution as a prior and an inference-time, cost-derived likelihood tilting it toward feasible, optimal trajectories. To sample from this posterior without any retraining of the base policy, we leverage the Feynman--Kac corrector framework, originally formulated for diffusion models, and extend it to deterministic flow-matching policies. The result is a unified, inference-time, retraining-free sampler for diffusion and flow policies. We validate the approach on pretrained Diffusion Policy, GR00T-N1.6, and \pi_0.5 checkpoints across simulated and real-world manipulation tasks, including planning around non-convex obstacles introduced at inference time, and show improvements over the base \(\pi_0.5\) on zero-shot tasks.

Abstract:
Diffusion policies with linearly composed scores have emerged as a promising approach to multi-task robotic learning by combining separate factors to enable modular adaptation, compositional generalization, and interpretable functional decomposition. However, current training objectives do not encourage factor specialization, often resulting in uninterpretable denoisers whose composed prediction averages over redundant copies, undermining their architectural premise. To address this, we propose a drop-in score-decorrelation auxiliary loss, grounded in projective composition theory. Our approach encourages orthogonality between component noise predictions with little architectural change or computational overhead. We demonstrate across multiple benchmarks that our decorrelation loss consistently reduces measurable double counting between components, improves held out compositional generalization, and produces behaviorally meaningful specialization without compromising multitask performance.

Abstract:
Robots that learn over long deployments must add new skills without losing the shared policy structure that makes earlier skills reusable. We study sequential robot skill learning, where previous trajectories and task losses may be unavailable, and the deployed policy must remain a single shared controller without task-specific heads, routing, or adapters. We identify skill-coupling collapse, a failure mode in which individual skill success remains non-trivial while reliability among related skills deteriorates. We propose Sleeping Robots, a wake-sleep framework that learns each new skill during wake and consolidates the shared policy offline during sleep using compact frozen skill memories: frozen critics with unordered state buffers for reinforcement learning and frozen actor snapshots with unordered observation buffers for imitation learning. During sleep, these memories define differentiable surrogate objectives whose gradients are combined through Nash bargaining, with adaptive anchoring and local excitability for stable consolidation. On Meta-World MT5, Sleeping Robots improves average success by 64% and pairwise reliability by 2.0× over the strongest non-oracle baseline, and on SurgicAI it improves average success and backward transfer relative to continual imitation baselines while remaining competitive on pairwise reliability.

Abstract:
Pruning redundant visual tokens is a promising direction for reducing the high inference cost of Vision-Language-Action (VLA) models. However, existing pruning criteria are largely inherited from generic Vision-Language Models (VLMs), relying on vision-text attention or static semantics that struggle to adapt to the stage-wise nature of embodied manipulation. As manipulation progresses, relevant objects and interaction regions evolve, requiring token importance to shift accordingly. To address this, we observe that a policy’s predicted actions naturally reflect short-horizon task progression and interaction dynamics. Based on this insight, we propose the Action-Centric Evaluation (ACE) framework, a training-free pruning approach that scores visual tokens by their influence on action generation. ACE estimates token importance by back-attributing predicted actions to visual tokens via gradient-weighted action relevance, enabling stage-aware visual token pruning for embodied control. Experiments show that ACE reduces FLOPs by up to 39.0%. On π-series backbones, ACE improves average success rates by 1.7%–2.0% while achieving 1.33× speedup on LIBERO and 1.36× in real-world tasks. On OpenVLA-OFT, ACE preserves competitive performance with 1.57× and 1.55× speedups in simulation and real-world settings, respectively. These results demonstrate the effectiveness of action-centric pruning for efficient embodied control. Codes are available at https://anonymous.4open.science/r/ACE-VLA-5A29/.

Abstract:
Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at hand. Current robot policies instead learn observation-to-action mappings from visual inputs and language instructions, without explicitly inferring the demonstrated task. Learning from human video thus remains largely trajectory-level: models can replay motions in near-identical scenes, but still struggle to imitate what the demonstrator intends rather than merely what they do. We introduce The Imitator Game, a four-level benchmark (L0--L3) that progressively widens the gap between the human demonstration and the robot's own scene, isolating where trajectory replay ceases to suffice and task understanding becomes necessary. We pair it with IG-10K, the largest environment-aligned paired human--robot dataset to date and the only one instantiated across all four levels in both real and simulated settings (20,000+ paired episodes, 50+ tasks, 6 domains), and Imitator Arena, an open platform for blind A/B human evaluation. Across nine state-of-the-art models, performance is stable from L0 to L2 but collapses at L3, identifying functional substitution -- achieving the same intent through a different object affordance -- as the decisive barrier to intent-level imitation. Human-video-conditioned models outperform caption-conditioned ones, yet every model falls below 13% zero-shot success on unseen tasks; fine-tuning an IG-10K-pretrained model with only 10 demonstrations yields large gains that grow with pretraining scale. The project website and access to Imitator Arena are available at https://imitator-game.github.io.

Abstract:
Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn reactive recovery behaviors, training a separate policy for every distinct failure mode is highly inefficient. We introduce Recovery-Driven Synthesis of Relational Concepts (ReSYNC), the first approach that progressively discovers and refines state abstractions (relational predicates) from failure-recovery experience to support abstract planning. Unlike purely reactive methods, ReSYNC jointly learns skills and concepts through an incremental dual-learning process. In the skill-learning phase, the robot uses RL to learn to recover from failures seen in training tasks. In the concept-learning phase, the robot discovers new relational predicates and refines its abstract planning model to explain and generalize the learned recovery behaviors. This interaction enables ReSYNC to convert local recoveries seen during training into global failure avoidance at test time. Across four simulated domains, we show that ReSYNC's ability to continually expand and refine its abstraction library allows it to solve long-horizon, previously unseen problems, outperforming strong baselines by over 50%. Additionally, we demonstrate sim-to-real transfer of ReSYNC, where it performs real-world non-prehensile manipulation skills and generalizes to unseen scenarios through abstract planning. Overall, ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure-aware planning in the physical world.

Abstract:
Robot policies inevitably encounter failures when deployed in real environments. Naive retries often repeat the same mistakes, while many existing recovery methods rely on human intervention. In this paper, we propose Failure-Aware Retry (FAR), a framework that enables robots to learn from previous failures at test time, adapt their behavior accordingly, and eventually complete the task autonomously. FAR combines Failure-Contrastive Preference Adaptation, which constructs preference learning data from failures to steer the policy away from previously unsuccessful behaviors, with lightweight action perturbations during retries to encourage local exploration. We further incorporate successful recovery trajectories into a training loop for continual policy improvement. Experiments in both simulation and real-world manipulation tasks show that FAR substantially improves success rates and robustness, with average gains of 17.6% over the standard diffusion policy in simulation and 11.7% in the real world. In addition, FAR significantly improves data efficiency under both reset and timestep budgets during continual policy improvement by exploiting informative failure cases.

Abstract:
Hierarchical robotic systems executing long-horizon manipulation tasks must make high-level semantic decisions that orchestrate stochastic low-level skills. In this setting, failed rollouts are ambiguous: a poor downstream state may reflect an invalid high-level decision, partial observation, or a valid decision whose physical execution failed. Traditional supervised learning lacks data for such recovery states, while reinforcement learning struggles with sparse rewards and non-local credit assignment. We propose MAGMA-GEN, an on-policy data-generation pipeline that converts ambiguous failed rollouts into validated recovery supervision. MAGMA-GEN first uses a privileged coach to hypothesize an early decision-level error and propose localized correction or recovery actions. Because this diagnosis is fallible, candidates are retained only if re-execution from the same state under matched conditions improves downstream progress. This produces supervised examples from the agent's own failure distribution without per-step human demonstrations. Evaluated on interactive long-horizon manipulation tasks, MAGMA-GEN improves task success and recovery capabilities, against distillation and trajectory-repair baselines under evolving task constraints in both simulation and real-robot execution.

Abstract:
Vision-Language-Action (VLA) models have become a powerful paradigm for robotic manipulation and show strong promise for real-world deployment, making adversarial robustness critical for reliable robot operation. Existing adversarial attacks on VLA-controlled robots mainly optimize prediction-level discrepancies or rollout-level deviations to disrupt manipulation. However, they rarely account for the fact that VLA-predicted commands are not executed directly, but must be realized through downstream control and robot embodiment. In this paper, we propose ExecGap, an adversarial patch attack framework that exploits this command-execution gap. Specifically, ExecGap biases VLA-predicted task-space commands toward low-executability directions, which are estimated from kinematic inefficiency and dynamic weak actuation. This induces execution degradation, where the predicted command becomes difficult to realize effectively, thereby impairing task completion. Extensive simulation experiments across multiple VLA policies show that ExecGap induces larger command-execution mismatch and weaker realized execution than existing patch attacks. Real-robot experiments further validate that the deployed patch can induce execution degradation on physical platforms. Code will be made public upon paper acceptance.

Abstract:
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation. Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors. Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.

Abstract:
Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions. However, existing embodied datasets remain fundamentally limited: internet-scale video data lack precise physical states and interaction grounding, while laboratory motion datasets provide high fidelity but only narrow behavioral coverage. This mismatch creates a critical bottleneck for scalable humanoid policy learning. We present HiPHI, a 600-hour high-precision whole-body human motion dataset designed to systematically maximize coverage of the human motion and interaction manifold. HiPHI is theoretically guided by FrameNet, a linguistic framework organizing human primitives. Captured using an optical motion capture pipeline, HiPHI provides sub-millimeter spatial tracking accuracy for full-body human motion and mesh-level object trajectories. We further introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications. Our analyses demonstrate that HiPHI significantly expands motion coverage compared to existing motion datasets while maintaining high-fidelity interaction quality. We believe HiPHI establishes a scalable data foundation for training, evaluating, and generalizing humanoid policies in real-world embodied environments.

Abstract:
Autonomous systems can fail in rare and heterogeneous ways, making real-world failure discovery difficult under limited testing budgets. Although cheaper proxies such as simulators, lower-fidelity systems, or related policies can be sampled extensively to find failures, proxy failures often do not transfer to the real world due to sim-to-real and system-to-system gaps. The key challenge is therefore to effectively leverage proxy system information for accurate prediction of severe target system failures. We propose an adaptive failure discovery method that combines proxy evaluations with limited target system results to guide scenario selection for target system testing. Our method learns a local predictor of target risk by correcting proxy failure signals using control-variate-inspired residual modeling. To find failures that are both likely and diverse, we combine this predictor with a support-aware mutual-information objective that favors realistic, well-supported regions while expanding coverage across failure modes. Across autonomous driving, manipulation, and quadruped velocity-tracking tasks, our method discovers up to 2x as many failures as random sampling and active-learning baselines, including severe and diverse failures missed by competing methods.

Abstract:
Dexterous grasping in cluttered scenes remains a central challenge in robotic manipulation, as it requires high-dimensional hand coordination under complex object-environment constraints. To address this problem, we introduce one of the largest simulation benchmarks to date, comprising around 0.5B grasp annotations across eight dexterous hands and approximately 50K objects. Furthermore, we propose T(R,O) Grasp++, a scalable diffusion-based framework for cross-embodiment dexterous grasp synthesis in cluttered environments. Extensive experiments demonstrate that our model significantly outperforms existing baselines, achieving an average success rate of 88% across eight dexterous embodiments. Meanwhile, our pipeline achieves only 0.3s inference latency while improving inference sample efficiency by over 10 times.

Abstract:
Large-scale dexterous grasp datasets encode rich priors over hand-object interaction, but their use has largely been confined to grasp generation and pick-and-place manipulation. We study whether such data can instead support functional dexterity in articulated tool use, where a robot must acquire a tool, maintain contact, and operate its functional moving parts. We adapt a hierarchical imitation learning framework that combines high-level hand sub-goal prediction with a low-level goal-conditioned controller. We construct a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and use it to pretrain the low-level controller. The controller is then fine-tuned on downstream task demonstrations. To evaluate this setting, we introduce DexCraft, a simulation benchmark with six articulated tool-use tasks requiring coordinated finger motion. Across simulation and real-world experiments, our approach outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch. In the real world, it improves full-task success by 33.3 percentage points over DP3. These results show that grasp datasets can serve not only as resources for grasp synthesis, but also as scalable pretraining data for contact-rich dexterous manipulation. Videos are shown on our anonymized website https://grasp2dexterity.github.io/ .

Abstract:
While multi-fingered dexterous robot hands have the potential to provide human-level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm-hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real-world learning. However, real-world demonstrations in the full arm-hand control space are costly and noisy because each trajectory must coordinate arm motion with high-DoF multi-finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off-nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm-hand coordination policy for arm control and hand-skill activation. This design enables effective real-time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real-world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances.

Abstract:
Robotic grasping remains a fundamental challenge, particularly when bridging the gap between simulation and real-world deployment. Although optimization-based simulation methods can generate candidate grasps with strong theoretical validity, their performance often degrades significantly in real-world environments. This degradation is especially severe under challenging conditions such as collisions, clutter, and severe occlusions. In such complex scenarios, grasps produced in simulation are frequently unreliable, leading to high failure rates during physical execution. To address this issue, we propose a system that enhances simulation-verified grasps through a rigorous real-world verification pipeline. Our approach employs a multi-camera setup to automatically validate the physical feasibility of grasps with minimal human intervention. By leveraging multi-view perception, the system can accurately assess grasp validity even under diverse environmental constraints where single-view methods typically fail. Furthermore, we also present a coverage-preserving grasp selection algorithm that ensures the dataset spans a wide range of geometric configurations. We show that this real-world verification process produces substantially more reliable grasps than simulation alone, while also increasing grasp diversity to better handle highly constrained environments. We demonstrate that our automatic pipeline is significantly more efficient and effective than human teleoperation, enabling the collection of a larger and more diverse set of valid grasps. We validate our system across 100 objects with diverse geometries, resulting in a large-scale dataset of firmly grasped, real-world verified objects.

Abstract:
Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects, noisy observations, or unmodeled contact dynamics. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1000 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, adapts effectively to deployment-time shift, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90% success rates on several challenging object categories after only 50 online grasp attempts.

Abstract:
Vision-based tactile sensing is widely used in perception, reconstruction, and robotic manipulation, yet collecting large-scale tactile data remains costly due to diverse sensor-object interactions and inconsistencies across sensor instances. Existing approaches to scaling tactile data—simulation and free-form tactile generation—often yield unrealistically rendered signals with poor transfer to highly dynamic real-world tasks. We propose ControlTac, a two-stage controllable tactile image generation framework that generates realistic tactile images conditioned on a single reference tactile image, contact force, and contact pose. By grounding generation in these important physical priors, ControlTac synthesizes realistic samples across different sensors while effectively capturing task-relevant variations. Across a series of downstream tasks and real-world experiments, such as object insertion, imitation learning, and object weighting, the augmented datasets using our approach consistently improve performance and demonstrate practical utility in dynamic real-world settings.

Abstract:
Compliance is essential for dexterous manipulation, yet existing solutions often rely on external tactile or force sensors that are costly, fragile, and difficult to deploy on low-cost robot hands. We propose a proprioception-driven framework that learns tactile-like contact feedback from motor current and joint states. Since motor current is closely related to actuator torque, it provides an intrinsic signal for perceiving contact force, object resistance, and grasp stability without additional sensing hardware. Rather than estimating external wrenches or commanding torque, our method predicts a compliance reference position: an ideal joint-position target for a standard PD controller whose induced position error generates appropriate grasping force. This position-based formulation is compatible with mainstream teleoperation and policy-learning pipelines, while enabling the robot to adapt interaction forces from real-time proprioceptive feedback. Thus, motor current serves not only as a force proxy but also as a learnable proprioceptive contact signal for compliance reference prediction. Experiments on multiple dexterous hands and contact-rich tasks, including fragile object handling, dynamic load adaptation, sustained surface contact, and thin-object retrieval, show stable compliant grasping, safer and more efficient teleoperation, and improved downstream policy learning without external tactile sensors.

Abstract:
Dexterous manipulation involves contact-rich and fine-grained interactions with the physical world, posing significant challenges for existing vision-language-action (VLA) models due to severe visual occlusions and complex contact dynamics. While recent works have incorporated tactile sensing into robotic manipulation, most approaches still rely on homogeneous multimodal fusion, lacking adaptive tactile integration and explicit modeling of physical dynamics. In this work, we present DeCAL, a physically-grounded dexterous vision-language-action model that unifies understanding, imagination and action generation for contact-rich dexterous manipulation. Built upon a Mixture-of-Transformers (MoT) architecture, DeCAL leverages specialized experts for each capability while enabling efficient information flow among them. To effectively leverage tactile information, we introduce Adaptive Visuo-Tactile Fusion that dynamically regulates tactile interactions via a contact-aware gating strategy. Furthermore, we propose Visuo-Tactile Latent Co-Imagination to jointly model visual and tactile dynamics, equipping the policy with implicit physical world knowledge. Experimental results show that DeCAL consistently achieves state-of-the-art performance across all tasks, attaining a 71% average success rate and an 83.4% progress success rate, while also demonstrating strong generalization to unseen scenarios.

Abstract:
Robots in contact-rich manipulation must decide which information to encode from vision and touch, while also determining the manipulation phases in which tactile information is sufficiently action-relevant to justify representation budget. We introduce Phase-Aware Variational Multimodal Information Bottleneck (PA-VMMIB), a visuo-tactile policy that suppresses tactile latents before contact and reallocates rate to touch during high-precision contact. The method is motivated by an Action-Conditioned Multimodal Information Bottleneck (AC-MMIB): within a modality-separated encoder class, and under explicit separability and conditional-noise assumptions, the analysis gives a contact-conditioned water-filling diagnostic in which active modalities equalize marginal rate-distortion slopes and the tactile branch activates only when contact makes touch action-relevant. A calibrated exponential-family approximation predicts an activation scale inversely related to the leading tactile Fisher mode, \beta_c \propto 1/\lambda_\max(\mathbfJ^\tau); in smooth non-ideal tactile models, this hard threshold becomes a finite-width crossover. Across Linear-Gaussian checks, MuJoCo+TACTO tasks, and a Franka+DIGIT peg-in-hole setup, PA-VMMIB improves most in high-precision regimes and matches vision-dominated policies when touch is less decision-critical. On the physical task, it improves over vision-only by 26 percentage points at 4 mm clearance and converges toward vision-only performance at 8 mm, supporting phase- and precision-dependent tactile allocation as a practical design principle rather than a universal optimality claim.

Abstract:
Achieving versatile and natural whole-body humanoid interaction control remains challenging due to the high cost of whole-body teleoperation data. We present ZeroWBC, a teleoperation-free framework that learns humanoid whole-body interaction from human egocentric videos paired with synchronized whole-body motion and text annotations. ZeroWBC adopts a generation-then-tracking formulation to tackle the static scene whole-body interaction control problem. Given an initial egocentric image and a language instruction, a fine-tuned Vision-Language Model generates future human whole-body motion tokens, which are decoded into continuous motions and retargeted to the humanoid. The resulting reference motions, together with root and key body-part trajectories, are then executed by a general interactive motion tracking policy. To improve interaction performance, we introduce an interaction-oriented tracking reward that prioritizes global root and key body-part trajectory alignment while preserving natural whole-body motion. Experiments on Unitree G1 humanoid robot show that ZeroWBC enables diverse scene-aware behaviors without robot teleoperation demonstrations. These results suggest a scalable paradigm for learning natural humanoid whole-body interaction from human egocentric data.

Abstract:
Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale. We address this bottleneck by generating vision-language-kinematics (VLK) supervision synthetically in reconstructed scenes. Our pipeline leverages 3D Gaussian Splatting to reconstruct metric-scale indoor environments, synthesizes navigation and object-interaction trajectories using privileged scene information, and renders paired egocentric observations after the fact. We produce 48,000 paired trajectories with no human intervention and train a VLK policy that predicts short-horizon whole-body kinematic trajectories. A whole-body tracker converts these predictions into actions on the physical humanoid. We evaluate on the physical Unitree G1 performing navigation and single-object transport, demonstrating that synthesized interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation.

Abstract:
Learning long-horizon humanoid loco-manipulation poses a dual challenge: it requires not only the robust execution of meta-skills but also their seamless, closed-loop chaining equipped with autonomous recovery. Existing approaches remain limited: explicit humanoid-object interaction representations offer precision but are notoriously difficult for high-level planning, whereas implicit skill embeddings are compact but lack the interpretability required for reliable composition. We propose OmniContact, a hierarchical framework centered on contact flow, a compact representation consisting of key body trajectories and time-series binary contact signals. Leveraging this shared interface, our low-level policy CF-Track learns a unified library of loco-manipulation skills, while our high-level module CF-Gen heuristically synthesizes future contact-flow sequences. Together, they enable robust execution, autonomous failure recovery, and flexible composition of meta-skills for long-horizon tasks. Experiments show that OmniContact significantly outperforms prior baselines in executing meta-skills, chaining multi-skill sequences, and solving long-horizon tasks. Furthermore, our framework naturally integrates with VLMs for semantic task decomposition, enabling complex, semantically grounded loco-manipulation behaviors, such as arranging scattered boxes into a heart shape.

Abstract:
We introduce \emphHumanoid Hanoi, a challenge problem for long-horizon humanoid skill composition that requires a robot to repeatedly sequence pickup, locomotion, and placement skills to rearrange stacked boxes in simulation and on hardware. This symbolically simple task exposes a weakness in a common modular humanoid architecture, where reusable high-level skills are learned on top of a shared, task-agnostic whole-body controller (WBC). While such a WBC provides a unified low-level interface for skill composition, it may not be sufficiently hardened to execute the full collection of learned skills under domain randomization and long-horizon closed-loop execution. We compare task-agnostic strategies for addressing this issue, including skill-specific WBC fine-tuning, residual low-level adaptation, and a rollout aggregation procedure that augments shared-WBC training with closed-loop rollouts from each individual skill under domain randomization. Our experiments show that the strongest long-horizon robustness is achieved not through skill-specific specialization, but by incrementally expanding the closed-loop coverage of the shared WBC while preserving its original task-agnostic tracking objective. Results in simulation and on the Digit V3 humanoid demonstrate improved long-horizon robustness and suggest that preserving low-level controller consistency can be as important as improving individual skills.

Abstract:
General Motion Tracking (GMT) controllers have advanced humanoid whole-body teleoperation, yet inconsistent foot-ground contact remains a recurring deployment artifact, appearing as foot skating, dragging, and mistimed transitions. Standard kinematic metrics do not fully capture this physical gap, and adapting a frozen GMT backbone to contact-sensitive failures creates a retention--adaptation trade-off shaped by the adaptation parameterization. We study this trade-off through controlled comparisons of full fine-tuning, additive residual adaptation, and low-rank updates under matched supervision and budget, and instantiate the resulting insight in Contact-Aware General Motion Tracking (CA-GMT). CA-GMT combines foot contact-grounded rewards with a low-rank update of a frozen GMT backbone, and includes a phase-conditional variant that restricts adaptation to swing frames. On a held-out evaluation corpus, CA-GMT reduces the False Contact Rate (FCR) from 0.401 to 0.235, while lowering tracking error by 16% and jerkiness by 37%. The phase-conditional variant further reduces FCR to 0.199 while remaining within the base policy's pre-training-distribution variance. Retention analysis across PT/FT/ID/OOD splits shows that the evaluated low-rank parameterization preserves out-of-distribution tracking better than the full fine-tuning and residual variants tested in this work. On a Unitree~G1, the phase-conditional variant again achieves the lowest hardware FCR.

Abstract:
Event cameras offer high temporal resolution and low latency, making them ideal sensors for high-speed robotic applications where conventional cameras suffer from image degradations such as motion blur. In addition, their low power consumption can enhance endurance, which is critical for resource-constrained platforms. Motivated by these properties, we present a novel approach that enables a quadrotor to fly through cluttered environments at high speed by perceiving the environment with a single event camera. Our proposed method employs an end-to-end neural network trained to map event data directly to control commands, eliminating the reliance on standard cameras. To enable efficient training in simulation, where rendering synthetic event data is computationally expensive, we propose Approximate Imitation Learning, a novel imitation learning framework. Our approach leverages a large-scale offline dataset to learn a task-specific representation space. Subsequently, the policy is trained through online interactions that rely solely on lightweight, simulated state information, eliminating the need to render events during training. This enables the efficient training of event-based control policies for fast quadrotor flight, highlighting the potential of our framework for other modalities where data simulation is costly or impractical. Our approach outperforms standard imitation learning baselines in simulation and demonstrates robust performance in real-world flight tests, achieving speeds up to 9.8ms^-1 in cluttered environments.

Abstract:
Quadrotor pursuit-evasion in cluttered environments requires the pursuer to jointly track the evader and avoid obstacles. Policies trained with local distance or visibility rewards can become short-sighted: actions that reduce the current distance may still lead to blocked routes, unsafe maneuvers, or states with poor future capture feasibility. We propose a differentiable framework that trains non-myopic pursuit policies through closed-loop rollouts. Specifically, we introduce Global Reachability Value (GRV), a training-only value that evaluates future capture potential, guiding the policy toward obstacle-aware states that facilitate long-term success. Furthermore, to handle intermittent target observations, we introduce the Visibility-Gated Residual Belief (VGRB). This mechanism maintains target state estimation by coupling a Kalman motion prior with a visibility-gated residual predictor. In simulation validation, the proposed framework achieves 89.65%, 86.52%, and 76.04% capture rates with zero collision in three representative cluttered scenarios. Ablations show that GRV reduces myopic route-selection failures, while VGRB improves target-state estimation under occlusion. Experiments in Gazebo and real-world further validate the superiority of our framework.

Abstract:
Pretrained video generation models encode rich priors about physical interactions, but leveraging them for robot control typically requires either resource-intensive video generation at test time or action heads trained from scratch that do not fully utilize the pretrained backbone. We introduce Tokenizing Actions as Video Patches in a Shared Transformer Policy (TAPESTRY), a world action model that repurposes a pretrained video diffusion transformer into a robotic policy without requiring video generation at inference. Our key idea is to treat robot actions as single-patch tokens in the video model's latent space. A lightweight linear projection maps each action timestep to the patch dimensions of the pretrained VAE, and the resulting tokens are processed by the same patchifying layers and transformer backbone used for video. The shared backbone is optimized through three diffusion objectives sampled per minibatch element: Video diffusion uses VAE latents from robot and human videos to establish dynamics priors. Joint video+action diffusion trains on robot demonstrations using a coupled diffusion timestep and a block-causal attention mask. Action-only diffusion denoises actions conditioned on clean image latents. At test time, the model generates a compact action token sequence in a single DDIM step from pure noise, producing smooth action chunks at 84 ms per chunk and avoiding the cost of full video generation. Across simulation benchmarks and real-robot experiments, TAPESTRY achieves strong performance with substantially lower inference cost than comparable joint video-action policies. More information can be found at https://tapestry-wam.github.io/.

Abstract:
Video world models are increasingly used to provide predictive visual representations, yet it remains unclear which pretraining signals induce action-relevant structure in their latent spaces. We study this question through a unified probe-based evaluation across diverse encoder families, including image-only self-supervision, video pretraining with and without latent prediction, reconstruction-based autoencoders, diffusion models, and shortcut-forcing dynamics models. Using a common inverse-dynamics probing objective, we find that action-relevant structure is driven primarily by temporal video pretraining rather than pixel reconstruction fidelity: models with strong pixel decoding quality can exhibit near-zero action recoverability, while video-pretrained self-supervised encoders consistently achieve the best Pareto trade-off between visual fidelity and action prediction. Comparing V-JEPA and VideoMAE further shows that most gains arise from natural-video temporal context, with feature-level latent prediction providing a smaller additional benefit. These trends transfer across robotic benchmarks, though CALVIN reveals that static-environment tasks can partially mask the importance of temporal structure by allowing strong image priors to suffice. Finally, inverse-dynamics supervision substantially improves robustness to visual corruption, suggesting that action-aware objectives regularize latent geometry beyond clean-setting performance. Our results identify temporal predictive structure---not reconstruction fidelity---as the primary ingredient underlying action-relevant video representations.

Abstract:
World action models (WAMs) are a category of policy models that explicitly predict both actions and future world states. They have been shown to outperform action-only models across various robotic manipulation tasks. However, representing the future with full-frame video prediction can be computationally expensive during both training and inference. This raises a central question: what is the minimal state representation that preserves the planning and supervision benefits of future prediction? In this paper, we propose BoxWAM, which replace dense future frames with oriented bounding box trajectories of task-relevant objects. By explicitly imagining object-centric future motion, our method provides a lightweight prediction target that captures the key physical changes induced by robot actions. We show that box trajectory prediction substantially reduces computational cost while improving action prediction performance, suggesting that object-centric compact states can serve as an efficient alternative to video-based world action modeling for robotic manipulation.

Abstract:
World action models (WAMs) provide a promising learning paradigm for robotic manipulation by jointly modeling future observations and robot actions. However, most existing methods operate in 2D image spaces, making it difficult to explicitly capture spatial scene dynamics, maintain cross-view geometric consistency, and effectively align visual observations with robot actions. To address this, we propose GSWAM, a world action model based on 3D Gaussian Splatting (3DGS), which jointly models scene dynamics and robot actions in a unified 3D space. We first compress 3DGS scenes into geometrically consistent latent representations using a Gaussian variational autoencoder (GSVAE). Furthermore, we align scenes and actions within a shared 3D-anchored space, enabling GSWAM to explicitly model the joint distribution between 3D scene dynamics and robot actions. Based on this representation, we design a 3D-anchor-based spatial diffusion transformer to model spatial interactions between scenes and actions. To accommodate their different generative characteristics, we further employ decoupled flow timesteps for the two modalities, allowing the model to jointly learn scene-action generation and action-conditioned future scene prediction. Experiments on both simulated and real-world robotic manipulation tasks demonstrate that GSWAM outperforms representative existing methods in both action generation and future scene prediction. Project website: https://anonymous.4open.science/w/GaussianWAM

Abstract:
Generative video foundation models exhibit strong compositional priors, yet world-action models (WAMs) and video-action models (VAMs) often lose these priors after finetuning on robotic action data. We refer to this discrepancy as the \bf video-action generalization gap. In this paper, we systematically investigate this gap by evaluating a comprehensive design space of VAMs, demonstrating that standard design choices yield no emergent explanation pattern. To explain this behavior, we introduce the Temporal Ratio (TR), an attention-based measure of how strongly the action head relies on future latent rollouts relative to the anchored current frame. TR has two key properties: first, a model's structural reliance on future-predictive latents, measured via TR, acts as a predictor of its compositional generalization capacity; second, it natively fluctuates based on task phase, shifting attention to future frames during planning and reverting to the present frame for precise manipulation. Finally, based on these findings, we propose an inference-time adaptive guidance method, which exploits this intrinsic feature attention pattern to dynamically amplify compositional video conditioning signals precisely when the policy relies on future rollouts. Evaluated on the LIBERO benchmark and real-world tasks, our approach mitigates the OOD-ID compositional generalization gap. More details: \urltemporal-ratio.github.io

Abstract:
Safe control is a prerequisite for real-world embodied intelligence, for which safe reinforcement learning has emerged as a promising paradigm. However, existing safe reinforcement learning methods either require costly real-world exploration or depend on hand-crafted safety functions. Neither scales to vision-language-action models deployed in open-world physical environments. We propose SafeDojo, the first model-based safe reinforcement learning framework for vision-language-action policies designed to learn safe actions through world model-based imagination. Specifically, SafeDojo performs online reinforcement learning on top of an interactive video world model. The world model generates action-conditioned future predictions, from which a tailored ResNet success classifier estimates per-step task progress from imagined frames and a lightweight safety head predicts per-step contact probability from latent context together with the proposed action chunk, enabling simultaneous assessment of task execution and trajectory safety. The decoupled task-reward and safety-cost signals are balanced through a Lagrangian-based constrained GRPO objective, enabling coordinated improvement of task success and safety under explicit constraints. Experiments on SafeLIBERO and real-world Franka deployment demonstrate that SafeDojo consistently outperforms inference-time safety, model-free RL, and model-based RL baselines on five key metrics, achieving over 7% average safe-success improvement across 21 tasks. Our results position world model-based safe reinforcement learning as a scalable and generalizable path toward safe embodied intelligence.

Abstract:
Current methods in robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and robot-operator labor. Towards eliminating these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g. video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82% success, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision.

Abstract:
World Action Models (WAMs) generalize better than standard Vision-Language-Action (VLA) policies to novel motions and environments, because a video-modeling objective lets them learn from abundant unlabeled video rather than scarce labeled robot demonstrations. This generalization is computationally expensive. To complete a task, a WAM runs over multiple inference chunks, and each chunk requires a costly denoising process. Existing acceleration methods reduce this cost by caching and reusing computation within a single chunk's denoising trajectory. Our empirical analysis reveals a substantial source of redundancy they overlook: redundancy across chunks. When a robot executes a smooth behavior, the residuals computed at a given denoising step are strongly correlated from one chunk to the next. We introduce C³ache, a training-free method that caches and reuses these residuals across inference chunks at the same denoising step. Experiments on benchmarks with a Fast-WAM backbone show that C³ache achieves up to a 2.5X speedup in total wall-clock inference time, with negligible degradation in task success rate.

Abstract:
Embodied world models have emerged as a promising paradigm in robotics by predicting how robot actions affect the surrounding scene. However, the rollout inference remains computationally expensive in pixel space, as long-horizon manipulation videos typically have to be generated frame by frame. This cost cannot be easily reduced by indiscriminately dropping frames, since downstream policies rely on complete preservation of sparse task-relevant events such as approach, contact, grasp, and release. To address this challenge, we propose Sparse Keyframe Interpolation Paradigm (SKIP), an event-preserving sparse-to-dense framework that avoids dense frame-by-frame generation. SKIP first identifies task-relevant keyframes by leveraging robot-aware multimodal features. It then synthesizes only these keyframes with a sparse video diffusion model. A learned gap predictor and an action-conditioned interpolator subsequently reconstruct the missing intervals according to the robot actions. On LIBERO, SKIP generates dense rollouts 4.16× faster than a dense baseline while improving visual fidelity and reducing aggregate FVD by 89.0%. Importantly, SKIP-generated videos are effective policy-training data. Even when they fully replace real demonstrations, \pi_0.5 success drops only 1.3 pp in LIBERO simulation and 6.7 pp on the real robot, whereas fully dense frame-by-frame generation collapses by 48 to 58 pp.

Abstract:
Learning dexterous manipulation requires demonstrations that preserve fine hand-object interactions while remaining executable at deployment. Existing pipelines either lose deployable dexterity through retargeting or embodiment conversion, or rely on robot-specific teleoperation that is costly to scale and often lacks intuitive, contact-aware control for dexterous data collection. We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module that integrates a lightweight dexterous hand, in-hand vision, and fingertip tactile sensing. A palm-side isomorphic teleoperation glove maps human finger inputs to robot-hand joint commands, enabling real-time, retargeting-free, intuitive, and precise hand control. The shared hand and sensing modules yield zero-gap end-effector data, with matched in-hand observations, tactile signals, contacts, and hand actions between collection and deployment. Across eight real-robot tasks spanning fine-grained, contact-rich, long-horizon, and bimanual manipulation, policies trained on RealDexUMI data achieve an average success rate of 88.75%, generalize to unseen initial poses, and transfer across three embodiments.

Abstract:
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present Contact Wrench Guidance from Human Demonstration in Robotic Dexterous Manipulation (CHORD), a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning. The key idea is object-centric contact wrench space guidance: we represent human and robot motions by the forces and torques they can induce on the object, enabling similarity to be measured by the induced instantaneous motions. This guidance makes reinforcement learning more scalable for contact-rich dexterous manipulation. We further introduce a large-scale simulation benchmark with 4739 bimanual dexterous manipulation tasks, constructed from motion-capture datasets and reconstructed in-house videos. Evaluated on 1831 benchmark tasks, CHORD achieves an average success rate of 82.12%, demonstrating strong scalability. CHORD also generalizes to whole-body manipulation from hand-only and third-person demonstrations, achieving a 90.77% success rate, and the learned policies transfer to the real world in both open-loop and closed-loop settings.

Abstract:
Robot manipulation has traditionally relied on end-effector-centric interactions. In contrast, humans routinely exploit whole-arm contacts to improve efficiency, robustness, and adaptability, especially when the hands are occupied or when precise grasping is unnecessary. Enabling similar capabilities in robots remains challenging: the robot must select where on its arm to make contact, generate effective non-prehensile interactions, and satisfy task-specific pose or safety constraints on the end-effector. We propose Whole-Arm Contact Learning (WACL), a hierarchical framework that carries each whole-arm contact task to a trajectory optimizer through three robot-agnostic representations: semantic keypoints that summarize the scene, a reference contact trajectory that specifies where and how the arm presses on the surface, and safety-aware path constraints, generated from language by vision-language models, that protect the end-effector's task. The robot's kinematics enters only in a downstream surface-contact optimizer, which grounds these representations onto the arm by jointly searching over which link makes contact, where on that link, and the joint configuration that realizes it. This decomposition lets WACL learn the contact trajectory from a handful of bare-hand human demonstrations, while a new end-effector constraint changes only the optimizer's input rather than any trained component. Real-world experiments on a 7-DoF arm across four whole-arm tasks show that WACL sustains contact under end-effector constraints where a strong keypoint-conditioned baseline fails (78% vs. 33% success), generalizes to new constraints without retraining, and learns without teleoperation. Website: https://wacl-robot.github.io.

Abstract:
Post-training Vision-Language-Action (VLA) models into policies that can be reliably deployed on real robots remains a major bottleneck. SFT and DAgger exploit failure signals only indirectly, and reward-based RL is bottlenecked by the difficulty of real-world reward design and of training reliable critics. We present FlowPRO, a reward-free offline reinforced fine-tuning framework for flow-matching VLAs. Algorithmically, we propose RPRO (Robotic Flow-matching Proximalized Preference Optimization), a preference-optimization objective tailored to the flow-matching action head of VLA models. RPRO pairs a contrastive optimizer with an explicit proximal regularizer that anchors the absolute magnitude of the implicit reward, thereby eliminating the reward-hacking failure mode of plain Flow-DPO. On the data side, a teleoperated intervention-and-rollback paradigm produces naturally paired positive and negative trajectories (\tau^w, \tau^l) on a real robot from a single operator action; a Smooth Interpolation procedure, combined with batch mixing, then converts these sparse corrections into dense per-state supervision while preserving the base policy's capabilities. On four long-horizon bimanual tasks, FlowPRO attains the highest success rate, outperforming four representative baselines, and ablations confirm the contribution of each loss component.

Abstract:
Scaling imitation learning requires large datasets, yet human teleoperation inevitably produces mixed-quality demonstrations containing hesitations and recoveries. Prior frame-level progress reward models supervise on absolute temporal progress proxies that suffer from label noise, or require costly human annotations to define subtask boundaries. We present WARP (Warp-Augmented Relative Progress), a novel fully self-supervised algorithm for learning dense, signed relative progress magnitudes directly from successful demonstrations. WARP generates per-frame progress targets via time-warp augmentations of demonstrations (variable playback speeds and reversals) and we train WARP-RM to predict the normalized elapsed time between input frames. Aggregating these predictions across overlapping windows yields a dense frame-level progress signal. We then introduce WARP-BC, which leverages these scalar reward estimates to upweight high-advantage action chunks during behavior cloning, where chunk-level advantage is obtained by aggregating per-frame rewards. We evaluate our approach on a physical bimanual robot system performing a long-horizon deformable object manipulation task: folding T-shirts from a random crumpled start. To evaluate policy robustness against suboptimal data, we construct training datasets of varying quality using episode length as a proxy for teleoperation sub-optimality. As the dataset is widened to admit more inefficiencies, WARP-BC maintains a 19/20 success rate compared to vanilla BC's collapse to 2/20, improving throughput by up to 18x.

Abstract:
Active exploration in cluttered environments is essential for reconstructing occluded 3D scenes, requiring both observation and physical interaction. Existing methods rely on belief maps that indicate where the scene is unresolved but not how each region can be resolved, estimating the gains of viewing and pushing only through costly lookahead or hand-tuned heuristics. We propose DiVE (Decomposed Visibility Estimator), a learned model that predicts two decomposed visibility maps: a view-resolvable map and a push-resolvable map, making explicit what each action mode can reveal. By converting these modality-specific predictions into a common visibility-gain scale, DiVE supports a hierarchical decision structure: the robot first chooses between observation and interaction, then selects the specific action through a mode-specific policy. In simulation, DiVE improves occupancy IoU by 11.3% on extreme-occlusion shelf scenes and runs 13.2× faster per step (19.8× faster in planning) than a state-of-the-art baseline. Deployed zero-shot on a real robot, it recovers 79.2% of initially occluded objects.

Abstract:
Simulation-based RL for contemporary robot control is increasingly organized around GPU-resident simulation: physics, rollout collection, and learning are placed on a single GPU-centric execution path. This paradigm has greatly improved training speed, but it has also encouraged a default assumption that efficient training requires physics to reside on the GPU. We revisit this assumption. Our view is that, in simulation-dominated robot control, the essential question is not which processor runs physics, but whether simulation throughput, policy learning, and runtime synchronization form an efficient end-to-end loop. We present UniLab, a heterogeneous CPU-simulation / GPU-learning architecture that decouples CPU-parallel simulation from GPU policy updates through a unified runtime for data movement, buffering, and synchronization. UniLab is implemented as a complete and extensible training system using MuJoCoUni and MotrixSim CPU-batched physics backends, supporting PPO, FastSAC, FlashSAC, and APPO. On representative simulation-based robot control tasks, UniLab improves end-to-end training efficiency by 3--10× under the same hardware configuration, while reducing dependence on the NVIDIA CUDA-based software stack and supporting cross-platform execution on the Apple macOS platform and the AMD ROCm and Intel XPU accelerator backends. These results show that GPU simulation is an effective path to efficient training, but not a necessary one, broadening the practical system choices available for robot RL training.

Abstract:
Open-world delivery requires mobile manipulators to follow free-form user instructions and manipulate potentially novel objects. Existing dual-system approaches use high-level grounding models to convert language into grounded visual prompts, but their low-level controllers can remain brittle under noisy perception, dynamic scenes, and contact-rich interactions. We instead use a pretrained flow-matching vision-language-action model as the low-level control interface, leveraging its reactivity and robustness to environmental changes while treating the grounding output as a spatial cue for policy steering. Our key insight is that the pretrained VLA already provides a strong manipulation prior, while the spatial cue supplies the missing target information needed to guide actions under novel language--object mappings. Concretely, we introduce a lightweight cue-conditioned adapter. The adapter is first trained with contrastive objectives to produce salient and spatially discriminative cue representations, and is then supervised to predict a diagonal affine transformation over the generated action chunk, aligning policy steering with the cued target. Across tabletop and mobile-base settings, our method improves instruction following and manipulation success on both in-domain and out-of-domain objects, achieving up to near 2x improvement in average task success rate with negligible inference overhead.

Abstract:
Learning-based surgical robot autonomy requires large-scale demonstrations with synchronized videos and robot actions, but such data are exceedingly rare in clinical or realistic tissue settings because robot kinematics are typically inaccessible outside controlled research systems. In contrast, phantom data collected on research platforms provide accurate action labels but lacks the visual diversity of real tissue. We propose SurgVIL, a framework for scaling surgical robot imitation learning using open-source surgical videos. SurgVIL combines kinematically labeled phantom robot demonstrations with surgical videos from open-source datasets and online sources for policy learning. Since these videos lack robot motion labels, we estimate approximate kinematics as weak supervision. We evaluate SurgVIL on two da Vinci robot tasks: needle pick-up and cholecystectomy cutting. Across ACT, Pi0, and Gr00t-H backbones, adding surgical videos substantially improves generalization to real-tissue and out-of-distribution settings, suggesting a scalable path from phantom training toward generalizable surgical robot policies.

Abstract:
We propose Ambient Diffusion Policy, a simple and principled method for imitation learning from suboptimal data in robotics. High-quality, task-specific robot data is expensive and time-consuming to collect, while suboptimal datasets with lower-quality or out-of-distribution demonstrations are abundant. Existing methods that co-train on both data sources in robotics often fail to separate the meaningful and the harmful features in suboptimal samples. In contrast, our method attempts to extract only the useful learning signal by introducing a new axis to co-training in robotics: noise-dependent data usage. Ambient Diffusion Policy restricts the contribution of suboptimal data during training to only the high and low diffusion times. We observe that robot action data exhibits a spectral power law, which induces two important properties on the optimal Diffusion Policy: locality and a global-to-local hierarchy. These properties rigorously justify our algorithm. Our experiments validate Ambient Diffusion Policy on four types of suboptimal action data (noisy trajectories, sim-to-real gap, task mismatch, and large-scale data mixtures) and six tasks. The results show that it effectively learns from arbitrary sources of suboptimal data. Notably, it outperforms existing baselines by up to 33% when scaled to Open X-Embodiment---a large real-world dataset with unstructured and heterogeneous distribution shifts. Overall, Ambient Diffusion Policy increases the utility of suboptimal demonstrations and expands the set of usable data sources in robotics.

Abstract:
Vision-Language-Action (VLA) models require large-scale video-action pairs, yet real teleoperation remains scarce. While generated robot videos offer a scalable alternative, existing methods treat them as real robot data by recovering pseudo-actions from synthesized pixels. We argue that deriving low-level control from generated visuals is a mismatched abstraction. A video captures only geometry: the spatial trajectory representing the where of a task. A real demonstration captures control: the exact motor commands representing the how. Human-to-robot video generation preserves these unequally: the visible geometry survives the generation process, while the underlying control signals are lost. This Asymmetric Preservation Principle dictates a clean rule: this surviving geometry should solely supervise visual perception, leaving control to real demonstrations. Following this principle, we propose GRA (Geometry-guided Representation Alignment), which extracts the geometric content as future 2D end-effector waypoints, computed from the source human video through pose estimation, retargeting, simulation, and calibrated projection, and routes them to the VLA vision backbone via an auxiliary 2D head. The action head is trained on real demonstrations only. During fine-tuning, the waypoint loss persists as a spatial representation anchor that prevents the backbone from losing its geometric grounding. On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations, suggesting that correctly routed geometry bridges generated videos to robot policies more reliably than recovered actions.

Abstract:
Learning generalizable and robust behavior cloning policies requires large volumes of high-quality robotics data. While human demonstrations (e.g., through teleoperation) serve as the standard source for expert behaviors, acquiring such data at scale in the real world is prohibitively expensive. This paper introduces ExpertGen, a framework that automates expert policy learning in simulation without any human demonstrations, while preserving strong sim-to-real transfer performance. ExpertGen first initializes a behavior prior in using a diffusion policy trained on synthetic data (e.g., Code as Policy). Reinforcement learning is then used to steer this prior in massively parallel simulation toward high task success by optimizing the diffusion model’s initial noise while keeping the denosing process frozen. In this way, ExpertGen regularizes exploration to remain within safe behavior manifolds, enabling stable learning from sparse rewards without additional reward engineering. Extensive experiments on challenging manipulation benchmarks demonstrate that ExpertGen reliably acquires high-quality policies with strong robustness and generalization. On long-horizon manipulation tasks, ExpertGen achieves success rates of up to 85%, while on industrial assembly benchmarks it achieves a 90.5% overall success rate, outperforming all baseline methods. To validate sim-to-real transfer, the learned state-based expert policies are further distilled into visuomotor policies via DAgger and successfully deployed on real hardware.

Abstract:
In robot imitation learning, influence functions provide a principled approach to quantify each demonstration's effect on robot task outcomes, yet scaling them to billion-parameter Vision-Language-Action (VLA) models is limited by computational and multitask bottlenecks. To this end, we propose ATHENA, an influence function framework tailored for multitask VLA data curation at billion-parameter scale. Concretely, it leverages the Kronecker structure of linear-layer gradients to reduce projection cost, and approximates dense Hessian inversion with a rank-r Random Truncated Approximation, achieving about a 313.4× speedup in influence computation. Furthermore, ATHENA formulates global and local interactive influence to balance data curation across 50 jointly trained tasks. Extensive evaluations on RoboTwin 2.0 and real-robot deployment, covering 9.34 and 6.90 hours of demonstrations, respectively, show that ATHENA matches or exceeds full-data joint fine-tuning using only 50% of demonstrations in simulation and 66% of data across six real-robot tasks. Overall, ATHENA demonstrates its effectiveness for data curation in billion-parameter multitask VLA fine-tuning. The code will be released upon acceptance.

Abstract:
Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset. Yet not all states are created equal: in a small number of critical states, even a slight deviation from the expert action can lead to a failure, while in many other states such errors are recoverable and inconsequential. We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state. We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy. SCRIM assigns each demonstration state a score reflecting how vulnerable the task outcome is to small, policy-relevant action errors. We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance. We further show that SCRIM enables data curation by selecting compact, high-value subsets of demonstrations that preserve performance while substantially reducing the amount of training data. We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success.

Abstract:
While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. The anonymized project website can be found here: https://anonymousperson12321.github.io/SCOUT/.

Abstract:
In real-world environments, fast and accurate adaptation of dynamics models is a fundamental challenge for model-based online control. This is exemplified in autonomous drifting, where effective vehicle parameters shift rapidly with tire temperature, surface friction, tire wear, and load transfer. Prediction errors from the dynamics model in this regime differ in scale across state channels and vary across driving regimes, making fixed squared-error objectives brittle. Likelihood objectives naturally account for this structure, but joint negative log-likelihood training of dynamics parameters and residual variances is often unstable. We address this limitation using a decoupled profiled-likelihood objective for training history-conditioned structured dynamics estimators directly from observed state-control transitions. Using this objective, we train a bidirectional transformer that aggregates long state-control histories through a learned query token and outputs parameters of a structured vehicle dynamics model in a single forward pass. The resulting estimator runs at over 100Hz and is integrated into an existing MPC framework. We evaluate the approach on two full-scale vehicles, a heavily modified Toyota Supra and a Lexus LC500, across multiple drifting experiments with varying operating conditions. Our estimator improves tracking over hand-tuned physics and neural-network dynamics baselines, achieving up to 2.4 times lower RMS path-tracking error. These results show that profiled likelihood-based parameter adaptation can reliably and efficiently improve closed-loop performance in real-world applications of model-based control.

Abstract:
Parallel-jaw grippers are the default manipulator choice in robotics because they are simple, robust, and inexpensive. Their limited in-hand mobility, however, often forces large arm motions and restricts dexterous manipulation in confined workspaces. We present a parallel-gripper upgrade: a double-belt-based finger module that preserves standard opening/closing while adding three in-hand degrees of freedom (DoF): translation, pitch, and roll. The mechanism is deliberately kept simple and engineered for inexpensive manufacturing and straightforward integration, preserving the reliability and precise control of traditional parallel grippers while greatly broadening the range of manipulation capabilities. To demonstrate the utility of the added DoFs, we integrate the gripper in two control pipelines. First, we adapt a model predictive controller for in-hand manipulation of known objects. Second, we introduce a lightweight teleoperation interface that enables simultaneous control of the robot arm and gripper (10 DoFs total) with minimal hardware. Across a suite of challenging manipulation tasks executed via teleoperation, MPC, and trained policies, the proposed gripper consistently improves dexterity and task feasibility compared to a conventional parallel gripper.

Abstract:
We introduce Accelerating DExterity via Pre-Training (ADEPT), a framework for learning high degree-of-freedom (DoF) arm-hand dexterity through large-scale RL. ADEPT pretrains a dexterous manipulation policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior on contact-rich downstream tasks. This enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots. We find that the pretrained policy generalizes and can solve the reposing phase of downstream tasks zero-shot. However, naive RL fine-tuning rapidly degrades this capability during task transfer. To address this, we propose a post-training framework that combines behavior-cloning distillation to adapt the actor to new observations, frozen-actor critic warm-up to calibrate it on the downstream reward, and conservative on-policy updates to prevent large policy drift. To safely exploit the full kinematic dexterity of an arm-hand robot, we introduce a joint configuration-space Geometric Fabric controller that mediates between the RL policy and the robot. Finally, we distill the post-trained teacher into a stereo-RGB student policy that zero-shot sim-to-real transfers on a 23 DoF Kuka-Allegro robot. The resulting system solves contact-rich, long-horizon manipulation tasks requiring grasping, reorienting, and insertion across a wide range of initial states.

Abstract:
Minimal-intervention target retrieval asks a robot to extract a target object from clutter while limiting non-target removals. The challenge is that a single scene may support multiple candidate grasps, and whether an object needs to be removed depends on the chosen grasp and the current execution stage, so scene-level clutter cues alone are insufficient. We address this with grasp-conditioned dependency graphs, where candidate grasps are action nodes and typed object-grasp edges represent approach- and lift-stage blockers. By tying blockers to concrete candidate grasps, the graph lets the planner distinguish objects that merely surround the target from those that are likely to block a particular grasp. To learn these dependencies, we run single-object intervention tests on 8,687 scene-target samples from 500 scenes, yielding 3,665,775 labeled edges, typed dependencies, and oracle minimal blocker sets. An action-centric edge-context model predicts object-grasp dependencies from observed scenes, and a short-prefix planner ranks blockers, selects a target grasp, and replans after each intervention. Stored-label planning provides a large-scale controlled test of the success-removal trade-off, closed-loop MuJoCo rollouts test the same policy under physical scene updates, and hardware trials validate the execution interface.

Abstract:
Slip detection is fundamental to dexterous manipulation, yet existing systems often lack precise detection latency characterization and cross-platform generalization. We present SlipSense, a multimodal tactile slip detection framework built on TacV5, a compact sensor integrating a 32x32 piezoresistive array (240 Hz) and a 3-axis MEMS accelerometer (8 kHz). The piezoresistive array captures spatial pressure distribution while the accelerometer captures friction-induced vibration, providing complementary slip cues. The framework performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. Experiments on a 1.4M-frame dataset spanning 37 objects demonstrate that the two modalities are crucially complementary: it achieves 96.7% Macro F1 with false-positive rate below 1.6%, detecting 76% of slip events within 23.1ms. Remarkably, trained purely on a UMI, SlipSense generalizes zero-shot to a Tesollo dexterous hand, transferring across unseen objects, distinct sensor units and new robotic platforms without any retraining.

Abstract:
Recent advances in humanoid whole-body motion tracking have enabled the execution of diverse and highly coordinated motions on real hardware. However, existing controllers are commonly driven either by predefined motion trajectories, which offer limited flexibility when user intent changes, or by continuous human teleoperation, which requires constant human involvement and limits autonomy.This work addresses the problem of how to drive a universal humanoid controller in a real-time and interactive manner. We present TextOp, a real-time text-driven humanoid motion generation and control framework that supports streaming language commands and on-the-fly instruction modification during execution. TextOp adopts a two-level architecture in which a high-level autoregressive motion diffusion model continuously generates short-horizon kinematic trajectories conditioned on the current text input, while a low-level motion tracking policy executes these trajectories on a physical humanoid robot. By bridging interactive motion generation with robust whole-body control, TextOp unlocks free-form intent expression and enables smooth transitions across multiple challenging behaviors such as dancing and jumping, within a single continuous motion execution. Extensive real-robot experiments and offline evaluations demonstrate instant responsiveness, smooth whole-body motion, and precise control.

Abstract:
High-precision humanoid control is limited by target-domain dynamics mismatch, where the same control objective can induce different realized motions under changes in terrain, payload, or actuator response. Existing methods either pursue zero-shot transfer through domain randomization or in-context adaptation without target-domain specialization, or require heavy adaptation pipelines that leverage target-domain data, such as model calibration, residual learning, or policy retraining. In this paper, we present FADA (Few-Shot Domain Adaptation via Dynamics Alignment), a three-stage Planner-Inverse Dynamics Model (Planner-IDM) framework for few-shot adaptation in humanoid control. FADA first trains an oracle policy with privileged information and then distills the oracle behavior into a deployable Planner-IDM student through DAgger. At deployment, FADA freezes the planner and finetunes only the IDM using approximately 2 minutes of target-domain rollouts with standard supervised learning. Rather than requiring optimal demonstrations or rewards, FADA uses the paired actions and observations that are observed during these rollouts as supervision, aligning the IDM's action generation with target-domain dynamics. Experiments show that FADA outperforms both in-context and end-to-end adaptation baselines, improving task performance under dynamics shifts and enabling real humanoid robots to execute diverse high-precision whole-body tasks.

Abstract:
In legged locomotion, divergent components of motion (DCMs) have emerged as characteristic states for balance control. They isolate the unstable mode of the dynamics, but only apply to reduced models such as the linear inverted pendulum. In this study, we show how divergent components of motion can be more generally cast as Koopman eigenfunctions, and derived from closed-loop data without committing to a kinematic model. Whereas Koopman analysis typically targets small eigenvalues that characterize conserved quantities and persistent dynamics, we deliberately search for unstable eigenpairs. The resulting Koopman DCMs are data-driven observables encoded from measurement histories (signals available on the real robot, requiring neither simulation groundtruth nor full-state estimation) and regularized by their coupling with control inputs through the DCM recursion. Applied to legged balancing, we show how Koopman DCMs learned from one hour of robot data improve tracking of reference walking patterns. We further discuss how Koopman DCMs apply to learning state-based viability constraints when combined with model predictive control.

Abstract:
Reinforcement learning has produced capable controllers across a broad range of legged-robot tasks, but a subset of these tasks fail to converge under standard training: those for which most exploration trajectories terminate before producing useful gradient signal. To address such tasks we introduce the Actuator Dynamics Curriculum, a procedure that initializes joint stiffness at a high value and anneals it toward the system-identified value as completed episode lengths grow. Using a cart-pole system as a representative example, we show that higher actuator natural frequency under critical damping enlarges the viability kernel of the underlying Markov Decision Process, and a larger viability kernel reduces the sample complexity of policy-gradient methods. We validate the kernel monotonicity on the cart-pole and apply the curriculum to a quadrupedal-to-handstand transition on the Boston Dynamics Spot, a narrow-viability task that fails to converge under fixed identified stiffness. The trained policy executes the transition in simulation across 10 seeds and transfers to hardware. More broadly, our results suggest that simulated actuator dynamics is a useful axis along which to design curricula for tasks in which exploration is bottlenecked by termination conditions rather than by reward signal.

Abstract:
Training locomotion policies for complex unstructured terrain requires a curriculum to avoid early exploration failures. However, since unstructured terrain lacks explicit difficulty ordering for curriculum design, existing methods resort to heuristic curricula over parameterized terrains. This abstraction limits generalization, as policies can overadapt to near-fixed perceptual patterns. To address this, we propose TACL, an Trajectory-level Automatic Curriculum Learning framework that generates training tasks directly from unstructured terrain maps. At each curriculum update, the evaluator learns a difficulty function for the current policy that maps a given trajectory task to a difficulty score. The sampler then proposes new trajectories guided by the learned evaluator as the curriculum for the next policy update. This forms a closed loop in which the curriculum is iteratively matched to the evolving policy. Quantitative and qualitative experiments show that \ourname continuously provides effective curricula on unstructured terrain, improving trajectory success rate by \(56.3%\) over direct training without curriculum. Compared with handcrafted curriculum learning, our method improves success rate by \(18.5%\) on the hardest terrain tasks and by up to \(39.74%\) when evaluating traversal from diverse approach directions on the same obstacle type.

Abstract:
Learning motion planning policies that remain reliable under closed-loop execution is a central challenge in Autonomous Driving (AD). Imitation Learning (IL) scales well to large AD datasets but suffers from distribution shift, causal confusion, and compounding errors under closed-loop execution. In contrast, Reinforcement Learning (RL) typically requires handcrafted rule-based rewards and massive simulation. We introduce a simple closed-loop IL framework that uses RL purely as an optimization tool, guided by a trajectory-distance reward derived from logged expert data. A lightweight MLP policy trained with our method achieves a score of 86.51 on the nuPlan Val14 Reactive benchmark, a +3.2 point improvement over the previous best IL planner (Flow Planner), with >15x faster inference. Our central finding is that training strategy plays a role at least as important as model complexity: closed-loop optimization with trajectory-based supervision is sufficient to outperform recent large-scale generative IL planners while dramatically reducing inference cost. Code and trained models will be made publicly available upon acceptance.

Abstract:
Vision-Language Navigation has increasingly emphasized high-level instruction reasoning, memory, global map construction, and instruction decomposition, while the low-level action representation remains comparatively underexplored. We propose CoFL-S, a low-level vision-language-action framework that predicts a language-conditioned flow field over the robot's local visible sector and generates continuous trajectories by rolling out the predicted field. To train this low-level representation, we convert each VLN-CE episode, originally a whole-episode instruction paired with an action sequence, into frame-level local supervision with aligned sub-instructions and matched action, trajectory, and dense flow-field targets. For evaluation, we introduce a continuous-time Habitat benchmark that isolates low-level action interfaces from instruction decomposition and executes all methods through a shared velocity-command controller, enabling decomposition-independent closed-loop comparison across different planner frequencies rather than fixed discrete forward-and-turn transitions in VLN-CE. Under matched encoders and training settings, CoFL-S consistently outperforms action-token and action-chunk baselines across planner frequencies in the continuous-time Habitat benchmark, and zero-shot real-world closed-loop deployment further shows its advantage over both baselines beyond simulation.

Abstract:
Driver alerting from dashcam video is a sequential decision problem under partial observability: a system must warn early enough to support driver response, while avoiding frequent nuisance alerts. Existing approaches often reduce this problem to binary risk prediction, forcing a thresholded silent-or-alert decision even when visual evidence is incomplete. We propose VLAlert, a vision-language framework that formulates driver alerting as a tri-action policy over Silent, Observe, and Alert. The Observe action serves as an internal evidence-gathering state, allowing the system to defer uncertain warnings and adapt its subsequent observation window before issuing a driver-facing alert. VLAlert adapts a 4B vision-language model to generate structured safety-evidence spans and uses hidden states pooled from these spans as compact belief representations for danger estimation and policy prediction. To evaluate alerting beyond threshold-free anticipation metrics, we construct VLAlert-Bench, a unified per-tick benchmark from multiple driving-alert datasets, and introduce DAUS, a driver-aware utility score that jointly measures ranking, video coverage, false-alert burden, and lead-time utility. Experiments show that VLAlert achieves the best overall operating-point utility among representative video and VLM baselines. Ablations further indicate that structured belief-span pooling improves alert separability and that retaining Observe as a separate action provides useful supervision for uncertain pre-alert states.

Abstract:
Vision-Language-Action (VLA) models have made rapid progress in grounding language-conditioned visual observations into robot action chunks. Yet the action interface directly shapes how these chunks are learned and executed on physical robots. Cartesian end-effector space provides an object-centric and geometrically meaningful representation of motion intent, but executing Cartesian actions on joint-controlled robots requires an additional inverse-kinematics layer, which can yield non-unique or less stable joint targets on redundant robots. Joint-space actions are directly compatible with low-level controllers, but can obscure task-level motion structure and require policies to internalize embodiment-specific kinematics. This creates a trade-off for single-space VLAs between clear motion intent and stable execution. To address this trade-off, we introduce DualSpace-VLA, a dual-space VLA framework that uses Cartesian-space motion intent to guide executable joint-space action generation. DualSpace-VLA predicts Cartesian motion-intent tokens from VLM representations under chunk-wise delta end-effector supervision and injects these tokens into a flow-matching joint-action expert through intent-guided cross-attention and gated fusion. Differentiable forward kinematics further maps generated joint trajectories to Cartesian end-effector trajectories, providing a trajectory-level geometric regularizer during training without inverse kinematics at inference time. Extensive experiments in RoboTwin simulation, real-world bimanual manipulation, and few-shot transfer to a low-cost embodiment show that DualSpace-VLA outperforms both end-effector-only and joint-only VLA baselines.

Abstract:
Vision-Language-Action (VLA) models remain brittle under visual distribution shifts, often relying on spurious correlations tied to domain-specific factors rather than task-relevant structure. We propose Domain-Invariant Latent Lookahead (DILL), a representation-learning framework that mitigates shortcut learning in VLA policies. Our key idea is to supervise policies with domain-invariant future latents learned from domain-transformed trajectory data. A Task-Domain Encoder is trained with contrastive objectives and Gaussian disentanglement regularization to separate task-relevant structure from domain-specific visual variation. The learned encoder then provides future latents for VLA policy learning through lookahead prediction and domain disentanglement, encouraging the policy to focus on task-relevant structure rather than incidental visual factors. Under counterfactual task--view compositions, DILL reduces shortcut degree while improving OOD success. On LIBERO-Plus visual perturbations, DILL achieves the highest average success rate of 69.1%, outperforming the strongest matched-input baseline by 11.4 absolute points, while limiting the average performance drop to 12.5 points. Latent-space diagnostics further show that DILL better preserves task-consistent structure while suppressing domain-specific variation.

Abstract:
Robotic foundation models have recently made substantial progress in multi-task capability, cross-embodiment transfer, and language-conditioned control. Yet deploying these models robustly across diverse real-world settings remains difficult, in part because policies often fail to distinguish between causally relevant visual structure and spurious scene-level correlations. For instance, a policy trained to pick up a mug may fail when the table is shifted, the room becomes dimmer, or previously unseen distractors appear nearby. We identify this failure mode as shortcut learning: the tendency of a model to exploit predictive but non-causal correlations in the training distribution rather than the task-relevant visual evidence that determines successful action. Although shortcut learning has been extensively studied in computer vision and broader machine learning, its role in robotic foundation models remains comparatively underexplored. In this paper, we study shortcut learning as a central obstacle to scalable robotic foundation models and propose Artificial Foveated Perception (AFP) as a direct intervention in the perception pipeline. AFP is a lightweight, policy-agnostic module that takes the same vision and language inputs as existing Vision-Language-Action and World Action Model pipelines and predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions. These masks can be used for visual preprocessing, token filtering, attention modulation, or auxiliary grounding losses, enabling AFP to suppress task-irrelevant visual evidence without modifying the core policy architecture. We evaluate AFP across state-of-the-art robotic foundation models and show that foveating perception reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations. Through ablations over integration modes and mask usage, we further show that these gains arise from directing policy learning and inference toward task-relevant visual regions. These results suggest that task-conditioned foveated perception is a practical mechanism for making robotic foundation models more robust, data-efficient, and scalable.

Abstract:
Visual bottlenecks that focus policy inputs on regions of interest (ROIs) can improve data-efficient visuomotor learning by separating where to look from how to act. Many ROI interfaces rely on external spatial labels, such as gaze, object classes, or affordance annotations. Label-free alternatives often derive crops from trajectories by detecting gripper or motion events and centering a fixed crop at the projected end-effector. Such action-derived crops are useful spatial priors that require no additional labels, but they encode fixed choices about event timing, proxy points, and crop scale. When the visual evidence needed for control lies away from the end-effector or changes continuously with task progress, these crops can become misaligned. We propose Seeker, a task- and state-conditioned readout that learns attention from action. Starting from frozen DINO features, Seeker iteratively updates a query with gathered visual evidence, producing progression-aware ROIs solely from action supervision. The learned ROI serves as a spatial interface for RGB cropping, mask-guided background augmentation, and point-cloud filtering. Across simulation and real-robot experiments, Seeker improves data efficiency and robustness over no-crop, augmentation, and action-derived crop baselines. On real robots, Seeker raises average in-domain success from the best baseline's 48.3 to 76.7% and success under lighting/background shifts from 20.0 to 60.0%.

Abstract:
We propose world-language-action (WLA) models as a new class of embodied foundation models. WLA takes textual instructions, images, and robot states as inputs to jointly predict textual subtasks, subgoal images, and robot actions, conjoining the \emphworld modeling interface to learn from extensive egocentric videos as in the world-action model (WAM) and the \emphlanguage reasoning capacities to solve complex long-horizon tasks as in vision-language-action (VLA) models. At the core of WLA lies an \emphauto-regressive (AR) Transformer backbone, instead of a bidirectional diffusion Transformer as in WAMs, to predict the \emphnext-state, comprising the \emphsemantic-level textual intention and complementary \emphfine-grained physical dynamics. The physical dynamics are supervised by the world modeling objective based on a dedicated World Expert, and are leveraged to ease the characterization of the state-action correlation for the Action Expert. WLA leverages meta-queries to make the world prediction \emphimplicitly impact the action generation so that the former can be disabled during inference. The world prediction can also be activated to enable test-time scaling for improved robot control. Our WLA-0 prototype, with 2B active parameters, achieves 40 ms per inference on an NVIDIA RTX 5090. Evaluations across simulated and real-world environments demonstrate that WLA-0 achieves state-of-the-art multi-task and long-horizon learning abilities, e.g., 92.94% success rate on RoboTwin2.0 Clean and 56.5% success rate on RMBench. WLA-0 also holds the promise to learn novel tasks directly from \emphcross-embodiment robot videos without action annotations.

Abstract:
Online reinforcement learning for flow-based manipulation policies is challenging because multi-step solver-based action generation makes the action likelihoods required by standard policy-ratio objectives difficult to access. Hopefully, NFT-style methods offer a promising likelihood-free route for local policy optimization, and recent embodied extensions have begun to adapt this idea to flow-based robot policies. However, several structural issues remain: trajectory-level outcomes provide noisy and misaligned credit for action chunks and their intermediate solver states; full-space mirrored comparisons ignore the compositional geometry of robot actions; and local updates may still follow noisy behavior-sampled continuations without explicitly identifying better alternatives, which may be guided toward the manifold of suboptimal actions. We present Preference-Aware Semantic NFT (PAWN), a unified NFT-style online RL framework that addresses these issues through structured local preference optimization. PAWN derives implicit chunk-level preference signals by lifting solver-step density comparisons to chunk-path scores, reorganizes mirrored local evidence into SE(3)-aware motion modes, and uses conditional pair guidance to favor locally better continuations. Experiments on flow-based manipulation benchmarks show consistent gains in adaptation efficiency, final success, and OOD robustness over strong online RL baselines.

Abstract:
Direct imaging of Earth-like exoplanets requires 10 milliarcsecond angular resolution and 10^-9 contrast; engineering a single telescope that simultaneously exhibits the aperture baseline and wavefront control needed to meet both requirements has proven challenging. We build on recent advances in distributed-aperture interferometer designs to recast the problem as a high-dimensional visuomotor optomechanical robotic control task: many physically separate mirror segments must be aligned to within a small fraction of the observing wavelength, given only science path focal plane imagery. We train recurrent visuomotor policies with PPO to phase such a telescope end-to-end from the interferometric focal-plane image. Direct application of these methods to the full forty-five-DOF, fifteen-segment problem is ineffective; instead we train fifteen per-segment alignment skill policies, each articulating three actuators, and compose them at inference into a single 240-step rollout. The same policy architecture, with a target-conditioning input and a modified reward, is able to perform directed wavefront shaping to suppress flux in designated dark-hole regions by approximately two orders of magnitude across many target regions. A coarse-to-fine variant of the same pattern co-phases a two-segment aperture from 2 arcsec initial differential motion in a median of fifteen control steps, approximately five orders of magnitude faster than an online-optimization baseline.

Abstract:
Learning embodied manipulation from egocentric human data offers a scalable path, reducing the cost of collecting robot-specific demonstrations. However, existing representation learning methods largely stop at perceptual understanding, such as extracting what a scene contains or what an action is. A capable embodied agent is more than understanding: it predicts the next motion. We propose that forecasting 3D human hand motion is one suitable pretraining task, as the representation jointly learns how to manipulate objects and how the motion will unfold next. Building on this hypothesis, we introduce SFHand, the first representation learning method via streaming egocentric 3D hand forecasting. SFHand learns by autoregressively predicting future 3D hand states, including hand type, 2D bounding box, 3D pose, and trajectory, from a continuous stream of video and language instructions. Its autoregressive architecture is equipped with an ROI-enhanced memory layer, capturing temporal context while focusing on salient hand-centric regions. To enable this research, we also introduce EgoHaFL the first open-source large-scale dataset pairing synchronized 3D hand poses with fine-grained language instructions (247K clips, 3.95M frames). Transferred to downstream embodied control, the learned representation shows state-of-the-art performance on both simulation and real-world manipulation benchmarks, improving success rate by up to 13.4%. Our method also achieves leading 3D hand forecasting accuracy, with 35.8% lower ADE than baseline methods. These results suggest that forecasting hand motion is a better pretraining task for representation learning in embodied manipulation. The code and data will be released.

Abstract:
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we introduce a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines. Project homepage: \hrefhttps://mof-policy.github.iomof-policy.github.io.

Abstract:
Visuomotor policies have advanced on manipulation tasks where the target object stays static during execution, but real deployments break this assumption: parts drift on conveyors and fruits sway in the wind. We introduce Static In, Dynamic Out (SIDO), a counterfactual action augmentation that enables a policy trained only on static object demonstrations to adapt to unseen object motion at test time. Our key idea is to factorize moving object manipulation into two sub-problems: predicting where the object will be, and reaching that predicted pose. SIDO displaces the object to a counterfactual future position and morphs the demonstrated action chunk to preserve the hand-object relative pose, yielding a goal-conditioned policy. At deployment an object pose predictor supplies the future position. Across three simulated tasks (Mug, Square, Stack) under five object motion patterns and two real-world tasks (Gantry, Peachtree), SIDO improves moving object success over the baselines while preserving static object performance. Project website: https://sido-staticindynamicout.github.io/

Abstract:
Autonomous long-horizon sidewalk navigation is essential for micro-mobility applications such as robotic food delivery and assistive electronic wheelchairs. Unlike autonomous driving on the road, long-horizon sidewalk navigation requires precise maneuvering through unpredictable sidewalk terrains and pedestrians, with a lightweight perception stack as minimal as a single monocular RGB camera. While imitation learning (IL) from demonstrations offers a practical solution, the resulting autopilot policy often suffers from compounding errors, a lack of social compliance on sidewalks, and deficiencies in counterfactual reasoning to handle complex situations. To address these challenges, we introduce FlowPilot, a mapless navigation policy that achieves robust and efficient long-horizon navigation performance using only a monocular RGB camera. We first propose to use anchored flow matching as an action representation for policy pre-training on large-scale robot fleet data and to capture the diverse, complex, multimodal distribution of sidewalk navigation behaviors. To bridge the gap between imitation and alignment, we further design a human-in-the-loop preference learning scheme to tune the policy on a small amount of human intervention data. It strengthens the model's counterfactual reasoning and social compliance on sidewalks. We evaluate FlowPilot through extensive simulation and real-world experiments in diverse sidewalk environments. FlowPilot achieves 42% success rate and 66% route completion in simulation, while FlowPilot-HP further improves real-world robustness and social compliance, reducing IR by 40.0% and NIR by 52.1% relative to the base model. Code and model will be made available.

Abstract:
Zero-shot object navigation (ZSON) requires robots to find target objects in unseen environments without task-specific fine-tuning or pre-built maps, a key capability for general-purpose service robots. Yet methods that perform well in simulation often degrade in cluttered real-world scenes with severe occlusion and latent hazards, where large unseen regions make single-scene inference brittle and unsafe. We propose Schrödinger's Navigator, a belief-aware framework that reasons at inference time over multiple \emphtrajectory-conditioned imagined 3D futures. Given candidate paths, a trajectory-conditioned 3D world model predicts hypothetical observations and maintains a superposition of plausible scene realizations rather than committing to one map. An adaptive occluder-aware sampler directs imagination to uncertainty-critical regions, while a \emphFuture-Aware Value Map (FAVM) aggregates imagined futures for robust, proactive action selection. Experiments in simulation and on a physical Go2 quadruped show that Schrödinger's Navigator outperforms strong ZSON baselines, improving hidden-target discovery and risk-aware waypoint selection in occlusion-heavy navigation scenarios. These results highlight imagined 3D futures as a scalable and generalizable strategy for zero-shot navigation in uncertain real-world environments.

Abstract:
Monocular depth estimation has achieved strong open-domain generalization, yet reliable robotic deployment remains difficult in transparent, reflective, and specular environments, where depth sensors often produce missing or biased depth. Existing methods often handle such optical failures with scene-specific preprocessing, auxiliary modules, or post-hoc fine-tuning. While effective in constrained settings, these designs increase architectural redundancy and can over-specialize general geometry models to narrow optical scenarios. We revisit this problem as a localized failure mode within base-model training and identify sensor-induced supervision bias as a key bottleneck: models inherit sensor failure patterns from biased real-depth supervision in optically challenging regions. We then introduce OptiGeo, a bias-aware training framework that rehabilitates biased real supervision using a clean-geometry teacher and residual-trimmed alignment. We redefine transparency-targeted rendering as a compact source of clean optical geometry, rather than a large domain-specific fine-tuning set. With only a small targeted rendering set, OptiGeo learns the geometric structure of transparent objects and regions, correcting local geometry distortions that real sensors cannot reliably supervise. Despite only 30M parameters, OptiGeo outperforms substantially larger 300M-scale monocular models and billion-scale multi-view baselines on transparent-scene benchmarks, while remaining competitive on general zero-shot depth and boundary sharpness. Real-world navigation cases further validate its practicality as an efficient perception module in optically challenging scenes.

Abstract:
Multi-modal robust localization requires estimating uncertainty over multiple pose hypotheses while adapting perception and motion models across environments and robotic platforms. Classical particle filters provide a principled Bayesian framework for this problem, but their manually specified models and discrete resampling operations limit integration into end-to-end trainable robotic systems, while learning-based localization approaches typically lack the modularity and generalizability. Differentiable particle filters promise to combine these two worlds by replacing filtering components with differentiable alternatives, but existing methods remain confined to either planar state spaces, visual or simulated observations, or single-platform evaluation. We present the first end-to-end differentiable particle filter capable of real-world cross-environment SE(3) LiDAR localization by using a unit dual quaternion formulation, 3D-LiDAR observations, and differentiable motion, observation, and resampling modules. Within this framework, we thoroughly compare different submodules and propose improvements that demonstrate increased robustness on held-out datasets and, in parallel, \qty89\percent latency reduction for the observation model. We evaluate the framework on real-world LiDAR data, demonstrating zero-shot RMSE of \qty0.398m or better in unknown environments, on different robotic platforms, or both, and we outperform SotA learning-based localization systems requiring 2.5 × less training time. Project Website: https://anonymous.4open.science/w/MaD-PF/

Abstract:
Neural inertial odometry has demonstrated strong potential for motion estimation in challenging environments, yet inertial-only preintegration remains sensitive to IMU bias and uncertainty. To this end, this paper introduces PLATO:~\emphPrivileged Learning from Accurate Trajectory Observations, a likelihood-based framework that leverages accurate trajectory observations to jointly learn IMU bias dynamics modeled by a neural ordinary differential equation~(NODE) and gyroscope and accelerometer noise covariances. Optimization exploits the sparse structure of the negative log-likelihood, with IMU noise-parameter gradients computed by forward differentiation. A tailored double-adjoint scheme couples a discrete invariant-error adjoint with a continuous-time adjoint for the bias NODE, enabling memory-efficient likelihood optimization over the nested bias-dynamics and IMU-preintegration rollouts. Validation on EuRoC shows improved performance, and underwater robot experiments demonstrate applicability under intermittent lighting failures and visual degradation.

Abstract:
Today’s dominant route to generalist robots scales robotics data to train end-to-end policies. We instead use a vision-language model (VLM) as a coding agent that writes robot programs using a curated library of perception, planning, and control tools. MAESTRO’s orchestration harness packages multi-step perception and action procedures into callable tools and guides their use through a plan–act–replan loop, using visual and execution feedback to verify outcomes and recover from failures. With this tool library and harness, MAESTRO matches or surpasses state-of-the-art vision-language-action (VLA) models zero-shot in our tabletop comparison and performs diverse mobile manipulation tasks, without collecting new teleoperation data or invoking a VLA tool. Ablations show that expanding the tool library improves average task progress with the full harness, while adding tools without it does not reliably improve progress. Additional case studies examine cross-trial code improvement, VLA tool use, and autonomous seed demonstrations for downstream data augmentation. We argue that such agentic systems offer a complementary, far less-explored axis to data scaling for general-purpose robots. Project site: maestro-robot.github.io. Code: github.com/Beethoven-Q/maestro.

Abstract:
Generative planners based on diffusion/flow matching can learn to synthesize long-horizon trajectories from demonstrations. However, real-world deployment requires (i) certified collision avoidance and (ii) tight online replanning at fast execution rates. Prior safe diffusion/flow planners generate the agent’s full trajectory at once, while repeatedly perturbing intermediate states to satisfy safety constraints. This approach is not only computationally intensive, but also introduces distribution shift since the learned sampling dynamics is distinct from the system’s execution dynamics. We propose SafeStreamingFlow, a goal-conditioned planner that aligns flow sampling dynamics with execution dynamics by sequentially integrating a learned state vector field. Importantly, we need to certify safety only for the executed step via high order control barrier functions. To improve robustness, we introduce a hierarchical network architecture that predicts the next position before predicting the next physical velocity. Across navigation, racing, and locomotion benchmarks, SafeStreamingFlow reduces planning latency and improves safety compared to existing methods, while main taining competitive goal-reaching success.

Abstract:
Reachability analysis is central to safety-critical control, robotics, and neural network verification, but classical computational methods, such as Hamilton--Jacobi reachability and set propagation, scale poorly with state dimension. Sampling-based methods have emerged as a promising alternative, often providing finite-sample guarantees that bound the probability-mass left uncovered. However, an explicit account of how the geometry of the initial set, the dynamics, and the sampling law affect the accuracy of the estimator is not fully available in the literature. We study this by casting sampling-based reachable-set recovery as geometric support estimation over a family of problems specified by an initial set, its dynamics, and a sampling law. First, we identify two regularity properties, positive reach of the initial set's complement and Lipschitz continuity of the dynamics, that together make recovery well-posed: a probability-mass coverage guarantee can be upgraded to accuracy r in Hausdorff distance. Second, we bound the resulting sample complexity: recovery is achievable with \tilde\mathcalO\!\big((e^3LT/r)^n\big) samples, exponential in both the state dimension and the time horizon. Third, we show that neither can be removed: an almost-matching minimax lower bound of \Omega\!\big((e^LT/r)^n\big) holds for every estimator, so the exponential dependence on dimension and the degradation over the horizon are both intrinsic, not artifacts of a particular method. Experiments on nonlinear systems confirm that adversarial sampling improves constants but not the scaling.

Abstract:
Recent studies suggest that diffusion models can recover geometric structure in the data manifolds they are trained on, yet the supporting evidence has so far come mostly from natural-image data, where the underlying geometry itself is unknown. We study this question in a setting where the geometry is analytically tractable: constrained inverse kinematics (IK). Each task-space constraint defines a configuration-space manifold with known intrinsic dimension, giving direct ground truth for evaluating the geometry learned by the model. For each of the 6-DoF UR5 and 7-DoF Franka, we train a single conditional diffusion model across seven constraint families, spanning solution manifolds from discrete IK branches to self-motion manifolds. Our empirical results reveal that the intrinsic dimension recovered from the model's score function matches the analytical degrees of freedom of the corresponding constraint manifold across both robots. Moreover, linear interpolation in the latent space leads to generated solutions that remain close to the appropriate constraint manifold, indicating that the learned representation further captures geometric structure of the constraint family beyond intrinsic dimension alone. Constrained IK therefore offers a controlled setting for studying the intrinsic geometry learned by diffusion models.

Abstract:
The success of generative models in language, image, and video generation has inspired extensive applications to generative robot planning. However, most existing works either focus on single-robot planning, or generate multi-robot trajectories in a sequential manner with iterative post-processing to resolve inter-robot conflicts. In this work, we investigate whether coordinated multi-robot trajectories, as a special spatiotemporal distribution, can be learned and generated with a generative model in a feed-forward manner. We propose Robots as Tokens (Roken), a unified diffusion transformer that directly generates multi-robot trajectories that satisfy both (individual) safety and (global) connectivity constraints. The core design of Roken is to represent each robot as a discrete token, allowing them to naturally interact with each other through self-attention, and cross-attend to map tokens for environment layouts. We further introduce several auxiliary tasks based on Bayes' theorem, specifically local occupancy reconstruction and long-horizon waypoint prediction, to provide multi-scale spatial-temporal supervision for efficient learning of the conditional distribution. In training, Roken absorbs diverse expert trajectories from different team sizes. During inference, Roken behaves as a versatile multi-robot planner that can handle single-robot planning, coordinated multi-robot trajectory generation, and conditional trajectory generation by fixing some robot tokens as conditions. Experiments in diverse cluttered environments show that Roken can generate coordinated multi-robot trajectories to perform connectivity-constrained goal navigation tasks with high success rates, outperforming the baseline method used to generate the training dataset. Roken also demonstrates good scalability after training with mixed team sizes, and shows generalization to unseen or partially observed environments, verifying its potential as a unified multi-robot planner that can learn from diverse data and perform versatile tasks.

Abstract:
Vision-language-action (VLA) models have emerged as generalist robotic controllers that map visual observations and language instructions to continuous actions. However, pretrained VLAs typically provide limited information about the reliability of their generated actions, making deployment challenging when execution failures must be anticipated. We introduce SentryVLA, a lightweight runtime wrapper for frozen VLA policies. At the action level, SentryVLA uses conformalized quantile regression to estimate a calibrated upper bound on the predicted deviation between sampled VLA actions and expert-like behavior under the calibration distribution. This score is used to select, among multiple stochastic action candidates, the action with the lowest estimated execution risk. At the execution-monitoring level, SentryVLA applies a Mahalanobis-distance-based anomaly score to the contextualized latent execution state extracted before final action prediction, providing a complementary signal for detecting abnormal execution states during closed-loop control. The framework does not require retraining or architectural modification of the underlying VLA policy. Experiments in simulation and on a physical robot show that SentryVLA improves action selection and provides useful runtime warning signals for abnormal execution states. These findings demonstrate the practical value of post-hoc reliability scoring for improving the runtime behavior of pretrained VLA policies.

Abstract:
Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emphwhen, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emphskip-update scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.

Abstract:
World Action Models (WAMs) have recently shown great promise in robotic control, benefiting from the future imagination capability of large-scale pretrained video models. However, most existing WAMs require explicitly imagining future states at test time, incurring substantial computational overhead and latency that make them unsuitable for real-time robotic control. In this paper, we propose W2A, a two-stage distillation framework for real-time robotic manipulation that distills intermediate denoising representations from a video diffusion model into an action model during training, while entirely discarding the video model at test time. We hypothesize that although video latent representations are high-dimensional, a compressed low-dimensional subspace is sufficient to provide effective supervision for action generation. This makes it possible to transfer such information effectively into an action model. Extensive experiments in both simulation and real-world settings demonstrate competitive success rates while significantly improving inference efficiency. On LIBERO, W2A achieves a 95.9% average success rate. More importantly, W2A achieves about 21X faster inference than the representative WAM baseline. Project website: https://anonymous.4open.science/w/W2A.

Abstract:
Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by combining semantic knowledge from pretrained vision-language models with expressive action-generation policies. Diffusion-based action generators are particularly effective for modeling temporally coherent action chunks, but these chunks are typically executed open-loop after inference. This limits responsiveness when objects move, contacts change, or the scene evolves during execution. We propose VLA-Feedback, a two-timescale architecture that combines low-frequency diffusion planning with high-frequency observation feedback. Rather than fully denoising an action chunk before execution, VLA-Feedback stops before the final denoising stage and refines the chunk online using observation-conditioned denoising velocity updates at control frequency. This design preserves the expressiveness of the diffusion planner while enabling real-time action correction without rerunning the full vision-language diffusion model. Experiments on stationary and non-stationary manipulation tasks show that VLA-Feedback preserves static-task performance while improving robustness in dynamic environments.

Abstract:
We present Cloak, a training recipe that endows a vision-language-action policy with zero-shot cross-embodiment transfer by cloaking the end-effector from its own wrist camera. The mask comes for free from the robot's geometry and camera calibration, projected into the wrist view in real time with no generative model and nothing to hallucinate, and is augmented during training so the policy does not overfit to the end-effector's geometry. At deployment, we mask an unseen end-effector the same way and retarget its state and actions through two tips, the gripper jaws or a hand's fingertips, by inverse kinematics. We demonstrate the recipe with Cloak-VLA, fine-tuned on a single parallel-jaw gripper. Without new data or retraining, it transfers zero-shot to end-effectors it never saw, from another gripper to a five-fingered dexterous hand, at little cost to its performance on the source. Masking the wrist view is what unlocks this transfer: strong baselines given the same retargeting but no mask collapse on the unseen embodiments, most sharply on the dexterous hand.

Abstract:
Expert demonstrations are often assumed to be the gold standard for robot imitation learning, yet for fine-grained manipulation tasks such as insertion, stacking, and alignment, we find a counterintuitive failure mode: fluent expert demonstrations can be poor teachers. Skilled teleoperators compress the most critical alignment and recovery behaviors into a brief temporal window, leaving imitation models dominated by redundant free-space motion and starved of supervision on the high-precision segments that actually determine success. We study this phenomenon through the lens of critical motion segments and propose three complementary remedies: (i) a deliberate demonstration strategy that slows down near alignment and exposes corrective micro-adjustments, (ii) a resampling scheme that upweights key segments around gripper state changes and task-critical transitions, and (iii) a dynamic feature representation that compresses short temporal neighborhoods of visual observations into predictive features, conditioning the action expert on local motion and alignment progress rather than isolated static frames. Together, these results show that robust robotic alignment benefits less from perfect trajectories than from demonstrations and representations that expose the structure of correction, pointing toward a more pedagogical view of robot data collection where demonstrations are optimized not only for human efficiency but for machine learnability.

Abstract:
Due to the highly deformable nature of garments, training a generalizable policy for robotic T-shirt folding and unfolding remains a significant challenge. In this work, we present a large-scale synthetic dataset for robotic T-shirt folding and unfolding, covering 6 robotic embodiments, 1K T-shirts, 1K environmental assets, and 120K episodes with rich annotations, which can be used to train a wide range of manipulation policies. We first follow the FoldNet pipeline to generate a large-scale dataset of physically simulatable T-shirts with diverse appearances and annotated semantic keypoints. Based on these semantic keypoints, we then generate manipulation demonstrations for different robotic embodiments through a unified rule-based framework. We use these demonstrations to train visuomotor policies, and experimental results demonstrate that models trained solely on our synthetic data can achieve over 90% end-to-end task success rates when directly deployed to unseen real-world environments and previously unseen T-shirts from arbitrary initial configurations.

Abstract:
Scaling generalist robotic policies is limited by the high cost of collecting embodied interaction data. Large-scale egocentric human videos offer a rich alternative, but they lack robot action labels, and existing action annotation methods are often noisy under real-world visual complexity and diverse manipulation dynamics. We introduce LAPT-Scale, a scalable latent action pre-training paradigm with two components: LAPT-Scale-LAM learns continuous latent actions by explaining frame-to-frame visual transitions in human videos and uses them as human-derived supervision, while LAPT-Scale-Policy trains a VLA policy through human-video latent-action pre-training, target-robot embodiment alignment, and task-specific post-training. Unlike prior vector-quantized LAMs, LAPT-Scale-LAM uses a continuous latent space to preserve fine-grained motion and interaction dynamics. We further combine perceptual, semantic, and optical-flow alignment signals to improve transition-level supervision beyond pixel reconstruction. Trained on 23.3K hours of egocentric video, LAPT-Scale-LAM provides implicit action supervision for policy pre-training, yielding consistent scaling gains and improving real-robot success from 63.3% to 85.0% as annotated human video scales from 1K to 5K hours.

Abstract:
Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non-transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning. Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4×, and 3D flow improves in-domain performance by 20–30%. More details: https://egowam.github.io/.

Abstract:
Action chunking---predicting and executing multiple future actions instead of a single action---has proven to be a critical component for learning effective robotic control policies. However, our precise understanding of \emphwhy action chunking improves performance has remained limited. In this work we seek to close this gap. Through rigorous experimental evaluations, we first show that, while existing hypotheses for the success of action chunking---in particular non-Markovian expressivity, horizon reduction, and representation learning---can each partially explain the success of action chunking, in many settings of interest \emphdelayed policies, which at each step predict a single action based on the state k steps in the past, can match or exceed the performance of action chunking, fully accounting for the benefits of these hypotheses. We then show that there exists an additional benefit of action chunking not accounted for previously that we refer to as \emphimplicit ensembling. In particular, by learning a diversity of temporal relationships (that is, a_t | o_t, a_t | o_t-1, \ldots), action-chunked policies exhibit behavior matching that of a model ensemble, increasing their robustness and generalization ability over policies that only learn a \emphsingle temporal relationship. Motivated by these findings, we propose a policy class that amplifies this effect of action chunking by explicitly instantiating an ensemble, and which we show significantly improves over the performance of action chunking in many domains.

Abstract:
Behavior-cloned policies often learn multiple behavior modes from demonstration datasets, including modes that are unsafe or otherwise undesired at deployment. For example, a policy trained on diverse handover demonstrations may learn to pass a knife blade-first. Standard remedies such as data curation and inference-time steering either require access to the original demonstrations for full retraining or add substantial inference-time overhead. To address this gap, we propose MoRE (Mode Redirection), which redirects policy rollouts toward desired behavior modes through a short "uncloning" step. Specifically, MoRE distills the redirection signal from a temporary mode classifier into the policy weights to steer behavior. A retain loss balances this edit by preserving desired-mode competence, allowing the standalone policy to suppress unwanted modes with zero inference-time overhead. Across eight simulated and real-world tasks, MoRE improves average deployment success rate (SR) by 43 percentage points over the original mixed-mode policy. Among all compared adaptation and steering baselines, MoRE achieves the strongest SR and approaches the filtered-data retraining reference, while preserving task competence and inference speed. MoRE also generalizes across robot policy backbones, including Diffusion Policy and the \pi_0.5 VLA, diverse task categories, and real-world deployments.

Abstract:
Acoustic information provides rich cues about object location, material properties, and changes caused by contact or motion. This paper introduces a new set of acoustic-aware manipulation tasks for imitation learning, in which robots must use auditory cues to determine manipulation targets. These tasks require sound source localization and identification for active exploration in robotic manipulation. Also, we propose a multimodal imitation learning framework, Spatial-Spectral Audio Action (S2A2), that integrates visual features with acoustic spatial and acoustic signal information for the acoustic-aware manipulation tasks. We implemented S2A2 models that integrates policies such as ACT, Diffusion Policy, VQ-BeT, and \pi_0, into our framework. Simulation experiments showed that the proposed method is the most effective for tasks requiring both position and timbre. Furthermore, real-robot experiments confirm the applicability of the proposed tasks and framework to real-world manipulation.

Abstract:
Vision-language-action (VLA) policies are effective generalist robot controllers, but in long-horizon tasks a visually plausible action can be inappropriate unless earlier subgoals have been completed. We introduce historical support, a reliability criterion asking whether task history justifies a candidate action at the current decision step. Based on this criterion, RAMT-VLA keeps the base VLA frozen and adds a lightweight post-policy assessor that scores candidate action chunks, outputs sparse evidence masks for each candidate, and reranks candidates by historical support. The assessor is trained without dense evidence annotations using wrong-history and missing-evidence counterfactuals, with sparsity and faithfulness regularization. With a frozen 0.5B base model, RAMT-VLA achieves 98.1% average success on LIBERO, 83.3% on SimplerEnv-WidowX, and 84.0% on real-world long-horizon tasks. Matched LIBERO ablations, candidate-selection controls, and transfer evaluations show that support-based reranking improves the frozen base policy, with the largest history-dependent gains on long-horizon tasks, beyond generic sampling or policy-confidence reranking.

Abstract:
We introduce HelpMeOut, an embodied AI benchmark for error-aware monitoring of household task execution. Existing embodied household benchmarks typically assume that humans execute tasks correctly, but this assumption does not hold for cognitively impaired persons or older adults whose everyday activities produce structured execution errors. To address this gap, \textscHelpMeOut operationalizes research on everyday action errors into 20,000 paired activity episodes, covering nine neuropsychologically motivated error categories. In each episode, agents observe a streaming human activity trajectory and must monitor whether the execution is unfolding correctly, diagnosing any structured errors as early as possible. We evaluate supervised sequence models and LLM prompting baselines. Results show that these methods perform poorly as error monitors, often missing when execution goes wrong. These results suggest a gap left by standard goal-inference and task-completion benchmarks, making HelpMeOut a testbed for error-aware assistive agents.

Abstract:
Dual-arm robotic grasping is essential for manipulating large, heavy, and geometrically complex objects that cannot be reliably handled using a single manipulator. These large objects often contain only sparse graspable regions determined by local geometric properties such as thickness, edge structure, and gripper clearance. Prior bimanual grasping methods assume access to a full point cloud of the object which inherently contains this geometric information, but may not be accessible in real scenarios. This work proposes PartialBiGrasp, a dual-arm grasp generation framework that operates directly on partial point cloud observations.Our model learns geometric features implicitly through convolutional occupancy networks, enabling local reasoning about graspability, collision-free contact regions, and object thickness. We leverage this understanding to generate force-closure compliant grasp pairs, which are further refined using a sampling-based optimization to correct for ambiguity caused by incomplete geometry. We evaluate our approach using analytical force-closure metrics, large-scale simulation experiments, and real-world testing on noisy point clouds of novel objects, demonstrating robust and physically stable dual-arm grasp generation for large-object manipulation under partial observations.

Abstract:
Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first warm-starts tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5–40% success to 85–100% within 40–80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies.

Abstract:
Human videos are an abundant source of dexterous manipulation behaviors, but they lack tactile information that is crucial for contact-rich interaction. This raises a fundamental question: can robots learn deployable visual-tactile dexterous manipulation policies directly from human video demonstrations? We present Dex-X, a framework for learning visual-tactile dexterous manipulation from human videos through simulation. Our key insight is that simulation can serve as a tactile completion engine. Given monocular human demonstrations, Dex-X reconstructs hand-object interactions in simulation, where physically grounded contact dynamics provide tactile supervision unavailable in the original videos. Leveraging this recovered tactile information, we train visual-tactile dexterous manipulation policies and distill them into deployable policies operating on point-cloud observations and tactile sensing. We demonstrate zero-shot sim-to-real transfer on dexterous hand-arm platforms across diverse grasping and contact-rich tool-use tasks. The teacher policy achieves 65.9% average success across eight tasks in simulation, while the distilled visual-tactile policy achieves a 53% success rate on cube picking and 23% on the challenging real-world table-cleaning task, which is not achieved by existing general sim-to-real manipulation systems. Our results suggest that tactile completion is a critical ingredient for scalable dexterous manipulation learning and provide a path toward learning real-world robot skills from Internet-scale human video data.

Abstract:
High-speed racket sports provide a demanding testbed for humanoid robots, where successful interaction requires time-critical decisions, precise striking, and dynamic whole-body coordination. This challenge becomes especially pronounced in badminton, as the shuttlecock follows highly nonlinear trajectories and each return requires tightly coordinated contact selection, racket control, and balance recovery within a short time window. Although human demonstrations offer valuable motion priors for such athletic skills, available badminton data is often limited and imperfect; meanwhile, purely task-driven reinforcement learning can optimize returns but tends to produce unnatural motions that are difficult to deploy on hardware. To address these challenges, we present a three-stage hierarchical framework for athletic humanoid badminton that combines human motion priors with task-driven optimization. The framework first learns a compact latent skill space from retargeted human demonstrations, then optimizes skill composition through reinforcement learning, and finally regularizes the learned behavior with a context-conditioned adversarial latent prior to promote human-like motion. When deployed on a real humanoid robot, our system demonstrates multi-skill badminton play with human players, including rallies and jump smashes. To the best of our knowledge, this is the first real-world demonstration of humanoid jump-smash skills in racket sports.

Abstract:
Enabling humanoid robots to perform agile and adaptive interactions has long been a core challenge in robotics. Current approaches are bottlenecked by either the scarcity of realistic interaction data or the need for meticulous, task-specific reward engineering, which limits their scalability. To narrow this gap, we present HumanX, a unified framework that compiles human video into generalizable, real-world interaction skills for humanoids, without per-task design. HumanX integrates two co‑designed components: XGen, an anchor-centric data generation pipeline that yields physically plausible robot interaction data from monocular video and augments it to create a diverse, scalable training dataset; and XMimic, an interaction-prioritized learning framework to achieve generalizable interaction skills from seconds of demonstration. Evaluated across five distinct domains—basketball, football, badminton, cargo pickup, and reactive fighting—HumanX successfully acquires 10 different skills and transfers them zero‑shot to a physical Unitree G1 humanoid. The learned capabilities include complex maneuvers such as pump‑fake turnaround fadeaway jumpshots without any external perception, as well as interactive tasks like sustained human‑robot passing sequences over 10 consecutive cycles—learned from a single video demonstration. Our experiments show that HumanX achieves over 8× higher generalization success than prior methods, demonstrating a scalable and task‑agnostic pathway for learning versatile, real‑world robot interactive skills.

Abstract:
Humanoids are advancing toward sharing everyday spaces with people, yet reactive humanoid--human interaction (HHI) remains an open question because existing controllers either treat the robot in isolation or engage only static objects and scenes. HHI is challenging because handcrafting an interaction reward is complicated, noisy two-human motion-capture demonstrations cannot be retargeted onto a humanoid without losing their inter-person contacts, and on-board close-range perception breaks down under occlusion and out-of-view human subjects. We present REACT, a learning pipeline that scales HHI to a large corpus of human--human demonstrations. It uses a vision-language model to detect plausible spatio-temporal contacts, and a contact-preserving retargeting stage maps each demonstration onto the humanoid--partner geometry. A privileged-state teacher trained with an interaction contact reward is then distilled into an ego-depth student, with a realistic human partner rendering in the loop so that training-time depth matches the deployed sensor. We also use generative human-pose-estimation auxiliary loss to handle the multimodality of close-range perception. On a real Unitree G1, REACT executes diverse, contact-rich interactions with a human partner, significantly outperforming a baseline that relies on an external pose-estimation module, which becomes inaccurate at close range.

Abstract:
Visual loco-manipulation of arbitrary in-the-wild objects requires accurate end-effector (EE) control and a generalizable understanding of the scene from visual inputs (eg, RGB-D images). Existing imitation and sim2real methods jointly learn both these aspects via monolithic end-to-end learning and are thus hard to scale. In this work, we bring to bear the best tools for each of these problems -- large vision models for generalizable scene understanding and simulated training for accurate EE control -- leading to an overall modular loco-manipulation system that exhibits strong generalization. Our core technical innovation is HERO, an accurate residual-aware EE tracking policy made possible by combining classical robotics with machine learning. It uses a) inverse kinematics to convert residual end-effector targets into reference trajectories, b) a learned neural forward model for accurate forward kinematics, and c) goal adjustment and replanning. Together, these innovations reduce the end-effector tracking error to 2.44cm, outperforming the strongest prior method by 5.5x. Our overall system operates in diverse real-world environments, from offices to coffee shops, where the robot reliably grasps various everyday objects (eg, mugs, apples, toys) on surfaces ranging from 43cm to 92cm in height. Systematic modular and end-to-end tests demonstrate the effectiveness of our proposed design. We believe our advances open up new ways of training humanoids to interact with daily objects.

Abstract:
We propose GeoXel, a novel framework for geo-referenced 3D reconstruction from long-duration aerial video. While recent feed-forward reconstruction models have demonstrated impressive efficiency in regressing camera trajectory and 3D geometry, they suffer from two critical limitations: cumulative trajectory drift and the lack of alignment with global geographic coordinates. To address these challenges, we exploit widely accessible geo-referenced geographic data---specifically Digital Orthophoto Maps (DOM), Digital Elevation Models (DEM), and OpenStreetMap (OSM)---as a spatial grounding for precise geo-aligned reconstruction. Specifically, our method decomposes long-sequence videos into overlapping chunks and employs a transformer-based visual geometry model to capture camera trajectory and points map. We then select anchor frames from each video chunk and establish accurate correspondences and geometric transformation constraints between anchor frames and geo-referenced maps to bridge visual reconstruction results and global geographic space. A key contribution of our work is the introduction of a tailored Test-Time Training (TTT) strategy. By fine-tuning a sparse, carefully selected subset of the model's layers, we successfully align the latent tokens with the geographic coordinate system. Our findings reveal that this TTT effectively preserves the reconstructed structural integrity while correcting spatial drift. Furthermore, we conduct intra-chunk and global pose graph optimization (PGO) to ensure local and global spatial consistency. Extensive evaluations on three aerial reconstruction benchmarks demonstrate that our method outperforms existing feed-forward reconstruction baselines by a large magnitude. Our code will be made publicly available upon the publication of this paper.

Abstract:
Humans learn not only how their bodies move, but also how the surrounding world responds to their actions. In contrast, while recent Vision-Language-Action (VLA) models exhibit impressive semantic understanding, they often fail to capture the spatiotemporal dynamics governing physical interaction. In this paper, we introduce Pri4R, a simple yet effective approach that endows VLA models with an implicit understanding of world dynamics by leveraging privileged 4D information during training. Specifically, Pri4R augments VLAs with a lightweight point track head that predicts 3D point tracks. By injecting VLA features into this head to jointly predict future 3D trajectories, the model learns to incorporate the evolving scene geometry within its shared representation space. This allows the action prediction components to leverage a more physically-aware context for precise control. Due to its architectural simplicity, Pri4R is seamlessly compatible with dominant VLA design patterns through minimal changes. During inference, we run the model using the original VLA architecture unchanged; Pri4R adds no extra inputs, outputs, or computational overhead during inference. Across simulation and real-world evaluations, Pri4R significantly improves performance on challenging manipulation tasks, including a +10% gain on LIBERO-Long and a +40% gain on RoboCasa. We further show that 3D point track prediction is an effective supervision target for learning action-world dynamics, while also validating our design choices through extensive ablations. Code and checkpoints will be released.

Abstract:
Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume the scene is stationary between observation and execution, so at any non-trivial object speed the resulting latency exceeds the time available to grasp. We close this gap with AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics), a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model. A small world model trained on manipulation video forecasts future patch tokens in the VLA’s feature space, conditioned on per-token velocity and acceleration from optical flow. A language-and-motion saliency mask concentrates prediction on task-relevant patches, and the model rolls forward for an adaptive horizon, halting when prediction uncertainty crosses a threshold. The frozen action decoder then receives the predicted future tokens in place of the current ones. AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios where the strongest baseline reaches 31 to 58%. On a physical UFactory xArm 7, AHEAD succeeds on 29/30 to 30/30 on three conveyor and rolling-ball tasks, 23/30 on paddle interception, and 19/30 on projectile catching where every baseline scores 0/30.

Abstract:
World models offer a promising avenue for capturing complex environment dynamics where simulators face challenges. However, large-scale world models required for complex real-world settings are computationally expensive to adopt in popular RL approaches. In this paper, we introduce a novel first-order RL method that enables policy training via a novel decoupled first-order gradient (FoG) method: a large-scale world model generates accurate forward trajectories, while a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation. Our coupled local and global world model formulation allows high-fidelity forward dynamics alongside computationally efficient differentiation needed for model-based RL. Through real-world experiments across a range of robotic tasks with increasing complexity, we demonstrate tractable RL and zero-shot deployment thanks to our world-model-based formulation. We show significantly better sample efficiency compared to PPO in a canonical real-world PushT benchmark and observe similar patterns in more complicated ego-centric object manipulation and grasping tasks. Together, these results demonstrate that learning agents entirely inside data-driven world models is a feasible and promising pathway for solving hard-to-model tasks in image space without reliance on hand-crafted physics simulators.

Abstract:
Latent world models are a promising approach for learning state representations and dynamics directly from high-dimensional observations, enabling robot control in hard-to-model settings. However, control performance ultimately depends on the latent representation encoding the required information for the task. In this work, we study latent-space safe control problems and show how partial observability can induce control failures when safety-relevant information is not preserved in the latent state. Specifically, we identify two world model failure modes: estimation gaps, where current observations do not reveal safety-critical quantities (e.g., temperature in a cooking task), and prediction gaps, where failures are observable once they occur but cannot be reliably anticipated from available observations. We introduce two diagnostics for these gaps: a mutual-information-based measure of safety observability and a rollout-based measure of future safety predictability. Finally, we present mitigation strategies for each failure mode: privileged multimodal supervision for estimation gaps and conformal risk calibration for prediction gaps. Across two hardware case studies---using unimodal RGB world models and multimodal RGB+Tactile and RGB+Thermal variants---we show that these mitigation strategies improve the safety of a Franka Research 3 manipulator on challenging cooking tasks under partial observability, albeit with increased conservativeness. More broadly, our work raises the question of when world model state representations are sufficient for reliable robot control.

Abstract:
Video world models (WMs) have shown promise for policy evaluation and improvement by imagining realistic future observations conditioned on ego-robot actions. While WMs can model distributions over futures, policy evaluation and improvement typically rely on nominal imaginations, which can miss high-impact outcomes of robot actions unless prohibitively many samples are drawn. To enable robust policy evaluation and improvement over WM imaginations, we propose StressDream, which steers imaginations toward high-impact yet plausible outcomes specified at inference time by optimizing the initial noise of diffusion-based WMs. However, optimizing high-dimensional noise is challenging: the optimization must reason about nuanced, scene-dependent target events in generated videos while avoiding out-of-distribution (OOD) noise that yields implausible imaginations. We address this with two complementary objectives: a semantic objective with a Vision-Language Model that provides informative gradients by reasoning about the generated video, and a plausibility objective that prevents the optimized noise from drifting OOD. With state-of-the-art video world models for autonomous driving and robotic manipulation, we show that StressDream effectively steers imaginations toward high-impact yet plausible outcomes specified by text at inference time, such as task failures, enabling robust policy evaluation and improvement by identifying actions whose plausible futures include undesirable outcomes. Video results can be found at https://stressdream.github.io/.

Abstract:
For robots to work safely in household environments, they need to be compliant and react to torque and force feedback during contact. However, the majority of existing data collection pipelines still lack the ability to capture force and torque data for learning active compliant policies. In this paper, we present Universal Manipulation Exoskeleton (UME), an upper-limb exoskeleton that provides real-time haptic torque feedback while recording whole-arm configurations and joint torque signals for teleoperation. With transparent torque feedback, human operators can even unsheathe kinematically constrained objects while blindfolded. UME is low-cost, lightweight, and portable. Equipped with an embedded IMU, it enables teleoperation for mobile manipulation. With our proposed universal retargeting algorithm, UME can teleoperate a range of robots, including the 7DoF OpenArm, 7DoF Franka, and 6DoF X-ARM. We demonstrate that this combination of capabilities enables learning bimanual, whole-body, and active compliant policies that operate effectively in highly constrained spaces. The learned robust autonomous policies achieve high success rates across a variety of tasks, including long-horizon mobile manipulation, force-mediated box flipping, visually occluded box pushing, and space-constrained tabletop manipulation.

Abstract:
Dexterous teleoperation requires reliable human-hand state estimations. However, common low-cost motion-capture gloves and markerless trackers often exhibit biases that vary across users, glove fit, and recording sessions, degrading retargeting and demonstration quality. We present YOCO, a fast few-shot, fine-tuning-free calibration framework that corrects biased hand-pose streams from a small set of paired raw and target poses. Instead of optimizing a separate model for every operator or session, YOCO conditions a calibration HyperNet on the paired examples and predicts LoRA-style updates for a frozen MANO hand-estimation module, turning per-user calibration into a lightweight feed-forward adaptation step while preserving the geometric prior of MANO and the efficiency of a compact estimator. We train YOCO with synthetic drift augmentations on InterHand2.6M and evaluate on augmented InterHand sequences, offline real glove data, and dexterous teleoperation tasks. Across these settings, YOCO improves calibration efficiency, hand-state estimation quality and teleoperation performance compared with uncalibrated input and standard calibration baselines.

Abstract:
Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.

Abstract:
How can we scalably generate data for robotic manipulation, especially on human-like platforms such as dexterous multi-fingered hands? Learning from watching human videos has recently emerged as a likely answer to this question. However, difficulties in estimating hand-object interaction and crossing the human-to-robot embodiment gap have hindered the adoption of abundant monocular RGB-only human videos as the primary source of robot manipulation data. In this work, we present Do as I Do, an algorithm to reconstruct and retarget monocular RGB human videos to multi-fingered dexterous robotic hands. Do as I Do reconstructs hand-object interactions from various egocentric and exocentric in-the-wild video sources. Do as I Do then retargets these hand-object interaction estimates into a sequence of actions executable in the real world, yielding robot-complete manipulation data from disparate human videos. Overall, Do as I Do outperforms previous state of the art in estimating hand-object interactions and extracting dexterous manipulation trajectories from RGB videos, as we show on experiments on datasets with ground truths and on a dataset of video clips collected online. Finally, our experiments enable us to propose an efficacy playbook for practitioners collecting human data for manipulation.

Abstract:
Recent advancements in learning from human demonstration have shown promising results in addressing the scalability and high cost of data collection required to train robust visuomotor policies. However, existing approaches are often constrained by a reliance on multiview camera setups, depth sensors, or custom hardware, and are typically limited to policy execution from third-person or egocentric cameras. In this paper, we present WARPED, a framework designed to synthesize realistic wrist-view observations and actions from human demonstration videos to facilitate the training of visuomotor policies using only monocular RGB data. With data collected from an egocentric RGB camera, our system leverages vision foundation models to initialize the interactive scene. A hand-object interaction pipeline is then employed to track the hand and manipulated object and retarget the trajectories to a robotic end-effector. Lastly, photorealistic wrist-view observations are synthesized via Gaussian Splatting to directly train a robotic policy. We demonstrate that WARPED achieves success rates comparable to policies trained on teleoperated demonstration data on five rigid-object tabletop tasks and one articulated-object task, while requiring 5-8x less data collection time.

Abstract:
Universal Manipulation Interface (UMI) enables scalable data collection without target-robot teleoperation, but UMI-style demonstrations are not directly suited for large-scale Vision-Language-Action (VLA) training. We identify the challenge of training VLA models from UMI-collected data as a dual adaptation problem: (i) wrist-mounted fisheye observations are misaligned with the visual distributions of pretrained vision-language models, and (ii) human-collected trajectories are not guaranteed to be executable under target-robot embodiment constraints. To mitigate this challenge, we present VISTA, a framework that adapts UMI-collected data to VLA training through fisheye-aligned visual grounding and embodiment-conditioned trajectory validation. UMI-VQA supplies vision-language supervision under wrist-mounted fisheye observations, while trajectory validation scores human-collected demonstrations by continuity, self-collision risk, and execution fidelity to select physically plausible action supervision before training. VISTA then uses the adapted visual-language data and validated trajectories in a two-stage VLA training procedure. Experiments show that UMI-VQA improves policy performance over action-only training and standard-view VQA, while validation scores are predictive of real-robot deployment success. Across UMI-style simulation and real-world manipulation tasks, VISTA consistently improves over strong VLA baselines including \pi_0.5, LingBot-VLA, and Wall-X. We will release the pipeline, datasets, and pretrained model.

Abstract:
Self-play reinforcement learning has recently taken the wheel in autonomous driving. The approach uses cheap, large-scale simulation to substitute expensive, large-scale human driving data used by imitation learning approaches. While cost-effective, policies trained through self-play often behave unpredictably around humans; they fail to coordinate because they tend to converge to effective but incompatible driving conventions. Previous works attempt to mitigate such behavioral incompatibilities through reward engineering and domain randomization techniques, which are brittle and labor-intensive. We build on the best of both approaches and instead train driving policies using 30 minutes of human driving data and over 60 years of self-play simulation with a minimal reward function that solely encodes safe goal reaching. We call this _spiced self-play_: just as a pinch of a well-chosen ingredient can transform a dish, a small amount of human data, added as a behavioral anchor, disproportionately improves coordination with human drivers. Results show that our policies achieve improved coordination with real human driving trajectories using 2,500× less human data than imitation learning baselines. The entire pipeline completes in 15 hours on a single consumer-grade GPU. Together, our results point toward a positive revision of the field's understanding of data scaling: a small quantity of human data may suffice to regularize the vast space of driving policies toward ones that fit the road.

Abstract:
Real-world reinforcement learning for robotic manipulation remains challenging, and this difficulty is amplified for flow matching policies: applying policy gradient methods to these policies is fundamentally limited by the need to backpropagate through time (BPTT) along the multi-step ODE that maps noise to actions, which is computationally prohibitive and numerically fragile. We propose FlowDPG, a DDPG-style method specifically designed for flow matching policies that distills the critic gradient into the velocity field at training time, bypassing BPTT entirely. Intuitively, FlowDPG combines two complementary vectors: the demonstration-driven velocity that keeps the action feasible, and the critic-driven correction that steers it toward higher value. Our contributions are threefold: (1) a BPTT-free distillation framework that enables stable DDPG-style policy improvement on flow matching policies, (2) a formal connection between the FlowDPG update direction and vanilla Deterministic Policy Gradient via three explicit approximations, and (3) real-world validation on a long-horizon, multi-stage, dual-arm AirPods assembly task, where FlowDPG attains a 92% end-to-end success rate, substantially outperforming recent RL methods spanning value-conditioning, auxiliary-module adaptation, and adjoint-based critic-gradient approaches. Videos and more results are provided on the project page https://flowdpg.github.io.

Abstract:
Recovering real-world scenes as interactive simulation environments can enable generalizable robot learning and reproducible policy evaluation. However, constructing scenes that are both physically stable and visually faithful remains slow and expensive. In this work, we present RoboSnap, a real-to-sim framework that turns a single RGB image into a simulation-ready scene. The key idea is a layered design that separates the physics-critical interaction area from the surrounding visual context: collision-aware foreground assets are refined for stable robot interaction, while a 3D Gaussian splatting visual layer preserves faithful background appearance under novel views. Experiments on DROID scenes and real-robot tasks show that RoboSnap achieves reliable trajectory replay in the recovered scenes, supports task-specific synthetic data generation for policy training, and yields meaningful sim-real correlation for policy evaluation. To further support real-to-sim research, we introduce DROID-Sim, a real-to-sim companion dataset constructed from 564 real-world scenes in DROID. Extensive experiments suggest that the value of real-to-sim methods lies not only in high-fidelity visual reconstruction, but in turning real environments into reusable infrastructure for robot learning and evaluation.

Abstract:
Training and evaluating robot policies in the real world is costly and difficult to scale. We introduce SimFoundry, a modular and automated system for zero-shot real-to-sim scene construction from a single image or video. SimFoundry generates sim-ready digital twins and supports object, scene, and task editing, enabling the automated generation of diverse digital cousins. Policies trained on SimFoundry data transfer zero-shot to challenging real tasks involving multi-step manipulation, articulated object interaction, and bimanual interaction, and its digital cousins facilitate generalization to new real-world conditions. Across 7 manipulation tasks and 5 policies, SimFoundry simulation evaluations strongly predict real-world performance, with mean Pearson correlation 0.928 and mean maximum ranking violation 0.018. When evaluating sim-trained policies zero-shot in the real world, policies trained with object, scene, and task cousins during sim-to-real training show average task success rate improvements of 17%, 21%, and 50%, respectively. Additional details at https://simfoundry.github.io/.

Abstract:
Real-world robot imitation learning is often limited by the cost of collecting physical demonstrations, which typically yield only one action label per visited state. We present CrowdMAPS, a web-based crowdsourcing paradigm for augmenting real-robot datasets with Multiple Actions per State (MAPS). After or during standard robot data collection, CrowdMAPS distributes visited robot states to remote annotators through a web-based 3D interface, where they specify alternative valid actions without requiring access to physical robot hardware. This expands robot datasets along a complementary “width” dimension — by increasing the number of actions per visited state — rather than only by collecting more trajectories. We evaluate CrowdMAPS using keyframe-centric goal-pose annotations supplied by non-expert annotators and train imitation policies on the resulting MAPS data. Across five real-world manipulation tasks, increasing MAPS width from one to ten actions per state improves sample efficiency and increases average success rates by 35.6% for keypose policies and 33.0% for trajectory policies. These results show that software-mediated human annotations can augment physical robot data to afford denser supervision, establishing CrowdMAPS as a scalable digital layer for real-robot data collection.

Abstract:
Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate Active Generative Simulation: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility simulation parameter distributions without human heuristics. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable.

Abstract:
Enabling robots to infer interaction-relevant physical properties from vision before contact would improve planning and control in unstructured environments. Vision provides useful pre-contact information, but appearance alone is often ambiguous: visually similar regions can induce different friction, stiffness, or sinkage responses upon contact. Existing approaches often collapse this ambiguity to a point estimate or aggregate experience through fixed semantic classes, limiting their ability to preserve distinct physical regimes. Instead, we propose a real-time estimator that replaces semantic class indexing with an interaction-grounded Gaussian-mixture posterior over latent physical parameters, conditioned directly on continuous visual embeddings. Our approach maintains interaction-derived Gaussian parameter estimates, typically from inverse modeling, as mixture components while using embedding similarity to retrieve and weight relevant past interactions for each new query region. A support-aware fusion mechanism incorporates a conditional generative prior model when interaction history is sparse, while a learned embedding refinement improves retrieval relevance without discretizing the representation. Our experiments show that the estimator recovers multimodal physical beliefs, rejects unrelated interaction history, avoids spatial map discretization, and evaluates at controller rates. In a high-fidelity autonomous racing simulation with unknown low- and high-friction surface regions, the posterior enables the controller to adapt over repeated laps as interaction evidence accumulates, improving performance while maintaining friction-aware behavior.

Abstract:
Cutting a piece of food looks effortless in human hands: we nudge the blade, adjust the angle, modulate pressure as the skin yields and cuts. For a robot, however, every cut is a high-contact interaction with a deformable, elasto-plastic object whose behavior changes with pose, contact mode, and cutting stage. Collecting real-world data is also inherently expensive and wasteful: each trial typically contaminates, damages, or destroys the food, so obtaining scalable real world data for dexterous cutting are expensive and difficult. Simulation is an appealing alternative, but a large sim-to-real gap exists in both physics and perception. This gap is especially severe for topology-changing interactions like cutting, where small errors in contact, friction, or material and elastic transformations response quickly compound. We propose DexNinja, a differentiable real2sim2real framework that turns a handful of real demonstrations into a dense and realistic training distribution. DexNinja (i) reconstructs object instances from real trajectories, (ii) randomizes physically meaningful parameters (e.g., mass, geometry, friction) under category-level constraints, and (iii) augmented dexterous manipulation episodes using a custom differentiable simulator that couples robot dynamics, tactile contact, and deformable cutting. We evaluate DexNinja on a dexterous food-slicing task show that (1) the augmented data consistently improves sim-to-real transfer, and (2) the resulting policies generalize to out-of-distribution objects such as strawberries with varying shapes and sizes.

Abstract:
We introduce Yielding Universal Bidigital Interface (YUBI), a finger-aligned gripper designed to enable intuitive, ergonomic, and scalable data collection for bimanual dexterous manipulation. While handheld data collection systems such as Universal Manipulation Interface (UMI) enable affordable data collection, their bulky pistol-grip designs can pose ergonomic and usability challenges for fine-grained, dexterous manipulation tasks. To address this, YUBI presents a distinct design principle: yielding, finger-driven actuation that directly maps human finger movements to gripper jaw motion. Using the YUBI devices, we set up a data collection system with integrated VR-based 6 DoF tracking of the gripper, ensuring high-fidelity trajectory data acquisition. We curate a UMI-based dataset of unprecedented scale: 8434 hours across 1.20M episodes and 119 tasks. Experiments show that YUBI offers advantages over the UMI gripper in versatility for complex bimanual tasks, dexterity, and operational efficiency. A single policy trained on the YUBI dataset transfers across multiple bimanual robots (UR, Franka, and ELEY) simply by mounting the gripper on each platform, confirming that the collected data are directly executable as policy supervision. We release the gripper hardware, data-collection software, and dataset as one integrated stack, offering the open community a reproducible path to large-scale data acquisition for advancing robotic foundation models.

Abstract:
End-to-end manipulation policies, combined with web-scale pretrained Vision-Language Models (VLMs), show the promise for generalizable and dexterous robotic manipulation. However, they inherit two key limitations from 2D foundation models: 1) the reliance on 2D RGB inputs that ignores the intrinsically 3D nature of manipulation; and 2) the lack of spatial 3D alignment between input-output spaces as well as across diverse robot embodiments, camera setups, and trajectory datasets. In this paper, we present a series of contributions to address these issues. First, we introduce aligned vertex map and vertex spectrum — a pixel-wise 3D representation that elevates 2D visual inputs to 3D, using camera calibration and optional depth. This novel input representation marries 3D awareness with the generalization of 2D large VLMs. Then, we propose to align the inputs and outputs of manipulation policies by expressing per-pixel 3D information of each camera view and robot actions to a shared coordinate. Based on this, we designate a canonical Bird's-Eye-View (BEV) alignment frame and innovatively propose to construct BEV images, producing a view-invariant representation robust to camera pose variations. To enable training and evaluation at scale, we develop a comprehensive data processing pipeline to perform such alignments; we also introduce a novel temporal alignment scheme for trajectories across diverse robots, human operators, and datasets. These contributions collectively mitigate input and output spatial-temporal misalignments, improving the consistency and generalization for real-world manipulation. Pretrained checkpoint, source code and data processing pipeline will be made publicly available.

Abstract:
Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. Recent methods based on vision-language models (VLMs) largely rely on coarse 2D perception and struggle to perform accurate reasoning over complex 3D geometry. To address this limitation, we propose AssemLM, a spatial multimodal large language model for robotic assembly that integrates assembly manuals, point clouds, and textual instructions to predict task-critical 6D assembly poses with explicit geometric understanding. To bridge raw 3D perception and high-level linguistic reasoning, AssemLM employs a specialized point cloud encoder to extract fine-grained geometric and rotational features for accurate 3D spatial reasoning in assembly tasks. In addition, we introduce AssemBench, a large-scale benchmark for assembly-oriented spatial reasoning with over 900K multimodal samples and precise 6D pose annotations, extending evaluation from 2D grounding to full 3D geometric inference. Extensive experiments and real-robot evaluations demonstrate that AssemLM achieves state-of-the-art 6D pose reasoning performance and effectively supports fine-grained, multi-step assembly tasks in real-world settings. Code, models, and the AssemBench dataset will be made publicly available.

Abstract:
Effective dexterous manipulation demands a seamless integration of long-term temporal foresight and rapid reactive control. Vision-Language-Action (VLA) models, while proficient in high-level reasoning, often lack a fine-grained understanding of physical dynamics and spatial perception. Conversely, World-Action Models (WAMs) typically suffer from high inference latency due to iterative generation. These deficiencies result in a critical temporal misalignment where the model's intent fails to adapt to rapid physical contact changes. To overcome this fundamental bottleneck, we propose LiMA, an asynchronous dual-system generative framework that systematically decouples intent planning from reactive execution. LiMA organizes computation into a multi-scale hierarchy: a slow system handles sparse long-horizon spatiotemporal intent generation, while a fast system focuses on dense high-frequency motion refinement. To align sparse intent predictions with dense action trajectories, we introduce a Latent Schrödinger Bridge Coupling mechanism that formulates refinement as an entropy-regularized probabilistic transport process. LiMA reduces inference latency by 45.8% compared with Cosmos-Policy via asynchronous decoupling. Evaluated across six bimanual dexterous manipulation tasks spanning multiple horizons, LiMA achieves an overall success rate of 70.8% and an average subtask success rate of 78.5%, while maintaining performance in unseen scenarios.

Abstract:
Robotic disassembly presents a key challenge in end-of-life (EoL) product remanufacturing automation, where a robot must perceive and interpret product states with uncertainty, satisfy prerequisite sequences, execute precise, contact-rich manipulation, and recover from failed attempts. Recent vision-language-action (VLA) models offer greater flexibility; however, they are largely applied to tabletop and daily-life manipulation rather than industrial settings. Supervised fine-tuning provides a practical path for adaptation, but limited and costly disassembly data cannot capture the distribution shifts arising from changing environments and product uncertainties. Human corrections expose the policy's own failure states, but the key challenge is how to turn a limited set of recovery trajectories into useful policy-improvement signals rather than treating them as ordinary demonstrations or coarse trial outcomes. Motivated by this, we introduce PACER, a Process-Aware Correction and Evidence Reweighting framework that preserves the native VLA training objective by converting correction trajectories and in-process robot evidence into weights or advantages. Using 70 human correction trajectories collected across seven positions, we evaluate PACER on real UR5e motherboard disassembly across five component classes and multiple configurations, achieving a 33% success-rate improvement on robotic disassembly tasks compared with direct SFT.

Abstract:
Implicit object properties that are difficult to directly infer from vision, such as material, container contents, or softness, can be revealed through tactile sensing and exploratory interactions. However, because tactile signals are transient and sparse, extracting informative tactile events and effectively incorporating them into robotic manipulation remains a challenge. In this work, we present an object-centric context-aware manipulation framework that learns task-agnostic object representations through tactile exploration. A token learner autonomously selects representative tactile segments from long-horizon exploration, while contrastive alignment with descriptive text embeddings enables a latent space that captures multiple physical object properties. These learned representations are then used as semantic context to guide object-centric manipulation policies and adapt strategies based on object properties. Experiments show that the learned representations achieve 90% accuracy for object property estimation for both seen and unseen objects. Evaluated on three tasks involving visually ambiguous objects, i.e. multi-object rearrangement, pouring, and box opening, the proposed framework improves both target selection and property-dependent manipulation adaptation, achieving an average task success rate of 85%, compared to 36% for baseline policies without object-context conditioning.

Abstract:
Contact-rich teleoperation logs expose a policy I/O design choice: demonstrations may contain the robot execution (E), leader command (C), or both. These signals are not interchangeable: E2E may discard contact-generating command offsets, whereas E2C preserves these offsets but omits the robot's execution response. We propose Dual-State Conditioning (EC2C), which conditions on both E and C while predicting future C, exposing command-execution mismatch as a cue for contact, latency, payload, and operator compensation; in quasi-static contact, this cue is often force-correlated. On a low-cost setup without force, tactile, or motor-current policy input, EC2C outperforms E2E and a strong E2C baseline across several real-world contact-rich, force-sensitive, and dynamic tasks. These results support EC2C as a practical default I/O setting for contact-rich imitation learning. We further formulate latency-adaptive inpainting as a temporal extension of this I/O choice for action-chunking policies, and discuss when long histories help dynamic inference or introduce causal confounding.

Abstract:
Quadruped robots have achieved remarkable locomotion, yet their behavioral repertoire remains confined to a few gaits—far from the expressive, companion-like presence long envisioned for them. Attempts to import the humanoid recipe of large-scale motion data have inherited one tacit assumption: that robot motion must first pass through an animal body, making data collection dependent on cooperative animals, reconstruction fragile across species, and retargeting ill-posed across incompatible morphologies. We propose Uni-Mo, a fully automated pipeline that removes the animal from the loop by reframing data scarcity as a generation problem: an LLM proposes motion prompts, a video diffusion model synthesizes the corresponding robot behaviors, and the generated videos are lifted into 3D reference trajectories used to train tracking policies deployed on a real Unitree Go2. To make naively-drifting generations reliably extractable, we introduce an Identity Consistency Loss that enforces appearance coherence across frames. We release Quad-Imaginarium, the resulting open-source dataset of 7488 language-annotated quadruped motions (18.5 hours) spanning acrobatic and performative behaviors. We validate 392 randomly sampled motions on a real Unitree Go2 with a 96.7% deployment success rate, complemented by a 97.6% success rate across the full dataset in simulation.

Abstract:
Model-free reinforcement learning has enabled impressive humanoid locomotion. Existing methods are dedicated to developing policies that can locomote on complex terrains, but locomotion on sloped terrain is seldom explored, requiring simultaneous stability and posture control. To address these challenges, we propose a two-stage framework with physics guidance for blind humanoid slope locomotion. In Stage I, we learn a general locomotion prior with a terrain-aligned apparent Zero Moment Point (ZMP) regularizer, evaluating balance on a local support plane rather than a world-horizontal plane. However, the trained policies on steep inclines often present inefficient crouched gaits and unnatural postures that overuse the low Center of Mass (CoM) for short-term stability. To handle these issues, we further introduce the Biomechanical Slope Gait Adapter (BSGA) in Stage II, which uses PCA-extracted macroscopic terrain descriptors as training-only critic inputs and reward-gating signals. It conditions CoM-height, joint-level gait, and swing-leg priors on the estimated slope geometry, encouraging hip-dominant uphill propulsion and knee-oriented downhill braking while keeping the deployed actor purely proprioceptive. To address the absence of simulators, we also develop a variety of uphill terrain environments for policy training based on Isaac Lab. Sim-to-real experiments on a real-world Unitree G1 show that our method mitigates crouched-gait degeneration and enables blind continuous traversal of outdoor grass slopes up to 25°, providing a physics-informed approach for robust humanoid locomotion on steep inclines. keywords: Humanoid Locomotion, Sloped Terrains, Reinforcement Learning

Abstract:
While recent advances in perceptive locomotion have enabled humanoid robots to traverse structured terrains, agile parkour in highly discontinuous environments remains an open challenge. In particular, crossing sparse footholds and narrow regions requires precise foothold selection, effective use of visual observations, and consistent alternating foot placement during fast transitions. In this paper, we present an end-to-end perceptive humanoid parkour framework that enables stable traversal across terrains with limited foothold availability using only onboard depth sensing. The framework features a saliency-guided temporal perception module that combines a foothold saliency prior with gated memory. It retains task-relevant depth features across frames, enabling reliable foot placement from partial observations. By introducing an alternation loss, our symmetry regularization encourages natural alternating gait patterns and improves traversal robustness. Extensive experiments show that our method significantly improves success rate and foothold accuracy on challenging parkour terrains in both simulation and real world.

Abstract:
Extending humanoid traversal to the open world is key to practical deployment in human environments, but remains challenging. The robot must use vision to ensure safe and reliable foot placement on heterogeneous terrain under highly dynamic motion, while producing coordinated, natural whole-body behaviors. We propose SSR, an efficient end-to-end framework for egocentric vision-based humanoid traversal that jointly learns these capabilities. SSR introduces imagined foothold guidance, which learns to model forthcoming swing-foot contacts and evaluates their support to guide pre-touchdown swings toward stable regions, reducing edge slips. It further employs equivariant latent-space symmetry augmentation to efficiently induce bilateral coordination under high-dimensional visual observations, and uses terrain-specific multi-discriminator motion priors to encourage human-like behavior across scenes. Extensive experiments show that SSR achieves safe, stable, and high-quality locomotion on diverse real-world terrains, including stairs with varied structures and extreme challenges such as wide gaps and high platforms, while enabling reliable long-horizon traversal in open outdoor environments.

Abstract:
Humanoid behavior foundation models aim to acquire reusable whole-body control policies from broad human motion priors, enabling a single controller to produce diverse and expressive behaviors. However, existing motion-centric foundation policies largely assume that the reference motion is already physically compatible with the robot's surroundings. This assumption breaks when the demonstrator, operator, and robot inhabit different environments: a human motion may specify the intended behavior, but not the footholds, clearance, body height, or contact timing required by the robot's local terrain. We introduce \emphPerceptive Behavior Foundation Model (Perceptive BFM), a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception. The model preserves raw kinematic motion references as the behavioral interface, while using local terrain observations to adapt contacts, posture, and timing. To provide scalable terrain supervision, we develop \emphterrain-conformal reference synthesis (TCRS), which converts locomotion-oriented human motion clips into terrain-consistent references through contact-aware foothold construction, foot-geometry-aware swing optimization, support-aware root reconstruction, collision repair, and multi-point inverse kinematics. We then train a blind adapted-reference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student through target-frame action alignment. The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways initialized to preserve the motion-tracking prior and trained to produce local corrections only when needed. Across quantitative TCRS analysis, matched-compute training ablations, and qualitative real-robot rollouts, a single policy tracks locomotion, stylistic motions, acrobatic maneuvers, and motion-capture teleoperation across stairs, slopes, sparse supports, recessed obstacles, grass, and irregular indoor/outdoor terrain. The results indicate that robot-centric perception can turn human motion priors into terrain-compatible whole-body behavior without changing the raw motion command interface.

Abstract:
Text-conditioned motion generation has become a promising interface for programming humanoid robots, but current generators are often trained on human motion datasets retargeted to robot morphology. While such data provides rich semantic and kinematic priors, it does not capture the nuances of the whole-body tracking controller, including balance, contact, actuation limits, and controller-specific failure modes. Therefore, generated motions can be semantically plausible yet difficult or impossible for the robot to execute. We propose TEXEDO, a test-time scaling framework for humanoid motion generation that improves motion quality without requiring a stronger underlying generator. Given a text prompt, our method samples multiple motions from the pre-trained text-conditioned generator and selects the best executable and task-aligned motion. The reward model combines a dynamic feasibility verifier, distilled from whole-body tracking rollouts to predict physical executability, with a semantic alignment verifier that measures text-motion alignment in a learned co-embedding space. Our pipeline treats dynamic feasibility as a hard constraint and semantic alignment as the selection objective within the feasible set. Across large-scale simulation studies and real-world deployment on a Unitree G1, we show that our test-time scaling strategy consistently improves both tracking fidelity and text alignment, demonstrating that grounded verification is an effective path toward deployable language-guided humanoid motion generation.

Abstract:
Responding to rhythm is a natural human behaviour but a challenge of high-dynamic whole-body control for humanoid robots. Existing motion generation models produce expressive but physically invalid trajectories. As a result, even an expert-trained tracker accumulates a feasibility gap on long sequences of generated dance. In this work, we present InfiniDanceRobot, a latent-guided closed-loop system that streams music-conditioned dance directly to a physical humanoid robot. We collect about 50 hours of dance: 30 hours of lab-controlled motion capture data and 20 hours filtered from public datasets. To provide continuous motion generation ability, we propose a planner building on diffusion forcing with a causal receding noise schedule and cross-modal alignment. Then the latent-guided closed-loop optimization framework is designed to correct next-token predictions from tracker feedback during both training and inference. Experiments show that InfiniDanceRobot can generate expert human dance for the robot, producing continuous, physically plausible, and high-dynamic motion across diverse music styles and dance genres.

Abstract:
Humanoid behavior learning uses either physical simulations, which generalize to diverse motions but have limited fidelity, or neural world models, which are highly expressive but data-hungry and brittle. Residual world modeling retains the strengths of both: it preserves the first-principles priors of the physics simulator and only learns the sim-to-real gap. A key challenge when training the residual model for humanoids is accurately initializing the floating base state and contact forces in the simulator to match recorded hardware states. Using the Unsupervised Actuator Networks (UAN) formulation, our method learns a residual correction without torque sensing or motion capture by using a novel proprioceptive state estimator to align initial contact forces and base velocities during open-loop playback of hardware trajectories. This alignment allows the model to capture physics discrepancies rather than artifacts from inconsistent simulator initialization. The resulting residual simulator constrains downstream policy learning to actuator-feasible motions, improving sim-to-real transfer robustness, and enabling efficient locomotion and precise end-effector force control on hardware.

Abstract:
We present a real-time framework for expressive legged locomotion that enables independent control over velocity and style. Unlike existing data-driven controllers which typically require explicit style labels for training, our method efficiently leverages large-scale unstructured motion data, enabling generalization to unseen styles. We disentangle style from functional locomotion through a tailored sampling strategy used to train a style encoder, which is fed to an autoregressive diffusion model. During inference, the user provides a style reference clip and a joystick velocity command. The diffusion model then generates a kinematic reference tracked by a reinforcement learning controller. To improve user controllability and stylization strength, we propose an attention-biasing mechanism and a velocity command correction strategy. We demonstrate our method on a physics-based humanoid character in simulation and on a physical robot, showcasing robust real-time control and continuous style modulation.

Abstract:
Humans expand their locomotion capability by wearing appropriate footwear, such as hiking boots for mountains or skates for ice. Footwear changes the ground-contact interface and enables traversal of otherwise difficult terrain. In contrast, locomotion controllers are typically trained for a fixed morphology, and their performance can degrade when shoes alter foot geometry, mass, and friction. We propose SHOES, a sample-efficient framework for adapting locomotion policies to footwear-induced morphology changes. SHOES freezes a simulation-trained nominal policy and fine-tunes only lightweight, identity-initialized modulation modules on the robot, preserving the initial behavior while enabling efficient hardware adaptation. We also introduce SafeBox, a workspace-aware way-point sampler that constrains online adaptation within a user-defined workspace. The simulation results with a shoe-equipped Unitree Go2 show that SHOES substantially recovers locomotion performance after footwear-induced morphology changes using limited additional adaptation. Under the same adaptation budget, SHOES outperforms FiLM-only, Linear Adapter-only, and residual-RL baselines. Real-world experiments on a Unitree Go2 fitted with human shoes further evaluate the effectiveness of SHOES for hardware deployment.

Abstract:
Reinforcement learning has become the leading paradigm in legged locomotion, enabling complex behaviors from backflips to parkour via massively parallel simulation. Under PPO's non-stationarity, shallow networks remain the de facto architecture, supported by carefully staged environments; nevertheless, the representations learned by these policies remain poorly understood, specifically the relation to motion produced on hardware. In this work, we empirically study locomotion policies through the effective rank of the network Jacobian and show that conditioning rank on gait phase exposes representation structures invisible from a global rank. In particular, we find that architectural choices, namely observation and layer normalization with residual connections, allocate more than one additional dimension of effective rank to swing-dominant states than to stance-dominant states, which is mostly absent in vanilla MLPs. Building on this, we propose practical recommendations to convert these into diagnostics for healthier and smoother policies. In practice, this results in lower joint jitter, up to 2x, not only in simulation, but also on a physical Spot. More broadly, our findings show that how a policy distributes its capacity across the gait cycle is closely tied to its health and sim-to-real transfer.

Abstract:
Recent perception-free end-to-end (E2E) autonomous driving methods bypass explicit perception outputs by compressing dense image patch tokens into compact scene tokens for downstream trajectory generation and scoring. While these scene tokens form a compact visual bottleneck for the planner, they receive supervision solely from the planning objective, providing limited constraints on the encoded visual information. To address this limitation, we introduce Neural Token Reconstruction (NTR), a representation learning framework to directly constrain the compact scene-token bottleneck in perception-free driving. NTR introduces a self-distillation masked latent reconstruction objective that reconstructs masked patch-level latent features using only compact scene tokens as reconstruction memory. This forces reconstruction gradients to pass exclusively through the scene-token bottleneck, encouraging scene tokens to preserve richer and less redundant visual representations for planning. We further introduce semantic priors derived from foundation-model annotations as a weak semantic interface biasing reconstruction targets toward driving-related structures without introducing explicit perception heads. All auxiliary reconstruction components are removed at inference time, leaving the deployed planner unchanged. NTR achieves state-of-the-art performance on three public autonomous driving benchmarks, including 8.0461 RFS on Waymo E2E and 94.1 PDMS / 90.9 EPDMS on NavSim1&2. The learned scene tokens exhibit lower pairwise redundancy and higher effective rank, indicating that effective bottleneck supervision improves both compact visual representation learning and planning performance. We further validate NTR on a private large-scale driving dataset and integrate it into a real-vehicle planning stack, demonstrating its practical applicability beyond public benchmarks. Real-world deployment videos are provided in the supplementary materials.

Abstract:
Generating safety-critical scenarios is essential for validating and improving autonomous driving systems, yet it inherently requires maximizing adversariality to expose failures while preserving realism. Existing methods usually manage this trade-off with handcrafted heuristics, confining generation to known priors and overlooking underexplored patterns. While recent open-ended agentic evolution can push this limit, unconstrained general agents lack strict simulator grounding and tend to collapse the multi-objective tension into single-scalar maximization. Here we present EvoDrive, the first automated, LLM-based agentic evolution framework for multi-objective scenario generation. EvoDrive employs a simulator-grounded actor-critic architecture where a memory-driven actor iteratively proposes improvements to the generators and critics filter out implausible candidates, and a self-evolving world evaluator routes promising proposals to optimize simulation budgets. EvoDrive further maintains a Pareto archive of evaluated candidates to preserve diverse attack-realism trade-offs and guide future evolution via simulation feedback. Benchmark results on MetaDrive and CARLA show that EvoDrive not only significantly expands the Pareto frontier across various generators, but also produces valuable scenarios for policy training.

Abstract:
Anomaly detection is essential for ensuring the safety and reliability of autonomous driving systems operating in open environments, where rare objects, abnormal behaviors, and long-tail events frequently arise. Conventional perception-based approaches often lack the contextual understanding necessary to interpret complex anomalies, while recent Large Language Model (LLM)-based methods incur substantial computational overhead and exhibit limited integration with geometric sensory modalities. To address these challenges, we propose SOFA, an adaptive slow-fast anomaly detection and reasoning framework that selectively activates multimodal reasoning only when anomalous events are detected. SOFA integrates three key components: 1) a fast LiDAR-based anomaly detection module for efficient real-time perception, 2) a slow VLM-based reasoning module for semantic understanding, and 3) a Time-to-Collision (TTC)-driven switching mechanism that dynamically coordinates the two modules to balance efficiency and reasoning fidelity. To enable systematic evaluation of multimodal anomaly detection and reasoning in real-world driving scenarios, we further construct MAQA, a Multimodal Anomaly Question Answering dataset comprising 16K anomalous objects, synchronized LiDAR-image observations, and reasoning-oriented language annotations. Extensive experiments on MAQA demonstrate that SOFA significantly outperforms baselines in detection accuracy, computational efficiency, and reasoning capability.

Abstract:
Vision-language-action (VLA) models have advanced generalist robotic manipulation, yet real-world deployment reveals a fundamental challenge: robots are equipped with diverse and heterogeneous sensor configurations, auxiliary sensors can fail unexpectedly during operation, and different robot embodiments often lack certain sensors by design. A unified policy that can exploit auxiliary perceptual inputs when available while remaining reliable under sensor absence — whether incidental or by design — is therefore essential for practical deployment. However, existing VLA policies couple action generation to a fixed sensor set through shared dense computation, making them brittle when sensors are missing and limiting their ability to specialize across diverse tasks and long-horizon behaviors. We propose CoRE-VLA, a scalable and robust VLA framework that formulates action generation as context-conditioned sparse computation. Sensor availability gates modality-specialized experts, enabling graceful degradation under missing sensors without retraining. Task intent further routes action-side representations to task-relevant experts, improving specialization across diverse tasks and long-horizon subgoals. While the framework is designed to accommodate different auxiliary sensors, we focus on depth as a representative and practically important auxiliary modality in our experiments. Experiments on LIBERO, RoboCasa GR1 Tabletop, and real-world dual-arm manipulation show that CoRE-VLA achieves strong results on long-horizon and multi-task benchmarks, and outperforms both a dense-action-generator ablation and a strong pretrained VLA baseline, including in zero-shot generalization to unseen scenarios. Modality analysis shows that CoRE-VLA can exploit auxiliary depth when available while remaining robust when depth is unavailable during deployment.

Abstract:
Latent action pretraining learns representations of visual change from pairs of observations, but existing methods typically encode each transition as a single unstructured representation that entangles transition extent and transition mode. We introduce Polar Latent Actions with Radial structure (PoLAR), which imposes a radial-direction structure on latent actions, encouraging radius to encode transition extent and direction to retain transition mode. PoLAR uses temporal offset between two observations as a weak proxy for transition extent, encouraging latent action from observation pairs separated by larger temporal gaps to occupy larger radii. We instantiate this structure in hyperbolic space, whose expanding volume with radius offers a natural fit for more diverse transition modes at larger extents. Across in-task and large-scale pretraining settings, PoLAR improves downstream policy performance in simulation and real-world robot experiments, outperforming latent action baselines and strong pretrained VLAs. These results suggest that the geometry of the latent action space is an important design choice for transferring visual pretraining to downstream robot policy learning.

Abstract:
Many essential manipulation tasks -- such as food preparation, surgery, and craftsmanship -- remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human notions of task quality. Using only 50--200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.

Abstract:
In Human-Robot Interaction, the standard approach to learn a reward model that represents human preferences for robot behavior consists of three steps. First, the robot collects limited direct evidence from human feedback (e.g., positive or negative binary feedback). Then, the robot utilizes the direct evidence to derive accepted or rejected labels to feasible but unchosen actions using fixed implication rules. Finally, the robot updates the reward model with both the direct and derived evidence. Unfortunately, the fixed rule can hinder preference learning: in a user study with two collaborative simulation environments, human-provided implication labels often differed from the standard fixed rule, and using the human labels substantially improved reward learning with the Preference Learning from Implicit and Explicit Feedback (PIE) algorithm. Consequently, we propose IMPLIED, an implication modeling method that treats fixed-rule implications as an initial guide while learning to infer and revise accepted and rejected action labels over time. Across evaluations on recorded human-robot interaction trajectories and a physical robot pizza-making study, IMPLIED predicts human implications more accurately than the fixed rule approach and LLM baselines, approaching the performance of a human-label oracle. In turn, IMPLIED reduces preference-estimation error and leads to robot actions that are more often rational with respect to a combined reward (which includes the true preference reward and a task-specific reward) compared to baselines. By learning to reason about the implications of human feedback, this work enables more faithful and efficient robot behavior adaptation during human-robot collaboration.

Abstract:
Offline-to-online reinforcement learning is a promising paradigm for generalizable robotic manipulation, but its full-stack complexity makes system behavior difficult to reproduce and diagnose. Within such systems, value estimation plays a central role by determining how heterogeneous data are prioritized for policy improvement. Despite its importance, a central question remains underexplored: how value-function reliability affects value-guided learning across offline pretraining and online adaptation. To answer this question, we ground our analysis in a complete offline-to-online reinforcement learning pipeline and introduce value-reliability metrics that quantify value-function quality along global task-level progress and local action-level preference, enabling us to analyze how value estimation affects downstream policy behavior. Across the extensive empirical study encompassing 240 hours of pretraining data and over 3,000 rollout trajectories, our experiments show that these metrics are closely aligned with downstream performance: More reliable value functions yield better action-quality estimates, enabling value-guided offline RL to scale more effectively than quality-agnostic behavior cloning. As value estimates become more reliable, online adaptation becomes more stable through better prioritization of high-quality rollout data. Integrating reliable value guidance through offline pretraining with online adaptation, our system achieves 86% success on millimeter-level precise chip insertion and 84% on generalizable block disassembly. Ultimately, we hope our findings could underscore the importance of value-guided data utilization for turning heterogeneous robotic experience into effective policy improvement. We will release the full offline-to-online RL framework and the corresponding pretraining data to promote reproducibility and further research.

Abstract:
Self-play reinforcement learning at scale has emerged as a powerful paradigm for training autonomous driving policies, producing agents that exhibit robust collision-avoidance and stable navigation in simulation. However, policies trained purely through self-play lack exposure to the distribution of real human driving behavior, resulting in trajectories that appear unnatural. To overcome this, we present Self-play with Human Imitation Fine-Tuning (SHIFT), a hybrid training framework that fine-tunes a pretrained self-play foundation policy on real-world driving demonstrations without changing the underlying reinforcement learning algorithm. SHIFT fine-tunes in closed-loop log-replay simulation, in which the ego vehicle is rolled out by the learned policy while the non-ego agents are replayed from logged human trajectories. Distance to the human expert trajectory is simply an additional reward term in the same PPO with advantage filtering algorithm used for pretraining. To enable extended fine-tuning without losing the base self-play performance, we use a KL regularization loss against the frozen self-play foundation policy. On the closed-loop nuPlan Val14 benchmark, SHIFT improves over the self-play baseline in human-likeness (ADE: 9.4 m → 8.0 m, FDE: 21.3 m → 18.5 m), safety (collision rate: 6.4% → 3.9%, drivable area violations: 2.5% → 1.7%, red light violations: 1.6% → 1.0%), and comfort (brake tap rate: 29% → 20%, oscillation rate: 9% → 4%), and achieves the best ADE, FDE, collision, and drivable area scores among all baselines, including extended self-play, behavior cloning, BC+RL, and human-regularized PPO. We deploy both policies on self-driving cars and find similar improvements in the real world (2.6× fewer safety takeovers: one every 0.57 → 1.46 miles). To our knowledge, this is the first evaluation of self-play and self-play with human imitation on public roads. Our results suggest that post-training self-play with human imitation, stabilized by KL regularization, is a promising direction for deployable autonomous driving systems. An open-source implementation, along with additional materials, will be available on the project page: https://rohit-ash.github.io/SHIFT/

Abstract:
Abstract: Mobile Manipulation (MoMa) is a cornerstone of autonomous robotic systems, yet real-world fine-tuning of such tasks remains a formidable challenge due to their high-dimensional action spaces and the intricate coupling between heterogeneous modules. In this paper, we propose HiRe-MoMa (Hierarchical Residual Mobile Manipulation), a systematic framework for stable and sample-efficient real-world fine-tuning of complex MoMa systems. At its core, HiRe-MoMa adopts a Hierarchical Residual Architecture that fine-tunes a frozen base policy. A Policy-Agnostic Residual learns additive corrections atop the base policy, while Hierarchical Decoupling splits the residual into mobile-base and manipulator-arm heads with unidirectional conditioning, mitigating gradient interference and aligning corrections with the platform’s physical hierarchy. To improve sample efficiency in data-expensive real-world settings, we integrate Expert-Guided Value Alignment, which leverages sparse human interventions with under 20% training coverage to reshape the critic’s value landscape, effectively guiding autonomous exploration through expert priors. Extensive real-world experiments on the TIAGo++ platform demonstrate that HiRe-MoMa consistently enhances task performance with only around 30 minutes of online interaction, regardless of the base policy architecture. To our knowledge, this is among the first frameworks to enable efficient on-robot fine-tuning for mobile manipulation in the real world, providing a practical paradigm for deploying complex robotic systems in dynamic, human-centric environments. Code and videos are available in the supplement.

Abstract:
Robotic foundation models achieve strong generalization by leveraging internet-scale vision-language representations, but their massive computational cost creates a fundamental bottleneck: high inference latency. In dynamic environments, this latency breaks the control loop, rendering powerful models unsafe for real-time deployment. We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution. Inspired by hierarchical control, AsyncVLA runs a large foundation model on a remote workstation to provide high-level guidance, while a lightweight, onboard Edge Adapter continuously refines actions at high frequency. To bridge the domain gap between these asynchronous streams, we introduce an end-to-end finetuning protocol and a trajectory re-weighting strategy that prioritizes dynamic interactions. We evaluate our approach on real-world vision-based navigation tasks with communication delays up to 6 seconds. AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines, effectively bridging the gap between the semantic intelligence of large models and the reactivity required for edge robotics.

Abstract:
Bird's-eye-view (BEV) images have been widely demonstrated to provide valuable prior information for navigation. Given the global information provided by such views, two key challenges remain: how to fully exploit this information and how to reliably use it during execution. In this paper, we propose a navigation system that uses BEV images as global priors and is designed for ground and near-ground robotic platforms. The system employs an image generation model to interpret human intent from natural language, identify the target destination, and generate traversability masks. During execution, we introduce cross-view localization to align the robot's odometry with the BEV map and mitigate long-term drift in conventional odometry. We conduct extensive benchmark experiments to evaluate the proposed method and further validate it on both quadruped robot and UAV platforms. Using only a conventional local motion planner, the robot successfully completes a 285-meter outdoor long-range navigation task. This work demonstrates how the world-understanding capabilities of foundation models can be transferred to embodied navigation, enabling robots to benefit from the strong generalization ability of existing image generation models.

Abstract:
Current navigation benchmarks focus on task success but do not capture the economic constraints essential for commercializing autonomous delivery systems. We introduce CostNav, an Economic Navigation Benchmark that evaluates physical AI agents on a cost-revenue and break-even analysis, pairing Isaac Sim's collision and cargo dynamics with industry-standard data such as Securities and Exchange Commission (SEC) filings and Abbreviated Injury Scale (AIS) injury reports. To our knowledge, CostNav is the first physics-grounded economic benchmark to use regulatory and financial data to quantify the gap between navigation metrics and commercial deployment, revealing that high task-success rates alone do not ensure economic viability. Evaluating seven baselines (two rule-based and five imitation-learning methods), we find no method economically viable: all yield negative contribution margins. CANVAS, using only an RGB camera and GPS, attains the highest task success and the least-negative margin among methods with non-zero Service-Level Agreement (SLA) compliance (-\28.40/run), outperforming LiDAR-equipped Nav2 w/ GPS (-\37.34/run). A sim-trained policy evaluated on a real delivery robot yields SLA compliance close to its simulation result, indicating that policy performance in CostNav's simulation transfers to real-world deployment. We challenge the community to achieve economic viability on CostNav, which scores methods by cost-revenue outcomes.

Abstract:
Robots are often deployed in environments where safety must be learned from experience rather than specified in advance. Demonstrations, operator interventions, near misses, and failed rollouts can reveal which behaviors are risky, but they provide only local evidence. The key challenge is not only to identify risky behavior, but to make that knowledge shape the distance a robot uses for long-horizon planning. Existing goal-conditioned methods can compose short behaviors into longer plans, but they usually assume the safety-relevant cost is already known. We introduce CoGeL, a constraint-to-geometry learning method that learns a soft safety signal from robot data and incorporates it directly into the goal-reaching distance used for planning. This makes the learned distance safety-aware, instead of treating safety as a separate classifier, hard mask, or post-hoc filter. We show that errors in the learned safety signal lead to controlled changes in the navigation geometry, that approximate distance consistency supports long-horizon composition through waypoints, and that calibrated safety scores limit the expected number of true violations. Experiments across different navigation tasks show that CoGeL substantially reduces unsafe events while preserving strong long-horizon goal-reaching.

Abstract:
Visual localization is critical for embodied agents, but deploying state-of-the-art deep feature matchers on edge devices is fundamentally limited by their massive computational cost. Standard deep matchers tightly interleave intra-image context building and inter-image matching at every layer, creating a severe "interaction bottleneck." Localizing a single query against K database images forces the system to execute the full network K independent times with no capacity for caching. To break this bottleneck, we advocate for a paradigm shift toward cacheable image matching. We realize this paradigm with LateMatch, a versatile approach that restructures existing matchers by deferring inter-image interaction to their final layers. By isolating intra-image processing into offline-cacheable Context Blocks, only a lightweight set of Interaction Blocks must be executed online. To further optimize LateMatch, we introduce E-SToRE (Entropy-guided Spatial Token Reduction and Expansion). E-SToRE leverages the decreasing Shannon entropy of the late Interaction Blocks to dynamically reduce token counts just prior to cross-image interaction. Evaluated across HPatches, ScanNet, MegaDepth, and ETH3D, LateMatch reduces effective computational cost by over 2x compared to standard deep matchers, while E-SToRE yields an additional 10% to 14% savings with negligible degradation in accuracy.

Abstract:
Long-horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and arm control. While recent Vision-Language-Action models excel at short-horizon tasks, they lack the hierarchical reasoning required for multi-stage objectives. Furthermore, existing hierarchical agents suffer from rigid sub-task mapping, inflexible replanning, and a lack of continuous learning. To address these limitations, we introduce MobiAgent, a dual-loop agentic framework that bridges robust deployment execution and continuous policy evolution. During deployment, the Inner Loop decouples high-level reasoning from low-level control through highly composable atomic skills. It employs Vision-Language models for receding-horizon planning and visual reflection, dynamically composing skills to ensure robust error recovery. These skills are executed by specialized flow-matching experts that share a unified VLM backbone, maximizing reusability while mitigating capacity interference. Concurrently, the Outer Loop drives automated lifelong learning by autonomously segmenting and verifying deployment rollouts, clustering them to discover atomic skills, and continuously fine-tuning the skill library without human annotations. Evaluations across the simulated BEHAVIOR-1K benchmark and real-world tasks demonstrate that MobiAgent significantly outperforms state-of-the-art monolithic VLAs, achieving a +22.5% higher success rate in simulation. Furthermore, it demonstrates robust error recovery against critical failures and enables steady, annotation-free policy evolution via autonomous data recycling.

Abstract:
The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike the large-scale web vision-language data, robotic data collection is an active process incurring prohibitive physical costs. As a result, the key challenge is not merely to collect more robot data, but to obtain training data that provides informative learning knowledge by broadening behavioral coverage and capturing meaningful variations in objects, scenes, and skills. However, how to automatically curate task proposals that maximize the utility of collected robot data remains a critical yet under-explored challenge. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse and informative manipulation tasks across heterogeneous robotic embodiments, spanning single-arm, dual-arm, and mobile robots. RoboGene integrates three core components: (1) diversity-driven sampling for broad task coverage, (2) agentic self-reflection for task-quality assessment, and (3) human-verified feedback memory for execution-grounded validation and continual refinement. We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories and introducing novel metrics to assess task quality in terms of feasibility and diversity. Results demonstrate that RoboGene significantly outperforms frontier foundation models (e.g., GPT-4o, Gemini 2.5 Pro). Furthermore, real-world experiments show that VLA models trained on data generated by RoboGene achieve higher success rates and stronger generalization, highlighting the importance of high-quality task generation. Our project is available at https://x-robogene.github.io/.

Abstract:
Modern robotic perception increasingly involves large-scale geometric optimization problems distributed across multiple robots or sessions. However, existing distributed solvers often depend on brittle hand tuning and primarily target rigid body pose graphs. To address this, we present DeepCORD, a learning-augmented framework for distributed factor graph optimization on general matrix Lie groups. By unfolding a parallel and accelerated Riemannian optimizer into differentiable iterations, DeepCORD learns a self-supervised feedback policy that dynamically adapts solver parameters according to the optimization phase and communication status. The resulting method enables adaptive distributed optimization over matrix Lie groups under both synchronous and asynchronous communication regimes. Extensive experiments on real-world \mathrmSE(3) pose graph optimization and \mathrmSL(4) projective submap alignment show that our method achieves lower objective values than existing distributed baselines on most benchmarks across realistic operating scenarios.

Abstract:
Safe motion planning during online mapping of unknown environments is critical for autonomous systems operating under limited and noisy perception. Neural Time Fields model arrival time as a learned continuous cost-to-go for motion planning, but existing formulations are typically deterministic and do not account for sensing uncertainty. In particular, Active Neural Time Fields lack mechanisms to ensure safety during online deployment, where limited and noisy observations can degrade the safety of planned trajectories and cause collisions. In this work, we introduce an uncertainty-aware neural time field formulation that models and propagates sensor-induced uncertainty in robot–obstacle distance into arrival-time computation via a risk-sensitive representation. We integrate this formulation into a real-time, closed-loop motion planning framework with chance-constrained trajectory optimization. Experimental results demonstrate reduced collision rates during online navigation while preserving path efficiency compared to deterministic and inflation-based baselines, with additional results demonstrating applicability to manipulator systems.

Abstract:
Collision-free path planning in cluttered, real-world environments relies on a representation of the collision-free space, and existing representations broadly fall into two categories. Explicit representations, such as unions of convex sets, can be plugged into optimization-based planners as hard collision-free constraints, but their parameters scale poorly with configuration-space dimension. Implicit representations, by contrast, are flexible and scale well to complex geometries, yet typically lack such guarantees. We bridge this gap with ILD (Invertible Latent Decomposition), a framework that jointly learns an invertible mapping and a union of explicit convex polytopes in the resulting latent space. Planning is carried out over these latent convex sets, and the invertible mapping decodes the resulting paths back to the original configuration space while preserving feasibility with respect to the refined explicit safe regions. We further propose Visibility-Guided Sampling (VGS) to keep the convex sets connected for path planning. Across 2D navigation, 6-DoF, and 14-DoF manipulation environments, ILD achieves broader coverage, better inter-set connectivity, and higher path-planning success rates than prior baselines, with zero observed false positives after test-time refinement. On a 14-DoF bimanual manipulator, we further demonstrate real-time collision-free planning, with test-time refinement adapting to scene-geometry changes during real-world deployment on a single 6-DoF arm.

Abstract:
Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents, with an emerging strategy of scaling test-time compute to improve capability. However, we observe that doing so increases latency, token usage, and FLOPs while yielding uneven, often diminishing gains in downstream success, limiting where embodied agents can be deployed. We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world. We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt, improving the success--cost Pareto frontier over fixed model selection. Across three dominant scaling axes, namely chain-of-thought depth, model size, and memory history, our experiments on VLABench and RoboMME show that test-time compute is not a uniform lever: different axes yield qualitatively distinct capability gains. We validate these insights on a physical Franka arm in a DROID setup spanning zero-shot manipulation and long-horizon chaining, where our router matches or exceeds a stronger model's success rate at up to 65% lower average latency. Ultimately, our results show that naively scaling test-time compute is wasteful, and that DIRECT can provide frontier-level embodied planning in robotic systems at a fraction of the cost. Project page can be found at direct-corl.github.io.

Abstract:
Generative control policies (GCPs), such as diffusion policies and flow-based vision-language-action models, enable test-time scaling in robot control. This compute can be allocated along two axes: sequential scaling, which increases denoising steps to refine actions, and parallel scaling, which samples multiple candidate actions to search across modes of the policy distribution. Existing methods typically rely on fixed conservative schedules that may over-compute in easy states and under-allocate in difficult ones. However, the optimal allocation is hard to know a priori as it is state-, task-, and policy-dependent: early stages of a grasp may benefit from broader parallel exploration, while near-contact phases may require more sequential refinement for precision. We present ELASTIC, an algorithm that learns state-dependent test-time compute schedules for GCPs. We formulate compute allocation as a meta-Markov Decision Process in which a meta-policy interacts with a frozen pretrained robot policy and selects sequential steps and parallel samples at each denoising iteration to maximize task success while minimizing compute. Using reinforcement learning, this meta-policy also learns adaptive compute schedules without access to the GCP's training data. Across simulated manipulation benchmarks with diffusion policies, ELASTIC Pareto-dominates fixed and single-axis scaling baselines at matched compute budgets. On real-world robot manipulation with the \pi_0.5 vision-language-action model, ELASTIC matches best-of-10 success while reducing wall-clock latency by 34%.

Abstract:
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimized physical spotlight attack framework that exploits these vulnerabilities via targeted illuminations, dropping baseline task success rates to zero without any access to model internals. While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations incorrectly condition VLA models to discard color as noise, collapsing their visual perception into a purely shape-biased processor. We expose this degradation through a diagnostic grayscale evaluation, in which the defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 47.5%, well below the undefended baseline. To address this, we propose ChromaGuard, a chroma-preserving adversarial training method. On a physical 6-DoF robotic platform, we demonstrate that ChromaGuard achieves 97.5% and 92.5% success rates in benign and attacked color-dependent tasks, respectively.

Abstract:
Despite growing interest in foundation models for robotic manipulation, systematic evaluation of their capabilities remains elusive. Existing benchmarks are either narrow and saturated, offering little signal on SOTA methods, broadly scoped, providing diversity at the cost of controlled experimentation, or require training on large, messy open-source datasets. We introduce MESA, an evaluation framework designed for precise, reproducible measurement of language-conditioned policy generalization. MESA provides five evaluation suites that isolate in-distribution performance and four distinct axes of generalization: spatial configuration, object instance, object category, and subtask composition, enabling researchers to diagnose where and why policies fail, not just whether they fail. To populate these suites at scale, we develop MESA-Gen, a pipeline for task and demonstration generation built by improving MimicGen. We instantiate the framework on two platforms and release both: the single-arm MESA suite, and BiMESA, the first large-scale benchmark for bimanual generalization. We evaluate seven baseline methods spanning diffusion policies, VLM-based approaches, and pretrained VLAs, finding that the strongest model achieves only 52% on MESA and 49% on BiMESA, providing meaningful signal while leaving ample room for progress. MESA rankings are broadly consistent with RoboArena scores, suggesting that MESA captures method-level differences relevant beyond simulation.

Abstract:
As robots are deployed across diverse environments, manipulation tasks require robot hands with different morphologies to satisfy varying task requirements, hardware capabilities, and cost constraints. However, transferring grasping policies learned on one robot hand to another with a different morphology remains a fundamental challenge due to variations in kinematic structures and degrees of freedom across embodiments. To address these limitations, we propose DiffMorphGrasp, the first diffusion-based framework that incorporates hand morphology-awareness into the generative process for cross-embodiment dexterous grasping that enables zero-shot generalization to unseen robot hand structures. The proposed approach maps grasps from diverse robotic hands into a unified canonical hand representation, providing a common space for learning. Grasp generation is then continuously conditioned on structured representations of hand kinematics, encoded as graphs derived from hand configurations, together with object geometry. In addition, we introduce a loss function that computes joint-level errors over the hand kinematic structure to constrain the generated grasps during training. To evaluate DiffMorphGrasp, we construct a new benchmark for assessing grasp generation performance across robot hands with different morphologies. Extensive experiments demonstrate that DiffMorphGrasp achieves strong zero-shot generalization to unseen hand morphologies, enabling scalable cross-embodiment grasp deployment.

Abstract:
Despite the success of vision-based generalist robotic policies, existing tactile-based policies remain tied to fixed embodiments and sensor setups. This is because tactile signals are highly heterogeneous across hardware, making cross-sensor generalization difficult. We present FTP-1, the first generalist foundation tactile policy pretrained to acquire transferable tactile manipulation abilities across diverse sensors and embodiments. FTP-1 supports varied tactile inputs, including image-, array-, and state-based signals, by using heterogeneous encoders to project them into unified morphology-aware latent tokens that are jointly modeled by a shared tactile Transformer expert. Pretrained on ~ 3,000 hours of tactile manipulation data aggregated from 26 data sources, spanning human and robot demonstrations across 21 sensors, FTP-1 learns tactile skills that transfer beyond the sensors seen during pretraining. Across downstream finetuning experiments spanning 5 hardware configurations, FTP-1 improves contact-rich manipulation on seen sensor setups by +17.2% and, surprisingly, transfers to two previously unseen tactile-sensor setups, achieving a +31% gain in success rate. FTP-1 establishes the first unified foundation baseline for tactile manipulation, providing future tactile policies with a shared model-level starting point. All model checkpoints, datasets, and training code will be released in the future.

Abstract:
Training vision-language-action (VLA) policies for humanoid loco-manipulation is constrained by the high cost and complexity of collecting human teleoperation demonstrations. VLA policies fine-tuned in simulators have so far transferred poorly to humanoid loco-manipulation tasks. We present LEGS (Loco-manipulation via Embodied Gaussian Splatting), a hybrid simulator that composites a mesh foreground (robot, objects, props) over a photorealistic 3D Gaussian Splatting (3DGS) background reconstructed from a handheld scene capture. LEGS uses a procedural motion-primitive generator to synthesize labeled demonstrations at scale without human teleoperation, and a two-stage color calibration that aligns renders to the deployment camera. On a Unitree G1 humanoid robot, across three pick-and-place tasks of increasing whole-body difficulty and three VLA backbones (psi0, pi0.5, GR00T N1.6), a policy trained purely on LEGS data matches or exceeds one trained on human teleoperation demos on every experiment. It also outperforms a mesh-only baseline trained on the same motions without the calibrated 3DGS frontend. Because motion is recorded independently of appearance, the same demonstrations can be re-rendered for a target scene and objects at over 15x lower cost than teleoperation. When both the objects and the scene change, a policy trained on LEGS-aug data re-rendered for that target maintains task success, while the teleop baseline fails entirely. Code and videos are available at https://hojjunekim.github.io/legs/.

Abstract:
Standardized benchmarks such as LIBERO and robosuite have played a central role in advancing robot manipulation learning, yet most focus on ground-supported manipulation systems, limiting their applicability to dynamics-critical domains such as aerial manipulation (AM). This field presents different system-level challenges, including environmental disturbances, coupled dynamics between the manipulator and floating base, and constrained degrees of freedom, causing task performance to depend jointly on robot embodiment, low-level control, and high-level policy design. We introduce AM-Bench, a modular simulation suite and benchmark for multirotor-based AM policy learning. AM-Bench includes representative embodiments spanning fully actuated and underactuated systems, 12 tasks across contact, transport, and constrained interaction, configurable disturbances that reflect real-world effects including aerodynamic effects and actuator saturation, standard low-level controllers, and baseline policy learning algorithms. Unlike prior manipulation benchmarks that primarily emphasize end-to-end policy performance, AM-Bench enables system-level evaluation of how embodiment, control, disturbances, and policy choices interact. We further demonstrate its diagnostic value through benchmark studies that reveal policy performance under AM conditions, different policy--control interface choices, and robot embodiment differences.

Abstract:
Scaling generalist manipulation policies is bottlenecked not by model capacity but by teleoperation cost: covering the full distribution of object placements, orientations, and scene contexts that a deployed robot must handle quickly becomes prohibitive. We argue this bottleneck is largely artificial. Human egocentric demonstrations—plentiful, cheap, and naturally diverse—already encode the visual and physical regularities needed to handle out-of-distribution (OOD) configurations; the missing piece is a recipe that injects this knowledge into a robot-specific action head without re-training the embodiment mapping. Building on LDA-1B, a 1B-parameter latent dynamics-action model pretrained jointly on cross-embodiment robot trajectories and human ego video, we propose a post-training recipe that keeps both data sources alive throughout fine-tuning. Specifically, we co-train with five complementary objectives—policy (P), forward dynamics (FD), inverse dynamics (ID), video generation (VG), and joint action-observation denoising (JD)—routed through embodiment-aware heads. Human ego data participates in FD, ID, VG, and JD but is excluded from policy supervision, so it shapes the shared visual-physical representation while leaving the embodiment-specific action head uncontaminated. We further unfreeze the VLM backbone during post-training, recovering cross-embodiment knowledge that a frozen-VLM recipe otherwise abandons. Across rigorous out-of-distribution evaluations and a suite of long-horizon dexterous manipulation tasks on two robot embodiments, our recipe consistently outperforms robot-only baselines and matches policies trained on substantially more teleoperated data—demonstrating that scarce real-robot demonstrations, combined with abundant egocentric human data, are sufficient for strong generalization.

Abstract:
World models provide robots with a predictive mechanism for reasoning about how actions change object states in manipulation tasks. However, existing image-based and dense 3D world models do not explicitly impose object-level rigid motion, leaving rigid-object geometry preservation to be learned implicitly and making multi-step rollouts susceptible to shape drift or object-level motion inconsistency. To address this, we propose ORD-WM, a 3D world model that learns object-centric rigid dynamics for robotic manipulation. ORD-WM learns a vector-neuron variational autoencoder supervised by signed distance fields, which encodes each object point cloud as a geometry-aware latent representation composed of 3D vector features, and uses an action-conditioned dynamics model to predict object-wise SE(3) motion increments. Rather than generating unconstrained next latent states, ORD-WM analytically applies predicted object-wise rigid transformations to the current latent states, separating persistent object geometry from pose changes. Experiments show that ORD-WM enables stable, geometry-preserving multi-step prediction and supports efficient real-time replanning for closed-loop model predictive control in simulated and real-world manipulation scenarios. More details and videos are available at: https://anonymous.4open.science/w/ORD-WM.

Abstract:
Reinforcement learning (RL) promises to unlock capabilities beyond imitation learning for Vision--Language--Action (VLA) models, but its requirement for massive real-world interaction prevents direct deployment on physical robots. Recent work attempts to use learned world models as simulators for policy optimization, yet closed-loop imagined rollouts inevitably suffer from hallucination and long-horizon error accumulation. Such errors not only degrade visual fidelity, but also mislead policy optimization by providing unreliable learning signals. We propose WoVR, a reliable world-model-based RL framework for post-training VLA policies. Instead of assuming a faithful world model, WoVR explicitly regulates how RL interacts with imperfect imagined dynamics. It improves rollout stability through a controllable action-conditioned video world model, reshapes imagined interaction to reduce effective error depth via Keyframe-Initialized Rollouts, and maintains policy--simulator alignment through World Model-Policy co-evolution. Extensive experiments demonstrate that WoVR enables stable long-horizon imagined rollouts and effective policy optimization, achieving superior LIBERO performance and consistent real-world gains across multiple robotic platforms. These results show that world models can serve as practical simulators for RL when hallucination is explicitly controlled.

Abstract:
Imitation learning for dexterous hands is bottlenecked by the difficulty of collecting contact-rich demonstrations that transfer faithfully to the robot. Prior wearable-exoskeleton systems record only on the human side and retarget via open-loop mappings calibrated in free space, which degrade under contact. We present SEED-UMI, a framework in which both the human and the robot wear the same exoskeleton: joint encoders become a physically shared measurement, and wrist cameras mounted to the exoskeleton observe the same outer mechanism during both human data collection and robot policy rollouts. This turns retargeting into paired cross-embodiment supervision and lets policies train directly on raw exoskeleton-centric wrist images, without segmentation or inpainting. On five contact-rich tasks, SEED-UMI achieves 3.0× greater data collection efficiency than exoskeleton-based teleoperation and a 70.0% average rollout success rate.

Abstract:
Generative models, such as diffusion and flow-based models, have shown strong promise for robot policy learning by capturing complex and multimodal action distributions from demonstrations. However, policies trained solely with imitation learning often suffer from imperfect demonstrations and distributional shifts, while further improvement typically requires additional expert data. Reinforcement learning offers a natural solution through environment interaction, but effectively finetuning generative robot policies remains challenging due to the intractability of likelihood estimation. In this work, we propose RoboFFT, a forward-process reinforcement learning framework for finetuning generative robot policies, which applies forward noising to sampled actions and uses the weighted score / flow matching loss to construct a surrogate policy ratio for PPO-style updates. We evaluate RoboFFT with popular generative robot policy on representative simulation benchmarks, including long-horizon planning and sparse reward settings. Extensive experiments and analysis demonstrate that RoboFFT consistently improves performance while achieving better stability and training efficiency. We further integrate RoboFFT into a real world RL framework and demonstrate its effectiveness in real world tasks.

Abstract:
Embodied foundation models for manipulation require scalable evaluation and data generation over diverse scenarios, with simulation serving as an important pathway. Automated scene generation offers a promising path, yet prior work has largely emphasized coarse-grained scene layouts rather than fine-grained functional object compositions. Motivated by this gap, we present GIF, an agentic Generation framework for Interactive and Functional object compositions. In this framework, we recast this problem as disentangled reconstruction followed by relative pose recovery. CoGen produces instance-disentangled meshes with coarse initial poses leveraging complementary strengths of 2D and 3D generative models, GPRM refines the relative pose under joint geometric and physical guidance, and a VLM verifier selects the candidate that best matches the structured specification. We further construct a benchmark spanning eight representative contact-geometry classes and compare with state-of-the-art generators; GIF improves both asset quality and relation matching, while reducing collision rate to below 1%. Finally, we synthesize data for policy learning, revealing diversity scaling in both simulation and real-world deployment.

Abstract:
Modern robotic imitation learning pipelines increasingly rely on large-scale datasets, prompting heavy engineering investments in high-performance data infrastructure under the assumption that dataset loading and online augmentation present primary bottlenecks. We challenge this assumption through a systematic, measurement-driven profile of end-to-end policy training within the LeRobot ecosystem across various spatial resolutions. We evaluate standard implementations against a custom, high-performance Rust data engine utilizing Apache Arrow for zero-copy memory access and a copy-on-write (COW) shared cache. Our profiling reveals three key findings: first, across all reachable configurations on consumer hardware, the data path accounts for a mere 1–4% of per-step training time; second, substantial data-path optimizations yielding massive isolated micro-benchmark speedups translate to zero measurable wall-clock improvement in full training; and third, severe data-path degradation stems solely from format-specific serialization flaws, such as unindexed sequential video decoding. Finally, we argue that future robotics data infrastructure must be evaluated against overall training step ratios rather than isolated benchmarks, shifting the community's optimization paradigm away from data loading and toward upstream model layers like visual feature caching.

Abstract:
Dexterous grasping in densely packed same-category clutter remains challenging for Vision-Language-Action (VLA) policies, in part because text-prompted VLAs provide little guidance for selecting among visually identical instances. Although external object masks are commonly provided as visual prompts, making the mask visible does not ensure that the policy maintains focused attention on the masked object---a failure mode especially consequential for dexterous-hand grasping, where small object-grounding errors directly affect contact and closure. We introduce FOCUS, a novel method for Foreground-aware, Object-anchored Continuous grasping that Uses Self-distillation to turn the mask from a passive visual prompt into a training-time alignment signal internalized by the action pathway. During training, FOCUS injects a mask-derived additive bias into the action expert's cross-attention and distills the resulting object-anchored representation into a bias-free pathway, encouraging control to be related to the mask-specified object region rather than object-specific appearance or illumination. To support sequential grasping across multiple objects, a lightweight progress query token co-attends with state and action tokens to trigger sub-task termination and provide phase-aware context. In real-robot experiments at two levels of inter-instance distinguishability, FOCUS achieves 67.5% average grasp success, surpassing all four VLA baselines by at least 26.9 percentage points, with the largest margin (+34.6 points) on the most challenging setting. Additional experiments confirm FOCUS's robustness to unseen object categories (58.4%) and unseen lighting conditions (60.1%), where all four baselines fall below 31%.

Abstract:
How should a robot learn to manipulate objects so fragile that sub-Newton contact forces can cause irreversible damage? Prior visuo-tactile policy learning typically treats tactile sensing as an additional modality to integrate into the policy, for example by fusing tactile inputs with vision or using execution-time adaptation. In direct-contact force-sensitive manipulation, however, the main bottleneck arises earlier: during teleoperated data collection, contact force is not directly observable, and visual feedback is too delayed to reliably maintain the narrow force range required for stable grasping. We address this problem with a tactile reflex controller used during collection, producing teleoperated demonstrations with controller-shaped grasping behavior. These shaped demonstrations form a better curriculum for tactile-free policy learning. The effect is clearest on Action Chunking with Transformers (ACT), where the learned policy matches the teacher's closing profile and achieves 95% stable grasps on the nominal plastic-cup task. The same data intervention also improves in-distribution stability and transfer to an unseen paper-cup variant on π₀.₅. Under randomized external disturbance, however, the same reflex-data π₀.₅ policy fails in 45% of policy-only trials, whereas deterministic reflex arbitration preserves the grasp with no failures. These results reveal a new role for tactile feedback in force-sensitive manipulation: rather than integrating tactile into the policy, we use it as a collection-time teacher that shapes grasping behavior in demonstrations for policy learning, while disturbance rejection remains controller-dependent, revealing the boundary of tactile-free policy.

Abstract:
KlaskTron is an open-hardware physical platform for two-player adversarial reinforcement learning, with a learned actuator model that supports zero-shot sim-to-real transfer of trained policies. Building on this platform, we contribute KlaskTron-IL: a multi-tier imitation learning dataset (80k human demonstration timesteps collected directly on the physical robot, 320k symmetry-augmented timesteps, and synthetic expert trajectories up to 5M steps generated by the zero-shot-validated PPO policy), a GPU-native Isaac Lab environment, and diagnostic baselines that expose three characteristic failure modes of standard IL methods under contact-rich adversarial dynamics: BC's data scaling plateau, GAIL's interpretable but unusable discriminator (Spearman \rho = -1.0 peg--ball proximity recovery with a logit gradient too flat to drive PPO), and an asymmetric BC-prior × reward-density interaction. Because the policy interface used throughout this paper is the same one the actuator model bridges to hardware, algorithms developed against these diagnostics are immediately deployable on the physical robot.

Abstract:
Quadruped robots are expected to operate reliably in dynamic and unstructured environments, where unexpected motor failures may occur and severely degrade locomotion performance. Traditional fault-tolerant control methods typically rely on explicit fault detection or handcrafted recovery strategies, which limit their responsiveness and generalization. In this work, we present a damage-aware adaptive control framework that enables quadruped robots to achieve zero-shot locomotion adaptation under sudden motor failures. At its core lies a pain-inspired reinforcement learning paradigm in which the reward explicitly constrains policy behavior, forcing the robot to exploit actuator-health cues from the ontology-sense module, and thereby promoting safer, more protective responses to faults. To improve robustness, we design a failure-probability curriculum that gradually increases the chance of motor failures during training. Evaluations under weakened-motor and locked-joint faults show that the resulting policy achieves lower velocity-tracking errors than SOTA reference across all evaluated fault categories. Comprehensive experiments in both the high-fidelity Isaac Gym simulator and real quadruped hardware demonstrate that our framework achieves robust, damage-aware locomotion, marking a step toward resilient and deployable fault-tolerant legged robots.

Abstract:
Recent advances in vision language action (VLA) models have shown remarkable potential for autonomous driving by directly mapping multimodal inputs to control signals. However, previous VLA-based methods have not explicitly exploited the critic capability of VLAs to refine driving decisions, even though such capability has been well demonstrated in other LLM-based domains, thereby limiting their performance in complex closed-loop scenarios. In this work, we present a theoretically inspired two-stage framework, CriticVLA, which extends the role of VLAs from acting to judging. CriticVLA first generates a rough trajectory and then refines it through multimodal evaluation and single-step optimization guided by a VLA-based critic, yielding higher-quality driving behaviors. To support this process, we construct a large-scale synthetic dataset of 12.9 million annotated trajectories covering diverse driving scenarios, which enhances the critic’s reasoning and refinement abilities. Extensive closed-loop experiments on the Bench2Drive benchmark show that CriticVLA significantly surpasses state-of-the-art baselines, achieving a 73.33% total success rate and delivering about 30% improvement in challenging scenarios.

Abstract:
Embodied Large Language Models (LLMs) are increasingly used as reasoning modules in robotic control pipelines to improve human-robot interaction, but their memory and generation latency make real-time deployment difficult. Pruning can reduce these costs, but for controllers that undergo multiple pre- and post-training phases, the crucial question is not only how much to prune, but when pruning should occur. In this work, we propose Before Parc Fermé (BPF), a pruning strategy performed during RL that compresses embodied LLM controllers while they are still being optimized for closed-loop behavior. This allows pruning decisions to account for the task-specific supervision and closed-loop feedback that shape the final controller. We propose two variants: BPF-RL, which performs iterative pruning during RL by removing part of the model at predefined training intervals, and BPF-SFT/RL, which first prunes part of the model structure during SFT and then further compresses it during RL using the same iterative strategy as BPF-RL until the target pruning ratio is reached. We evaluate BPF on RobotxR1, an LLM-based autonomous-driving control pipeline, using an established LLM pruning framework (LLM-Pruner), and compare it against post-training pruning, post-training pruning with RL recovery, SFT-stage pruning, and smaller dense models from the same family. Our results show that BPF provides the best task-performance vs. memory and throughput trade-off among the considered pruning strategies. When compressing the larger RobotxR1 models, BPF-SFT/RL achieves a 1.69× better size-end-to-end performance trade-off than directly selecting a smaller dense model from the same family, measured as removed parameters per lost percentage point of control adaptability. On the Jetson AGX Orin mounted on the target robotic platform, the compact models improve decode throughput by up to 27%. Finally,we further show that pruning must be evaluated end-to-end, since changes in generated output tokens can affect downstream latency and closed-loop behavior.

Abstract:
Adapting general Vision-Language Models (VLMs) to Autonomous Driving (AD) is thought to require heavy 3D spatial priors or intrusive architectural overhauls to compensate for a lack of spatial grounding. In this work, we present a counterintuitive pilot study: off-the-shelf general VLMs already exhibit remarkable adaptability across diverse driving tasks, while precise metric depth remains the primary missing link, a bottleneck readily addressable via onboard sensors and depth foundation models. This insight reveals that instead of complex over-engineering, general VLMs merely require a targeted geometric catalyst to bridge the spatial gap. To this end, we introduce the Triplet Incremental Training Framework, a non-intrusive paradigm that seamlessly instills 3D geometric awareness into a general VLM backbone while fully preserving its open-world capabilities. Driven by automatically constructed (image, depth, QA pair) triplets, our framework operates via a unified tri-stream training pipeline: an unfrozen primary VLM stream to adapt to driving tasks, an auxiliary depth stream that guides the visual backbone to implicitly imagine 3D scene geometry via explicit geometric constraints, and a frozen twin VLM stream employing logit-level KL distillation to prevent semantic drift. Crucially, the auxiliary depth branch is discarded during deployment, ensuring the enhanced geometric awareness comes with zero inference overhead. We evaluate our framework across four major benchmarks targeting four diverse capabilities spanning 3D spatial perception, driving-oriented language reasoning, end-to-end planning, and general multimodal understanding. Our approach outperforms state-of-the-art driving VLMs, rivals domain-specific expert models in 3D perception, and successfully retains its original general reasoning capabilities. This demonstrates that Triplet Incremental Training offers a unified, deployment-ready paradigm for transforming general foundation models into robust driving intelligences.

Abstract:
Reinforcement learning for robot manipulation is bottlenecked by reward design: hand-crafted rewards are brittle and prohibitively task-specific, while learned reward models typically demand additional task data and handle failures poorly, leaving them vulnerable to reward hacking. To address this, we introduce the Reference-Anchored Reward Model (RARM), a lightweight visual comparator that leverages a single reference demonstration to generate dense, progress-aware rewards with no per-task reward engineering. RARM couples a DINOv3 backbone with low-rank adapters, a temporal aggregator, and an asymmetric cross-attention comparator, trained once with a contrastive objective on ordinary videos and requiring no robot-specific data. At policy-training time, each rollout clip is matched against reference clips from the demonstration, and the reward is based on confidence-gated progress change from the matched clip, anchoring learning to that single demonstration. Across 10 simulated tasks (LIBERO, MetaWorld) and 4 real-world tasks, including long-horizon settings, RARM achieves the best overall success rates while adding negligible per-step reward generation overhead.

Abstract:
Solving complex tasks using reinforcement learning (RL) from sparse rewards and demonstrations is appealing since one can avoid reward shaping. However, past approaches often require heavy curriculum engineering, extensive training time, and/or infeasible amounts of demonstrations. In this paper, we investigate how scalable RL algorithms, high-throughput GPU simulations, and demonstration-based auto-curricula can address these challenges. We show that our method, FlashRFCL, can solve a variety of high-dimensional robotic tasks faster in addition to relying on just a single demonstration each. We perform extensive ablation studies on the design choices of FlashRFCL, highlighting how each component impacts training speed and performance compared to prior work.

Abstract:
General-purpose robots must perform a wide range of tasks from agile locomotion to dexterous manipulation, but current sim2real RL pipelines depend on per-task structural priors to train such as shaped rewards and demonstrations. Recent work has shown that diverse simulator resets, combined with massively parallel simulation, can replace much of this engineering. Yet, uniformly sampling over this distribution wastes a growing fraction of environments on task configurations the policy has already mastered or cannot yet attempt, so returns flatten when scaling the number of parallel environments. We introduce a simple adaptive sampler, Success Guided Sampling (SGS), that tracks the policy's frontier of capability, concentrating experience on task configurations where success rate sits in a productive band. This results in continued performance gains up to 1M+ parallel environments, 16× beyond prior work, under simple reward structures and no demonstrations. We demonstrate this on two settings where prior approaches required per-task engineering. For agile locomotion, a single policy acquires multiple long-horizon skills that previously demanded per-task priors and separate training runs. For dexterous manipulation, we present the first fully-RL solutions to the nut-and-bolt and rod assembly tasks from the NIST board.

Abstract:
Hierarchical VLM-based robotic systems often use sparse points or bounding boxes as intermediate spatial representations, which are inadequate for region-level grounding. This is particularly evident in instructions such as place near the bowl, which specify a feasible region rather than a single target point. We present RoboMAP, a VLM-based robotic grounding framework that uses adaptive affordance heatmaps as a language-conditioned intermediate interface between spatial reasoning and downstream modules. RoboMAP predicts dense score maps for continuous and object-free target regions, exposing feasible spatial support that downstream segmentation, grasping, and control modules can use beyond sparse prompts alone. To enable scalable training without manual dense labels, RoboMAP synthesizes heatmap supervision from heterogeneous annotations, including points, boxes, and robot trajectory data. Across four spatial grounding benchmarks evaluated with standard point-extraction metrics, RoboMAP obtains the best reported accuracy on three benchmarks while maintaining a 0.04 s grounding-stage forward pass. In 50 real-world dual-arm manipulation trials spanning five tabletop task types, it achieves an 82% success rate using heatmap-guided segmentation and grasp proposal. We further provide qualitative cross-embodiment demonstrations across manipulation and navigation scenarios.

Abstract:
Existing robot planning systems rely on appearance-based reasoning, where visual observations are encoded into latent spaces organized around object appearances (e.g., recognizing a "cart" based on how it looks). However, planning requires reasoning about task-relevant functionalities of objects (e.g., whether an object is "movable"), which appearance-based latent spaces do not capture. As a result, existing approaches struggle to generalize to novel robot-object interactions. We address this limited generalizability through affordance reasoning, enabling planning based on task-relevant object functionalities instead of appearance alone. We introduce A4D, which maps visual observations into a shared latent space structured around affordances (e.g., "movable"). By projecting visual observations into this functional latent space and measuring their proximity to affordances, A4D infers functionalities relevant to the observed object. Furthermore, we introduce an affordance discovery mechanism that expands the latent space to handle unseen scenarios where existing affordances are insufficient. A4D uses proximity in the functional latent space to quantify uncertainty in affordance inference and selectively triggers affordance discovery. We evaluate A4D across several planning tasks involving diverse and unseen affordances. A4D achieves 94% inference accuracy on existing affordances outperforming state-of-the-art approaches by over 15% points, improves new-affordance inference accuracy from ~70% to over 90% with fewer than ~10% of the original training data, and enables 100x faster inference. Code, videos, and data available at: https://A4Dance-reasoning.github.io.

Abstract:
Online 3D Gaussian Splatting (3DGS) is emerging as a practical way to build a representation for robotics applications. However, deploying it on real-world platforms exposes multiple challenges. Rapid maneuvers of a robot induce motion-blurred images, which corrupt the reconstructed geometry and cause camera pose drift. Furthermore, the number of Gaussians grows without bound during continuous online mapping. In this study, we present RoMo-GS, an online 3DGS system designed to address these challenges. Instead of averaging N discrete sub-frames as in prior motion blur-aware 3DGS, we analytically integrate each projected Gaussian over the exposure interval, yielding an O(1) complexity per Gaussian. Building on this, we bound the Gaussians' growth with a Hessian-based pruning score, while jointly refining pose, velocity, and Gaussians with Adam and an online LM step over the Gaussians. On multiple public datasets and a self-collected dataset, RoMo-GS matches or exceeds offline deblurring 3DGS methods in rendering quality while training 15--150x faster, and operates online at competitive quality with about 20% fewer Gaussians than the online 3DGS baseline.

Abstract:
Learning open-world mobile manipulation policies requires vast data to achieve spatial generalization, long-horizon robustness, and scene generalization. Current prevailing data collection paradigms, teleoperation and UMI, would require dramatic human effort and cost at scale. We introduce WANDA: learning open-World mobile mANipulation from one demonstration via a synthetic DAta engine. WANDA first reconstructs background Gaussian splats and robot-object interaction trajectories from source RGBD observations, as a world substrate for later planning and rendering. It then rearranges contact-rich robot-object interaction segments into extensive spatial configurations, utilizing whole-body motion planning to chain them into new trajectories. To enhance long-horizon robustness, it applies Corrective State Expansion to increase the robot and object state diversity at different stages of mobile manipulation trajectories. To unlock scene generalization, novel trajectories are synthesized on generated 3D worlds from daily captured photos. We then combine Gaussian splatting for backgrounds and Isaac Sim rendering for reconstructed objects to render photo-realistic visual observations. We evaluate our data generation pipeline on extensive simulation and real-world tasks across environments. The policies trained from our synthetic data engine achieve strong spatial and scene generalization from one real demonstration.

Abstract:
Vision-Language-Action (VLA) models are a promising paradigm for field robotics, but their deployment is bottlenecked by strict edge SWaP-C constraints. Compression techniques such as quantization, distillation, and pruning are well-established for LLMs and VLMs. Pruning, however, remains particularly underexplored for VLA models. We surprisingly find that pruning VLA models yields drastic performance degradation and increased safety violations. Through spectral analysis, we identify structural properties of VLA weights that explain this fragility. Based on this insight, we introduce GLUESTICK, a post-pruning recovery method that restores much of the original model's capability while preserving sparsity benefits. GLUESTICK, computes a one-time corrective term by interpolating between the dense and pruned models in weight space. At inference, each pruned layer applies this correction to recover lost performance with minimal overhead. Our approach requires no additional training and is agnostic to the pruning algorithm. Across diverse VLA architectures and domains, GLUESTICK, recovers up to 100% of the success rate and restores safety to near-dense levels, while saving up to 5.36 GB of memory and achieving up to 1.75x speedup.

Abstract:
Vision-Language-Action (VLA) models leverage large-scale vision-language pretraining for flexible robot manipulation, yet at test time they remain brittle along two axes: spatial generalization, when object positions differ from those seen during training, and task generalization, when a familiar scene is paired with a different language instruction than the one seen in training. A growing family of methods addresses this brittleness by endowing a policy with the spatial and task-aware information such as 2D pixel-coordinate for object localization and placement. However, we find that existing representation through language prompting or visual prompting does not address the limitations; in contrast, exploiting a 3D point-based representation and feeding it directly to the action head leads to substantial improvements—revealing that how the grounding signal is represented and injected into the VLA is the true game changer. Thus, we propose a lightweight, model-agnostic module that represents the grounding signal in 3D, computes its relative displacement to the gripper, and injects the resulting spatial embedding directly into the action head through adaptive layer normalization. The entire module is a two-layer MLP that requires no changes to the VLA backbone or pretraining pipeline. On LIBERO-PRO, our method improves the average success rate of GR00T-N1.6 from 31.2 to 77.5 points under task perturbation and from 28.1 to 60.2 points under position perturbation (gains of 46.3 and 32.1 points, respectively). Comparable gains are achieved for \pi_0.5 as well, demonstrating that the mechanism is backbone-agnostic. We further validate practical applicability with real-world experiments. Together, these results support our central finding: given adequate grounding lifted into 3D, injecting it directly into the action head is what unlocks both spatial and task generalization in VLAs—achievable with nothing more than a lightweight module on top of a pretrained backbone.

Abstract:
Diffusion trajectory planners with domain-specific experts face a practical question in unseen regions: can they rely on selecting among existing experts, or must the planner itself adapt? We study this question using a planner with a shared adapter and per-domain expert adapters. Across three real-world held-out maps, none of eight test-time routing schemes improves the fleet-mean lateral offset over the static default, which uses the same expert for every scene. Even an oracle that selects the best expert for each scene reduces the mean offset by only 0.9 m. In contrast, reward-based fine-tuning on just 20 target-region scenes yields substantially larger gains. On matched evaluation slices, joint replay achieves a fleet-mean offset of 6.9 m, compared with 12.4 m for the static default. Comparisons under the same target-region training budget show that target-region training drives the improvement, while replaying training scenes limits forgetting on the original ODDs. On nuPlan, fine-tuning across training cities also improves open-loop offsets, although closed-loop performance deteriorates. Further diagnostics reveal that two test-time adaptation baselines achieve low offsets by producing degenerate trajectories. These results show the limited headroom for selecting among existing experts and the greater open-loop gains available from adapting the planner.

Abstract:
Generative robot policies typically begin action generation from an observation-independent standard Gaussian distribution, leaving the choice of source distribution underexplored. This work asks a simple question: where should action generation begin? We propose LeaP, a Learnable source Prior that replaces the standard Gaussian with a proprioception-conditioned diagonal Gaussian over action chunks. Parameterized by a lightweight MLP, LeaP jointly predicts the mean and state-adaptive variance of the source distribution, while keeping the downstream generator architecture and inference solver unchanged. This design provides an observation-informed yet stochastic initialization, allowing the generator to focus on precise action refinement rather than transporting samples from an uninformed noise source. On 15 RoboTwin manipulation tasks, LeaP achieves an average success rate of 81.6%, outperforming four representative baselines—including deterministic-source methods, a no-prior counterpart, and a diffusion-bridge policy—by 6.5 to 25.5 percentage points. The same prior consistently improves both flow-matching and diffusion-bridge generators, while using fewer parameters and converging faster. The advantage carries over to real-world deployment, where LeaP attains the best performance. These results suggest that the source distribution is an independent and reusable design axis for generative robot policies, complementary to the choice of generative dynamics.

Abstract:
Dexterous grasping across diverse object scales requires contact modes ranging from two-finger pinches to bimanual grasps. Existing dexterous grasp synthesis methods reduce the high-dimensional optimization space with manually designed expected contacts and initialization heuristics, which struggle to balance synthesis success rate and diversity. We present HUGS, a human-prior-guided framework for unified dexterous grasp synthesis across modes and scales. Instead of directly retargeting human demonstrations, HUGS learns an object-conditioned human prior that captures human grasp preferences and guides downstream force-closure-aware optimization. The prior is trained on a compact self-collected human grasp dataset with 1.8K grasps over 304 objects, providing broad coverage of object scales and contact modes. During synthesis, HUGS adaptively proposes contact modes and wrist initializations, substantially improving the balance between contact-mode coverage and synthesis success rate over heuristic-based methods. With HUGS, we synthesize 3.2M robotic grasps over 157K scenes, spanning object half-diagonal lengths from 2 cm to 30 cm and modes from two-finger to bimanual grasps. Models trained on the synthesized dataset autonomously select appropriate contact modes in the real world, enabling grasping from screws to large boxes.

Abstract:
Robotic manipulation has seen rapid progress with vision-language-action (VLA) policies. However, visuo-tactile perception is critical for contact-rich manipulation, as tasks such as insertion are difficult to complete robustly using vision alone. At the same time, acquiring large-scale and reliable tactile data in the physical world remains costly and challenging, and the lack of a unified evaluation platform further limits policy learning and systematic analysis. To address these challenges, we propose UniVTAC, a simulation-based visuo-tactile data synthesis platform that supports three commonly used visuo-tactile sensors and enables scalable and controllable generation of informative contact interactions. Based on this platform, we introduce the UniVTAC Encoder, a visuo-tactile encoder trained on large-scale simulation-synthesized data with designed supervisory signals, providing tactile-centric visuo-tactile representations for downstream manipulation tasks. In addition, we present the UniVTAC Benchmark, which consists of eight representative visuo-tactile manipulation tasks for evaluating tactile-driven policies. Experimental results show that integrating the UniVTAC Encoder improves average success rates by 17.1% on the UniVTAC Benchmark, while real-world robotic experiments further demonstrate a 25% improvement in task success.

Abstract:
Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we focus on tennis and propose AdaPT, an Adaptive motion Planning and Tracking framework that learns professional serving and rally styles directly from broadcast videos. This hierarchical design is based on the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal impact on high-level planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades in real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perceptions. To address this, our adaptation mechanism improves tracking robustness by learning to track randomized motion speeds, while conditioning the planner on a learned motion-speed adapter to mitigate the compounding errors. Both simulation and real-world experiments validate the effectiveness of the proposed framework in bridging the sim-to-real gap, exhibiting professional motion styles and task performance.

Abstract:
Vision-language-action policies are a promising foundation for general robot control, but long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, occlusion, and recovery. We study this problem as object-state divergence: the object state used to condition a whole-body action can differ from the state used to decide whether the action achieved the intended physical relation. We propose Persistent Object Tokenization (POT), which maintains role-indexed 3D object records from RGB-D observations and converts them into object tokens for a whole-body action expert. Instantiated as POT-VLA, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system in which object state is both actionable and verifiable. On a Unitree G1, POT-VLA improves a matched direct GR00T-N1.7 baseline from 39/80 to 71/80 successes over eight real-world task families. In an external Being-0-aligned reference, POT-VLA achieves 44/50 successes on aligned service tasks, compared with the 37/50 success reported by the Being-0 paper. The largest gains occur on tasks requiring maintained 3D relations, suggesting that persistent object-centered state is a useful abstraction for verifiable humanoid VLA execution.

Abstract:
Vision-Language-Action (VLA) models such as \pi_0 have demonstrated remarkable generalization across diverse fixed-base manipulators. However, transferring these foundation models to aerial platforms remains an open challenge due to the fundamental mismatch between the quasi-static dynamics of fixed-base arms and the underactuated, highly dynamic nature of flight. In this work, we introduce AirVLA, a system that investigates the transferability of manipulation-pretrained VLAs to aerial pick-and-place tasks. We find that while visual representations transfer effectively, the specific control dynamics required for flight do not. To bridge this "dynamics gap" without retraining the foundation model, we introduce a Payload-Aware Guidance mechanism that injects payload constraints directly into the policy’s flow-matching sampling process. To overcome data scarcity, we further utilize a Gaussian Splatting pipeline to synthesize navigation training data. We evaluate our method through a cumulative 460 real-world experiments which demonstrate that this synthetic data is a key enabler of performance, unlocking 100% success in navigation tasks where directly fine-tuning on teleoperation data alone attains 81% success. Our inference-time intervention, Payload-Aware Guidance, increases real-world pick-and-place task success from 23% to 50%. Finally, we evaluate the model on a long-horizon compositional task, achieving a 62% overall success rate. These results suggest that pre-trained manipulation VLAs, with appropriate data augmentation and physics-informed guidance, can transfer to aerial manipulation and navigation, as well as the composition of these tasks.

Abstract:
World Action Models (WAMs) commonly rely on video generation to bridge visual world modeling and robot control. However, video-based WAMs face three coupled limitations: dense multi-frame future tokens make inference costly, full video prediction spends capacity on action-irrelevant temporal and appearance details, and long-horizon future imagination may introduce errors that mislead action prediction. These issues raise a simple question: Does world action model really need video generation? We propose ImageWAM, a simple WAM framework that repurposes pretrained image editing models for robot action prediction. In contrast to video generation, image editing provides a better-matched prior: it only needs to model a target-frame transformation, focuses on action-relevant current-to-target visual differences, and grounds task instructions to localized visual changes through edit pretraining. In practice, ImageWAM does not decode the target frame at inference time; instead, it conditions a flow-matching action expert on the KV caches produced by image-editing denoising, using them as a compact world-action context. ImageWAM outperforms standard VLA baselines and matching competitive WAMs without additional policy pretraining across different simulator and real-world experiments. It also reduces FLOPs to 1/6 and latency to 1/4 of video-based WAMs. Attention analysis further shows that editing caches focus on task-relevant change regions, supporting image editing as an effective alternative to video-based world-action modeling.

Abstract:
Dexterous manipulation remains a fundamental capability for embodied intelligence, yet its scalability is severely bottlenecked by expensive data acquisition and hardware fragmentation. The current ecosystem lacks a unified data paradigm, preventing the aggregation of demonstrations across structurally diverse robotic platforms. Furthermore, shared visuomotor policies often suffer from visual entanglement, where task-critical spatial semantics are confounded with embodiment-specific appearances. To overcome these barriers, we present AdvDex, a unified Vision-Language-Action framework for scalable cross-embodiment dexterous manipulation. First, we introduce OmniShare, a massive multimodal human demonstration dataset that provides high-fidelity priors to bypass teleoperation bottlenecks. Second, we propose the Joint-Aligned Action Space (JAAS), a canonical SE(3) wrist and 15-finger-joint representation that seamlessly harmonizes human hands, dexterous robotic hands, and parallel grippers. Finally, we employ domain-adversarial learning to explicitly decouple embodiment-specific visual features from universal task skills. Extensive physical experiments demonstrate that AdvDex consistently outperforms single-embodiment baselines, enabling robust zero-shot policy transfer and rapid few-shot adaptation across heterogeneous hardware and unseen environments.

Abstract:
In this paper, we introduce World In Your Hands (WIYH), a large-scale ego-centric dataset comprising over 1,000 hours of ego manipulation data collected in the wild with millimeter-scale motion accuracy. WIYH is built around three components: (1) Oracle Suite, a wearable data-collection kit with an auto-labeling and validation pipeline for accurate motion capture; (2) the WIYH Dataset, which contains diverse multimodal manipulation demonstrations across hundreds of skills and real-world scenarios; and (3) comprehensive annotations, including calibrated visual observations, 3D hand and wrist trajectories, depth, masks, task and sub-task language, and vision-language supervision. We evaluate WIYH through a preliminary real-robot co-training study and a main pre-training study in RoboCasa and on real robots. The results show that ego-centric data improves manipulation policies when robot data are accurate but narrow and that verified WIYH annotations provide substantially stronger pretraining signals than unverified ego data, even when the latter contains more samples. WIYH provides a foundation for studying scalable ego-centric data collection and annotation-quality-aware robot policy learning. All data and documents will be open-sourced.

Abstract:
Contemporary visuo-motor dexterity models often rely on expressive policy classes with diffusion and transformer backbones to achieve strong performance. However, these architectures require significant data and computational resources, and remain far from reliable, particularly for multi-fingered dexterity. Importantly, they model skills as reactive mappings and rely on fixed-horizon action chunking, creating a rigid trade-off between temporal coherence and reactivity. To address these issues, we first introduce Unified Behavioral Models (UBMs), a framework to represent dexterous skills as coupled dynamical systems that capture how visual features of the environment (visual flow) and proprioceptive states of the robot (action flow) co-evolve. As such, UBMs ensure temporal coherence by construction rather than heuristic averaging. Unlike world models that attempt to predict the impact of arbitrary robot actions on the environment, UBMs target behavioral dynamics that encode how demonstrated robot behavior is related to desired impacts on the environment. A UBM can be viewed as a pseudo planner: given an initial condition, it computes the desired robot behavior over the entire skill horizon, while simultaneously ``imagining" the resulting flow of visual features. To operationalize UBMs, we propose Koopman-UBM, a first instantiation of UBMs as a structured latent linear system. K-UBM is computationally efficient, enabling reactivity and adaptation via an online replanning strategy: the model acts as its own runtime monitor, automatically triggering replanning when predicted and observed visual flow diverge beyond a threshold. Across seven simulated tasks and four real-world tasks, our approach matches or exceeds the performance of state-of-the-art baselines, while offering considerably faster inference, smooth execution, robustness to occlusions, and flexible replanning.

Abstract:
Achieving robust humanoid locomotion in complex, unstructured environments requires transitioning from reactive proprioception to proactive perception. However, integrating exteroception remains a significant challenge: existing approaches often fail to achieve high-speed, stable locomotion across diverse, unstructured terrains. In this work, we present Hiking in the Wild, a scalable, end-to-end perceptive legged parkour framework designed for robust humanoid hiking. Our approach maps depth inputs and proprioception directly to joint actions, without relying on external localization and mapping. To ensure safety and training stability, we introduce two key mechanisms: a foothold safety mechanism combining scalable Terrain Edge Detection with Foot Volume Points to reduce catastrophic slippage on edges, and a Flat Patch Sampling strategy that mitigates reward hacking by generating feasible navigation targets. Extensive experiments on the Unitree G1 humanoid robot demonstrate that our policy enables robust traversal of complex terrains at speeds up to 2.5 m/s. The training and deployment code will be open-sourced to facilitate further research and deployment on real robots with minimal hardware modifications.

Abstract:
Perceptive bipedal locomotion over sparse terrain remains a difficult challenge: model-based methods are precise but brittle to uncertainty, while model-free methods are robust but struggle to discover the precise, constrained motions required for safety-critical locomotion where small errors can cause catastrophic failures. We propose a model-assisted reinforcement learning (RL) framework that combines both perspectives in three steps: (1) generate a safe reference trajectory using simplified models; (2) train a privileged teacher policy guided by a control Lyapunov function (CLF) reward built around the safe reference trajectory; and (3) distill the teacher into a vision-based student policy. We show that this model-assistance procedure produces physically grounded locomotion, improving sample efficiency, reducing the need for a complex learning curriculum, and achieving smoother locomotion behavior alongside stepping stone performance comparable to model-free baselines. We validate our approach in simulation and demonstrate successful deployment on a Unitree G1 humanoid robot navigating sparse footholds with lateral constraints.

Abstract:
We present ADAPT, an end-to-end framework for interactive, text-conditioned humanoid whole-body control. Unlike dominant text-to-motion pipelines that generate kinematic motions for a separate tracker, ADAPT solves language control with an end-to-end closed-loop control framework, where the robot must continuously respond to changing commands while maintaining balance, natural motion, and smooth transitions. ADAPT learns a diffusion-based action prior from text-labeled humanoid state-action trajectories, enabling diverse motion skills to be directly executed from language commands. To improve long-horizon robustness and smooth prompt switching, we train a lightweight residual reinforcement learning policy on top of the frozen diffusion controller. We further show that the same diffusion policy can be reused as a steerable text-conditioned motion prior for downstream task adaptation. Experiments demonstrate robust language-grounded skill execution, smooth interactive transitions, and style-preserving downstream control.

Abstract:
Humanoid robots have recently achieved remarkable success in motion tracking. However, during real-world deployment, robots inevitably encounter dynamics gaps—arising from either extra physical disturbances or inaccurate simulation modeling—resulting in a significant training-deployment gap. Therefore, effectively leveraging real-world data to fine-tune policies represents the crucial ``last mile'' for robust deployment. To bridge this gap, we propose HumanoidNDM, an online fine-tuning framework for humanoid motion tracking driven by a real-world-trained neural dynamics model. First, in the data collection stage, we introduce a low-frequency reference perturbation method with an adaptive intensity mechanism, balancing deployment feasibility and exploration efficiency. Second, for the model training stage, we train a robust history-based, proprioceptive neural dynamics model in Euclidean space by incorporating differentiable forward kinematics, eliminating the reliance on costly motion capture systems. Finally, in the policy fine-tuning stage, we directly fine-tune the policy via backpropagated model gradients, achieving high data efficiency. Through physical validation on the Unitree G1 robot, we demonstrate that HumanoidNDM enables rapid real-world adaptation to various perturbations, representing the first online, MoCap-free fine-tuning framework for humanoid tracking policies.

Abstract:
While robot foundation models are growing increasingly capable, the strongest models are typically trained on proprietary data and have traditionally remained closed-source, limiting downstream users' ability to adapt them to new tasks, embodiments, and deployment settings. Following the LLM community, an emerging access paradigm for closed-weight robot foundation models is the managed supervised fine-tuning (SFT) API, where users submit training data and receive a tuned policy without access to model weights, gradients, or training internals. While such APIs let downstream users leverage powerful proprietary foundation models, they restrict policy improvement to pure imitation, ruling out classical reinforcement learning and other closed-loop methods that rely on internal training signals. This limitation is particularly acute for agile, contact-rich humanoid manipulation, where the gap between policy outputs and deployed behavior is large due to novel states, action tracking dynamics, latency, and controller-specific failure modes. In this work, we study how effective this new managed-API regime is for humanoid adaptation, and how closed-loop improvement can be realized within it to push policies toward task mastery. We conduct one of the first empirical studies of managed-API adaptation on a real humanoid, instantiated on Gemini Robotics On-Device (GROD). Direct SFT through the API already substantially outperforms a leading open-weight VLA trained on the same demonstrations, yet still falls short of deployment-level mastery on agile, contact-rich tasks. To close this gap, we introduce CLIFT: Closed-Loop Iterative Fine-Tuning, which turns deployment-time reward feedback into API-compatible supervised data and enables closed-loop policy improvement without accessing weights, gradients, likelihoods, or losses---pushing GROD to near-perfect success after two flywheel cycles, all without ``opening the model box.'' Video results can be found at Project Website.

Abstract:
Universal humanoid motion tracking is essential for enabling humanoid robots to perform diverse tasks in human environments. To capture the wide diversity of human motions, mixture-of-experts (MoE) policies have been widely adopted, leveraging dynamic routing to enhance policy expressiveness. However, we identify that divergence between old and updated policies can be misestimated during policy-gradient updates of MoE policies, due to their dynamic routing. Because most policy-gradient algorithms regulate policy updates based on the divergence, this misestimation can lead to either overly conservative or excessively large updates, which can yield a suboptimal policy. In this paper, we propose Mixture of Experts with Alternating Policy Optimization (MAPO), which alternately optimizes the router and the experts to avoid the divergence misestimation. As a result, MAPO achieves about 1.8× higher reward than an MoE policy trained with an existing policy-gradient method. MAPO also attains the highest tracking success rate among the compared methods and the lowest tracking error across most error metrics, demonstrating its superior performance. Videos can be found on the project website: https://mapo-corl2026.github.io/MAPO/

Abstract:
Long-horizon embodied tasks require agents to learn, reason, and generalize robustly across complex real-world environments. However, existing approaches often struggle with partial observability, fragmented spatial reasoning, and inefficient integration of heterogeneous memories, limiting their capacity for long-horizon adaptation. To address this, we introduce RoboMemory, a brain-inspired framework that unifies Temporal, Spatial, Semantic, and Episodic memory for efficient long-horizon planning and interactive learning. Its core innovations are a comprehensive memory system designed for embodied tasks, and a dynamic spatial knowledge graph for scalable, consistent memory updates. Extensive experiments on EmbodiedBench show that RoboMemory improves the average success rate of Qwen2.5-VL-72B-Ins by 26.5 percentage points and achieves performance comparable to strong closed-source VLM agents. Real-world trials further suggest RoboMemory's potential for interactive environmental learning by reusing accumulated experience across repeated task attempts. These results demonstrate RoboMemory as a scalable memory-augmented framework for embodied agents, improving long-horizon planning and cumulative learning in complex interactive environments.

Abstract:
Flow matching is now the dominant approach for vision-language action (VLA) robot policies, thanks to its expressiveness and simple training objective. However, inference for standard flow matching requires iteratively solving an ODE, limiting the control frequency at deployment. Whilst several works have proposed novel methods to allow for single-shot action generation, we note that the prevailing VLA training recipe already relies on rectified flow matching, which makes one-step and multi-step integration equivalent at the global optimum. We show that across six flow-matching VLAs, one-step action generation largely matches multi-step sampling at scale: the multi-step refinement that these models are typically evaluated with confers little to no consistent advantage at inference. As supported by the theory, this holds as long as models are sufficiently trained, with enough capacity. Practically, the implication is that for large VLAs, the single-step policy comes essentially for free from the multi-step model, enabling low-latency deployment without retraining, distillation, or auxiliary objectives.

Abstract:
We propose LURE, a framework that lets users teach a quadruped robot new navigation skills in cluttered scenes by physically luring it with a teaching rod, so that the robot then responds to high-level gesture and verbal commands. Achieving this raises two challenges. First, physical teaching is data-burdensome, and social cues vary substantially across users. Therefore, the system should be able to quickly adapt to each new user. Second, the choice of social-cue modality is critical, as verbal commands specify the action while gestures specify the spatial target, where neither alone is sufficient in our setting. LURE addresses both challenges by combining data augmentation via scene reconstruction and multimodal conditioning. For the data burden, we reconstruct each recorded interaction scene in simulation and learn to recover from deliberately injected perturbations, expanding a few minutes of physical interaction into a larger training distribution. For the modality challenge, we condition the policy on both gesture and verbal cues, and show that the two complement each other, jointly outperforming either modality used alone. Overall, LURE achieves a 97.15% success rate in reconstructed-simulation evaluations of six navigation scenarios in cluttered scenes, using less than one hour of real-world demonstrations. We further show that LURE adapts to a novel user from under five minutes of additional real-world interaction, raising sim-evaluated success by 22.23% on average. We additionally deploy LURE on real hardware for a multi-obstacle course combining training scenes unseen together during training.

Abstract:
Robots deployed in human-centric environments routinely receive natural-language descriptions of spatial information (“I left my backpack on the table”) that reference parts of the world beyond their perceptual field of view. Traditional metric-semantic mapping ignores this signal, while off-the-shelf multimodal models remain limited in 3D spatial reasoning and are not directly amenable to fusion with other sensor modalities. To convert language observations into a calibrated spatial distribution, we train a Language Sensor Model (LSM) that maps each utterance and its scene-graph context to a multimodal distribution, with mixture weights encoding referential ambiguity (e.g., “which table”) and component covariances encoding spatial uncertainty (e.g., where “on the table” the target lies). We then introduce VL-Map (Vision-Language Metric-Semantic Mapping), a probabilistic framework that treats these language predictions as stochastic observations and fuses them with onboard perception within a unified belief map. On the VLA-3D benchmark as well as on a real-world mobile robot, LSM is the only language predictor whose covariance estimates remain within the calibrated regime; fused into VL-Map, it leads to more accurate predictions of the target object location (~70% more probability mass on the true target compared to the strongest foundation-model baseline).

Abstract:
Vision-Language-Action (VLA) models combine a pretrained vision encoder, a language backbone, and an action head, but no controlled measurement exists of which design choices actually drive performance. We fix the backbone (SigLIP2-So400m and Qwen2.5) and the pipeline (DROID, LIBERO, LIBERO-Plus), and train 63 configurations that vary the action head along four axes (decoder, loss, initialization, inference passes) and the per-module scale, with measured RTX~5090 latency on every cell. Three findings come out of the experiment. First, the action head should be initialized from the last transformer layers of the language backbone; this one choice adds 7.1 points at no latency cost and flips the preferred value of every other axis. Second, the action head is the highest-return module to scale, but only under that initialization. Third, returns saturate at a model size close to the one today's \pi-series VLAs already use, so growing the model further wastes compute. The three together specify \textscEffVLA, a ～3.75B model that matches the strongest open-source VLAs on standard LIBERO, leads on most LIBERO-Plus perturbation axes at lower latency, and transfers without modification to a real SO-ARM101 arm.

Abstract:
Recent visual imitation learning systems have widely adopted multi-camera setups with wrist-mounted cameras as the de facto standard. However, manipulation from a single global view remains challenging, as the policy should capture fine-grained interaction details and identify task-relevant regions without local wrist views. To address this challenge, we present Spatially Conditioned Diffusion Policy (SCDP), a diffusion-based visuomotor policy that achieves precise and robust manipulation in a single-camera setting. Our key idea is that end-effector trajectories can serve as visual attention anchors that reflect task-relevant regions. Building on this idea, SCDP consists of two key components: (i) a visual encoder that produces multi-scale feature maps to capture both broader context and fine-grained visual features, and (ii) a spatial conditioning module that samples point-wise features along intermediate end-effector trajectories in the diffusion loop. Extensive simulation experiments show that SCDP consistently outperforms strong single-view baselines and achieves performance comparable to multi-camera baselines. Real-world experiments further demonstrate precise manipulation and robustness to visual distractors, highlighting the potential of single-camera imitation learning.

Abstract:
Vision-language-action (VLA) policies can deviate from nominal trajectories during manipulation, even when tasks remain physically feasible. Recovering from these deviations is challenging, as they push the policy into unfamiliar state spaces where direct re-planning frequently destabilizes action sequences. We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven VLAs that leverages future visual conditioning as a recovery interface. Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation. At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal. This enables the VLA to robustly map off-trajectory observations back to a familiar future. On failure-injected LIBERO, under controlled recovery timing aligned with the injected failure, B2FF increases the average success rate of a baseline VLA from 56.3% to 74.0%, demonstrating that pre-imagined milestones can guide recovery without fine-tuning the low-level action generator.

Abstract:
Learning from Observation (LfO) is a fundamental robotic capability that replicates how humans and animals socially learn from each other. Beyond its biological parallels, this modality provides a practical solution for data scaling in sample-inefficient and data-starved domains like robotics. Recent work has demonstrated promising results in learning manipulation skills from human videos, yet progress in this area remains difficult to assess. Existing methods vary widely in assumptions, hardware choices, and environment setups making it difficult to draw meaningful comparisons and identify advances in the field. To address these challenges, we introduce RoboReel: a unified benchmark for evaluating models that learn policies from human videos. RoboReel consists of bundled real-world human demonstration videos, simulated robot trajectories, and evaluation environments on ten manipulation tasks. We develop four test suites to evaluate the models’ performance on multiple axes, including the robustness to visual distractors and the ability to complete long-horizon tasks. Our benchmark covers learning-from-observation models from different categories, and studies the effectiveness of multiple representation choices in our benchmark evaluation that covers over seven state-of-the-art algorithms (including our VLA based variants) in the field of LfO. Finally, we present an analysis of the different types of algorithms showing that long-horizon tasks and tasks with low tolerances are still challenging for current models.

Abstract:
We study the problem of scaling dynamic loco-manipulation from a single free-rolling sphere to multiple spheres transported simultaneously on the back of a wheel-legged quadruped, without fences, grippers, or mechanical stops. Multiple identical free-rolling spheres form an unordered set with no persistent identity: their ordering may change independently at each history frame, creating a \emphper-frame permutation symmetry that standard history-concatenation set encoders do not explicitly enforce---these encoders impose only a shared, diagonal permutation symmetry over the full history. We show that this symmetry mismatch leads to a concrete failure mode in curriculum-based reinforcement learning. Within the same PPO training budget, flat MLPs and branch-wise encoders plateau at or below the two-sphere stage, while a history-concatenation Deep Sets baseline (\HCDS) fails to progress past the two-sphere stage in our runs unless ball-to-slot assignments are randomised during training, suggesting that it exploits slot indices as a curriculum shortcut rather than learning identity-free multi-sphere dynamics. We propose Per-Frame Deep Sets (\PFDS), which performs permutation-invariant pooling within each history frame before temporal readout; we prove that \PFDS is \Gframe-invariant and universally approximates continuous \Gframe-invariant policies. A 2×2 ablation over encoder architecture and slot randomisation separates the architectural and data-augmentation pathways, and \PFDS reaches the five-sphere stage with 100% no-drop transport in simulation across all five random seeds. We further distill the \PFDS teacher into \TactSet via DAgger, replacing privileged sphere-state observations with a 16×16 Boolean union contact map, yielding a compact and naturally \Gframe-invariant tactile representation.

Abstract:
General motion tracking and teleoperation offer a promising path to scalable humanoid skill acquisition, yet most existing frameworks are validated on compact platforms or without real payload interaction, leaving full-size humanoids with real payloads largely unexplored. Scaling to full-size humanoids introduces two compounding challenges: their larger inertia and tighter balance margins make tracking highly sensitive to noise, drift, and retargeting errors from commodity VR trackers, while their payload potential remains largely underutilized. We present HEFT, a heavy-payload full-size humanoid teleoperation framework that addresses both challenges. HEFT learns from deployable noisy VR references with physically plausible reconstructed references through Privileged Motion Guidance (PMG), and uses a Windowed Payload Curriculum (WPC) with expert-guided payload caps to acquire robust heavy-payload tracking. We deploy HEFT on L7, a 175cm, 65kg humanoid. The robot tracks motions including turns, forward/backward locomotion, and squats under payloads up to 24kg.

Abstract:
Pretrained video diffusion models (VDMs) offer a powerful foundation for World Action Models (WAMs), which jointly predict robot actions and future observations. Building such models requires injecting low-dimensional robot signals (e.g., proprioception, actions, and values) into a latent space shaped by natural videos, and a key design choice is the interface through which these signals enter: existing schemes treat it as a formatting detail, spending a separate latent frame on each signal or bolting on dedicated encoders and decoders. We argue that signal injection is instead a distribution-alignment problem: how well a representation matches the prior of the pretrained VDMs during denoising. Using the relative reconstruction error of a frozen VDM denoiser as a probe, we find that conventional spatial encodings sit far off this prior, whereas frequency-domain packing yields latents that the denoiser corrects far less. Motivated by this diagnostic result, we propose MetaWAM, which folds all robot signals into a single unified meta-frame: each signal is assigned to a disjoint frequency-domain channel band and lifted to the spatial latent by an inverse DCT, yielding a decodable interface aligned with the pretrained VDM prior without any added module or architectural change. A fixed band layout together with slot-aware routing lets the same meta-frame support policy, world-model, and value prediction over a shared set of robot variables. On RoboCasa, MetaWAM reaches 68.9% success while collapsing the per-signal frames into one and cutting inference latency by 39%, and attains 85.0% success on three real-world Franka tasks.

Abstract:
Unrecongized anomalies in critical infrastructure can rapidly cascade into catastrophic physical or operational failures. Current anomaly detection approaches often rely on direct access to the monitored system by utilizing onboard sensors, such as motor encoders, to detect anomalous states. However, legacy hardware often lacks these sensors, or strict security protocols may prohibit access altogether. We therefore present Adaptive World-model for Anomaly REcognition (AWARE), a world model framework that is capable of anomaly detection using only external camera monitoring via system dynamics estimation and state prediction. Evaluated across 240 real-world robotic crane trajectories, including payload collisions and motor degradations, AWARE demonstrates a 26% improvement in anomaly classification over our strongest baseline under noisy, camera-only state estimation. Furthermore, the inferred dynamics parameter can be decoded to identify the source of faults, localizing the degrading motor with 88% accuracy. Project page and video available at sites.google.com/view/aware-submission-745.

Abstract:
Enhancing the generalization of robotic learning in diverse unseen environments remains a fundamental challenge. Existing approaches often rely on large-scale pretraining, which is labor-intensive and time-consuming, or semantic data augmentation methods that assume flawless upstream object detection in real-world scenarios. In this work, we propose RoboAug, a novel generative data augmentation framework that reduces reliance on large-scale pretraining and perfect visual recognition by requiring only a single image with bounding box annotations for dataset construction. Leveraging this minimal supervision, RoboAug employs pretrained generative models for precise semantic augmentation and introduces a plug-and-play region-contrastive loss to guide attention toward task-relevant regions, thereby enhancing generalization and task success rates. We conduct extensive real-world experiments on three robotic platforms—UR-5e, AgileX, and Tien Kung 2.0—covering over 35k rollouts. Experimental results show that RoboAug consistently outperforms state-of-the-art augmentation baselines. In unseen environments with varying backgrounds, distractors, and lighting conditions, RoboAug improves success rates over the no-augmentation baseline from 0.09 to 0.47 on UR-5e, 0.16 to 0.60 on AgileX, and 0.19 to 0.67 on Tien Kung 2.0. These results demonstrate the superior generalization and effectiveness of RoboAug in real-world manipulation tasks.

Abstract:
Learning contact-rich, whole-body manipulation for soft continuum robots is held back by the lack of simulation infrastructure that rigid robots take for granted. Existing soft robot simulators are physically grounded but lack the contact handling, actuation support, or learning integration needed for contact-rich manipulation; rigid-body approximations offer these capabilities but sacrifice physical grounding. We bridge this gap for tendon-driven continuum robots (TDCRs) by deriving a continuum-mechanics-informed discretization that places the soft robot natively inside MuJoCo, unifying tendon forces, body contact, and dynamics in a single physics pipeline. We validate the simulator against a Cosserat rod reference (static and dynamic) and real TDCR hardware. We then train state-based imitation learning policies via teleoperation in simulation and deploy them zero-shot to a physical 3-segment TDCR on a 7-DoF Franka arm across two contact-rich manipulation tasks. To our knowledge, this is the first demonstration of sim-to-real transfer for contact-rich manipulation with continuum robots.

Abstract:
While low-cost passive leader arms enable scalable data collection for imitation learning, their lack of restoring forces causes unconstrained redundant degrees of freedom, resulting in highly inconsistent joint-space trajectories. To address this without relying on expensive active actuators, we propose a novel, low-cost passive joint mechanism featuring an adjustable mechanical clutch and a 3D-printed spiral spring. This mechanism introduces ``Hardware-as-Regularization'' by providing adjustable mechanical compliance that acts as a physical prior. By naturally guiding redundant joints back to a user-defined zero-point, our system significantly improves kinematic consistency without increasing the operator's cognitive load. Experimental results demonstrate that our purely mechanical approach drastically reduces joint-angle variance compared to a naive passive arm, achieving trajectory consistency on par with a more complex, active virtual spring system. Furthermore, in downstream policy learning, our mechanism stabilizes demonstrations by eliminating manual drift compensation, thereby matching the task success rates of the active setup while preserving extreme cost-effectiveness.

Abstract:
We present a learning-based framework that enables a full-scale bipedal humanoid to sprint at over 7 m/s with motion-prior-guided gait and perform aggressive cornering at high speed with a pronounced trunk lean of up to 25°. Achieving this combination is challenging: above approximately 4 m/s, periodic foot impacts induce severe, high-frequency yaw-rate oscillations intrinsic to high-speed gaits. This high-frequency temporal structure corrupts the history of proprioceptive feedback that flat-MLP policies rely on, while rendering conventional yaw-rate tracking rewards counterproductive, as they fight against the robot's natural oscillatory dynamics. Moreover, a single style prior shared across skills cannot tell which skill is commanded and inherently compromises the low-speed skill fidelity. To overcome these limitations, we adopt a causal multi-head attention (MHA) module to explicitly model the high-frequency temporal feedback. In place of the oscillatory yaw rate, we design a heading-direction reward that supervises only the low-frequency velocity direction, allowing agile turning at high speed without suppressing natural gait oscillations. We further introduce skill-conditioned multi-adversarial-motion-prior (MAMP) discriminators that condition the style reward on the commanded skill. Ablation experiments confirm that each mechanism is essential: replacing structured temporal encoding with a flat-history MLP caps the top speed; reverting to yaw-rate tracking prevents high-speed turning; and a single shared discriminator makes in-place turning collapse into a quasi-static stance. Our framework bridges the gap between raw sprinting speed and reference-motion fidelity, demonstrating stable high-speed sprinting and aggressive cornering in both simulation and real-robot deployment (videos: https://youtu.be/YWf62XkiApc).

Abstract:
Multi-Modal Motion Planning (M^3P) requires joint reasoning over continuous motions and discrete mode transitions, making it difficult to solve efficiently. For instance, a bipedal robot may walk to a target location and then use its arms to grasp an object. This scenario captures both mode transitions and continuous dynamics, yielding feasible paths that neither purely discrete nor continuous planners can handle. While Mixed-Integer Programming (MIP) offers a principled framework, constructing tractable formulations for non-convex problems is typically manual and domain-specific, especially in the \emphapproximate, discretization-based MIP regime needed for non-convex robotic tasks. We propose M^3P-R1, a reinforcement learning method that fine-tunes large language models (LLMs) to decompose M^3P tasks into MIP variables, constraints, and objectives. Instead of directly outputting answers—often prone to hallucination—the model generates executable Python code using MIP optimization libraries and constraint interfaces. This enables solver-backed execution for robust and verifiable solutions. Trained with an outcome-driven reward against the solver, M^3P-R1 learns to compose modality-level discretization primitives and synthesize cross-modal coupling constraints, producing executable MIP programs for complex M^3P tasks. We further introduce LLMM^3P Bench, a benchmark of 1,316 problems spanning 5 single-modal and 8 multi-modal compositions, and use it to evaluate the proposed method. Experiments show that M^3P-R1 achieves 91.7% single-modal and 80.9% multi-modal task success rate, improving over the strongest prompt-only LLM baseline (GPT-5.2) by 26.4 and 40.6 percentage points, respectively. M^3P-R1 further transfers to real-world hardware at 87.5% success rate. These results demonstrate the effectiveness of solver-grounded LLM reasoning for reliable motion planning.

Abstract:
A navigable agent must understand both high-level semantic instructions and precise spatial structure. Multimodal Large Language Models (MLLMs), with their strong generalization, are a promising foundation for such agents. We propose a decoupled design that separates low-level spatial state estimation from high-level semantic planning. Unlike prior methods that rely on predefined, oversimplified textual maps, we introduce an explicit metric world representation that maintains rich and consistent information, allowing the MLLM to query and reason over it for decision-making. We further introduce counterfactual reasoning to elicit the MLLM's capacity, while the world representation ensures the physical validity of the produced actions. We conduct comprehensive experiments in both simulated and real-world environments. Our method establishes a new zero-shot state-of-the-art, achieving 48.8% Success Rate (SR) on R2R-CE and 46.2% on RxR-CE. To validate the versatility of our representation, we further demonstrate sim-to-real transfer across diverse embodiments, confirming that the decoupled framework deploys robustly on real robots.

Abstract:
Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle planner suited to a single nominal robot. This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations. Post-training the policy with reinforcement learning (RL) offers a principled remedy.However, previous RL for diffusion approaches lead to only marginal improvements. This is because the intractable likelihood of diffusion policies renders policy gradients unstable in addition to inefficient policy exploration. To address these challenges, we propose a data-efficient diffusion RL post-training framework - GQRM (Group Q-score Reweighted Matching). Our framework introduces two complementary designs: (i) a self-bootstrapped exploration strategy with behavior perturbation that preserves the pretrained policy prior, and (ii) a group Q-score normalization mechanism that computes per-trajectory values on each state for efficient reweighted score matching. By conducting distributed online RL training across heterogeneous embodiments, the resulting fine-tuned policy, X-NavDP, achieves state-of-the-art cross-embodiment visual navigation performance, improving the overall success rate from 61.20% to 84.28% in simulation and 10% to 65% in real-world hard cases.

Abstract:
Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, but many existing methods still rely on direct mappings from language and visual observations to dense actions. This formulation can weaken the semantic reasoning capability inherited from pre-trained Vision-Language Models (VLMs), which are mainly optimized for visual-linguistic understanding rather than low-level control, and becomes fragile under spatial variations, including changes in object positions, scene layouts, robot embodiments, and camera viewpoints. To address these limitations, we propose H-VLA, a hierarchical VLA framework that decouples high-level key-action reasoning from low-level motion generation. H-VLA combines a Key-Action Model for predicting structured key-actions as semantic sub-targets, a Motion Planning Model for generating dense future actions conditioned on the predicted key-action, and a Unified Camera-Centric Action Space for consistent representation across datasets, embodiments, and viewpoints. We further adopt a two-stage training strategy that emphasizes key-action reasoning during pre-training and dense motion generation during fine-tuning. Experiments show that H-VLA achieves strong performance on SimplerEnv, reaching 91% on Google Robot visual matching, 84% on Google Robot variant aggregation, and 81% on WidowX visual matching. On Agilex real-robot tasks, H-VLA improves over the strongest baseline by 10, 52, and 16 percentage points under in-distribution, out-of-distribution position, and out-of-distribution scene/object settings, respectively.

Abstract:
Cross-embodiment data have become central to training robotic foundation models. To leverage such heterogeneous data, we focus on flow-based object manipulation, where robot flows (robot velocity fields) serve as embodiment-agnostic motion representations. Previous studies do not formulate robot flows as dense velocity fields, but as displacements of sparse keypoints, while such velocity fields better match the continuous-time nature of motions. We propose Flow as Flow, a framework that models robot flows as probability flows based on a flow matching formulation. By naturally modeling such velocity fields within this formulation, our method achieves efficient and high-quality robot flow generation. Across standard benchmarks, our method outperforms representative baseline methods on standard metrics, while achieving approximately 33× faster generation. Furthermore, through real-world experiments evaluating 9 methods with 260 trials per method across 13 manipulation tasks, we show that our method achieves a higher average success rate than the baseline methods.

Abstract:
Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than grounding language. We introduce a diagnostic framework that localizes this failure to individual instruction factors, e.g., reusable semantic components such as color, verb, object, size, and spatial attribute. Our framework formalizes instruction factor bias, the tendency of fine-tuned policies to over-rely on dominant factors as shortcuts, and quantifies it through two metrics: Factor Dominance Rate (FDR), capturing pairwise bias between factors, and Factor Dominance Hierarchy (FDH), aggregating these into a global ranking. Evaluation on six foundation policies reveals broadly consistent ordering, i.e., color \geq object \geq spatial \geq verb \geq size, with color dominant and verb and size most under-grounded. We further show the diagnosis is actionable: a bias-aware data collection strategy that reallocates a fixed budget toward under-grounded factors outperforms baselines in simulation and on a real robot using half the demonstrations, thereby enabling more sample-efficient and generalizable policy learning.

Abstract:
Humanoid soccer requires a robot to seamlessly combine agile locomotion, ball manipulation, and impact-rich contact dynamics across a continuous, dynamic behavioral spectrum. Yet existing approaches either cover only a limited set of skills, or forego motion priors entirely and produce unnatural behaviors, leaving the combination of a unified skill set and human-like motion quality unaddressed. We present SoccerBot, a hierarchical reinforcement learning framework that resolves both limitations through a structured latent motion prior distilled from large-scale human soccer motion capture data. By constraining policy learning within this dynamically plausible motion manifold, a single task-conditioned policy naturally learns to perform moving, dribbling, trapping, and kicking in a unified formulation, while inherently producing human-like motion and fluid transitions during commanded skill switching. Augmented by robustness-oriented simulation design, SoccerBot transfers zero-shot to the Unitree G1 humanoid robot, demonstrating agile, natural, and robust execution of the full soccer skill repertoire under real-world conditions.

Abstract:
The most widely-adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments. In contrast, unstructured human videos provide a scalable alternative. They contain diverse manipulation demonstrations across objects, scenes, and strategies, but are not directly connected to robot action. We propose LUCID, a two-stage framework that learns task intent from unstructured human videos drawn from internet-scale datasets and learns robot control in massively-parallel simulation. The intent model predicts short-horizon intent (what should happen next in the scene) from the current observation in closed loop. An embodiment-specific tracking policy converts this intent into robot actions. The intent interface is shared across controllers, so the same intent model can be applied to different embodiments, from our primary dexterous hand to a parallel-jaw gripper. We evaluate LUCID on five real-world manipulation tasks: stirring, wiping, and binning supervised by only internet video, with zero-shot transfer to novel scenes and object instances; and push-T and cable routing supervised by 1 hr each of self-collected smartphone video.

Abstract:
Robot manipulation has recently achieved significant progress through large-scale learning from teleoperated demonstrations, but collecting such data remains expensive, labor-intensive, and difficult to scale across diverse environments and long-horizon tasks. We present SkillWeaver, a scalable framework for autonomous manipulation data generation that combines diverse scene generation, reusable RL-based interaction skills, and VLM-guided hierarchical tree search. Given a task and a generated scene, SkillWeaver adaptively decomposes the task into subgoals, executes contact-rich interactions with closed-loop skills, and collects long-horizon demonstrations with failure recovery and re-planning. In contrast to alternative pipelines that rely on predefined sequencing of hand-designed controllers, grasp predictors, and motion planners, our framework adaptively reasons over object geometries and environment configurations, enabling more flexible and robust behavior generation across diverse tasks and scenes. We evaluate \modelSkillWeaver on LIBERO-PRO and a real Franka robot setup, showing that SkillWeaver effectively scales high-quality manipulation data and that large-scale simulation training enables stronger generalization across novel scenarios.

Abstract:
While traditional modeling effectively supports a robot's intrinsic mass, providing active anti-gravity assistance for the combined mass of a coupled human-exoskeleton system remains notoriously difficult. Challenges such as non-rigid interfacing, unmodeled biological torques, and the static indeterminacy of closed kinematic loops severely limit the efficacy of analytical modeling. In this paper, we present a novel, efficient data collection paradigm that bypasses these complexities to directly learn effective feedforward assistance torques. By implementing a velocity-limited admittance controller, we establish a kinematic constraint that actively suppresses inertial dynamics, Coriolis effects, and voluntary human reflexes. This generates a clean dataset isolating configuration-dependent assistive requirements based on slow passive yielding. We use this data to produce an empirical model which successfully generalizes gravity-compensation scaling to novel subjects. Our model drives absolute torque prediction errors below 10% and significantly outperforms state-of-the-art analytical baselines derived from sparse wearable sensors. Finally, we design a few-shot calibration protocol that approximates user-specific assistive profiles consistent with soft-tissue interference, allowing our personalized model to scale to high-acceleration dynamic tasks, providing a robust, empirical foundation for high-force assistive applications.

Abstract:
Multi-fingered dexterous manipulation remains a frontier for real-world reinforcement learning (RL) due to the high-dimensional action space and the prohibitive cost of hardware failures. While human-in-the-loop (HIL) RL allows operators to intervene before failures occur, current pipelines often treat these interventions as reactive corrections, discarding the rich safety signal inherent in the operator's decision to take control. In this paper, we ask: How can we learn from what a human would avoid? We present WHIRL, a safety-aware RL framework that transforms binary human interventions into forward-predictive signals for proactive risk avoidance. Our approach centers on an intervention-aware latent world model with four prediction heads: dynamics, reward, termination, and a novel per-state intervention-probability head that learns to predict the likelihood of a human takeover at future states. This head provides an actor-side risk-shaping term that discourages the policy from entering "intervention-prone" regions, modeling the operator's internal safety threshold. We evaluate our framework on a 16-DoF LEAP Hand across tasks spanning convex and irregular object grasping, prismatic manipulation, and long-horizon multi-stage tasks. Our results show that predictive risk-shaping enables the system to achieve a 96.7% success rate on complex grasping tasks while reducing the operator intervention burden by up to 84% in step-weighted terms. By closing the loop between human intuition and predictive world modeling, this work provides a scalable safety-aware blueprint for training complex dexterous agents in the real world without the prohibitive costs of human fatigue or hardware failure.