TRO2026

Abstract:
Robot simulators are indispensable tools across many fields, and recent research has significantly improved their functionality by incorporating additional gradient information. However, existing differentiable robot simulators suffer from nondifferentiable singularities, when robots undergo substantial shape changes. To address this, we present the shape-differentiable robot simulator (SDRS), designed to be differentiable under significant robot shape changes. The core innovation of SDRS lies in its representation of robot shapes using a set of convex polyhedrons. This approach allows us to generalize smooth, penalty-based contact mechanics for interactions between any pair of convex polyhedrons. Using the separating hyperplane theorem, SDRS introduces a separating plane for each pair of contacting convex polyhedrons. This separating plane functions as a zero-mass auxiliary entity, with its state determined by the principle of least action. This setup ensures global differentiability, even as robot shapes undergo significant geometric and topological changes. To demonstrate the practical value of SDRS, we provide examples of robot co-design scenarios, where both robot shapes and control movements are optimized simultaneously.

Affiliations: Department of Electrical and Computer Engineering, University of California, Riverside, CA, USA; Department of Mechanical and Aerospace Engineering, University of California San Diego, San Diego, CA, USA; Department of Planning and Control, Zoox Inc., Foster City, CA, USA; Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA; Department of Mechanical Engineering, Yale University, New Haven, CT, USA; Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA

Abstract:
Koopman operator theory offers a rigorous treatment of dynamics, emerging as a robust alternative for learning-based control in robotics. By representing nonlinear dynamics as a linear, higher dimensional operator, it provides a fresh lens for modeling complex systems. Its ability to support incremental updates and low computational cost makes it particularly appealing for real-time applications and online learning. This review delves deeply into the foundations, systematically bridging theoretical principles to practical robotic applications. In this article, we explain mathematical underpinnings, approximation approaches for inputs, data collection strategies, and lifting function design. We explore how Koopman models unify tasks, such as model-based control, state estimation, and motion planning. The review surveys cutting-edge research across domains ranging from aerial and legged platforms to manipulators, soft robots, and multiagent networks. We also present advanced theoretical topics and reflect on open challenges and future research directions. To support adoption, we provide a hands-on tutorial with code at https://github.com/sunnyshi0310/KoopmanRobo/tree/main.

Abstract:
Kinodynamic motion planning for a multirobot system with different dynamics and actuation limits is a challenging problem. The difficulty increases with the presence of aerodynamic interaction forces that occur when aerial robots fly in close proximity. Due to these complexities, existing planners either rely on simplified assumptions (like ignoring robot dynamics and interaction forces) or produce highly suboptimal solutions. This article presents a kinodynamic motion planner for a heterogeneous team of robots that respects robot dynamics, scales well to 16 robots, and directly reasons about interaction forces between aerial robots operating in close proximity. Our method, db-ECBS, generalizes the multiagent path-finding method Enhanced Conflict-Based Search (ECBS) to the continuous domain by using the single-robot kinodynamic motion planner discontinuity-bounded A. The planner db-ECBS operates on three levels. Initially, individual robot trajectories are computed using a graph search that allows bounded discontinuities between precomputed motion primitives. The second level identifies interrobot collisions or interaction force violations and resolves them by imposing constraints on the first level. The third and final level uses the resulting solution with discontinuities as an initial guess for a joint-space trajectory optimization. The procedure is repeated with a reduced discontinuity bound, resulting in an anytime, probabilistically complete, and asymptotically bounded suboptimal planner. We provide a benchmark of 65 problems with six different dynamics. We demonstrate that db-ECBS produces trajectories that are less than half the cost of existing planners. We show that the interaction-awareness is particularly important for very dense scenarios.

Abstract:
Event cameras in motion tend to detect object boundaries or texture edges, which produce lines of brightness changes, especially in man-made environments. While lines can constitute a robust intermediate representation that is consistently observed, the sparse nature of lines may lead to drastic deterioration with minor estimation errors. Only a few previous works, often accompanied by additional sensors, utilize lines to compensate for the severe domain discrepancies of event sensors along with unpredictable noise characteristics. In this article, we propose a method that can stably extract tracks of varying appearances of lines using a clever algorithmic process that observes multiple representations from various time slices of events, compensating for potential adversaries within the event data. We then propose geometric cost functions that can refine the 3-D line maps and camera poses, eliminating projective distortions and depth ambiguities. The 3-D line maps are highly compact and can be equipped with our proposed cost function, which can be adapted for any observations that can detect and extract line structures or projections of them, including 3-D point cloud maps or image observations. We demonstrate that our formulation is powerful enough to exhibit a significant performance boost in event-based mapping and pose refinement across diverse datasets, and can be flexibly applied to multimodal scenarios. Our results confirm that the proposed line-based formulation is a robust and effective approach for the practical deployment of event-based perceptual modules.

Abstract:
With continuously increasing requirements for physical human–robot interaction (pHRI), cable-driven parallel robots (CDPRs) have emerged as outstanding systems for its implementation due to the sufficient motion workspace and inherent cable flexibility. In particular, their modular structure facilitates straightforward reconfiguration. Inspired by this, this article aims to enhance the dynamic characteristics of CDPRs during pHRI through online reconfiguration, so as to achieve the interaction performance improvement based on human intent. A novel metric, the mixed interaction wrench margin, is first proposed to determine the optimal reconfiguration. This metric is devised by integrating the interaction force characteristics with CDPR inherent workspace properties, while explicitly considering the leading role of human intent. Subsequently, an adaptive-interaction-based reconfiguration strategy is established that the configuration can be arbitrarily changed by cable anchors, enabling compliant and adaptive pHRI. Informed by the actual interaction frequency, the strategy implements an asynchronous adjustment with different periods for configuration change and platform movement to achieve online optimization for reconfiguration. Finally, simulations and experiments conducted on different CDPR configurations with multiple pHRI tasks indicate that this strategy provides humans with more freedom, allowing them to exert more casual interaction forces, receive a quicker interactive response, and operate in a larger workspace.

Abstract:
Imitation-learning-based visuomotor policies excel at manipulation tasks but often produce suboptimal action trajectories compared to model-based methods. Directly mapping camera data to actions via neural networks can result in jerky motions and difficulties in meeting critical constraints, compromising safety and robustness in real-world deployment. For tasks that require high robustness or strict adherence to constraints, ensuring trajectory quality is crucial. However, the lack of interpretability in neural networks makes it challenging to generate constraint-compliant actions in a controlled manner. This article introduces differentiable policy trajectory optimization with generalizability (DiffOG), a learning-based trajectory optimization framework designed to enhance visuomotor policies. By leveraging the proposed differentiable formulation of trajectory optimization with transformer, DiffOG seamlessly integrates policies with a generalizable optimization layer. DiffOG refines action trajectories to be smoother and more constraint-compliant while maintaining alignment with the original demonstration distribution, thus avoiding degradation in policy performance. We evaluated DiffOG across 11 simulated tasks and two real-world tasks. The results demonstrate that DiffOG significantly enhances the trajectory quality of visuomotor policies while having minimal impact on policy performance, outperforming trajectory processing baselines such as greedy constraint clipping and penalty-based trajectory optimization. Furthermore, DiffOG achieves superior performance compared to existing constrained visuomotor policy.

Abstract:
This study informs the design of future multiagent pathfinding and multirobot motion planning (MRMP) algorithms by guiding choices based on constraint classification for constraint-based search algorithms. We categorize constraints as conservative or aggressive and provide insights into their search behavior, focusing specifically on vanilla conflict-based search and conflict-based search with priorities. Under a hybrid grid-roadmap representation with varying resolution, we observe that aggressive (priority constraint) formulations tend to solve more instances as agent count or resolution increases, whereas conservative (motion constraint) formulations yield stronger solution quality when both succeed. Findings are synthesized in a decision flowchart, aiding users in selecting suitable constraints. Recommendations extend to MRMP, emphasizing the importance of considering topological features alongside problem, solution, and representation features. A comprehensive exploration of the study, including raw data and map performance, is available in our public GitHub Repository.

Abstract:
Efficient rigid-body dynamics algorithms are instrumental in enabling high-frequency dynamics evaluation for resource-intensive applications (e.g., model-predictive control, large-scale simulation, and reinforcement learning), potentially on resource-constrained hardware. Existing recursive algorithms with low computational complexity are mostly restricted to kinematic trees with external contact constraints or are sensitive to singular cases (e.g., linearly dependent constraints and kinematic singularities), severely impacting their practical usage in existing simulators. This article introduces two original low-complexity recursive algorithms: the loop-constrained articulated body algorithm and proximal BBO (Brandl, Bae, and others), both based on a proximal dynamics formulation for forward simulation of closed-loop mechanisms. These algorithms are derived from first principles using nonserial dynamic programming, exhibit linear complexity in practical scenarios, and are numerically robust in the face of singular cases. They extend the existing constrained articulated body algorithm to handle internal loops and the pioneering BBO algorithm from the 1980s to singular cases. Both algorithms have been implemented by leveraging the open-source Pinocchio library, benchmarked in detail, and demonstrate state-of-the-art performance for various robot topologies, including over 6× speed-ups compared to existing nonrecursive algorithms for high-degree-of-freedom systems with internal loops, such as recent humanoid robots.

Abstract:
There exist numerous ways to achieve multitasking control in kinematically redundant robots to accomplish several goals simultaneously. In all approaches, regardless of the specific type of controller, one has to make a choice about the closed-loop inertia and consequently the dynamic task couplings. Here, we introduce a new control strategy that combines two fundamentally different properties that have not yet been brought together. First, we fully and dynamically decouple all individual subtasks, which cannot be achieved with classical passivity-based or hierarchical approaches. Second, we provide high robustness in practice, which is structurally not possible with any inverse dynamics approaches enforcing a decoupled but constant closed-loop inertia. Besides formal proofs of stability and passivity, we compare our approach with the other categories in various simulations and experiments. Since the proposed controller is grounded in the fundamental property of full natural task-space decoupling, this underlying strategy and its benefits can also be transferred to other design methods such as quadratic programming, model-predictive control, or learning-based approaches.

Abstract:
In safety-critical robot planning or control, manually specifying safety constraints or learning them from demonstrations can be challenging. In this article, we propose a certifiable alignment method for a robot to learn a safety constraint in its model predictive control policy with human online directional feedback. To the best of authors’ knowledge, it is the first method to learn safety constraints from human feedback. The proposed method is based on an empirical observation: human directional feedback, when available, tends to guide the robot toward safer regions. The method only requires the direction of human feedback to update the learning hypothesis space. It is certifiable, providing an upper bound on the total number of human feedback in the case of successful learning, or declaring the hypothesis misspecification, i.e., the true implicit safety constraint cannot be found within the specified hypothesis space. We evaluated the proposed method using numerical examples and user studies in two simulation games. In addition, we implemented and tested the proposed method on a real-world Franka robot arm performing mobile water-pouring tasks. The results demonstrate the efficacy and efficiency of our method, showing that it enables a robot to successfully learn safety constraints with a small handful (tens) of human directional corrections.

Abstract:
Soft robotic manipulators are generally slow despite their great adaptability, resilience, and compliance. This limitation also extends to current soft robotic micromanipulators. Here, we introduce FilMBot, a 3-DOF film-based, electromagnetically actuated, soft kinematic robotic micromanipulator achieving speeds up to 2117°/s and 2456°/s in α and β angular motions, with corresponding linear velocities of 1.61 m/s and 1.92 m/s using a 4-cm needle end-effector, 0.54 m/s along the Z-axis, and 1.57 m/s during Z-axis morph switching. The robot can reach ∼1.50 m/s in path-following tasks, with an operational bandwidth below ∼30 Hz, and remains responsive at 50 Hz. It demonstrates high precision (∼6.3 μm, or ∼0.05% of its workspace) in path-following tasks, with precision remaining largely stable across frequencies. The novel combination of the low-stiffness soft kinematic film structure and strong electromagnetic actuation in FilMBot opens new avenues for soft robotics. Furthermore, its simple construction and inexpensive, readily accessible components could broaden the application of micromanipulators beyond current academic and professional users.

Abstract:
In this article, we present stratified topological autonomy for long-range coordination (STALC), a hierarchical planning approach for multirobot coordination in real-world environments with significant inter-robot spatial and temporal dependencies. At its core, STALC consists of a multirobot graph-based planner which combines a topological graph with a novel, computationally efficient mixed-integer programming formulation to generate highly coupled multirobot plans in seconds. To enable autonomous planning across different spatial and temporal scales, we construct our graphs so that they capture connectivity between free-space regions and other problem-specific features, such as traversability or risk. We then use receding-horizon planners to achieve local collision avoidance and formation control. To evaluate our approach, we consider a multirobot reconnaissance scenario where robots must autonomously coordinate to navigate through an environment while minimizing the risk of detection by observers. Through simulation-based experiments, we show that our approach is able to scale to address complex multirobot planning scenarios. Through hardware experiments, we demonstrate our ability to generate graphs from real-world data and successfully plan across the entire hierarchy to achieve shared objectives.

Abstract:
Inspired by soft creatures and structures in nature, continuum robots exhibit remarkable flexibility, safe interaction, and ease of miniaturization, showcasing vast application potential. However, their flexible structure renders analytical methods inadequate for precise modeling and control, while existing data-driven approaches suffer from low data efficiency and unproven theoretical control performance. This article aims to achieve data-efficient modeling and reliable control of continuum robots through innovative algorithms, exploring the performance of the new method from both theoretical and experimental perspectives. Specifically, we utilize neural ordinary differential equations (NODE) to achieve data-efficient modeling of continuum robots and investigate the performance of the modeling method. Then, we propose a novel predefined-time-synchronized stable zeroing neurodynamics (PTSS-ZND) model. By combining the NODE method and the PTSS-ZND method, we propose a reliable data-driven control system. Through rigorous theoretical analysis, we prove the stability and predefined-time convergence of the data-driven control system. Finally, through simulations and physical experiments, we validate the feasibility and convergence of the novel method and its advantages over existing data-driven methods. Experiments on one-segment and three-segment continuum robots indicate that the proposed method achieves a root mean square position error (e.g., 2.5 mm for the three-segment robot) of less than 1% of the robot length using fewer than 100 data samples. Our method also demonstrates robust performance under various external and internal disturbances. In addition, it can potentially be extended for end-effector pose control.

Abstract:
Underwater acoustic communication, characterized by limited bandwidth, high latency, and low reliability, poses significant challenges for data exchange in bathymetric collaborative simultaneous localization and mapping (CSLAM). In this article, we introduce a novel vector quantization (VQ) method called ID(O) for mapping data compression in bathymetric CSLAM. ID(O) encodes the map into an index map (\mathbb I), a central depth map (\mathbb D), and an orientation map (\mathbb O). To accommodate strict communication constraints, orientations can be partially or fully excluded from transmission, and we propose a method to estimate these orientations during map restoration. Moreover, we integrate ID(O) within a feature-based bathymetric CSLAM framework named TTT CSLAM. Extensive experiments on two large-scale sea trial datasets demonstrate that ID(O) achieves about 40% higher restoration accuracy than the baseline method using principal component analysis. TTT CSLAM with ID(O) can match that with lossless compression regarding mapping accuracy and efficiency, and it is robust against 40% packet loss and large dead reckoning drift errors across diverse environments. To the best of the authors’ knowledge, ID(O) is the first VQ method for bathymetric data compression, and TTT CSLAM with ID(O) is the first bathymetric CSLAM tested within an underwater communication network employed by acoustic modems.

Abstract:
This article presents a correspondence-free, function-based sim-to-real learning method for controlling deformable freeform surfaces. Unlike traditional sim-to-real transfer methods that strongly rely on marker points with full correspondences, our approach simultaneously learns a deformation function space and a confidence map—both parameterized by a neural network (NN)—to map simulated shapes to their real-world counterparts. As a result, the sim-to-real learning can be conducted by input from either a 3-D scanner as point clouds (without correspondences) or a motion capture system as marker points (tolerating missed markers). The resultant sim-to-real transfer can be seamlessly integrated into a NN-based computational pipeline for inverse kinematics and shape control. We demonstrate the versatility and adaptability of our method on two vision devices and across four pneumatically actuated soft robots: a deformable membrane, a robotic mannequin, and two soft manipulators.

Abstract:
Conforming to safety standards often limits collaborative robots’ performance and size, restricting their applications despite their capabilities. Planning their motions in human environments involves a tradeoff between optimal trajectory planning and quick adaptation to dynamic, unstructured spaces. Traditional trajectory planning methods either use simplified robot models and sacrifice robot’s abilities for computational efficiency, or exploit robots’ abilities fully but have high computational complexity and rely on substantial precomputation. This article introduces an approach for trajectory planning that exploits robot’s full motion abilities while planning on-the-fly. In each step of the trajectory execution, it evaluates robot’s movement ability using polytope algebra and calculates a time-optimal Trapezoidal Acceleration Profile (TAP) on the remaining trajectory. The method is shown to be near time-optimal (around 5% slower trajectories) by benchmarking it against the state-of-the-art time-optimal method TOPP-RA. The method allows reaching higher velocities (able to plan up to 100% of the robot’s kinematic limits) while at the same time lowering the tracking error (under 4 mm) than traditional Cartesian Space planning methods. A mock-up experiment demonstrates its efficiency in collaborative waste sorting using a Franka Emika Panda robot.

Abstract:
Due to their stability and robustness properties, stable dynamical systems (SDSs) have received considerable attention as a means of representing motions in learning from demonstration tasks. Designing vector fields that fit complex trajectories while ensuring stability still remains a key challenge; although recent deep-learning-based methods have shown substantial progress in this direction, their tendency to overfit to demonstration trajectories often leads to undesirable behaviors, particularly as tasks deviate from demonstrations. At a fundamental level, the only reliable way to address this lack of generalization is to provide supervision in out-of-demonstration regions. Focusing on two types of general behaviors, mimicking and contracting, we propose a behavior-controllable stable dynamics model (BCSDM), a one-parameter family of SDS that allows users to adjust the system’s overall behavior depending on user intent. We show how to extend the BCSDM to accommodate demonstrations of multiple tasks, and also propose a deep operator vector field for memory-efficient encoding of multiple dynamical systems. Extensive experiments on tasks that involve mimicking or contracting behaviors demonstrate the advantages of BCSDMs over existing state-of-the-art SDS learning methods.

Abstract:
The wheeled three-link snake robot is a well-known example of an underactuated system modeled using nonholonomic constraints, preventing lateral slippage (skid) of the wheels. A kinematically controlled configuration assumes that both joint angles are directly prescribed as phase-shifted periodic input. In another configuration of the robot, only one joint is periodically actuated while the second joint is passively governed by a visco-elastic torsion spring. In our work, we constructed the two configurations of the wheeled robot and conducted motion experiments under different actuation inputs. Analysis of the motion tracking measurements reveals a significant amount of wheels’ skid, in contrast to the assumptions used in standard nonholonomic models. Therefore, we propose modified dynamic models, which include wheels’ skid and viscous friction forces, as well as rolling resistance. After parameter fitting, these dynamic models reach good agreement with the motion measurements, including effects of input's frequency on the mean speed and net displacement per period. This illustrates the importance of incorporating wheels’ skid and friction into the system's model.

Abstract:
Engineered assembloids fabricated from tissue spheroids hold immense promise for developmental biology, disease modeling, and regenerative medicine. However, fabricating heterogeneous assembloids with precise spatial patterning remains a critical bottleneck, often reliant on manual pipetting that lacks scalability and reproducibility. While magnetic microrobotics, which transforms spheroids into controllable robots, offers a noninvasive alternative, it faces a fundamental challenge: the global input of magnetic actuation. A single command moves all robots simultaneously, leading to coupled motion and frequent assembly failures. Here, we present a collaborative control framework that overcomes this limitation by leveraging local constraints and a novel motion decoupling strategy. We reformulate the high-dimensional, coupled multirobot planning problem into a low-dimensional aggregate space, effectively transforming the assembly task into a dynamic sequential decision problem. This framework, coupled with a graph-based dynamic path optimization algorithm, enables deterministic, collision-free assembly. Experimental validation demonstrates a 78.57% higher success rate and a 33.20% higher assembly efficiency compared to conventional strategies. This work establishes a foundational engineering principle for assembloid fabrication, transitioning the process from a qualitative experiments to a controllable and programmable engineering discipline, thereby unlocking the potential for deterministic construction of complex biological structures.

Affiliations: Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA; Department of Aeronautics and Astronautics, Stanford University, Stanford, CA, USA; Google DeepMind, San Francisco, CA, USA; Technical University of Denmark, Kongens Lyngby, Denmark; School of Management, Technical University of Munich, Munich, Germany; Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland; Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA, USA

Abstract:
Autonomous mobility-on-demand (AMoD) systems, powered by advances in robotics, control, and machine learning (ML), offer a promising paradigm for future urban transportation. AMoD offers fast and personalized travel services by leveraging centralized control of autonomous vehicle fleets to optimize operations and enhance service performance. However, the rapid growth of this field has outpaced the development of standardized practices for evaluating and reporting results, leading to significant challenges in reproducibility. As AMoD control algorithms become increasingly complex and data-driven, a lack of transparency in modeling assumptions, experimental setups, and algorithmic implementation hinders scientific progress and undermines confidence in the results. This article presents a systematic study of reproducibility in AMoD research. We identify key components across the research pipeline, spanning system modeling, control problems, simulation design, algorithm specification, and evaluation, and analyze common sources of irreproducibility. We survey prevalent practices in the literature, highlight gaps, and propose a structured framework to assess and improve reproducibility. While focused on AMoD, the principles and practices we advocate generalize to a broader class of cyber-physical systems that rely on networked autonomy and data-driven control. This work aims to lay the foundation for a more transparent and reproducible research culture in the design and deployment of intelligent mobility systems.

Abstract:
This study presents and experimentally validates an adaptive control method for human-exoskeleton interaction through online adaptation of desired joint trajectories. Leveraging gait phase and human-exoskeleton interaction torque estimators, our approach enables seamless assistance adaptation to varying walking patterns and speeds. Specifically, a pretrained neural network approximates the exoskeleton’s dynamics, enabling real-time interaction torque estimation from kinematic measurements and commanded motor torques alone. These estimates drive a gradient-descent update of the joint reference trajectories, minimizing a cost function that penalizes both interaction torques and trajectory modification, ensuring bounded convergence and stability without user-specific parameter tuning. We compared our adaptive controller with a fixed-trajectory gait-phase-based controller during overground and treadmill walking at three self-selected speeds ranging from 0.4 to 0.8 m/s. In 16 participants, the adaptive controller significantly reduced the hip and knee interaction torques by 51.2% \pm11.1 and 63.9% \pm29.7, respectively, during overground walking. Muscular effort significantly decreased in Bicep Femoris (21.0% \pm34.5) and Rectus Femoris (28.1% \pm34.6), while remaining unchanged in other muscles. Cadence and gait speed increased by 7.6% \pm5.2 and 10.7% \pm8.3, respectively, indicating that participants could walk faster with less effort due to trajectory adaptation. Postadaptation trajectories more closely resembled those of walking without the exoskeleton, and exoskeleton-torques aligned more closely with human biological torques. Our proposed adaptive controller, which requires only exoskeleton kinematics, also maintained performance during treadmill walking across speeds, demonstrating speed-invariant behavior compared to the nonadaptive controller.

Abstract:
The problem of planning sensing trajectories for a mobile robot to collect observations of a target and predict its future trajectory is known as active target tracking. Enabled by probabilistic motion models, one may solve this problem by exploring the belief space of all trajectory predictions given future sensing actions to maximize information gain. However, for multimodal motion models the notion of information gain is often ill-defined. This article proposes a planning approach designed around maximizing information regarding the target’s homotopy class, or high-level motion. We introduce homotopic information gain, a measure of the expected high-level trajectory information given by a measurement. We show that homotopic information gain is a lower bound for metric or low-level information gain, and is as sparsely distributed in the environment as obstacles are. Planning sensing trajectories to maximize homotopic information results in highly accurate trajectory estimates with fewer measurements than a metric information approach, as supported by our empirical evaluation on real and simulated pedestrian data.

Abstract:
Fast-flying aerial robots promise rapid inspection under limited battery constraints, with direct applications in infrastructure inspection, terrain exploration, and search and rescue. However, high speeds lead to severe motion blur in images and induce significant drift and noise in pose estimates, making dense 3-D reconstruction with neural radiance fields (NeRFs) particularly challenging due to their high sensitivity to such degradations. In this work, we present a unified framework that leverages asynchronous event streams alongside motionblurred frames to reconstruct highfidelity radiance fields from agile drone flights. By embedding event image fusion into NeRF optimization and jointly refining event-based visual inertial odometry priors using both event and frame modalities, our method recovers sharp radiance fields and accurate camera trajectories without groundtruth supervision. We validate our approach on both synthetic data and realworld sequences captured by a fastflying drone. Despite highly dynamic drone flights, where RGB frames are severely degraded by motion blur and pose priors become unreliable, our method reconstructs high-fidelity radiance fields and preserves fine scene details, delivering a performance gain of over 50% on real-world data compared to state-of-the-art methods.

Abstract:
Following human instructions to explore and search for a specified target in an unfamiliar environment is a crucial skill for mobile service robots. Most of the previous works on object goal navigation have typically focused on a single input modality as the target, which may lead to limited consideration of language descriptions containing detailed attributes and spatial relationships. To address this limitation, we propose VLN-Game, a novel zero-shot framework for visual target navigation that can process object names and descriptive language targets effectively. To be more precise, our approach constructs a 3-D object-centric spatial map by integrating pretrained visual-language features with a 3-D reconstruction of the physical environment. Then, the framework identifies the most promising areas to explore in search of potential target candidates. A game-theoretic vision-language model is employed to determine which target best matches the given language description. Experiments conducted on the Habitat-Matterport 3-D dataset demonstrate that the proposed framework achieves state-of-the-art performance in both object goal navigation and language-based navigation tasks. Moreover, we show that VLN-Game can be easily deployed on real-world robots. The success of VLN-Game highlights the promising potential of using game-theoretic methods with compact vision-language models to advance decision-making capabilities in robotic systems.

Abstract:
We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interprets multimodal inputs (sketch, vision, and language) as spatial-semantic primitives to generate executable robot actions requiring no prior maps or models. Novel components include multimodal fusion for understanding and a hierarchical policy for robust action generation. Efficacy is shown via extensive evaluations. Quantitative benchmarks on the large-scale dataset showing high accuracy in interpreting diverse sketch-based commands across various simulated domestic scenes. Real-world validation on two distinct robotic platforms, a statically mounted 7-DoF assistive arm (KUKA LBR iiwa) and a dual-arm mobile manipulator (Realman RMC-AIDAL), performing representative tasks such as targeted wiping and area cleaning, confirming the system’s ability to ground instructions and execute them reliably in physical environments. A comprehensive user study involving diverse demographics (elderly, simulated nonverbal, and low technical literacy) demonstrating significant improvements in usability and task specification efficiency, achieving high task completion rates (85.7%–96.4%) and user satisfaction. AnyUser bridges the gap between advanced robotic capabilities and the need for accessible nonexpert interaction, laying the foundation for practical assistive robots adaptable to real-world human environments.

Abstract:
Tilt-rotor aerial robots are more dynamic and versatile than fixed-rotor platforms, since the thrust vector and body orientation are decoupled. However, the coordination of servos and propellers (the allocation problem) is not trivial, especially accounting for overactuation and actuator dynamics. We incrementally build and present three novel allocation methods for tilt-rotor aerial robots, comparing them to state-of-the-art methods on a real system performing dynamic maneuvers. We extend the state-of-the-art geometric allocation (GA) into a differential allocation, which uses the platform’s redundancy and does not suffer from singularities. We expand it by incorporating actuator dynamics and propeller power dynamics. These allow us to model dynamic propeller acceleration limits, bringing two main advantages: balancing propeller speed without the need of nullspace goals and allowing the platform to selectively turn off propellers during flight, opening the door to new manipulation possibilities. We also use actuator dynamics and limits to normalize the allocation problem, making it easier to tune and allowing it to track 70% faster trajectories than a GA.

Abstract:
Recently, 3-D Gaussian splatting (3DGS)-based RGB-D simultaneous localization and mapping (SLAM) displays remarkable performance of high-fidelity 3-D reconstruction. However, 3DGS suffers from a lack of depth rendering consistency, which leads to suboptimal geometric reconstruction in 3DGS-based SLAM. In addition, 3DGS-based SLAM methods typically lack efficient loop closure, limiting their ability to build globally consistent maps online. In this article, we present 2-D Gaussian splatting (2DGS-SLAM), an RGB-D SLAM system using 2-D Gaussian splatting as map representation. By leveraging the depth-consistent rendering property of the 2-D variant, we propose an accurate camera pose optimization method and achieve geometrically accurate 3-D reconstruction. In addition, we implement efficient loop detection and camera relocalization by leveraging MASt3R, a feedforward 3-D reconstruction model, and achieve efficient map updates by maintaining a local active map. Experiments show that our 2DGS-SLAM approach achieves superior tracking accuracy, higher surface reconstruction quality, and more consistent global map reconstruction compared to existing rendering-based SLAM methods, while maintaining high-fidelity image rendering and improved computational efficiency.

Abstract:
Robotic ultrasound (US) systems represent an emerging frontier in medical imaging. A fundamental component of these systems is the rigid body transformation between the robot flange and the attached US probe, which enables mapping of visual data from image space to the robot’s reference frame. Traditionally, calibrating this transformation has been a tedious process, complicated by equipment demands and operational constraints arising from the probe’s narrow field of view. This work presents a novel calibration strategy based on conic sections geometry, which offers several key simplifications over existing approaches: first, it requires no external equipment beyond a single cone phantom; second, It operates with a small input size and imposes no strict alignment or motion constraints on the US scan plane during calibration; and third, it employs a straightforward pattern analysis pipeline to process images acquired from phantom scans. Experimental validation results show that the proposed method achieves accuracy comparable to existing state-of-the-art approaches while delivering superior precision, thereby demonstrating enhanced calibration reproducibility enabled by its streamlined workflow. These advantages make this method particularly suitable for application in clinical scenarios that require frequent and efficient calibration.

Abstract:
We present a neural radiance field (NeRF)-based large-scale reconstruction system that fuses lidar and vision data to generate high-quality reconstructions that are geometrically accurate and capture photorealistic texture. Our system adopts the state-of-the-art NeRF representation to additionally incorporate lidar. Adding lidar data adds strong geometric constraints on the depth and surface normals, which is particularly useful when modeling uniform texture surfaces which contain ambiguous visual reconstruction cues. A key contribution of this work is a novel method to quantify the epistemic uncertainty of the lidar-visual NeRF reconstruction by estimating the spatial variance of each point location in the radiance field given the sensor observations from the cameras and lidar. This provides a principled approach to evaluate the contribution of each sensor modality to the final reconstruction. In this way, reconstructions that are uncertain (due to, e.g., uniform visual texture, limited observation viewpoints, or little lidar coverage) can be identified and removed. Our system is integrated with a real-time pose-graph lidar simultaneous localisation and mapping (SLAM) system, which is used to bootstrap a structure-from-motion reconstruction procedure. It also helps to properly constrain the overall metric scale, which is essential for the lidar depth loss. The refined SLAM trajectory can then be divided into submaps using spectral clustering to group sets of covisible images together. This submapping approach is more suitable for visual reconstruction than distance-based partitioning. Our uncertainty estimation is particularly effective when merging submaps, as their boundaries often contain artifacts due to limited observations. We demonstrate the reconstruction system using a multicamera, lidar sensor suite in experiments involving both robot-mounted and handheld scanning. Our test datasets cover a total area of more than \text20 000 m^2, including multiple university buildings and an aerial survey of a multistorey building. Quantitative evaluation is provided by comparing with maps produced by a commercial tripod scanner.

Abstract:
User intention estimation is essential in shared control systems for powered wheelchairs. It enables seamless navigation assistance that enhances safety, efficiency, and usability, while preserving user autonomy and reducing effort. This article presents diffusion-based wheelchair user intention estimation (DIWIE), a novel multimodal learning framework that leverages a denoising diffusion probabilistic model to forecast multiple plausible future trajectories, addressing uncertainty in human behavior. DIWIE conditions on diverse inputs, including obstacle information, user attention cues from eye gaze and head pose, semantic context, wheelchair kinematics, and joystick commands, operating without predefined maps or target destinations. Evaluated on a large new dataset of natural navigation by multiple drivers, DIWIE outperforms state-of-the-art methods, achieving lower displacement errors and collision rates, making it a valuable component for integration into shared control systems. This work also analyzes the relevance of different data sources for intention estimation and aligns evaluation metrics with related fields to foster reproducibility.

Abstract:
Automating the manipulation of granular materials poses significant challenges due to complex contact dynamics, unpredictable material properties, and intricate system states. Existing approaches often fail to achieve efficiency and accuracy in such tasks. To fill the research gap, this article studies the small-scale and high-precision granular material digging task with unknown physical properties. A key scientific problem addressed is the feasibility of applying first-order gradient-based optimization to complex differentiable granular material simulation and overcoming associated numerical instability. A new framework, named differentiable digging robot (DDBot), is proposed to manipulate granular materials, including sand and soil. Specifically, we equip DDBot with a differentiable physics-based simulator, tailored for granular material manipulation, powered by GPU-accelerated parallel computing and automatic differentiation. DDBot can perform efficient differentiable system identification and high-precision digging skill optimization for unknown granular materials, which is enabled by a differentiable skill-to-action mapping, a task-oriented demonstration method, gradient clipping and line search-based gradient descent. Experimental results show that DDBot can efficiently (converge within 5 to 20 minutes) identify unknown granular material dynamics and optimize digging skills, with high-precision results in zero-shot real-world deployments, highlighting its practicality. Benchmark results against state-of-the-art baselines also confirm the robustness and efficiency of DDBot in such digging tasks.

Abstract:
With the promise of greater safety and adaptability, modular reconfigurable uncrewed air vehicles have been proposed as unique, versatile platforms holding the potential to replace multiple types of monolithic vehicles at once. State-of-the-art rigidly assembled modular vehicles are generally 2-D configurations in the shape of a “flight array,” where the rotors are coplanar. We introduce the Dodecacopter, a new type of modular rotorcraft where all modules take the shape of a regular dodecahedron, allowing the creation of richer sets of configurations beyond flight arrays. In particular, we show how the chosen module design can be used to create 3-D and fully actuated configurations. We justify the relevance of these types of configurations in terms of their structural and actuation properties with various performance indicators. Given the broad range of configurations and capabilities that can be achieved with our proposed design, we formulate tractable optimization programs to find optimal configurations given structural and actuation constraints. Finally, a prototype of such a vehicle is presented along with results of performed flights in multiple configurations.

Abstract:
Heterogeneous Bayesian decentralized data fusion captures the set of problems in which two or more robots must combine probability density functions over nonequal, but overlapping sets of random variables. In the context of multirobot dynamic systems, this enables robots to take a “divide and conquer” approach to reason and share data over complementary tasks instead of over the full joint state space. For example, in a target tracking application, this allows robots to track different subsets of targets and share data on only common targets. This article presents a system by which robots can each use a local factor graph to represent relevant partitions of a complex global joint probability distribution, thus allowing them to avoid reasoning over the entirety of a more complex model and saving communication as well as computation costs. From a theoretical point of view, this article makes contributions by casting the heterogeneous decentralized fusion problem in terms of factor graphs, analyzing the challenges that arise due to dynamic filtering, and then developing a new conservative filtering algorithm that ensures statistical correctness. From a practical point of view, we show how this system can be used to represent different multirobot applications and then test it with simulations and hardware experiments to validate and demonstrate its statistical conservativeness, applicability, and robustness to real-world challenges.

Abstract:
Narrow passage path planning is a prevalent problem from industrial to household sites, often facing difficulties in finding feasible paths or requiring excessive computational resources. In this article, we propose a homotopy optimization method tailored for the narrow passage problem, utilizing a novel collision constraint interpolation method using signed distance functions. The framework begins by decomposing the environment into convex objects and representing it as a simplicial complex based on their connectivity. This representation enables topological analysis to induce an easy-to-hard sequence of collision constraint interpolation that preserves homotopy equivalence. Using this collision constraint interpolation, the optimization proceeds through a series of subproblems, gradually guiding the path to the final solution. Several examples are presented to demonstrate how the proposed framework addresses narrow passage path planning problems.

Abstract:
The enhanced mobility brought by legged locomotion empowers quadrupedal robots to navigate through complex and unstructured environments. However, optimizing agile locomotion while accounting for the varying energy costs of traversing different terrains remains an open challenge. Most previous work focuses on planning trajectories with traversability cost estimation based on human-labeled environmental features. This human-centric approach is insufficient because it does not account for the varying capabilities of the robot locomotion controllers over challenging terrains. To address this, we introduce a novel real-world learning pipeline that unifies offline demonstrations, online reinforcement learning, and multimodal perception to achieve robust legged navigation. The framework employs multiple training stages to develop a planner that guides the robot in avoiding obstacles and hard-to-traverse terrains while reaching its goals. We first develop a novel traversability estimator in a robot-centric manner. The training of the navigation planner is directly performed in the real world using a sample efficient reinforcement learning method. With the proposed method, a quadrupedal robot learns to perform traversability-aware navigation through real-world interactions in diverse offroad and unstructured environments. Moreover, the robot demonstrates the ability to generalize the learned navigation skills to unseen scenarios.

Abstract:
Vector fields are advantageous in handling nonholonomic motion planning, as they provide the robot with reference orientation across the workspace. However, additionally incorporating curvature constraints presents challenges due to the interconnection between the design of the curvature-bounded vector field and the tracking controller under limited actuation. In this article, we present a novel framework to co-develop the vector field and the control law, guiding the nonholonomic robot to the target configuration with curvature-bounded trajectory. First, we formulate the problem by introducing the target positive limit set, which allows the robot to either converge to or pass through the target configuration, depending on its dynamics and the specific tasks. Next, we construct a curvature-constrained vector field (CVF) via blending and embedding elementary flows in the workspace. To track such a CVF, a saturated control law with dynamic gains is proposed, under which the tracking error’s magnitude decreases even when saturation occurs. Under the control law, the kinematically constrained nonholonomic robot is guaranteed to track the reference CVF and converge to the target positive limit set with bounded trajectory curvature. Numerical simulations show that the proposed CVF method outperforms other vector-field-based algorithms. Experiments on Ackermann uncrewed ground vehicles and semiphysical fixed-wing uncrewed aerial vehicles demonstrate that the method can be effectively implemented in real-world scenarios.

Abstract:
A standardized robot-testing algorithm should satisfy accuracy, efficiency, and very importantly, repeatability—consistently yielding similar outcomes across multiple executions by different stakeholders. Achieving repeatability grows challenging with increasing complexity, intelligence, diversity, and inherent stochasticity in testing methods, robotic platforms, and environments. While existing efforts address repeatability through ethical, hardware, or procedural means, this study specifically targets algorithm-level repeatability, focusing on statistical query (SQ) algorithms commonly used in standardized evaluations. We propose a lightweight, adaptive modification for any SQ-based routine, including Monte Carlo, importance sampling, and adaptive sampling, guaranteeing provable repeatability with bounded accuracy and efficiency. Effectiveness is demonstrated across three cases: standardized manipulator testing, intelligent risk assessment for automated vehicles, and performance evaluation of humanoid robot locomotion.

Abstract:
Many robots are not equipped with a manipulator and many objects are not suitable for prehensile manipulation (such as large boxes and cylinders). In these cases, pushing is a simple yet effective nonprehensile skill for robots to interact with and further change the environment. Existing work often assumes a set of predefined pushing modes and fixed-shape objects. This work tackles the general problem of controlling a robotic fleet to push collaboratively numerous arbitrary objects to respective destinations, within complex environments of cluttered and movable obstacles. It incorporates several characteristic challenges for multirobot systems, such as online task coordination under large uncertainties of cost and duration, and for contact-rich tasks, such as hybrid switching among different contact modes, and under actuation due to constrained contact forces. The proposed method is based on combinatorial hybrid optimization over dynamic task assignments and hybrid execution via sequences of pushing modes and associated forces. It consists of the following three main components: first, the decomposition, ordering, and rolling assignment of pushing subtasks to robot subgroups; second, the keyframe guided hybrid search to optimize the sequence of parameterized pushing modes for each subtask; third, the hybrid control to execute these modes and transit among them. Last but not least, a diffusion-based accelerator is adopted to predict the keyframes and pushing modes that should be prioritized during hybrid search; and further improve planning efficiency. The framework is complete under mild assumptions. Its efficiency and effectiveness under different numbers of robots and general-shaped objects are validated extensively in simulations and hardware experiments, as well as generalizations to heterogeneous robots, planar assembly, and 6-D pushing.

Abstract:
Traditional visual simultaneous localization and mapping (SLAM) systems focus solely on static scene structures, overlooking dynamic elements in the environment. Although effective for accurate visual odometry in complex scenarios, these methods discard crucial information about moving objects. By incorporating this information into a dynamic SLAM framework, the motion of dynamic entities can be estimated, enhancing navigation whilst ensuring accurate localization. However, the fundamental formulation of dynamic SLAM remains an open challenge, with no consensus on the optimal approach for accurate motion estimation within a SLAM pipeline. Therefore, we developed DynoSAM, an open-source framework for dynamic objects SLAM that enables the efficient implementation, testing, and comparison of various dynamic SLAM optimization formulations. We further propose a novel formulation that encodes rigid-body motion model in object pose estimation as well as an error metric agnostic to object frame definition. DynoSAM integrates static and dynamic measurements into a unified optimization problem solved using factor graphs, simultaneously estimating camera poses, static scene, object motion or poses, and object structures. We evaluate DynoSAM across diverse simulated and real-world datasets, achieving state-of-the-art motion estimation in indoor and outdoor environments, with substantial improvements over existing systems. In addition, we demonstrate DynoSAM’s contributions to downstream applications, including 3-D reconstruction of dynamic scenes and trajectory prediction, thereby showcasing potential for advancing dynamic object-aware SLAM systems. Code is open-sourced online.

Abstract:
Artificial muscles embody human aspirations for engineering lifelike robotic movements. This article introduces an architecture for inflatable fluid-driven origami-inspired artificial muscles (IN-FOAMs). A typical IN-FOAM consists of an inflatable skeleton enclosed within an outer skin, which can be driven using a combination of positive and negative pressures (e.g., compressed air and vacuum). IN-FOAMs are manufactured using low-cost heat-sealable sheet materials through heat-pressing and heat-sealing processes. Thus, they can be ultra-thin when not actuated, making them flexible, lightweight, and portable. The skeleton patterns are programmable, enabling a variety of motions, including contracting, bending, twisting, and rotating, based on specific skeleton designs. We conducted comprehensive experimental, theoretical, and numerical studies to investigate IN-FOAM’s basic mechanical behavior and properties. The results show that IN-FOAM’s output force and contraction can be tuned through multiple operation modes with the applied hybrid positive–negative pressure. In addition, we propose multilayer skeleton structures to enhance the contraction ratio further, and we demonstrate a multichannel skeleton approach that allows the integration of multiple motion modes into a single IN-FOAM. These findings indicate that IN-FOAMs hold great potential for future applications in flexible wearable devices and compact soft robotic systems.

Abstract:
Learning control policies from high-dimensional robot demonstrations remains challenging due to the need for large datasets, limited generalization, and high model complexity. These limitations hinder data-efficient learning and reactive deployment from limited demonstrations. This article introduces Laplacian Eigenmap-based modeling of nonlinear dynamical systems (LEMON-DS), a compact and data-efficient framework for one-shot learning of stable control policies from high-dimensional demonstrations. LEMON-DS proceeds in the following three stages: first, a graph Laplacian-based spectral embedding that reveals a quasi-linear latent structure from a single trajectory, second, defining a globally asymptotically stable dynamical system (DS) in the latent space, and third, learning a diffeomorphic mapping that reconstructs the demonstrated behavior. Collectively, this pipeline yields a lightweight reactive control policy in the form of a DS that is parameter-efficient, globally asymptotically stable, and faithfully reproduces the demonstrated behavior generalizing across initial conditions. We show empirical performance across diverse robot tasks up to 23 dimensions, outperforming prior approaches in accuracy and model compactness.

Abstract:
Flexible sensors are increasingly employed in soft robotics and wearable devices to provide proprioception of freeform deformations. Although supervised learning can train shape predictors from sensor signals, prediction accuracy strongly depends on sensor layout, which is typically determined heuristically or through trial-and-error. This work introduces a model free, data-driven computational pipeline that jointly optimizes the number, length, and placement of flexible length-measurement sensors together with the parameters of a shape prediction network for large freeform deformations. Unlike model-based approaches, the proposed method relies solely on datasets of deformed shapes, without requiring physical simulation models, and is therefore broadly applicable to diverse robotic sensing tasks. The pipeline incorporates differentiable loss functions that account for both prediction accuracy and manufacturability constraints. By co-optimizing sensor layouts and network parameters, the method significantly improves deformation prediction accuracy over unoptimized layouts while ensuring practical feasibility. The effectiveness and generality of the approach are validated through numerical and physical experiments on multiple soft robotic and wearable systems.

Abstract:
Data scaling has driven remarkable success in foundation models for natural language processing (NLP) and computer vision (CV), yet the principles of effective data scaling in robotic manipulation remain insufficiently understood. In this work, we investigate the nuanced role of data diversity in robot learning by examining three critical dimensions—task (what to do), embodiment (which robot to use), and expert (who demonstrates)—challenging the conventional intuition of “more diverse is better.” Throughout extensive experiments on various robot platforms, we reveal that first, task diversity proves more critical than per-task demonstration quantity, with scene diversity playing a more important role than skill diversity for robustness and generalization under distribution shifts, second, multiembodiment pretraining data are nonessential for cross-embodiment transfer—models trained on high-quality single-embodiment data can efficiently transfer to different platforms, showing desirable scaling property during fine-tuning and its potential of replacing large-scale multiembodiment pretraining, and third, expert diversity, arising from individual operational preferences and stochastic variations in human demonstrations, can be confounding to policy learning, with action rate multimodality emerging as a key contributing factor. Based on this insight, we propose a distribution debiasing method to mitigate action rate ambiguity, the yielding GO-1-Pro achieves substantial performance gains of 15%, equivalent to using 2.5× pretraining data. Collectively, these findings provide new perspectives and offer practical guidance on how to scale robotic manipulation datasets effectively. The code will be released.

Abstract:
Designing an effective and scalable scene primitive representation is fundamental for large-scale point cloud registration. Existing studies that predominantly rely on dense point clouds or single-type geometric primitives struggle to scale to scenes characterized by massive data volume, structural diversity, and wide viewpoint variations. To address these, this article introduces QuadricsReg, a novel point cloud registration framework based on quadric primitives for large-scale environments. We compactly model diverse scene structures within a unified semantic quadric formulation, achieving high compression while preserving geometric richness and discriminability. This representation enables efficient quadric matching initialization via intrinsic similarity and robust correspondence pruning by maximizing geometric consistency in a multilevel graph, ensuring reliable associations even under large viewpoint variations. Furthermore, we design a factor graph based on degeneracy-aware quadric residual to estimate the transformation, ensuring accurate alignment in heterogeneous scenes. We evaluate QuadricsReg on 5 public datasets, where its exceptional registration performance with low overhead demonstrates strong scalability for large-scale scenarios. With a compact representation of ～ 29.5 \mathrmKB/scan on KITTI, nearly 100% registration success rate is achieved for point cloud pairs within 10 \mathrmm. Real-world testing on the self-collected dataset further validates its robustness and generalization ability across different LiDAR sensors and robot platforms.

Abstract:
Neighborhood finders and nearest neighbor queries are fundamental components of sampling-based motion planning algorithms. Using different distance metrics or otherwise changing the definition of a neighborhood produces different algorithms with unique empirical and theoretical properties. In his textbook on planning algorithms, LaValle suggests a neighborhood finder for the rapidly-exploring random tree algorithm, which finds the nearest neighbor of the sampled point on the swath of the tree, that is, on the set of all of the points on the tree edges, using a hierarchical data structure. In this article, we implement such a neighborhood finder and show, theoretically and experimentally, that this results in more efficient algorithms.

Abstract:
This article proposes a model-free magnetic servoing technique for closed-loop pose control of capsule robots. The system employs an internal permanent magnet (IPM) inside the capsule and an external permanent magnet (EPM) manipulated by a robotic arm. Control feedback is directly derived from magnetic field measurements (via a sensor array) and optical tracking of the EPM’s pose. To enable model-free control, we develop a feature extraction method that condenses the IPM’s magnetic sensor data into a compact representation. Based on this magnetic feature, we derive a Jacobian matrix that directly maps the feature space to the robotic arm’s joint configuration space. To adaptively handle system nonlinearities, the Jacobian matrix is iteratively updated using an unscented Kalman filter, bypassing the need for an explicit dynamic model. Experimental results demonstrate the effectiveness of our approach, with average tracking errors of 0.62 mm in position and 0.75^\circ in orientation during simultaneous pose control tasks.

Abstract:
Multirobot task planning (MRTP) at scale in robotic mobile fulfillment systems (RMFS) remains a challenge due to the curse of dimensionality and complex dynamic properties. Aiming to solve these challenges, we construct an end-to-end scalable multirobot task planner capable of scaling to large-scale systems by learning hierarchical planning policies. In this planner, we design a centralized hierarchical temporal task planning framework to mitigate the curse of dimensionality while ensuring timely dynamic response. Following this framework, we propose a novel cycle-constrained asynchronous temporal graph to provide foundation for modeling the system dynamics. Based on the graph representation, we formulate the MRTP problem as a semi-Markov decision process (SMDP) that focuses solely on critical interaction points to improve computational and sampling efficiency. The policies in SMDP are parameterized via a hierarchical temporal attention network with temporal embedding layers to enhance spatio-temporal feature extraction. In addition, the decoder masks in this network naturally ensure that the generated actions strictly satisfy the required dynamic hard constraints. The above hierarchical policies are jointly optimized using an efficient hierarchical REINFORCE with rollout counterfactual baseline method. To further enhance generalization performance on unlearned instances while preventing catastrophic forgetting, we extend it with region expansion curricula. Experiments demonstrate that our planner outperforms state-of-the-art methods on different MRTP instances across simulated and real-world RMFS. It successfully scales to instances with up to 200 robots, 1000 retrieval racks on unlearned maps while maintaining performance advantages.

Abstract:
Visual imitation learning has achieved remarkable progress in robotic manipulation, yet generalization to unseen objects, scene layouts, and camera viewpoints remains a key challenge. Recent advances address this by using 3-D point clouds, which provide geometry-aware, appearance-invariant representations, and by incorporating equivariance into policy architectures to exploit spatial symmetries. However, existing equivariant approaches often lack interpretability and rigor due to unstructured integration of equivariant components. We introduce canonical policy, a principled framework for 3-D equivariant imitation learning that unifies 3-D point cloud observations under a canonical representation. We first establish a theory of 3-D canonical representations, enabling equivariant observation-to-action mappings by grouping both seen and novel point clouds to a canonical representation. We then propose a flexible policy learning pipeline that leverages geometric symmetries from canonical representation and the expressiveness of modern generative models. We validate canonical policy on 12 diverse simulated tasks and 4 real-world manipulation tasks across 16 configurations, involving variations in object color, shape, camera viewpoint, and robot platform. Compared to state-of-the-art imitation learning policies, canonical policy achieves an average improvement of 18.0% in simulation and 39.7% in real-world experiments, demonstrating superior generalization capability and sample efficiency.

Abstract:
Consistent localization of cooperative multirobot systems during navigation presents substantial challenges. This article proposes a fault-tolerant, multimodal localization framework for multirobot systems on matrix Lie groups. We introduce novel stochastic operations to perform composition, differencing, inversion, averaging, and fusion of correlated and noncorrelated estimates on Lie groups, enabling pseudopose construction for filter updates. The method integrates a combination of proprioceptive and exteroceptive measurements from inertial, velocity, and pose (pseudopose) sensors on each robot in an extended Kalman filter (EKF) framework. The prediction step is conducted on the Lie group \mathbb SE_2(3) × \mathbb R^3 × \mathbb R^3, where each robot’s pose, velocity, and inertial measurement biases are propagated. The proposed framework uses body velocity, relative pose measurements from fiducial markers, and inter-robot communication to provide scalable EKF update across the network on the Lie group \mathbb SE(3) × \mathbb R^3. A fault detection module is implemented, allowing the integration of only reliable pseudopose measurements from fiducial markers. We demonstrate the effectiveness of the method through experiments with a network of wheeled mobile robots equipped with inertial measurement units, wheel odometry, and ArUco markers. The comparison results highlight the proposed method’s real-time performance, superior efficiency, reliability, and scalability in multirobot localization, making it well-suited for large-scale robotic systems.

Abstract:
Trajectory planning involving multiagent interactions has been a long-standing challenge in the field of robotics, primarily burdened by the inherent yet intricate interactions among agents. While game-theoretic methods are widely acknowledged for their effectiveness in managing multiagent interactions, significant impediments persist when it comes to accommodating the intentional uncertainties of agents. In the context of intentional uncertainties, the heavy computational burdens associated with existing game-theoretic methods are induced, leading to inefficiencies and poor scalability. In this article, we propose a novel game-theoretic interactive trajectory planning method to effectively address the intentional uncertainties of agents, and it demonstrates both high efficiency and enhanced scalability. As the underpinning basis, we model the interactions between agents under intentional uncertainties as a static Bayesian game, and we show that its agent-form equivalence can be represented as a potential game under certain assumptions. The existence and attainability of the optimal interactive trajectories are illustrated, as the corresponding static Bayesian Nash equilibrium can be attained by optimizing a unified optimization problem. In addition, we present a distributed algorithm based on the dual consensus alternating direction method of multipliers tailored to the parallel solving of the problem, thereby significantly improving the scalability. The attendant outcomes from simulations and experiments demonstrate that the proposed method is effective across a range of scenarios characterized by general forms of intentional uncertainties. Its scalability surpasses that of existing centralized and decentralized baselines, allowing for real-time interactive trajectory planning in uncertain game settings.

Abstract:
Cross-embodiment learning enables robots to acquire manipulation skills by learning from demonstrations provided by different embodiments. However, most existing research on cross-embodiment learning focuses on transferring similar manipulation behaviors. For the same task, different embodiments may need different behaviors, e.g., it may be easier for a human to push an object to a goal position, while a robot may use a pick-and-place to perform the same task. Making use of such cross-embodiment and cross-behavior demonstrations becomes essential for large-scale imitation learning. In this work, we propose a novel framework for cross-behavior learning based on object flow. Object flow represents the task in a manner that is independent from the embodiment and the particular behavior used during the demonstration. By shifting the focus from the manipulator to objects, our framework enables learning from cross-behavior manipulation data rather than merely imitating the behavior of a specific embodiment. Our results on task representations show that, with only 20 robot demonstrations, integrating object flow prediction improves success rates by up to 91% in-domain and 87% out-of-domain. Adding 40 human demonstrations in addition to the 20 robot demonstrations further boosts out-of-domain performance by 143%.

Abstract:
The recent development of connected and automated vehicle (CAV) technologies has spurred investigations to optimize dense urban traffic, maximizing vehicle speed and throughput. This article explores advisory autonomy, in which real-time driving advisories are issued to human drivers, thus achieving near-term performance of automated vehicles. Due to the complexity of traffic systems, recent studies of coordinating CAVs have leveraged deep reinforcement learning (RL). Coarse-grained advisory is formalized as zero-order holds, and we consider a range of hold durations from 0.1 to 40 s. However, despite the similarity of the higher frequency tasks for CAVs, a direct application of deep RL fails to generalize to advisory autonomy tasks. To overcome this, we employ zero-shot transfer, training policies on a set of source tasks—specific traffic scenarios with designated hold durations—and then evaluating the efficacy of these policies on different target tasks. We introduce temporal transfer learning (TTL) algorithms to select source tasks for zero-shot transfer, systematically leveraging the temporal structure to solve the full range of tasks. TTL selects the most suitable source tasks to maximize the performance of the range of tasks. We validate our algorithms on diverse mixed-traffic scenarios, demonstrating that TTL more reliably solves the tasks than baselines. This article underscores the potential of coarse-grained advisory autonomy with TTL in traffic flow optimization.

Abstract:
As robots integrate into human society, ensuring safe robot-environment interaction, particularly in the event of collisions, has emerged as a growing design priority. A promising solution is introducing proper compliance into existing rigid robots, akin to musculoskeletal systems, to absorb impacts in various directions. However, mimicking longitudinal compliance seen in biological joints to absorb compressive impacts along limbs, such as the role of cartilages like menisci, remains a challenge shadowed by the complexity of joint architecture. Here, exploring and adapting the elastic longitudinal movement structure of knee, we incorporated traditional mechanical hinges with a compact buffer structure to enable both simple rotational motion and effective longitudinal impact absorption. Under longitudinal loading, the buffer structure functions as a mechanism transmitting the limited compression to amplified deformations of elastic elements, thus producing resistance against load. The resistive load-displacement relationship is tailorable by tuning diverse elastic components, allowing for a high-static-low-dynamic stiffness to improve energy absorption efficiency. Drop tests and walking robot demonstrations confirm that the proposed knee-inspired hinge not only mitigates acceleration transmitted to the robot’s main body but also reduces ground reaction forces, thereby improving robot-environment interaction safety. This work highlights the design paradigm of adapting natural solutions to mechanical systems, and holds potential for direct integration into diverse robots, exoskeletons, and prostheses.

Abstract:
This article addresses survey missions involving multiple uncrewed aerial vehicles (UAVs) over complex, varying terrain. The methodology integrates a probabilistic model of target’s position uncertainty with UAV flight dynamics, camera properties, and a machine learning-based detection system. It estimates undetected target probability and overall search performance, feeding into a feedback loop that combines 2-D ergodic search with model predictive control (MPC) of UAV altitude and velocity. Trial trajectory optimization accounts for sensing characteristics and operational constraints, producing terrain-aware, collision-free trajectories that balance area coverage with target detection. Simulations demonstrate the integration of MPC and ergodic search, enabling dynamic altitude adjustments to enhance the search performance. The control algorithm operates in real time and performs reliably under uncertainty. Field experiments provided training data, validated the method, and confirmed compliance with motion constraints. Detection rates closely match model predictions, demonstrating stable performance even under significant deviations from ideal conditions. The framework, thus, offers a reliable solution for autonomous multi-UAV search operations in real-world environments.

Abstract:
Sim-to-real transfer in robotic manipulation tasks has emerged as a crucial area of research, addressing the challenge of bridging the gap between simulated environments and real-world applications. This article presents a brief review of current methodologies and introduces novel approaches to enhance the efficacy of sim-to-real transfer. We propose a new framework, adaptive domain randomization (DR) with progressive neural architecture search, which combines adaptive DR techniques with neural architecture search to optimize both the simulation parameters and the neural network architecture for improved transfer. We evaluate our approach against multiple state-of-the-art baselines, including standard DR, adaptive DR, model-agnostic meta-learning, randomized-to-canonical adaptation networks, and ensemble policy learning. Our experiments on a diverse set of manipulation tasks demonstrate significant improvements in the transfer performance, with up to 35% reduction in reality gap compared to state-of-the-art methods. Furthermore, we introduce a novel metric, the transfer efficiency index, to quantify the effectiveness of sim-to-real transfer across different tasks and methodologies.

Abstract:
Hybrid wearable robotics, combining soft and rigid elements, offer a promising solution for upper limb assistance by balancing comfort and functionality. This study evaluates a hybrid soft-rigid elbow exosuit designed for elbow movement support in dynamic and endurance tasks, focusing on biomechanical impact, user perception, and ergonomics. A theoretical model guided the design, optimizing device dynamics and assistive force delivery. A generic interaction controller, combined with fixed system parameters, provided effective assistance across participants without the need for subject-specific tuning, simplifying deployment in practical settings. During dynamic tasks, the exosuit significantly reduced biceps muscle activity compared to the unassisted condition, with an average reduction of 28.40% . In endurance tasks, it reduced infraspinatus activity by an average of 44.55% and mitigated increased muscle activation during load-carrying. Subjective assessments indicated lower perceived physical demand, while the system usability scale confirmed high usability and acceptance. These findings highlight the exosuit’s ability to reduce physical effort and improve muscle coordination while maintaining comfort. The robust mechatronic design enabled effective assistance through a generic controller, emphasizing the importance of hardware reliability.

Abstract:
This article presents EROAM, a novel event-based rotational odometry and mapping system that achieves real time, accurate camera rotation estimation. Unlike existing approaches that rely on event generation models or contrast maximization, EROAM employs a spherical event representation by projecting events onto a unit sphere and introduces event spherical iterative closest point, a novel geometric optimization framework designed specifically for event camera data. The spherical representation simplifies rotational motion formulation while operating in a continuous spherical domain, enabling enhanced spatial resolution. Our system features an efficient map management approach using incremental k-d tree structures and intelligent regional density control, ensuring optimal computational performance during long-term operation. Combined with parallel point-to-line optimization, EROAM achieves efficient computation without compromising accuracy. Extensive experiments on both synthetic and real-world datasets show that EROAM significantly outperforms state-of-the-art methods in terms of accuracy, robustness, and computational efficiency. Our method maintains consistent performance under challenging conditions, including high angular velocities and extended sequences, where other methods often fail or show significant drift. In addition, EROAM produces high-quality panoramic reconstructions with preserved fine structural details.

Abstract:
This article explores the challenge of optimal routing for a mobile robot navigating a dynamic and shared human environment. The primary goal is to minimize the risk of performance degradation during motion, such as delays in completing tasks due to the need for safe or acceptable human– robot encounters. The problem is formulated as a graph whose edge costs become progressively known only as the robot moves through the environment. We model this problem as a Markov decision process (MDP), enabling an offline evaluation of the expected cost of alternative routes based on statistical information about human spatial distributions and possible observations at each intersection. This compact state representation scales linearly with the number of intersections in the map. Since the memoryless property of the MDP may induce loops during online execution, we compute an offline policy and introduce an online policy adaptation mechanism to prevent cyclic behaviors. Extensive simulations across environments of different complexity, and using data collected from real-world experiments, demonstrate that our approach outperforms reactive and advanced state-of-the-art planners in terms of either performance or scalability.

Abstract:
This article presents a hierarchical control framework for quadrupedal locomotion that unifies the complementary strengths of model-based optimization and reinforcement learning. We develop a convex quadratic programming (QP) solver based on the primal-dual Chambolle–Pock algorithm, enabling both massively parallel policy training and real-time deployment through efficient handling of constrained optimization problems. Our hierarchical framework employs learned policies for robust high-level control to handle real-world perturbations, while ensuring instantaneous constraint satisfaction and energy efficiency through a low-level whole-body controller powered by the proposed solver. Extensive benchmarks and experimental validation demonstrate quantifiable improvements in energy consumption, constraint satisfaction, and task transferability across simulated and real-world environments.

Abstract:
Learning from demonstration is a powerful method for robotic skill acquisition. Nevertheless, a critical limitation lies in the substantial costs associated with gathering demonstration datasets, typically action-labeled robot data, which create a fundamental constraint in the field. Video data offer a compelling solution as an alternative rich data source, containing diverse behavioral and physical knowledge. This study introduces G3M, an innovative framework that exploits video data via Graph-to-Graphs Generative Modeling, which pretrains models to generate future graphs conditioned on the graph within a video frame. The proposed G3M abstracts video frame into graph representations by identifying object and visual action vertices for capturing state information. It then effectively models internal structures and spatial relationships present in these graph constructions, with the objective of predicting forthcoming graphs. The generated graphs function as conditional inputs that guide the control policy in determining robotic behaviors. This concise method effectively encodes critical spatial relationships while facilitating accurate prediction of subsequent graph sequences, thus allowing the development of resilient control policy despite constraints in action-annotated training samples. Furthermore, these transferable graph representations enable the effective extraction of manipulation knowledge through human videos as well as recordings from robots with different embodiments. The experimental results demonstrate that G3M attains superior performance using merely 20% action-labeled data relative to comparable approaches. Moreover, our method outperforms the state-of-the-art method, showing performance gains exceeding 19% in simulated environments and 23% in real-world experiments, while delivering improvements of over 35% in cross-embodiment transfer experiments and exhibiting strong performance on long-horizon tasks.

Abstract:
Monocular vision-based target motion estimation is a fundamental challenge in numerous applications. This work introduces a novel bearing-box approach that fully leverages modern 3-D detection measurements that are widely available nowadays but have not been well explored for motion estimation so far. Unlike existing methods that rely on restrictive assumptions, such as isotropic target shape and lateral motion, our bearing-box estimator can estimate both the target's motion and its physical size without these assumptions by exploiting the information buried in a 3-D bounding box. When applied to multirotor micro aerial vehicles (MAVs), the estimator yields an interesting advantage: it further removes the need for higher order motion assumptions by exploiting the unique coupling between MAV's acceleration and thrust. This is particularly significant, as higher order motion assumptions are widely believed to be necessary in state-of-the-art bearing-based estimators. We support our claims with rigorous observability analyzes and extensive experimental validation, demonstrating the estimator's superior performance in real-world scenarios

Abstract:
This article presents a formal specification framework for planning and control of autonomous robots, focusing on the challenge of managing complex tradeoffs among multiple potentially conflicting objectives. These include hierarchical relationships and noncomparable objectives, some of which may be too complex to be captured by standard additive cost functions. We leverage the rulebook formalism to represent such objectives and their relationships and formulate two control synthesis problems: single-strategy synthesis, which seeks one optimal strategy, and complete synthesis, which computes the full set of optimal strategies with respect to a rulebook, analogous to the Pareto front in multiobjective planning. We show that our formulation generalizes existing temporal logic-based and optimization-based planning and control, providing a unifying framework across robotics, formal methods, control theory, and operation research. For single-strategy synthesis, we identify tractable subclasses and present a polynomial-time algorithm that accommodates richer combinations of objectives than prior work. For complete synthesis, we introduce an algorithm to compute all optimal solutions and analyze its computational complexity. In both cases, we present case studies that include complex multiobjective planning problems and demonstrate the practical effectiveness of our approach compared to existing methods.

Abstract:
Light detection and ranging (LiDAR)-based global localization (LGL) is an essential ingredient for autonomous robots. However, existing LGL methods typically consider only partial information (e.g., geometric features) from LiDAR observations or are designed for homogeneous LiDAR sensors, overlooking the uniformity in LGL. In this work, a uniform LGL method is proposed, termed UniLGL, which simultaneously achieves spatial and material uniformity, as well as sensor-type uniformity. The key idea of the proposed method is to encode the complete point cloud, which contains both geometric and material information, into a pair of bird’s eye view (BEV) images (i.e., a spatial BEV image and an intensity BEV image), thereby transforming the LGL problem into a cascaded LiDAR place recognition (LPR) and pose estimation problem from the perspective of image fusion. An end-to-end multi-BEV fusion network is designed to extract uniform features, equipping UniLGL with spatial and material uniformity. To ensure robust LGL across heterogeneous LiDAR sensors, a viewpoint invariance hypothesis is introduced, which replaces the conventional translation equivariance assumption commonly used in existing LPR networks and supervises UniLGL to achieve sensor-type uniformity in both global descriptors and local feature representations. Moreover, UniLGL introduces a pipeline that leverages a pretrained single-image vision foundation model for feature extraction to enhance the multi-BEV fusion LPR network, enabling strong generalization with only a few LiDAR data for fine-tuning. Finally, based on the mapping between local features on the 2-D BEV image and the point cloud, a robust global pose estimator is derived that determines the global minimum of the global pose on \textSE(3) without requiring additional registration. To validate the effectiveness of the proposed uniform LGL, extensive benchmarks are conducted in real-world environments, and the results show that the proposed UniLGL is demonstratively competitive compared to other SOTA LGL methods. Furthermore, UniLGL has been deployed on diverse platforms, including full-size trucks and agile micro-aerial vehicles (MAVs), to enable high-precision localization and mapping as well as multi-MAV collaborative exploration in port and forest environments, demonstrating the applicability of UniLGL in industrial and field scenarios.

Abstract:
Avoiding the usage of joint accelerations during robot online identification and control is significant in improving modeling and tracking accuracy. Regressor filtering is feasible to achieve acceleration-free online identification, where the robot dynamics is linearly parameterized and filtered to obtain an acceleration-free filtered regressor. But existing calculation methods of the filtered regressor are either applicable only to robots with low degrees of freedom (DoFs) or restricted by robot modeling techniques. We propose an acceleration-free analytical regressor filtering (AF-ARF) method to obtain the filtered regressor without restrictions on the number of DoFs or robot modeling techniques, where joint accelerations are eliminated by integration by parts, and the filtered regressor is derived by using the skew-symmetric property of the inertia matrix and some matrix operations. An acceleration-free composite learning robot control strategy based on AF-ARF is developed for exact online identification and control, where closed-loop exponential stability with parameter convergence is established under a weakened condition of interval excitation. Simulative and experimental comparisons based on a seven-DoF industrial robot have validated the superiority of our method over state-of-the-art methods in online identification, model prediction, and tracking control under reduced computing burden.

Affiliations: National Engineering Research Center of Robot Visual Perception and Control Technology, School of Artificial Intelligence and Robotics, Hunan University, Changsha, China; School of Architecture and Art, Central South University, Changsha, China; School of Computing and Communications, Lancaster University, Lancaster, U.K.; Department of Computer Science and Software Engineering, The University of Western Australia, Crawley, WA, Australia; School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

Abstract:
Pose estimation-guided unseen object 6-degree of freedom (6-DoF) robotic manipulation is a key task in robotics. However, the scalability of current pose estimation methods to unseen objects remains a fundamental challenge, as they generally rely on computer-aided design (CAD) models or dense reference views of unseen objects, which are difficult to acquire, ultimately limit their scalability. In this article, we introduce a novel task setup, referred to as SinRef-6D, which addresses 6-DoF absolute pose estimation for unseen objects using only a single pose-labeled reference RGB-D image captured during robotic manipulation. This setup is more scalable yet technically nontrivial due to large pose discrepancies and the limited geometric and spatial information contained in a single view. To address these issues, our key idea is to iteratively establish pointwise alignment in a common coordinate system with state space models (SSMs) as backbones. Specifically, to handle large pose discrepancies, we introduce an iterative object-space pointwise alignment strategy. Then, Point and RGB SSMs are proposed to capture long-range spatial dependencies from a single view, offering superior spatial modeling capability with linear complexity. Once pretrained on synthetic data, SinRef-6D can estimate the 6-DoF absolute pose of an unseen object using only a single reference view. With the estimated pose, we further develop a hardware–software robotic system and integrate the proposed SinRef-6D into it in real-world settings. Extensive experiments on six benchmarks and in diverse real-world scenarios demonstrate that our SinRef-6D offers superior scalability. Additional robotic grasping experiments further validate the effectiveness of the developed robotic system.

Abstract:
This article presents a novel approach for collision avoidance in optimal and model-predictive control, in which the environment is represented by a large number of points and the robot as a union of padded polygons. The conditions that none of the points shall collide with the robot can be written in terms of an infinite number of constraints per obstacle point. We show that the resulting semi-infinite programming optimal control problem can be efficiently tackled through a combination of two methods: local reduction and an external active-set method. Specifically, this involves iteratively identifying the closest point obstacles, determining the lower level distance minimizer among all feasible robot shape parameters, and solving the upper level finitely constrained subproblems. In addition, this article addresses robust collision avoidance in the presence of ellipsoidal state uncertainties. Enforcing constraint satisfaction over all possible uncertainty realizations extends the dimension of constraint infiniteness. The infinitely many constraints arising from translational uncertainty are handled by local reduction together with the robot shape parameterization, while rotational uncertainty is addressed via a backoff reformulation. A controller implemented based on the proposed method is demonstrated on a real-world robot running at 20 Hz, enabling fast and collision-free navigation in tight spaces. An application to 3-D collision avoidance is also demonstrated in simulation.

Abstract:
In this article, we design a distributed feedback optimization strategy, embedded into a modular robotic operating system (ROS) 2 control architecture, which allows a team of heterogeneous robots to cooperatively monitor and encircle a target while patrolling points of interest. By relying on the aggregative feedback optimization framework, we handle multirobot dynamics while minimizing a global performance index depending on both microscopic (e.g., the location of single robots) and macroscopic variables (e.g., the spatial distribution of the team). The proposed distributed policy allows the robots to cooperatively address the global problem by employing only local measurements and neighboring data exchanges. These exchanges are performed through an asynchronous communication protocol ruled by locally verifiable triggering conditions. We formally prove that our strategy steers the robots to a set of configurations representing stationary points of the considered optimization problem. The effectiveness and scalability of the overall strategy are tested via Monte Carlo campaigns of realistic Webots ROS 2 virtual experiments. Finally, the applicability of our solution is shown with real experiments on ground and aerial robots.

Abstract:
Multisensor fusion has been widely accepted as the mainstream approach for autonomous driving and robotic tasks, which is founded on precise spatiotemporal calibration. However, significant differences in sampling frequency and modality among various sensors, such as LiDAR, IMU, and cameras, present considerable challenges to calibration, especially under severe motion conditions. To this end, we propose M^\mathbf2-Calibr, a targetless spatiotemporal calibration for multisensor systems via a multioutput Gaussian process (MOGP). First, continuous-time motion states are modeled via MOGP regression. Building on this, Point-MOGP is proposed to refine per-point LiDAR motion states, reducing motion-induced errors and producing an undistorted point-cloud map. This map is then utilized to recover the absolute scale of camera motion, further enhancing calibration accuracy. Finally, to further enhance the calibration performance under rapid motion conditions, a trajectory correlation analysis method incorporating dynamic time warping (DTW) is proposed, which computes the minimal distortion path and the maximum trajectory correlation between motion timestamps, enabling fast and accurate estimation of temporal offsets. Extensive experiments on benchmark and custom datasets demonstrate that M^\mathbf2-Calibr performs well in accuracy.

Abstract:
Robust flight in complex and windy environments is critical for both single and multiple quadrotors. Existing methods either learn disturbance model at high computational cost or use error-based adaptive control with a speed-stability tradeoff that makes tuning difficult. To address these issues, this article proposes a future-trend-aware filter-based proportional-derivative model reference adaptive control (PD-MRAC) for single quadrotor and a distributed PD-MRAC for multiple quadrotor formation. By embedding a trend-aware derivative term in the adaptive update laws, the controller obtains anticipatory information about the error evolution, enabling rapid adaptation while mitigating oscillations. For more disturbance-sensitive multiquadrotors, we design a robust distributed protocol under a directed graph, improving resilience to disturbances. The approach maintains low computational cost and supports fast adaptive updates. Extensive simulations and real-world experiments validate improvement. For single quadrotor, the root mean square error (RMSE) reduced by around 57% versus the baselines and by around 12% versus the DJI Mavic 2. For multiquadrotors, formation results show enhanced robustness in simulation and effective real-world indoor/outdoor experiments under strong winds.

Abstract:
Humans regularly throw projectiles with high speed and accuracy; some animals, including chimpanzees and elephants, also throw objects occasionally. In comparison, robots are currently lagging behind, despite having lower communication latency and more accurate motor control. To understand this paradox and ultimately achieve ubiquitous throwing robots, one of the major obstacles is the lack of high-fidelity and tractable physical models of the transient release dynamics, where the momentum exchange between the hand and the object occurs within tens of milliseconds via the frictional interface. In this work, we try to establish a physical model for the release dynamics. We first demonstrate that the conventional model, which combines rigid-body dynamics and patch friction [limit surface (LS)], struggles to capture the release dynamics and exhibits pathological behaviors, such as Zeno-like oscillations, leading to poor accuracy in predicting throwing outcomes. To mitigate this, we formulate a viscous-smoothed variant of the limit surface model solved via implicit integration (ILS), which achieves high predictive fidelity but incurs significant computational cost. On the other hand, motivated by the dominant effect of in-hand pivoting in release dynamics, we propose a sliding pivot model that simplifies the contact dynamics by capturing the sticking–pivoting–sliding behavior emerging under vanishing normal force. This model achieves accuracy comparable to ILS, with only 10% higher error while offering over 20× faster computation. Compared to conventional LS models, our method reduces horizontal velocity prediction error by 40% and angular velocity prediction error by 63%, achieving 2.4-cm mean absolute error (MAE) for landing position and 15.4^\circ MAE for landing orientation. These results provide a robust physically grounded foundation for future scalable robot throwing systems.

Abstract:
This study presents an innovative solution for integrating a human and a robotic knee prosthesis symbiotically for walking. Achieving this requires human–robot cross-joint coordination to provide personalized walking assistance. Our approach uses inverse reinforcement learning to identify control objectives for reinforcement learning controller. Unlike existing methods that optimize performance of human or robot alone, our approach considers both human (thigh segmental angle) and robot (knee joint kinematics) aspects. This bilevel optimization method was evaluated on three nondisabled participants and two people with amputation. Results showed that the approach personalized the objective function and resulted in a robust policy, completing optimization within a duration of 3.5 min. Compared to previous approaches focusing only on robot states, this symbiotic approach increased stance time and step length on the prosthesis side for most participants. Our results highlight the potential of integrating human state into prosthesis control personalization, enhancing the functionality and health of people with amputation.

Abstract:
The agility of visual servo systems is essential for efficient operation in dynamic environments. However, this agility is limited by uncertainties, perceptual limitations, and the failure to integrate perception and control in conventional frameworks. In this article, a bioinspired reinforced active-inference-driven agile visual servo (RAID-AgiVS) technology is presented to systematically elude the above limitations. Central to RAID-AgiVS is the reinforced sensing mechanism that augments sparse visual measurements with fast soft measurements generated by alternating prediction and observation; thereby producing high-bandwidth data that surpasses the native sampling rate of the visual sensor. In addition, the reinforced data are employed by an active inference controller—embedded within a reciprocal perceptual control structure—to introduce perceptual inference and active control actions that bolster adaptability and agility in visual servo tasks. By fusing biological adaptability with advanced data generation and analytic modeling, the RAID-AgiVS framework markedly improves performance, particularly under sensory limitations and dynamic uncertainties. Comparative evaluations on a 6-DoF manipulator—and an indoor quadrotor unmanned aerial vehicle (UAV)—demonstrate the superior accuracy, agility, and adaptability of the proposed method.

Affiliations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China; Division of Intelligent and Biomechanical Systems, State Key Laboratory of Tribology, Department of Mechanical Engineering, Tsinghua University, Beijing, China; School of Science, Harbin Institute of Technology, Shenzhen, China; Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Hong Kong, Hong Kong; Neurological Rehabilitation Center, Beijing Rehabilitation Hospital Affiliated to Capital Medical University, Beijing, China; Division of Intelligent and Biomechanical Systems, State Key Laboratory of Tribology, Department of Mechanical Engineering, Tsinghua University, Haidian, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Abstract:
Soft robotic devices, known for their high compliance, are increasingly being used in assistance and rehabilitation. However, the limited force output of soft actuators has hindered their broader adoption. In this study, a lobster-tail-inspired high-force-output soft pneumatic bending actuator (SPBA) is developed, featuring a soft deformable body and a rigid kirigami limiting shell. The SPBA, with a radius of 10 mm, can generate forces of approximately 22 N at an internal pressure of 0.1 MPa and 36.43 N at 0.16 MPa. An analytical model based on the Euler–Bernoulli beam theory, incorporating a hyperelastic material model, has been constructed to predict the deformation and force of the actuated SPBA. This model demonstrates good agreement with simulated and experimental results. For assistance, a soft robotic gripper with four SPBAs can lift a weight of 5.38 kg at 0.26 MPa. For rehabilitation, an SPBA-based hand exoskeleton has been developed, demonstrating significant effectiveness in mitigating hand spasticity following strokes. This study introduces a novel SPBA design with promising potential for future applications in grasping, assistance, and rehabilitation.

Abstract:
Surface finishing is a time consuming, dangerous task, difficult to automate despite its necessity in many manufacturing processes. Its automation, particularly through robotics, increases productivity and relieves workers from health-critical tasks. However, challenges remain, as automated offline planning tools can result in certain areas being either neglected or overly processed. Ergodic control offers the possibility to cover target probability distributions in an online manner, by taking into account the observed coverage history. However, existing ergodic control approaches provide little flexibility in designing and adapting coverage strategies. Moreover, they come with simplifying assumptions, such as point-based dynamics, which are no longer valid for tasks where the robot is in contact with strongly varying curvatures on nontrivial surface geometries. In this work, we introduce a closed-form ergodic control framework that includes the tool imprint in the system modeling while simultaneously permitting the intuitive transfer of finish strategies, namely preferred motion directions. We build on the Spectral Multiscale Coverage approach, augmenting it with a tool imprint model, as well as both target distributions and state-dependent movement directions extracted from human demonstrations. Through evaluations in a surface finishing task using a torque-controlled, 7-DoF, robot arm we show that our approach optimally covers surfaces according to the tool contact area, with robust error convergence.

Abstract:
Robotic dexterous manipulation is a challenging problem due to high degrees of freedom (DoFs) and complex contacts of multifingered robotic hands. Many existing deep reinforcement learning-based methods aim at improving sample efficiency in high-dimensional output action spaces. However, existing works often overlook the role of representations in achieving generalization of a manipulation policy in the complex input space during the hand-object interaction. In this article, we propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects for dexterous manipulation skill learning. Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e., grasping, in-hand reorientation, bimanual handover, and extensive experiments are conducted to verify the effectiveness. In simulation, for grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories, significantly surpassing existing work trained with thousands of objects; for the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%. The grasp policies with DexRep are deployed to the real world under multicamera and single-camera setups and demonstrate a small sim-to-real gap.

Abstract:
A deformable, flexible sheet that is held by multiple robots can be used for manipulating and transporting an object. For such multirobot collaborative transporting system, forward kinematics provide object's potential positions where it might remain stationary on the sheet. Some of these forward kinematics solutions are, however, demonstrated unstable under object's position or robot formation perturbations. This article presents stability criteria of the kinematics solutions of the object on the deformable sheet held by multirobot system. We capture the sheet deformation and object position using the virtual variable cable model. A constrained quadratic problem is formulated to obtain the forward kinematics solution for a given multirobot configuration. Linear dependence of active constraints and stability multipliers are used to assess the stability of the kinematics solutions. Two stability criteria are proposed and analyzed under object position or robot formation perturbations. These criteria are related to the properties and conditions of the stability multipliers. We present an efficient computational algorithm to determine stable kinematics, which are a small fraction of feasible solutions. Experimental results and case studies are presented to validate and demonstrate the effectiveness and efficiency of the analyses and algorithms.

Abstract:
We propose a novel initialization and online spatial–temporal calibration method for visual–inertial odometry (VIO), which decouples rotation and translation estimation to achieve higher accuracy and better robustness. Existing initialization methods suffer from limited accuracy or robustness (e.g., in scenarios with small translational motion) and rarely integrate simultaneous spatial–temporal calibration during initialization, despite its considerable practical value. Our proposed method leverages rotation–translation decoupling constraints to enable simultaneous estimation of gyroscope bias, extrinsic rotation, and camera–inertial measurement unit (IMU) time offset—even under pure rotational motion. Moreover, we are the first to conduct observability analysis on rotational constraints in rotation–translation decoupling methods, experimentally identifying the unobservable state-space directions under three degenerate motions within our approach. We also perform extensive experiments to delineate practical parameter solution boundaries for our method, with both efforts substantially enhancing the overall practical applicability of decoupling-based methods. Extensive experiments on simulated and real-world datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy and robustness while maintaining computational efficiency. Furthermore, experiments verify that it significantly improves convergence in VIO systems.

Abstract:
Mapping and understanding complex 3-D environments is fundamental to how autonomous systems perceive and interact with the physical world, requiring both precise geometric reconstruction and rich semantic comprehension. While existing 3-D semantic mapping systems excel at reconstructing and identifying predefined object instances, they lack the flexibility to efficiently build semantic maps with open-vocabulary during online operation. Although recent vision-language models (VLMs) have enabled open-vocabulary object recognition in 2-D images, they haven’t yet bridged the gap to 3-D spatial understanding. The critical challenge lies in developing a training-free unified system that can simultaneously construct accurate 3-D maps while maintaining semantic consistency and supporting natural language interactions in real time. In this article, we develop a zero-shot framework that seamlessly integrates GPU-accelerated geometric reconstruction with open-vocabulary VLMs through online instance-level semantic embedding fusion, guided by hierarchical object association with spatial indexing. Our training-free system achieves superior performance through incremental processing and unified geometric-semantic updates, while robustly handling 2-D segmentation inconsistencies. The proposed general-purpose 3-D scene understanding framework can be used for various tasks including zero-shot 3-D instance retrieval, segmentation, and object detection to reason about previously unseen objects and interpret natural language queries.

Abstract:
This study introduces a novel mobile robot inspired by the pill bug that offers dual locomotion modes, shape morphing, and protective sliding curved shells. The proposed design includes two primary mobility modes: walking and active rolling, providing flexibility and adaptability to navigate various terrains. Through a systematic approach involving type and dimensional synthesis, an underlying morphing mechanism is created to replicate the distinctive shapes of a pill bug’s body. This mechanism features a single-input-multiple-output structure designed using multiloop coupled mechanisms that mimic the natural body motion of pill bugs. To ensure surface protection while allowing relative sliding between shell segments, curved shells are attached to the tracer points of the mechanism. For active switching between rolling and walking modes, two adaptive one-degree-of-freedom legs are integrated into the morphing mechanism. By manipulating speed differences between these legs, steering capabilities are achieved to ensure smooth transitions between locomotion modes.

Abstract:
Mobile robots in unknown cluttered environments with irregularly shaped obstacles often face energy and communication challenges which directly affect their ability to explore these environments. Existing heuristic and learning-based map prediction methods are unable to generalize to irregular obstacles and uneven terrain, as they rely on single-pass architectures that cannot iteratively refine map predictions or incorporate uncertainty under limited communication and energy constraints. On the other hand, diffusion models perform multipass denoising to reconstruct high-fidelity maps from partial observations, enabling accurate predictions in resource constrained settings. In this article, we introduce a novel deep learning architecture, confidence-aware contrastive conditional consistency model (4CNet), for robot map prediction during decentralized, resource-limited multirobot exploration. 4CNet uniquely incorporates: 1) a conditional consistency model for map prediction in unstructured unknown regions, 2) a contrastive map-trajectory pretraining framework for a trajectory encoder that extracts spatial information from the trajectories of nearby robots during map prediction, and 3) a confidence network to measure the uncertainty of map prediction for effective exploration under resource constraints. We incorporate 4CNet within our proposed robot exploration with map prediction architecture, 4CNet-E. We then conduct extensive comparison studies with 4CNet-E and state-of-the-art heuristic and learning methods to investigate both map prediction and exploration performance in environments consisting of irregularly shaped obstacles and uneven terrain. Results showed that 4CNet-E obtained statistically significant higher prediction accuracy and area coverage with varying environment sizes, number of robots, energy budgets, and communication limitations when compared to database and learning-based methods. Hardware experiments were performed and validated the applicability and generalizability of 4CNet-E in both unstructured indoor and real natural outdoor environments.

Abstract:
Over the past two decades, research in the field of simultaneous localization and mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from handcrafted methods, through the era of deep learning, to more recent developments focused on neural radiance fields and 3-D Gaussian Splatting representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this article aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths, and limitations and serves as a fundamental reference to highlight the dynamic progress and specific challenges.

Abstract:
Robotic grasping plays a critical role in robotics, with widespread applications across various domains. The stability of a grasp is crucial for subsequent operations, making accurate and robust stability assessments essential. While existing methods predominantly rely on visual data, the absence of tactile signals and quantitative stability metrics often leads to unreliable grasp execution. This article makes three fundamental contributions to address these limitations. First, a visual-tactile grasp dataset generation framework is proposed using NVIDIA Isaac Sim, which synthesizes large-scale multimodal grasping scenarios with stability degree labels through a physics-informed three-stage pipeline. Second, this article introduces the grasp margin matrix, a novel computational model that quantifies directional force margins and rotational moment margins to evaluate grasp robustness. This matrix simplifies traditional grasp wrench space analysis by decoupling complex friction cone calculations into interpretable mechanical metrics, achieving 87.72% stability classification accuracy via our stability assessment network. Third, a vision-based grasp perception network that predicts contact force distributions and object centroids without physical tactile sensors is developed, enabling real-time stability inference through the grasp margin matrix. This perception–evaluation–decision integration links grasp pose generation with stability assessment to inform grasp selection, achieving an 88.15% success rates in real robotic grasping scenarios. Experimental validations demonstrate significant improvements compared to force closure methods, with accuracy improving from 56.52% to 87.72%, offering both theoretical rigor and practical feasibility for industrial robotic systems.

Abstract:
Transporting a heavy payload using multiple aerial robots (MARs) is an efficient manner to extend the load capacity of a single aerial robot. However, existing planning schemes for the MAR transportation system (MARTS) still lack the capability to generate a collision-free and dynamically feasible trajectory in real time. Therefore, they are limited to low-agility transportation in simple environments. To bridge the gap, we propose a complete planning scheme for the MARTS, achieving safe and agile aerial transportation (SAAT) of a cable-suspended payload in complex environments. Flatness map for the motor’s revolutions per minute of the aerial robot, considering the complete kinematic constraint and the dynamical coupling between each aerial robot and payload, is derived. To improve the responsiveness for the generation of the safe, dynamically feasible, and agile trajectory in complex environments, a real-time spatiotemporal trajectory planning scheme is proposed for the MARTS. Besides, we break away from the reliance on the state measurement for both the payload and cable, as well as the closed-loop control for the payload, and integrate a fully distributed control scheme to track the agile trajectory that is robust against imprecise payload mass, nonpoint mass payload, wind disturbances, and communication delays. The proposed schemes are extensively validated through benchmark comparisons, ablation studies, and simulations. Finally, extensive real-world experiments are conducted on practical MARTSs containing different numbers of aerial robots with onboard computers and sensors. The result validates the efficiency and robustness of our proposed schemes for the SAAT in complex environments.

Abstract:
Microrobot swarms with locomotion dexterity and shape reconfigurability show immense potential in biomedical applications. Automatic control strategies are critical for the navigation of swarms in unstructured environments. Existing control methods mainly focus on initial and final states of swarms, and swarm process control designed to avoid undesired states throughout the control process is yet investigated. In this work, we develop a deep learning-based process control strategy for swarms guided by phase diagrams. Two deep neural networks are respectively built to model the swarm shape and kinematics. Control approaches based on precise swarm models are designed to automatically tune multiple swarm parameters. A phase diagram-based controller is proposed to guide the swarm reconfiguration while eliminating the coupling effects between swarm parameters. The swarm is enabled to accurately track predefined trajectories while performing continuous reconfiguration with desired states during the entire process. By integrating the process control of swarm pattern and locomotion, the swarm can dynamically adapt to constrained unstructured spaces and achieve robust collision avoidance.

Abstract:
Slip detection plays a critical role in robotic manipulation, enabling stable, and reliable grasp control. Event cameras produce sparse asynchronous event streams, which spiking neural networks (SNNs) can efficiently leverage for energy-efficient computation with high temporal precision. However, their recurrent neural network-like structure limits training speed and scalability on long sequences. To address this issue, we propose the Probabilistic Spiking Structured State-Space Network (PS3N), which reformulates SNN dynamics into linear time-invariant state-space equations. This enables efficient parallel training via a convolutional formulation inspired by Structured State Space Sequence (S4) models. We further introduce local decaying kernels for fast and scalable processing while preserving temporal precision. Moreover, Existing event-based slip detection studies often overlook challenging scenarios involving low-texture, reflective, or transparent materials under varying lighting conditions. To fill this gap, we present the event-based visual-tactile slip (EVTS) dataset, a real-world benchmark for event-based visual-tactile slip detection. EVTS covers 30 objects across common, metal, and transparent categories under diverse lighting conditions and motion patterns. Furthermore, we propose dynamic temporal multimodal fusion (DTMF), which adaptively weights modalities based on their time-varying reliability, allowing more reliable modalities to contribute more to the final prediction. Experiments on EVTS show that PS3N achieves a strong performance–efficiency trade-off across all categories and modalities, outperforming standard baselines, with over 28× faster training than SNN baselines. DTMF further achieves superior multimodal fusion performance over existing methods.

Abstract:
This article presents a method for comprehensive 6-D estimation of tip wrench for generally deflected static elastic rods. Current methods for load estimation are restricted to estimating lateral (point or distributed) forces for (quasi-)planar deformation; estimation of tangential force and moment, i.e., the full 6-D wrench, remains largely unreliable due to the ill-posed nature of the problem. To address this challenge, this article begins by proposing a high-fidelity static rod model that leverages the Koopman operator theory. Building on this model and utilizing shape feedback, a computationally efficient three-step wrench estimator is proposed as follows: a Koopman unscented Kalman filter local moment observer, a static equilibrium solver, and a rod model propagator. Then, a 2-D wrench screw system, identified as the insensible wrench in the initial estimation, elucidates error sources and informs strategies to enhance accuracy by incorporating additional feedback, such as the rod tip material frame and base axial force. Ultimately, the framework delivers accurate 6-D tip wrench estimation with quantified uncertainty. Simulation and experimental evaluations validate its effectiveness, demonstrating mean errors of 53.14 mN (1.94%) and 2.65 mNm (7.18%) for a 159-mm-long Nitinol tube undergoing complex out-of-plane deformations, outperforming three replicated state-of-the-art methods. In addition, its applicability to complex continuum robots is demonstrated via load estimation on a parallel continuum robot.

Abstract:
In recent years, multicamera systems have been increasingly popular in various robotic tasks due to the growing demand for wider fields of view. A key challenge is to locate the system accurately and efficiently. The common approach involves solving the generalized perspective-n-points (gPnP) problem, which estimates the pose of a calibrated multicamera system using 3D–2D point correspondences. Unfortunately, existing methods still face a tradeoff between accuracy and efficiency. Specifically, the efficient null-space-analysis-based (NSA-based) method compromises accuracy due to its suboptimal NSA framework, while the accurate optimality-based methods incur heavy computational loads from handling sophisticated polynomial systems. To bridge this gap, we propose EA-GPnP, a novel NSA-based gPnP method that achieves superior accuracy and further enhances efficiency. The core of our approach consists of three targeted optimizations to the NSA framework, concerning its formulation, overall structure, and polynomial solving. In addition, we develop two specialized solvers tailored for polynomials with 6 quadratic equations in 3 or 4 unknowns (6Q3 or 6Q4), yielding up to four or nine solutions, respectively. Extensive real and synthetic experiments demonstrate that our method not only outperforms existing methods in numerical performance and noise resilience but also achieves the fastest speed. It achieves up to a 3.5× speedup over the most accurate state-of-the-art method, while concurrently delivering better performance.

Affiliations: META Fundamental AI Research (FAIR), Menlo Park, CA, USA; NVIDIA AI, Santa Clara, CA, USA; Harvard University, Cambridge, MA, USA; Berkeley AI Research (BAIR), Berkeley, CA, USA; School of Interactive Computing, Georgia Tech, Atlanta, GA, USA; Machine Learning Department in Carnegie Mellon University, Pittsburgh, PA, USA; Computer Science Department of the Technical University of Darmstadt, Darmstadt, Germany; Computer Science Department, Technical University of Darmstadt, Darmstadt, Germany

Abstract:
Learning from demonstrations, the field that proposes to learn robot behavior models from data, is gaining popularity with the emergence of deep generative models. Although the problem has been studied for years under names, such as imitation learning, behavioral cloning, and inverse reinforcement learning, classical methods have relied on models that do not capture complex data distributions well or do not scale well to large numbers of demonstrations. In recent years, the robot learning community has shown increasing interest in using deep generative models to capture the complexity of large datasets. In this survey, we aim to provide a unified and comprehensive review of the last year’s progress in the use of deep generative models in robotics. We present the different types of models that the community has explored, such as energy-based models, diffusion models, action value maps, and generative adversarial networks. We also present the different types of applications in which deep generative models have been used, from grasp generation to trajectory generation or cost learning. One of the most important elements of generative models is the generalization out of distributions. In our survey, we review the different decisions the community has made to improve the generalization of the learned models. Finally, we highlight the research challenges and propose a number of future directions for learning deep generative models in robotics.

Abstract:
In continuum robotics, real-time robust shape estimation is crucial for planning and control tasks that involve physical manipulation in complex environments. In this article, we present a novel stochastic observer-based shape estimation framework designed specifically for continuum robots. The shape state space is uniquely represented by the modal coefficients of a polynomial, enabled by leveraging polynomial curvature kinematics to describe the curvature distribution along the arclength. Our framework processes noisy measurements from limited discrete position, orientation, or pose sensors to estimate the shape state robustly. We derive a novel noise-weighted observability matrix, providing a detailed assessment of observability variations under diverse sensor configurations. To overcome the limitations of a single model, our observer employs the interacting multiple model (IMM) method, coupled with extended Kalman filters, to mix polynomial curvature models of different orders. The IMM approach, rooted in Markov processes, effectively manages multiple model scenarios by dynamically adapting to different polynomial orders based on real-time model probabilities. This adaptability is key to ensuring robust shape estimation of the robot’s behaviors under various conditions. Our comprehensive analysis, supported by both simulation studies and experimental validations, confirms the robustness and accuracy of our methods.

Abstract:
Passivity-based control is a cornerstone of control theory and an established design approach in robotics. Its strength is based on the passivity theorem, which provides a powerful interconnection framework for robotics. However, the design of passivity-based controllers and their optimal tuning remain challenging. We propose here an intuitive design approach for fully actuated robots, where the control action is determined by a “virtual-mechanism” as in classical virtual model control. The result is a robot whose controlled behavior can be understood in terms of physics. We achieve optimal tuning by applying algorithmic differentiation to ordinary differential equation simulations of the rigid body dynamics. Overall, this leads to a flexible design and optimization approach: stability is proven by passivity of the virtual mechanism, while performance is obtained by optimization using algorithmic differentiation.

Abstract:
A key open challenge in agile quadrotor flight is how to combine the flexibility and task-level generality of model-free reinforcement learning (RL) with the structure and online replanning capabilities of model predictive control (MPC), aiming to leverage their complementary strengths in dynamic and uncertain environments. This article provides an answer by introducing a new framework called Actor–Critic MPC. The key idea is to embed a differentiable MPC within an actor–critic RL framework. This integration allows for short-term predictive optimization of control actions through MPC, while leveraging RL for end-to-end learning and exploration over longer horizons. Through various ablation studies, conducted in the context of agile quadrotor racing, we expose the benefits of the proposed approach: it achieves better out-of-distribution behavior, better robustness to changes in the quadrotor's dynamics and improved sample efficiency. In addition, we conduct an empirical analysis using a quadrotor platform that reveals a relationship between the critic's learned value function and the cost function of the differentiable MPC, providing a deeper understanding of the interplay between the critic's value and the MPC cost functions. Finally, we validate our method in a drone racing task on different tracks, in both simulation and the real world. Our method achieves the same superhuman performance as state-of-the-art model-free RL, showcasing speeds of up to 21 m/s. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out-of-distribution behavior.

Affiliations: Institute of Robotics and Intelligent Systems, Dalian University of Technology, Dalian, Liaoning, China; School of Mechatronics Engineering and Automation, Shanghai University, Shanghai, China; Department of Mechanical and Materials Engineering, Queen’s University, Kingston, ON, Canada; School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China; Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada

Abstract:
Untethered capsules are capable of entering the gastrointestinal (GI) tract and collecting fluid samples containing microbial communities from specific locations, facilitating the study of chronic diseases. However, existing sampling capsules are designed for single-site sampling, making it challenging to gather samples from multiple targets. This article reports a magnetic-driven capsule for multiple sampling within the GI tract and an on-demand magnetic-triggered fluid sampling strategy. The capsule consists of a body, a magnetic-triggered negative pressure unit, and a reservoir unit. Composed of an elastic membrane and Magnet I, the negative pressure unit controls pressure change inside the capsule cavity on demand to pump the sample by switching the magnetic field, while the embedded Magnet I also enables real-time magnetic localization for regional targeting and position tracking. The reservoir unit integrates three sampling papers for fluid absorption, two waterproof layers that maintain contamination levels below 25% to ensure reliable multisite sampling, and a rotating arm embedded with Magnet II for posture adjustment of the sampling paper. The pumping and storage performance of the capsule was systematically evaluated and optimized. Meanwhile, the capsule, actuated by an external magnetic field, was evaluated for its active locomotion performance. Finally, the feasibility of using the capsule to perform active navigation and multitarget sampling in a porcine intestine was validated via ex vivo experiments.

Abstract:
Recent visual autonomous perception systems achieve remarkable performances with deep representation learning. However, they fail in scenarios with challenging illumination. While event cameras can mitigate this problem, there is a lack of a large-scale dataset to develop event-enhanced deep visual perception models in autonomous driving scenes. To address the gap, we present the event-enhanced autonomous perception (eAP) dataset, the largest dataset with event cameras for autonomous perception. We demonstrate how eAP can facilitate the study of different autonomous perception tasks, including 3-D vehicle detection and object time-to-contact (TTC) estimation, through deep representation learning. Based on eAP, we demonstrate the first successful use of events to improve a popular 3-D vehicle detection network in challenging illumination scenarios. eAP also enables a devoted study of the representation learning problem of object TTC estimation. We show how a geometry-aware representation learning framework leads to the best event-based object TTC estimation network that operates at 200 FPS. The dataset, code, and pretrained models will be made publicly available for future research.

Abstract:
Reliable communication in micro air vehicle (MAV) swarms is challenging in environments, where conventional radio-based methods suffer from spectrum congestion, jamming, and high power consumption. Inspired by the waggle dance of honeybees, which efficiently communicate the location of food sources without sound or contact, we propose a novel visual communication framework for MAV swarms using motion-based signaling. In this framework, MAVs convey information, such as heading and distance, through deliberate flight patterns, which are passively captured by event cameras and interpreted using a predefined visual codebook of four motion primitives: vertical (up/down), horizontal (left/right), left-to-up-to-right, and left-to-down-to-right, representing control symbols (“start", “end", “1", “0”). To decode these signals, we design an event frame-based segmentation model and a lightweight spiking neural network for action recognition. An integrated decoding algorithm then combines segmentation and classification to robustly interpret MAV motion sequences. Experimental results validate the framework’s effectiveness, which demonstrates accurate decoding and low power consumption, and highlights its potential as an energy-efficient alternative for MAV communication in constrained environments.

Abstract:
Planning paths that maximize information gain for robotic platforms has wide-ranging applications and significant potential impact. To effectively adapt to real-time data collection, informative path planning must be computed online and be responsive to new observations. In this work, we present Incremental and Adaptive Tree-based Information Gathering Using Informed Sampling (IA-TIGRIS), which is an incremental and adaptive sampling-based informative path planner designed for real-time onboard execution. Our approach leverages past planning efforts through incremental refinement while continuously adapting to updated belief maps. We additionally present detailed implementation and optimization insights to facilitate real-world deployment, along with an array of reward functions tailored to specific missions and behaviors. Extensive simulation results demonstrate IA-TIGRIS generates higher-quality paths compared to baseline methods. We validate our planner on two distinct hardware platforms: a hexarotor uncrewed aerial vehicle (UAV) and a fixed-wing UAV, each having different motion models and configuration spaces. Our results show up to a 38% improvement in information gain compared to baseline methods, highlighting the planner's potential for deployment in real-world applications.

Abstract:
We present Century Maze, a simulated environment for cooperative navigation of groups numbering over 100 robots with primarily visual observations. The underlying task of large-scale object search is designed to push the scalability and efficiency of decentralized multirobot systems relying on communication to overcome the limitations of separate units. To enable the training of robotic agents in large-scale visual embodied applications such as Century Maze, we propose two reinforcement learning (RL) components that facilitate the emergence of effective communication: Differentiable Interagent Belief Learning (DIABL), which provides a clear learning signal by optimizing the agents' beliefs about their goals and propagating gradients back over a differentiable communication channel to include the appropriate observers, and informative event replay (infer), which compensates for the scarcity of samples relevant to DIABL by maintaining a dedicated buffer for auxiliary learning. We demonstrate our combined method through an actor-critic neural network architecture encompassing a decentralized multiagent system of variable size with individual goal-based belief modules. Results on Century Maze show that our method is able to significantly improve performance through cooperation over 100 agents against comparable baselines and maintains its effectiveness in realistic conditions.

Abstract:
This article introduces a novel framework for reactive control in dual-arm cooperative robotic systems, addressing the significant challenges posed by high-dimensional, nonconvex optimization demands, intricate kinematic, multimodal distribution, the need for precise, and synchronized coordination. The core of our approach is a two-stage sampling-based model predictive control, which integrates k-means, dual quaternion, and null space into a cohesive system. This integration enhances the system's ability to manage complex coordination tasks, such as obstacle avoidance and holding a water cup, while mitigating risks associated with local optima and reducing control jitter. Our framework not only improves performance and reliability, but also overcomes the traditional computational bottlenecks inherent in dual-arm coordination. These advancements are validated through extensive simulations and experiments, demonstrating the robustness and efficiency of our proposed methodology.

Abstract:
Task-agnostic controllers for partial-assist lower limb exoskeletons aim to reliably mimic biological torque while seamlessly adapting to changing movement patterns. However, current approaches relying on hidden state estimators or neural networks lack explainability and safety guarantees, while force amplification methods risk instability with an inherent tradeoff between sensitivity and robustness to control inputs. Energy shaping control uses a kinematic model-based framework to provide predictable, stable assistance, though its traditional passive form limits biomimetic performance. Previous work relaxed the strict passivity requirements to improve biomimicry but reduced the stability guarantees. This article presents an optimization-based extension of the energy-shaping control framework that combines the stability benefits of energy shaping with the intuitive biomimicry of force amplification. Our framework enables controlled tradeoffs between sensitivity to changing human impedance and high performance through adjustable cost contributions of force amplification and model-based terms. We provide theoretical guarantees of closed-loop stability to an invariant set under human joint impedance control, supported by empirical validation of stability characteristics of an ankle exoskeleton under varying controller passivity constraints. A study of ten non-disabled participants using bilateral ankle exoskeletons demonstrates that the biomimetic controller reduced biological ankle torque by 19.1% across various activities of daily life.

Abstract:
Magnetic localization, owing to its immunity to line-of-sight occlusion and noncontact nature, is considered a promising technology for medical applications. While it is intuitive that localization performance degrades as the target moves farther from the sensor array, uncertainty analysis has long been overlooked, which is essential for quantifying localization quality. In this work, we present a pose estimation and uncertainty analysis framework on SE(3) for magnetic source localization using sensor arrays, which enables concise formulation of the problem and quantitative assessment of the results. The volume of the uncertainty ellipsoid is used to characterize localization confidence, while the surface shape in Cartesian space is used for visualization. This also provides insight into the effective workspace of the magnetic localization system prior to deployment. Guided by this analysis, we designed a movable magnetic sensor array to expand the limited sensing volume and mitigate localization uncertainty, thereby enhancing overall localization performance. Simulations and pose tracking experiments validate the effectiveness of this framework. By dynamically moving the sensor array to minimize the volume of the uncertainty ellipsoid, localization errors are reduced compared with static and projection-based strategies by 67.04%, 16.51% in position and 43.87%, 20.73% in orientation, respectively. Furthermore, phantom experiments on distal locking screw alignment and magnetic capsule endoscope tracking demonstrate the system’s capability in improving localization accuracy (reducing alignment errors by 71.88%) and expanding the effective workspace by several folds.

Abstract:
In this article, we present mixture of discrete-time Gaussian processes (MiDiGaPs), a novel approach for flexible policy representation and imitation learning in robot manipulation. MiDiGaP enables learning from as few as five demonstrations using only camera observations and generalizes across a wide range of challenging tasks. It excels at long-horizon behaviors, such as making coffee, highly constrained motions, such as opening doors, dynamic actions, such as scooping with a spatula, and multimodal tasks, such as hanging a mug. MiDiGaP learns these tasks on a CPU in less than a minute and scales linearly to large datasets. We also develop a rich suite of tools for inference-time steering using evidence, such as collision signals and robot kinematic constraints. This steering enables novel generalization capabilities, including obstacle avoidance and cross-embodiment policy transfer. MiDiGaP achieves state-of-the-art performance on diverse few-shot manipulation benchmarks. On constrained RLBench tasks, it improves policy success by 76 percentage points and reduces trajectory cost by 67%. On multimodal tasks, it improves policy success by 48 percentage points while improving sample efficiency 7-fold. In cross-embodiment transfer, it more than doubles policy success. We make the code publicly available.

Abstract:
Long-term autonomy requires robust navigation in environments subject to dynamic and static changes, as well as adverse weather conditions. Teach-and-repeat (T&R) navigation offers a reliable and cost-effective solution by avoiding the need for consistent global mapping; however, existing T&R systems lack a systematic solution to tackle various environmental variations, such as weather degradation, ephemeral dynamics, and structural changes. This work proposes LiDAR teach radar repeat (LTR^2), the first cross-modal, cross-platform LiDAR-teach-and-radar-repeat system that systematically addresses these challenges. LTR^2 leverages LiDAR during the teaching phase to capture precise structural information under normal conditions and utilizes 4-D millimeter-wave radar during the repeating phase for robust operation under environmental degradations. To align sparse and noisy forward-looking 4-D radar with dense and accurate omnidirectional 3-D LiDAR data, we introduce a cross-modal registration (CMR) network that jointly exploits Doppler-based motion priors and the physical laws governing LiDAR intensity and radar power density. Furthermore, we propose an adaptive fine-tuning strategy that incrementally updates the CMR network based on localization errors, enabling long-term adaptability to static environmental changes without ground-truth labels. We demonstrate that the proposed CMR network achieves state-of-the-art CMR performance on the open-access dataset. Then, we validate LTR^2 across three robot platforms over a large-scale, long-term deployment (40+ km over 6 months), including challenging conditions, such as nighttime smoke. Experimental results and ablation studies demonstrate centimeter-level accuracy and strong robustness against diverse environmental disturbances, significantly outperforming existing approaches.

Abstract:
Magnetic microrobots capable of autonomous operation hold significant promise for critical cell and small creature manipulation tasks, including trapping, transportation, sorting, etc. Although conventional microrobot navigation methods have shown notable performance, they often lack adaptability to novel environments. To address these limitations, we propose a learning-based framework for real-world microrobot navigation and dynamic target tracking. Our approach employs spatial-temporal transformer reinforcement learning (STTRL) with a deterministic velocity obstacle (DVO) that processes historical navigation states and virtual light detection and ranging scans to predict optimal actions. The key innovation lies in the model’s ability to extract and utilize contextual information from observation histories, enabling adaptive behavior even in previously unseen environments. Through large-scale model-free reinforcement learning trained in randomized simulation environments, our method achieves remarkable real-world performance with zero-shot transfer capability. Experimental results demonstrate superior navigation agility with an 89.8% success rate in the base environment, representing a 7.4% improvement over state-of-the-art algorithms. Furthermore, the method exhibits robust generalization in diverse unseen environments, validating its adaptability to different environmental characteristics.

Abstract:
Social robots are increasingly used in healthcare and education, but technological gaps, fears of human replacement, and moral or social beliefs can limit their acceptance. In collaborative settings, the activities to complete may influence users’ willingness to participate, raising the question of how moral and social attitudes shape human–robot interaction. This article studies the effect of the social judgment theory on social robotics to analyze which factors affect the users’ willingness to complete the robot’s requests. The methodology classifies the activities requested by the robot into assimilation (activities people typically accept), noncommitment (activities people usually reject), and the contrast (activities some people accept) groups. We conducted a user study with 63 participants interacting with the Mini social robot in a collaborative session where it requested some actions from the user. We analyze whether the kind of activity requested by the robot, its expressiveness, and demographic, moral, social, and robot factors influence the user behavior. Results show that the social judgment theory can be extended to social robotics since the kind of activity affects the user’s willingness to complete it. Besides, the results indicate that an expressive robot convinced users more than a nonexpressive robot and that participants who lied about their completed activities were more easily persuaded. We also found that participants with moderate knowledge of robotics completed more activities than those with low knowledge, and individuals with previous experience interacting with Mini were more likely to comply with its requests. However, demographic factors such as age or gender do not seem to influence robot persuasion despite previous studies suggesting they are important in human–robot collaboration.

Abstract:
Magnetic compression anastomosis (MCA) offers a promising solution for minimally invasive anastomosis surgery. However, current MCA schemes lack safe, real-time localization, and guidance for compression magnets, hindering surgeons' ability to control the compression magnets effectively in complex, unstructured endoluminal environments. To address these limitations, this article introduces the MagsL-HUD endoscopic system, a novel solution that enables multimagnetic six-degree-of-freedom (six-DoF) localization and head-up display (HUD) guidance within the endoscopic view (EV). Specifically, the system integrates a developed Endo-MagCap device with an orthogonal magnet configuration, along with a magnetic sensor array, to achieve real-time full-pose localization. An endoscopic camera model is incorporated for HUD visualization, enhancing intuitive interaction for surgeons' better-informed decisions. Eventually, the effectiveness of the MagsL-HUD endoscopic system is validated through laboratory experiments and ex vivo animal trials. The system demonstrates six-DoF tracking accuracy with average errors of 0.0070 m and 0.1437 rad, and 0.0071 m and 0.1721 rad in the designed trajectory cases for two compression magnets, respectively. Additionally, ex vivo porcine tests confirm the system's feasibility and applicability, successfully performing a stomach-colon MCA surgery with a final compression gap of approximately 0.00247 m. Further comparative studies demonstrate that the MagsL-HUD method has a compression success rate of 71.4% versus 42.9% of the non-HUD approaches in the designed tests. This work represents a significant step toward the clinical adoption of magnetic-assisted endoscopy for minimally invasive anastomosis surgeries, holding substantial practical significance for improving the safety and efficacy of MCA procedures in complex, unstructured endoluminal environments.

Abstract:
Underactuated robotic hands are extensively used in remote manipulation due to their ability to adapt to various object sizes and shapes. Their structural simplicity and small number of actuators required for operation make them highly versatile and responsive, which is crucial for effective teleoperation. In addition to grasping performance, haptic feedback, which integrates force and tactile sensing, is essential for dexterous manipulation. This study proposes a solution using fiber-optic tendons embedded with fiber Bragg gratings (FBGs), combining sensing and actuation to simultaneously perform power transmission, along with force and tactile sensing. Each finger employs a fiber-optic tendon with three FBGs: one measures tendon tension, and the other two at the fingertip detect contact force and temperature. The tendon is placed on the volar side of the finger and routed to an actuation module with a servomotor at the wrist for power transmission. This tendon enables finger flexion, while a passive extension mechanism with linear springs on the dorsal side facilitates extension. Experimental results demonstrate the feasibility of this approach, showing the hand’s multifunctional capabilities, including haptic feedback and power transmission, as well as its potential for teleoperation. This approach improves the robotic hand’s ability to provide real-time feedback, improving dexterity in remote manipulation.

Affiliations: Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, China; School of System Design and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China; College of Mechanical and Energy Engineering, Beijing University of Technology, Beijing, China; School of Engineering, University of Manchester, Manchester, U.K.; Department of Mechanical Engineering, City University of Hong Kong, Hong Kong, SAR, China

Abstract:
Despite extensive investigations into the multiuser haptic-enabled robotic system (M-Hers), achieving scalable control design in the presence of nonpassive human operators remains a key challenge. This is primarily due to the increasing complexity of stability conditions and interaction coupling as the number of operators grows. In this study, we address this challenge in two steps. First, we introduce the individual interaction environment (IIE) to isolate the passivity violations, which facilitates the independent control design for each human–robot subsystem, thereby enhancing the scalability with respect to the number of subsystems. Second, within the IIE framework, we identify passivity-violating components caused by partners' active behaviors and propose a novel augmented tank-based controller (ATBC) to guarantee passive IIE while maintaining high rendering accuracy. Specifically, the ATBC employs an energy-related power regulation strategy to enhance interaction safety and a time-varying control gain to mitigate the negative effects of power regulation on rendering fidelity. We validated the proposed method through collaborative haptic tasks on a customized M-Hers composed of three robots in four different scenarios. Comparative studies demonstrate that our approach effectively ensures IIE passivity in the presence of active human behaviors, while ensuring high reproducibility and achieving a favorable balance between passivity and rendering accuracy.

Abstract:
Soft robots’ ability to safely navigate complex environments motivates the development of algorithms for accurate environmental interaction assessment, enabling greater autonomy. Specifically, strain-based shape and force estimation of continuum robots with embedded soft sensors poses an open challenge mainly owing to continuous softness, anisotropic deformation, and nonlinear properties. Mathematical description of deformable soft bodies and accurate estimation of external forces are crucial for achieving controllable and intelligent behaviors of these robots. In this article, a kinetostatic strain-based modeling for rod-driven soft robots with embedded stretch sensors is proposed, which incorporates local strains, actuation variables, and external interactions. The strain model enables full shape estimation of the robot and prediction of strain variations in soft bodies. Building on this, we develop a force estimator based on predicted and measured sensor and actuator lengths to evaluate 3-D external forces, accounting for both orthogonal and tangential components relative to the backbone. Moreover, we introduce a methodology using a novel ellipsoid representation to handle tangential forces that may become insensitive in certain singular configurations. This estimator allows us to either disregard such forces when they do not influence deformation or estimate them when they become observable. Our simulations and experiments demonstrate how this approach can be used to analyze the robot’s configuration and successfully estimate external forces. Finally, it is demonstrated that when the continuum arm follows trajectories with higher strain sensitivity, tangential force estimation is significantly improved.

Abstract:
For unmanned aerial vehicle (UAV) swarms operating in large-scale unknown environments, lightweight long-range mapping is crucial for enhancing safe navigation. Traditional stereo cameras constrained by a short fixed baseline suffer from limited perception ranges. To overcome this limitation, we present flying collaborative stereo (flying co-stereo), a cross-agent collaborative stereo vision system that leverages the wide-baseline spatial configuration of two UAVs for long-range dense mapping. However, realizing this capability presents several challenges. First, the independent motion of each UAV leads to a dynamic and continuously changing stereo baseline, making accurate and robust estimation difficult. Second, efficiently establishing feature correspondences across independently moving viewpoints is constrained by the limited computational capacity of onboard edge devices. To tackle these challenges, we introduce the flying co-stereo system within a novel collaborative dynamic-baseline stereo mapping (CDBSM) framework. We first develop a dual-spectrum visual-inertial-ranging estimator to achieve robust and precise online estimation of the baseline between the two UAVs. In addition, we propose a hybrid feature association strategy that integrates cross-agent feature matching—based on a computationally intensive yet accurate deep neural network—with intra-agent, optical-flow-based lightweight feature tracking. Furthermore, benefiting from the wide baselines between the two UAVs, our system accurately recovers long-range covisible 3-D sparse points. We then employ a monocular depth network to predict up-to-scale dense depth maps, which are refined using accurate metric scales derived from the triangulated sparse points via exponential fitting. Extensive real-world experiments demonstrate that the proposed flying co-stereo system achieves robust and accurate dynamic baseline estimation in complex environments while maintaining efficient feature matching with resource-constrained computers under varying viewpoints. Ultimately, our system achieves dense 3-D mapping at distances of up to 70 m with a relative error between 2.3% and 9.7%. This corresponds to up to a 350% improvement in maximum perception range and up to a 450% increase in coverage area compared to conventional stereo vision systems with fixed compact baselines.

Abstract:
Perception in granular media remains challenging due to unpredictable particle dynamics. To address this challenge, we present SandWorm, a biomimetic screwactuated robot augmented by peristaltic motion to enhance locomotion, and sandworm tactile sensor (SWTac), a novel event-based visuotactile sensor with an actively vibrated elastomer. The event camera is mechanically decoupled from vibrations by a spring isolation mechanism, enabling high-quality tactile imaging of both dynamic and stationary objects. For algorithm design, we propose an IMU-guided temporal filter to enhance imaging consistency, improving masked signal-to-noise ratio (MSNR) by 24%. Moreover, we systematically optimize SWTac with vibration parameters, event camera settings, and elastomer properties. Motivated by asymmetric edge features, we also implement contact surface estimation by U-Net. Experimental validation demonstrates SWTac’s 0.2 mm texture resolution, 98% stone classification accuracy, and 0.15 N force estimation error, while SandWorm demonstrates versatile locomotion (up to 12.5 mm/s) in challenging terrains, successfully executes pipeline dredging and subsurface exploration in complex granular media (observed 90% success rate). Field experiments further confirm the system’s practical performance.

Abstract:
Current numerical methods for solving kinematic identification (KI) and inverse kinematics (IK) are limited in accuracy, convergence rates, and robustness, necessitating further enhancement. This article presents a modified Halley method for solving the KI and IK problems of serial robots based on the product of exponentials formula, achieving quintic convergence. Specifically, a general error model is first established based on exponential coordinates, and KI and IK are reformulated as root-finding problems. Next, the modified Halley method, which is proven to be a fifth-order method and incorporates a damping strategy, is proposed to resolve the singularity issue and enhance robustness. Subsequently, the Jacobian and Hessian matrices required for the proposed method are analytically derived based on the time differential of exponentials. Furthermore, highly simplified and explicit formulas for these matrices are presented for the IK problem. Simulations on serial robots with various configurations validate the proposed method’s accuracy, convergence rates, and robustness in solving KI and IK problems, as well as its advantages over the state of the art. In addition, experimental validation of KI on two physical robots further demonstrates the effectiveness of the proposed method. Our custom-written MATLAB and C++ codebases are made publicly available for download.

Abstract:
This article presents a novel sample-based algorithm, called C^, for real-time coverage path planning (CPP) of unknown environments. C^ is built upon the concept of a rapidly covering graph (RCG), which is incrementally constructed during robot navigation via progressive sampling of the search space. By using efficient sampling and pruning techniques, the RCG is constructed to be a minimum-sufficient graph, where its nodes and edges form the potential waypoints and segments of the coverage trajectory, respectively. The RCG tracks the coverage progress, generates the coverage trajectory, and helps the robot escape from the dead-end situations. To minimize coverage time, C^ produces the desired back-and-forth coverage pattern, while adapting to the traveling salesman problem-based optimal coverage of local isolated regions, called coverage holes, which are surrounded by obstacles and covered regions. It is analytically proven that C^ provides complete coverage of unknown environments. The algorithmic simplicity and low computational complexity of C^ make it easy to implement and suitable for real-time on-board applications. The performance of C^ is validated by, first, extensive high-fidelity simulations and, second, laboratory experiments using an autonomous robot. C^ yields near optimal trajectories, and a comparative evaluation with seven existing CPP methods demonstrates significant improvements in performance in terms of coverage time, number of turns, trajectory length, and overlap ratio, while preventing the formation of coverage holes. Finally, C^ is comparatively evaluated on two different CPP applications using, first, energy-constrained robots and, second, multirobot teams.

Abstract:
Twists and wrenches, as defined in screw theory, are commonly used tools to establish the input/output velocity relationship of linkages, where the output velocity is written as the product of a Jacobian matrix and the input velocity array. However, for parallel mechanisms and, more generally, mechanisms with arbitrary topologies, this Jacobian matrix is typically derived through at least one numerical matrix inversion. This step may introduce numerical errors, particularly due to inconsistent units across the matrix components. This article presents a new method for explicitly formulating the Jacobian matrix of a broad class of linkages. This class includes all planar and spherical linkages, but also parallel mechanisms whose end-effector may only translate or rotate about a fixed point, regardless of topological complexity. Central to this method is a new matrix called the twist matrix, constructed from appropriately chosen wrenches. Using this matrix, the Jacobian of any body within the mechanism, not just the moving platform, can be derived explicitly as a combination of elementary twist matrices, independently from the mechanism’s complexity.

Abstract:
Compliance control enables flexible joint robots (FJRs) to interact with unknown environments, but joint friction may significantly degrade control performance and backdrivability. While several model-free friction observers for FJRs have been studied in recent decades, current approaches still face challenges when the robot interacts with stiff environments. To tackle this, this article proposes a new friction observer based on an \mathcal L_1 adaptive framework. The main advantage of the proposed method is that it overcomes a fundamental tradeoff in the state-of-the-art method between accurate friction compensation and natural environmental interactions. Moreover, the proposed approach enables the use of extremely high gains, which yield several additional benefits. First, unlike the conventional methods, which require feedback of so-called nominal signals obtained through simulation, measured motor signals can be fed back into the controllers, leading to a simpler implementation. Second, we provide performance analysis showing that increasing the gain improves performance and results in near-zero steady-state error. Third, the observer’s performance can be adjusted using only a single parameter. Lastly, the numerical issues arising from extremely high gains are alleviated by employing a stable numerical method. The above theoretical findings are validated through simulations, and the effectiveness of the proposed approach is further evaluated with real-world experiments using both single- and seven-joint FJR systems. The results demonstrate that the proposed approach enables robots to interact with stiff environments more naturally, while achieving enhanced friction compensation performance.

Abstract:
We present an effective solution for computing locally optimal Reeds–Shepp distances and paths for kinematically constrained vehicles in environments represented as obstacle-rich 2-D Cartesian occupancy grids, addressing the reliance of current methods on discretization and approximation techniques. Our solution leverages a visibility-based marching architecture with continuous analytical expressions for propagating the Reeds–Shepp distance function. We introduce a model to identify reachable and unreachable regions for Reeds–Shepp vehicles, accompanied by a comprehensive representation of the Reeds–Shepp distance function in both cases. Unlike existing approaches, our method computes locally optimal distances and smooth paths globally without discretizing vehicle orientations, motion primitives, or the PDE, and without gradient descent (GD) backtracking, ensuring both accuracy and computational efficiency. Extensive simulations in various environments demonstrate effective improvements over state-of-the-art methods, particularly in complex obstacle-rich scenarios. To facilitate adoption, we provide an open-source solver implemented in C++.

Abstract:
In this article, we present a fast and accurate initialization algorithm of monocular visual-inertial odometry (MVIO). Our algorithm enables MVIO initialization within under-second motion time (0.45 s for Q1 and 0.7 s for Q2) without divergence for 1500 random runs in the EuroC-MAV and TUM-VI dataset. Our algorithm is based on almost-exact one-dimensional (1-D) approximation of the loosely coupled MVIO (LC-MVIO) problem, which is obtained by approximately decoupling optimization of vision rotation error from the original LC-MVIO optimization. Thanks to the efficiency of 1-D domain exploration, this approximated LC-MVIO is feasible without the initial guess, therefore can be used as an initializer of the original MVIO. Based on this 1-D formulation, we design our initializer with exhaustive correctness guarantee, a novel estimator guarantee we propose in this work, that certifies the absence of multiple, ambiguous solution modes with similar likelihood cost. Our algorithm is validated through Monte-Carlo simulation, real-world dataset experiment, and real-vehicle experiment. An open-source MVIO pipeline with the suggested initialization algorithm is also provided as a supplement.

Abstract:
Caterpillar-type robots are widely used for medium- and large-sized pipe inspections. However, existing prototypes smaller than 80 mm lack both an active variable-diameter capability and a contact force sensing function, which are crucial for safe and automatic exploration of unknown rigid/soft pipes (e.g., the colon). This study develops a variable-diameter caterpillar-type miniature robot (VCMR) featuring a small size of Φ34.6 mm × 41 mm, a large variable-diameter range of 34.6–89.6 mm, and an integrated contact force sensing function. The VCMR actively adapts to pipe diameter changes using contact force feedback, demonstrates high load capacity in both vertical and horizontal rigid/soft pipes, and traverses a 150-cm colon phantom with sharp bends at an average velocity of 3.07 ± 0.48 cm/s. It holds promise for exploring tiny variable-diameter rigid/soft pipes and delivering cargoes through such pipes.

Abstract:
While many animals exhibit impressive hopping capabilities, machines have struggled to match their performance. Current hopping robots face limitations in power density, energy efficiency, and control stability. Here, we present a parallel-elastic actuation mechanism with a reactive latch that optimizes energy transfer, enabling a legged robot to achieve hopping heights and continuity previously unattainable. This mechanism efficiently stores and releases energy, extending the actuation period over the aerial phase while minimizing stance time. Our robot achieves a maximum hopping height of 3.6 m, surpassing both human and animal records while demonstrating sustained, high-frequency hopping cycles with minimal power requirement. By integrating inertia-based onboard sensorimotor autonomy, we demonstrate stable, controlled hopping in environments without external aid. These results represent a step toward bridging the performance gap between biological and robotic locomotion, with potential to influence the design of future legged systems.

Abstract:
As learning-based methods for legged robots rapidly grow in popularity, it is important that we can provide safety assurances efficiently across different controllers and environments. Existing works either rely on a priori knowledge of the environment and safety constraints to ensure system safety or provide assurances for a specific locomotion policy. To address these limitations, we propose an observation-conditioned reachability-based (OCR) safety-filter framework. Our key idea is to use an OCR value network (OCR-VN) that predicts the optimal control-theoretic safety value function for new failure regions and dynamic uncertainty during deployment time. Specifically, the OCR-VN facilitates rapid safety adaptation through two key components: a LiDAR-based input that allows the dynamic construction of safe regions in light of new obstacles and a disturbance estimation module that accounts for dynamics uncertainty in the wild. The predicted safety value function is used to construct an adaptive safety filter that overrides the nominal quadruped controller when necessary to maintain safety. Through simulation studies and hardware experiments on a Unitree Go1 quadruped in agile planar navigation tasks, we demonstrate that the proposed framework can automatically safeguard a wide range of hierarchical quadruped controllers, adapts to novel environments, and is robust to unmodeled dynamics without a priori access to the controllers or environments—hence, “One Filter to Deploy Them All.”

Abstract:
Due to their continuous electromechanical deformation, rate-dependent viscoelasticity, and complex mechanical vibration, dynamic modeling and high-speed tracking control of dielectric elastomer actuators (DEAs) remain elusive, significantly limiting their working bandwidth. In this work, we propose a physics-informed token prediction (PITP) that enables accurate modeling of DEA dynamics and high-speed feedforward tracking control. The PITP framework consists of two key components: a physics-informed encoder and a dynamic decoder. The physics-informed encoder is designed based on a simplified equivalent linear model and trained through the hierarchical optimization training method, which embeds the global dynamic characteristics into tokens, minimizing the need for extensive data and training. Then, the dynamic decoder is developed by using these tokens as state-dependent parameters, capable of describing complex dynamic responses through the autoregressive solution. Finally, by taking advantage of the model’s reversibility, a direct inverse compensator is established to linearize the input–output relationship. Experimental results of several DEAs with different configurations and payloads demonstrate that, based on our PITP framework, the complex nonlinear dynamic responses of all DEAs can be precisely described and eliminated within their natural frequency, validating its generality and versatility. By leveraging fast modeling (< 30 min) and high-speed feedforward tracking control, our PITP framework may accelerate DEAs’ practical applications.

Abstract:
This article describes a solution for pick-and-place tasks that do not require high precision throughout the execution, using a robotized gantry crane. Two common issues that occur when using a crane are addressed, and solutions are provided. First, the lack of rotational controllability is overcome by designing a gripper that passively aligns itself with the handle of the payload using a simple, robust control strategy. Second, the active workspace is expanded by using controlled, dynamic motions, based on a variable-length pendulum model. Thus, the workspace is no longer limited to positions directly accessible from above, as is the case with quasi-static control methods. The robustness and effectiveness of the proposed solution was validated by a grasping, and a shelf insertion experiment. The robot was able to grasp the payload during all 48 trials. Failures were detected and recovery strategies were implemented. The handle could also be reliably ungrasped. During the shelf insertion, imperfect trajectory tracking caused a significant error during the unobservable part of the trajectory. Nevertheless, the actual placement position was always close to the desired position.

Abstract:
This article investigates the reliable navigation problem, which requires that the ego vehicle navigates itself to the destination with the maximized stochastic on-time arrival (SOTA) probability in a given uncertain topological transportation network. One distinctive characteristic of the SOTA problem explored in this article is the inherent uncertainty stemming from the underlying network's topology, i.e., some of the edges might become untraversable during navigation. To the best of our knowledge, almost all conventional SOTA solutions presume that the network topology remains the same during the ego vehicle's navigation process. However, when facing uncertainties in the network's topology, these algorithms may experience a significant performance degradation. To address the challenge of uncertain network topology, we first formulate the special reliable navigation problem into the variational Markov decision process framework, and then initiate a new reinforcement learning-based algorithm, namely sample efficient variational actor critic (SEVAC) as its solution. SEVAC comprises the variational policy gradient (VPG) module, which optimizes the vehicle's routing policy, and the masked temporal difference (MTD) module, which approximates the underlying routing policy's SOTA probability. Both modules are extended to their off-policy counterparts, namely off-policy VPG and off-policy MTD, to improve the algorithm's sample efficiency. SEVAC are compared with several conventional SOTA solutions as well as Canadian traveller problem algorithms in a variety of commonly used transportation test networks, and achieve the best overall SOTA performance. Furthermore, we validate SEVAC's application to real-world scenarios by navigating a physical robot in a self-constructed indoor environment as well as a real-world building environment with uncertain topology.

Abstract:
Inspecting large-scale industrial surfaces like aircraft fuselages for quality control requires precise, high-resolution 3-D geometry. Vision-based tactile sensors (VBTSs) offer high local resolution but require slow “press-and-lift” measurements for large areas. Sliding or roller/belt VBTS designs provide continuous measurement but face significant challenges: sliding suffers from friction/wear, while both are speed-limited by camera frame rates and motion blur. Thus, a rapid, continuous, high-resolution method is needed. In this article, we introduce a novel neuromorphic tactile roller sensor. It uses a modified event-based multiview stereo (EMVS) algorithm for 3-D reconstruction, leveraging high temporal resolution and motion blur robustness. This reconstruction is most effective for surfaces with distinct edges or sharp features, which are often the most critical for defect detection in industrial inspection tasks. We demonstrate 0.5 m/s scanning speeds with MAE below 100 \mathrm\mu m (11× faster than prior methods). A multireference Bayesian fusion strategy reduces MAE by 25.2% (versus EMVS) and mitigates curvature errors. We also validate high-speed Braille reading 2.6× faster than previous approaches.

Abstract:
Soft grippers leveraging bioinspired fin-ray effect (FRE) have garnered significant attention for handling geometrically complex objects across diverse scenarios. Despite their benefits, the fundamental mechanics governing morphological transformation under external loading remain insufficiently understood, hindering practical grasping performance. To address this gap, this article derives, presents, and validates a generalized theoretical model for FRE employing co-rotation concept, providing comprehensive insights for design optimization. First, the co-rotation modeling and force–displacement relationship are derived, eliminating the small deformation assumption while providing high-fidelity predictions with enhanced computational efficiency. Second, we present comprehensive insights into FRE gripper design, encompassing univariate ablation studies of four key design parameters and Bayesian-optimization-integrated structure optimization. Third, the approach is validated via comprehensive simulations and physical experiments. Simulation results demonstrate excellent agreements with finite-element analysis, with an average error of 6% and greatly enhanced efficiency, as fast as 0.035 s. Physical experiments under point, multipoint, and distributed contact validate the model's generalizability and robustness, with relative deviations within approximately 2%–6% under different loading and contact conditions. Finally, grasping evaluation with diverse daily objects demonstrates the framework's capacity to advance compliant robotic systems through synergistic mechanics modeling and data-driven optimization.

Abstract:
The application of autonomous robotics to close-contact healthcare tasks, such as the nasopharyngeal (NP) swab test, has clear potential for reducing infection risks to staff and improving efficiency. We propose a control system that performs the NP swab test with a collaborative manipulator arm, guided by an instrumented end-effector that measures force and visual information. We assume a scenario where the patient is unrestrained, with hardware general enough for other types of close-contact tasks. The system employs visual servo control to align the swab with the nostrils. A compliant joint velocity controller inserts the swab into the nasal cavity, following a planned trajectory adjusted with force feedback. Fuzzy logic systems are designed to detect when the swab reaches the nasopharynx and enforce safety criteria. We validate the system using a second robotic arm that holds a nasal cavity phantom and simulates natural head motions. Extensive experiments identify controller configurations capable of effectively performing the NP swab test even with significant head motion.

Abstract:
Amplitude tuning is an important strategy in animals employing traveling wave patterns, enhancing their adaptability to unstructured environments. This article proposes, for the first time, an amplitude tuning method that leverages the compliance of a soft corkscrew by twisting its ends. An amplitude tunable traveling wave robot (ATWBot) is developed, consisting of a soft corkscrew housed in a high-degree-of-freedom cage and driven by only two servos. The soft corkscrew and cage are monolithically 3D-printed. The ATWBot achieves a wide range of active amplitude tuning with passive compliance adaptation and can extend its morphology to a coiled configuration, enabling clamping and rolling. A comprehensive model is built for the twisted soft corkscrew geometry, proving that the robot’s speed is decoupled from amplitude variations during twisting. A genetic algorithm is used to optimize the soft corkscrew for achieving the fastest speed while matching the cage geometry. Experiments demonstrate that the combination of active amplitude tuning and passive body compliance enables the robot to adapt to unstructured terrains including slits, steps, gaps, converging tunnels, slopes, and swimming.

Affiliations: State Key Laboratory for Novel Software Technology, Nanjing University Suzhou Campus, Suzhou, China; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany; School of Artificial Intelligence, Beijing Institute of Technology, Beijing, China; School of Artificial Intelligence, Sun Yat-sen University, Guangzhou, China; College of Engineering, China Agricultural University, Beijing, China; School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China; Department of Computer Science and Technology, Tsinghua University, Beijing, China

Abstract:
Flexible spines are critical to the remarkable agility and speed of animals. Translating this biological advantage to quadruped robots presents a significant control challenge, particularly in coordinating the spine and limbs for maximal velocity. In this work, we utilize reinforcement learning (RL) to develop high-speed locomotion for a bioinspired mouse robot with a lateral flexible spine. The resulting controller achieves motor performance that demonstrably surpasses nonspined and model-based methods. More importantly, our analysis reveals the principles behind this performance: the emergence of two distinct motion templates. For high-speed walking, the robot learns a “whip-like” spinal oscillation to increase leg swing frequency, while for agile turning, it adopts a dynamic “bend-and-straighten” pattern. These findings demonstrate the capability of RL to not only generate high-performance controllers but also to produce emergent strategies that, upon analysis, reveal underlying principles of high-speed, spine-driven locomotion.

Abstract:
In this article, we propose a third-order, i.e., white-noise-on-jerk, Gaussian process trajectory representation (TR) framework for continuous-time motion estimation (ME) tasks. Our framework features a unified TR that encapsulates the kinematic models of both \mathrmSO(3)× \mathbb R^3 and \mathrmSE(3) pose representations. This encapsulation strategy allows users to use the same implementation of measurement-based factors for either choice of pose representation, which facilitates experimentation and comparison to make a better choice for the ME task. In addition, unique to our framework, we derive the kinematic models with the closed-form temporal derivatives of the local variables of \mathrmSO(3) and \mathrmSE(3), which so far has only been approximated based on Taylor expansion in the literature. Our experiments show that these kinematic models can improve the estimation accuracy in high-speed scenarios. All analytical Jacobians of the interpolated states with respect to the support states of the TR, as well as the motion prior factors, are also provided for accelerated Gauss–Newton optimization. Our experiments demonstrate the efficacy and efficiency of the framework in various ME tasks, such as localization, calibration, and odometry, facilitating fast prototyping for ME researchers. We release the source code for the benefit of the community.

Abstract:
The aspiration to replicate the capabilities of the human hand has driven innovations in the design of soft robotic hands. Despite these advancements, many existing designs of soft hands still lack effective in-hand vision and the ability for each finger to achieve active multidegree-of-freedom motion. This article proposes a cable-driven soft robotic hand that can achieve dexterous grasping and manipulation, vision-guided grasping, vision-based slip detection and compensation, as well as visually servoed in-hand manipulation. The hand has five soft fingers, each capable of independent flexion/extension motion and bidirectional ad/abduction motion. A red–green–blue-depth (RGB-D) camera is integrated into the palm of the soft hand to enable in-hand vision capability. Modeling of the soft hand is established to analyze its kinematics, statics, and manipulability. A series of experiments are conducted to demonstrate its dexterous grasping and manipulation capabilities on a variety of objects. Using 3-D point cloud data from the in-palm camera, an effective vision-guided grasping strategy is developed to grasp objects on a table. The in-hand vision also enables slip detection and compensation during grasping to maintain the grasp stability. Furthermore, a hierarchical, visually servoed controller is developed to perform closed-loop in-hand object manipulation. With its high dexterity and visual feedback capabilities, the soft hand will find important applications such as household object manipulation and food picking/sorting, and may also be used as a prosthetic hand or an auxiliary hand for humans.

Abstract:
Soft robots can revolutionize several applications with high demands on dexterity and safety. When operating these systems, real-time estimation and control require fast and accurate models. However, prediction with first-principles (FP) models is slow, and learned black-box models have poor generalizability. Physics-informed machine learning offers excellent advantages here, but it is currently limited to simple, often simulated systems without considering changes after training. We propose physics-informed neural networks (PINNs) for articulated soft robots (ASRs) with a focus on data efficiency. The amount of expensive real-world training data is reduced to a minimum—one dataset in one system domain. Two hours of data in different domains are used for a comparison against two gold-standard approaches: In contrast to a recurrent neural network, the PINN provides a high generalizability. The prediction speed of an accurate FP model is exceeded with the PINN by up to a factor of 467 at slightly reduced accuracy. This enables nonlinear model predictive control of a pneumatic ASR. Accurate position tracking with the MPC running at 47 Hz is achieved in six dynamic experiments.

Abstract:
Endoluminal surgery offers a minimally invasive option for early-stage gastrointestinal and urinary tract, but is limited by basic surgical tools and a steep learning curve. Robotic systems, particularly continuum robots, provide flexible instruments that enable precise, intuitive tissue resection in confined spaces, potentially improving outcomes. This article presents an integrated visual perception platform for a continuum robotic system in endoluminal surgery. Our objective is to leverage monocular endoscopic image-based perception algorithms to accurately identify the position and orientation of flexible instruments and measure their distances from surrounding tissues. This thorough understanding of continuum robots and surgical scenes enhances the robustness of robotic procedures. We introduce 2-D and 3-D learning-based perception algorithms and develop a physically realistic simulator that models the dynamics of flexible instruments. This simulator features a pipeline for generating realistic endoluminal scenes, enabling control of flexible robots in a realistic environment and substantial data collection. Using a continuum robot prototype, we conducted extensive evaluations, including module assessments and system-level evaluation of the perception platform. Results demonstrate that our perception algorithms significantly improve control of flexible instruments, reducing manipulation time by over 70% for trajectory-following tasks and enhancing the understanding of complex surgical scenarios, leading to robust endoluminal surgeries.

Affiliations: Intelligent Robotics Institute, School of Mechatronical Engineering, and the Key Laboratory of Biomimetic Robots and Systems, Ministry of Education, Beijing Institute of Technology, Beijing, China; Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, SAR, China; School of Artificial Intelligence, Beijing Institute of Technology, Beijing, China; Thrust of Robotics and Autonomous Systems, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China; Department of Micro-Nano Systems Engineering, Nagoya University, Nagoya, Japan

Abstract:
Magnetic soft millirobotic-grippers, equipped with agile pivot-walking motions and adaptive enveloping morph abilities, hold great promise for biomedical target acquisition tasks. However, deploying these millirobotic-grippers in highly constrained, disturbed, and variable terrains to realize multi-sequence target acquisition and transport tasks remains challenging. Here, we introduce a hierarchical multimodal motion control method for pivot-walking magnetic millirobotic-grippers, which enhances adaptive locomotion capabilities and enables high-precision motion control, facilitating autonomous target acquisition in complex terrains. The millirobotic-gripper utilizes a centrosymmetric three-pivot design, enabling adaptive soft enveloping deformation and robust multimodal locomotion. A hierarchical control architecture is proposed, comprising: first, an upper level event-based finite state machine planner that dynamically orchestrates transitions between motion modes according to environmental feedback and task-specific conditions; and second, a lower level sliding mode controller integrated with Gaussian process-based gait parameter optimization, significantly improving motion accuracy and robustness against environmental disturbances. Experimental results demonstrate that our proposed method allows millirobotic-grippers to efficiently navigate morphing tunnels, leap across gaps exceeding three times their body length, accurately follow arbitrary paths with errors less than 5% of their body length, and reliably perform grasping and transport of three types of targets. Furthermore, the biomedical application potential of our system is initially validated through an ex vivo porcine gastrointestinal tract experiment with ultrasound guidance.

Abstract:
The use of cables in aerial manipulation offers a lightweight and flexible alternative to rigid grasping mechanisms. However, achieving autonomous tying and secure transportation of objects with cables remains a significant challenge. In this work, we present a novel method for autonomously securing and transporting objects using multilayer hitches formed in midair by a team of aerial robots. Building upon prior work on polygonal hitch formation, we extend the framework to include layered cable interlacing that increases frictional grip and enables secure object grasping. We introduce two new manipulation actions: multilayer tying and autonomous object release, completing the pipeline for aerial grasping and transportation. We develop a capstan-based analytical model that establishes an exponential scaling law for the effect of cable layers and provides a conservative guideline. The formation algorithm operates in parallel, ensuring scalability to large teams with constant execution time. We validate our system through simulation and hardware experiments, demonstrating fully autonomous object tying, lifting, and releasing using cables alone without human intervention.

Abstract:
Fast and safe motion is crucial for the successful deployment of physically interactive robots. Parallel robots (PRs) offer the potential for higher speeds while maintaining the same energy limits due to their low moving masses. However, they require methods for contact detection and reaction while avoiding singularities and self-collisions. We address this issue and present SafePR—a unified approach for the detection and localization, including the distinction between collision and clamping to perform a reaction that is safe for humans and feasible for PRs. Our approach uses information from the encoders and motor currents to estimate forces via a generalized-momentum observer. Neural networks and particle filters classify and localize the contacts. We introduce reactions with redundancy resolution to avoid self-collisions and type-II singularities. Our approach detected and terminated 72 real-world collision and clamping contacts with end-effector speeds of up to 1.5 m/s, each within 25–275 ms. The forces were below the thresholds from ISO/TS 15066. By using built-in sensors, SafePR enables safe interaction with already assembled PRs without the need for new hardware components.

Abstract:
Semantic segmentation based on RGB-Thermal (RGB-T) data fusion has made great progress in the field of autonomous driving. However, in this article, we find that most existing RGB-T semantic segmentation methods exhibit inferior performance in image central regions, in which segmentation performance is critical for driving safety. We refer to this phenomenon as spatial bias. To discover the reason for spatial bias, we design a series of experiments. The results challenge the common knowledge that more training data lead to better segmentation performance, and reveal a close causal relationship between segmentation performance and object complexity as well as image quality. We also provide a theoretical interpretation for the causal relationship using information theory and feature space analysis. Based on the findings, we propose a Gaussian-guided regional balancing masking method to balance segmentation performance across different image regions. Moreover, we introduce a spatial-weighted loss to further enhance the overall segmentation performance. Experimental results on two public datasets demonstrate the effectiveness of our method in mitigating spatial bias and improving balanced performance.

Abstract:
Advanced dexterous manipulation requires identifying and controlling multiple simultaneous contacts, including interactions with compliant objects where deformations are large. Raw optical tactile images are information rich but lack calibrated physical meaning, limiting cross-sensor use and real-world deployment. In this article, we present TensorTouch, a calibration framework that combines finite element analysis and learning to infer dense deformation and stress/force fields from a single tactile image. Across real sensors, TensorTouch achieves contact localization errors under 1.29 mm and mean force errors under 0.139 N per axis. In a multiobject selective grasp task with two simultaneously contacted objects (including identical cables), the system achieves up to 90.0% success. We further demonstrate robustness under repeated loading, yielding 38.295 dB peak signal-to-noise ratio between the initial tactile image and the image after 20 000 contacts. The learned model runs in real time at 95 Hz on an RTX 5090, proving to be suitable for contact-rich, dexterous manipulation.

Abstract:
The integration of robots into human environments is advancing rapidly, driven by the demand for systems that combine robot accuracy and repeatability with human flexibility and adaptability. In this context, human-centered manipulation applications must address uncertainties arising from human, robotic, and environmental factors. As a result, effective robotic manipulation requires both accurate pregrasping motions and precise grip strength control, especially in tasks where robotic devices are remotely controlled to grip objects with fine, desired, and adjustable grip force. The present work tackles the challenges posed by uncertainties and nonideal conditions in surface electromyography-driven human-in-the-loop (HITL) robot hand control applications. In this regard, a novel probabilistic shared autonomy framework for fine grip strength regulation is introduced, leveraging hidden Markov models applied to tactile data to encode the HITL grasping action into proper, probabilistically consistent phases. These phases are then exploited to modulate the level of shared autonomy between the human operator and the robot hand, enabling precise control over grip strength. The presented shared autonomy framework was evaluated under multiple experimental conditions, testing grip force regulation performance with a group of 10+10 intact-limb participants and a participant with amputation in both static (fixed hand–object configuration) and dynamic (pick-and-place and recipe preparation) grasping tasks, with differentiated goals inspired by real-world requirements. Moreover, to explore generalizability, experiments were conducted with both anthropomorphic and industrial robotic hands, properly equipped with tactile sensors. Experimental outcomes are supported by statistical analysis and show, for the considered sample, the effectiveness of the proposed shared autonomy control architecture in achieving fine, smooth, and controllable grip strength regulation with respect to the baseline case in absence of our approach.

Abstract:
Object-centric motion generation (OCMG) plays a key role in a variety of industrial applications—such as robotic spray painting and welding—requiring efficient, scalable, and generalizable algorithms to plan multiple long-horizon trajectories over free-form 3D objects. However, existing solutions rely on specialized heuristics, expensive optimization routines, or restrictive geometry assumptions that limit their adaptability to real-world scenarios. In this work, we introduce a novel, fully data-driven framework that tackles OCMG directly from 3D point clouds, learning to generalize expert path patterns across free-form surfaces. We propose MaskPlanner, a deep learning method that predicts local path segments for a given object while simultaneously inferring “path masks” to group these segments into distinct paths. This design induces the network to capture both local geometric patterns and global task requirements in a single forward pass. Extensive experimentation on a realistic robotic spray painting scenario shows that our approach attains near-complete coverage (above 99%) for unseen objects, while it remains task-agnostic and does not explicitly optimize for paint deposition. Moreover, our real-world validation on a 6-DoF specialized painting robot demonstrates that the generated paths are directly executable and yield expert-level painting quality. We additionally provide empirical evidence that our approach remains complementary to downstream trajectory optimization methods, and applicable to tasks beyond spray painting.

Abstract:
Inconsistency issue is one crucial challenge for the performance of extended Kalman filter (EKF)-based methods for state estimation problems, which is mainly affected by the discrepancy of observability between the EKF model and the underlying dynamic system. In this work, some sufficient and necessary conditions for observability maintenance are first proved. We find that under certain conditions, an EKF can naturally maintain correct observability if the corresponding linearization makes unobservable subspace independent of the state values. Based on this theoretical finding, a novel affine EKF (Aff-EKF) framework is proposed to overcome the inconsistency of standard EKF by affine transformations, which not only naturally satisfies the observability constraint but also has a clear design procedure. The advantages of our Aff-EKF framework over some commonly used methods are demonstrated through mathematical analyses. The effectiveness of our proposed method is demonstrated on three different simultaneous localization and mapping applications and one 3-D cooperative localization problem. Specifically, following the proposed procedure, the naturally consistent Aff-EKFs can be explicitly derived for these problems. The consistency improvement of these Aff-EKFs are validated by Monte Carlo simulations. The MATLAB code of the algorithm is made publicly available.

Abstract:
Designing an anthropomorphic robotic hand with few actuators while replicating the dexterous motion capabilities of the human hand remains a challenging problem. In this work, a kinematic synergy analysis method is proposed to identify joints with high motion independence and to quantify the synergistic motion characteristics of joints with strong motion dependence. Based on these findings, design principles for intrafinger and interfinger coupling–compliance mechanisms are developed, enabling the mechanical embodiment of joint kinematic synergies within and between fingers. Furthermore, a design principle for anthropomorphic robotic hands that reduces actuator count while retaining diverse motion functions is formulated. This principle is embodied in a 12-actuator, 21-joint prototype named SX-Hand, featuring a flexible thumb and an articulated palm. Experimental evaluations demonstrate that SX-Hand achieves the maximum score in the Kapandji test, performs all 33 grasp types defined in the comprehensive GRASP taxonomy, and executes dexterous in-hand manipulations. The proposed approach provides an effective conceptual and technical route for reproducing rich motion functions with reduced actuation, offering valuable guidance for the development of efficient anthropomorphic robotic hands and other bionic systems.

Abstract:
Despite decades of research in magnetic resonance imaging (MRI)-compatible robotic technologies, the existing MRI-safe needle drivers rarely feature simultaneously high compactness, large insertion force, and motion versatility, all of which are critical to facilitate clinical translation in intraoperative MRI-guided percutaneous procedures. This article presents an MR-safe needle driver that for the first time offers all these desired qualities. It measures only 2.2 × 5.3 × 3.8 cm (length × width × height), facilitating its adoption in in-bore skull-mounted or body-mounted MRI-guided procedures. It is driven by a single hydraulic bellows-based actuator, which provides good water sealing, smooth motion, and high expansion ratio, and a preclamped gripper design that offers large insertion force (> 10 N). A compact passive rotation mechanism, together with a motion decoupling and switching mechanism, was introduced, allowing the needle to move with three motion types: independent translation, translation with passive rotation, and independent rotation. The passive rotation motion reduces needle deformation and tissue resistance during insertion, while the combination of independent translation and independent rotation enables flexible needle steering. An adaptive elastic fuzzy controller was further developed to address the limitation of encoder resolution and the unknown nonlinear dynamics of the hydraulic bellows-based actuator to provide accurate needle tip control (< 0.06 mm overshoot and 0.17 mm root-mean-square repeatability error). The compact and highly versatile needle driver design with its high motion accuracy represents a significant breakthrough in MRI-safe robotic technology, facilitating in-bore real-time MRI-guided interventions that could save MRI operation time, improve workflow efficiency, and enable more autonomy integration.

Abstract:
Lower limb exoskeletons (LLE) have proven to be effective tools in rehabilitation for diminishing gait impairments. However, modern LLE control systems based on the assistance-as-needed concept face challenges in providing appropriate interaction torques to the user. This is especially true when it is necessary to make the exoskeleton transparent, i.e., with zero user–robot interaction torques (\tau _\textUR), even during walking speed variations, when no assistance is required. To address these limitations, in this work we designed a novel transparent control system that utilizes a baseline torque controller, which generates a torque profile based on \tau _\textUR from previous strides, and drives the actuators, leading to a lower \tau _\textUR, along with a synchronization layer comprising a pool of adaptive oscillators to estimate the user's gait phase and adapt the controller to different walking speeds. In addition, a zero-torque controller was used in parallel to the main torque controller to enhance users' volitional control and balance. The proposed controller was experimentally tested on a six-degrees of freedom LLE with eight healthy participants who walked for 200 gait strides at three walking speeds. The baseline torque controller was shown to adapt to different walking speeds and successfully improve the exoskeleton transparency in terms of user–robot interaction, reducing the average \tau _\textUR by 40% of the entire exoskeleton and 70% of the hip joint at 0.8 m/s, and gait kinematics compared to our prior controller based on a zero-impedance model.

Abstract:
In this article, a new model for 3D motion planning, applicable to aerial vehicles, is proposed to connect an initial and final configuration subject to pitch rate and yaw rate constraints. The motion planning problem for a curvature-constrained vehicle over the surface of a sphere is identified as an intermediary problem to be solved, and it is the focus of this paper. In this article, the optimal path candidates for a vehicle with a minimum turning radius r moving over a unit sphere are derived using a phase portrait approach. We show that the optimal path is CGC or concatenations of C segments through simple proofs, where C = L, R denotes a turn of radius r and G denotes a great circular arc. We generalize the previous result of optimal paths being CGC and CCC paths for r \in (0, \frac12 ]\bigcup \lbrace \frac1\sqrt2\rbrace to r \leq \frac\sqrt32 to account for vehicles with a larger r. We show that the optimal path is CGC, CCCC, for r \leq \frac1\sqrt2, and CGC, CC_\pi C, CCCCC for r \leq \frac\sqrt32. Additionally, we analytically construct all candidate paths and provide the code in a publicly accessible repository.

Abstract:
Work-related musculoskeletal disorders (WMDs) affect a high percentage of operators performing repeated weight lifting and load carrying in industrial scenarios. Since upper limb muscles are affected in the process, the assistance provided by upper body exoskeletons is increasingly needed to prevent WMDs and their consequent cost to the health system. This article presents the evaluation of Flexos, a portable, bilateral, shoulder exoskeleton prototype designed to assist logistic and industrial operators in performing occupational tasks. An in-lab assessment was conducted on twelve healthy subjects–9 males, 3 females–to evaluate Flexos capability in assisting the user during the execution of isometric, dynamic, and carrying-load tasks. Different metrics were extracted from time-series signals to assess the effort related to five targeted muscles surrounding the shoulder complex. Despite the limited experimental size and the prototypal level of the device, Flexos managed to cover almost all the shoulders range of motion–89.2% flexion/extension, and 88.4% internal/external rotation–and to globally decrease muscular activity in occupational activities, particularly when isometric contractions are required for a prolonged time, with average reductions of −27.2% for the static task, −18.6% for the dynamic task and −23.4% for the carrying-load task.