arXiv Papers with Code in Computer Science (October 2026)
Authors:Yuhan Liu, Lianhui Qin, Shenji Wan
Abstract:
Training generalizable embodied agents requires realistic, varied, and simulation‑ready indoor environments. Language‑guided methods support semantic planning and object‑level reasoning, but precise spatial grounding remains difficult. Visual generative models offer rich priors over plausible arrangements, yet their outputs require geometric reasoning to become physically valid 3D scenes. We introduce SceneBloom, an agentic framework that combines visual layout sampling with contract‑guided 3D compilation. A hierarchical sampler uses diffusion priors to generate furniture arrangements within functional zones. A compiler then uses relations extracted from each proposal to guide geometric fitting and local pose correction. Across five room types, SceneBloom attains the highest visual quality and physical stability among the compared methods and is the only method with no detected mesh collisions and out‑of‑bounds objects. Its layouts are also closer to the distribution of real rooms than those of all compared baselines. Object navigation policies trained in these realistic scenes achieve a higher success rate in held‑out real scans than those trained on an equal number of real training scans. Code is available at https://github.com/Tinaliu0123/scenebloom.
Authors:Philipp Lindenberger, Paul-Edouard Sarlin, Ondrej Miksik, Marc Pollefeys
Abstract:
To estimate correspondences across images, existing approaches match either sparse interest points, which is highly efficient but limited by repeatability, or dense pixel grids, which is robust but expensive. Furthermore, these approaches are trained with only matching pairs and thus often fail to reject unrelated pairs that exhibit visual symmetries, which is equally important in applications like Structure‑from‑Motion (SfM). We bridge these gaps by introducing FlexGlue, a neural network for asymmetric sparse‑to‑dense matching and disambiguation. Given two images and coordinates of points in the first image, FlexGlue estimates the coordinates of the corresponding points in the second image while simultaneously rejecting the pair if non‑matching. By searching for a limited set of correspondences among dense pixels, our approach is both fast and effective at both matching and disambiguation: it combines the sub‑pixel accuracy of dense matching with the speed of sparse matching, while rejecting incorrect pairs with high precision. Experiments on multiple datasets show that FlexGlue pushes the Pareto frontier in relative pose estimation and significantly outperforms existing approaches for SfM and visual localization. FlexGlue is publicly available at https://github.com/cvg/flexglue .
Authors:Amber Xie, Jiawei Fu, Xiaofeng Guo, Carmelo Sferrazza, Guanya Shi, Haozhi Qi
Abstract:
Collecting high‑quality demonstrations remains a central bottleneck for learning dexterous manipulation. Teleoperation records executable robot actions, but it requires a complete robot setup and places the burden of coordinating many finger joints on the operator. Human demonstrations are easier to collect, but their motions must be estimated and retargeted to the robot, and they do not directly provide robot actions. In this paper, we introduce DexPort, a portable data‑collection interface for high‑DoF robot hands. It uses the same hand hardware for collection and deployment, avoiding a hand embodiment gap, and records both commanded joint targets and measured joint states. Our key observation is that a small set of skills can support a wide range of dexterous tasks. During collection, the operator moves the hand‑held interface to specify the robot wrist motion and selects or adjusts skills that coordinate the finger motion. This removes the need to control each joint independently while retaining behaviors such as precise grasping and in‑hand manipulation. We collect demonstrations across several real‑world tasks and in unstructured environments without a robot arm. We then use behavior cloning to train visuomotor policies directly on the recorded wrist motions and hand commands, without skill labels. We evaluate across 4 tasks: pick‑and‑place with diverse objects, screwing with in‑hand tool use, drawer stowage which requires simultaneous grasp and pulling, and a bimanual scanning task. We collect over 2,500 demonstrations with our portable device, in both laboratory and in‑the‑wild settings. Across our tasks, we see an improvement from 11% to 64% when using recorded robot commands instead of states, and an improvement from 36% to 78% for task success when using in‑the‑wild data.
Authors:Zimo He, Tianyi Bian, Zizhong Yang, Hongjie Li, Nan Jiang, Jieming Cui, Baoxiong Jia, Yizhou Wang, Siyuan Huang, Yixin Zhu
Abstract:
Learning humanoid control from monocular human videos offers a scalable way to acquire diverse whole‑body behaviors. Yet, existing reconstruction‑retargeting‑tracking pipelines are limited in system‑level scalability by separately optimized intermediate stages and cannot directly optimize reference motion executability using downstream tracking feedback. We present ViT‑H, a video‑to‑humanoid control framework following a generation‑and‑tracking paradigm. ViT‑H directly generates humanoid reference motions from videos with a diffusion model, bypassing human motion reconstruction and retargeting, and uses motion‑tracking outcomes to post‑train the generator with direct preference optimization. We further introduce an autoregressive variant that enables real‑time humanoid video teleoperation with sub‑400 ms latency. Extensive experiments on the Unitree G1 demonstrate that ViT‑H outperforms state‑of‑the‑art reconstruction‑retargeting‑tracking pipelines in both motion quality and executability. We further show that ViT‑H scales favorably with model size and with both pre‑training and post‑training data scale, while motion‑tracking feedback consistently improves executability across different motion trackers.
Authors:Bernardo Perrone Ribeiro, Erkut Akdag, Egor Bondarev
Abstract:
Few‑shot industrial anomaly detection requires modeling category‑specific normality from only a handful of defect‑free images. Common approaches tie inference to stored exemplars, impose rigid linear structure, or risk reproducing anomalous content. Flownomaly instead casts the problem as normal‑counterpart recovery in frozen DINOv2 feature space. Given normal supports, a velocity field is learned through conditional flow matching over frozen patch tokens to transport Gaussian noise toward normal features. Along straight paths, a single recovery step using one evaluation of the optimal field yields the posterior mean of the clean normal token without trajectory integration. The operating time governs how strongly recovery modifies a query token. With no anomalous validation samples available for tuning, a normal‑only Jacobian gain‑to‑noise criterion selects this time from the supports alone. Squared residuals between query tokens and their recovered counterparts yield anomaly signals for both dense localization and image‑level detection. Across 1‑, 2‑, and 4‑shot benchmarks on MVTec‑AD and VisA, together with sample‑shot evaluations on Real‑IAD, Flownomaly achieves state‑of‑the‑art performance against recent category‑specific methods. In the 1‑shot setting, image‑ and pixel‑level AUROC reach 97.8% and 98.0% on MVTec‑AD and 95.3% and 98.7% on VisA, with a 2.1‑point image‑level AUROC improvement over the strongest baseline on VisA. Code is available at https://github.com/b‑rbmp/Flownomaly.
Authors:Xinrui Li, Pawan Sasanka Ammanamanchi, Siddharth Bhat, Rianna Jamdar, Yongxi Lin, Stella Biderman
Abstract:
Statement autoformalization translates mathematics from natural language into a proof assistant such as Lean 4, and models now do it at scale. As provers get stronger, the burden of correctness shifts from the proof to the statement, that is, to whether the formal statement faithfully translates its source. Given a trusted reference formalization, the natural test is logical equivalence, and it proves too much. P \leftrightarrow Q holds trivially for any two statements P and Q that the library proves, because a prover can prove each separately and pass through P \leftrightarrow \mathrmTrue \leftrightarrow Q. The stronger the prover, the more unrelated pairs pass. Earlier methods restrict the prover and check that each proof names the other statement. We introduce SemantEq, a more principled relation that instead requires each statement to turn into the other only through a chain of allowed steps, such as logical rearrangement and rewriting with the library's own equations. Our programs judge a proof by the proof term Lean checks, not by its tactic script. SemantEq‑Eval, a search without a model, reaches 100% precision and 65.6% recall on the ProofNetVerif test split under our adjudicated labels, against 95.8% and 61.8% for BEq+, and an 8.9‑fold increase in its CPU time adds at most three pairs. A language model finds more proofs, but under two of four prompts it proves both directions of about two in five of the non‑equivalent pairs. SemantEq‑ProofAudit audits each model‑written proof under a policy derived from the relation and accepts none of the 104 non‑equivalent pairs, and it raises recall on 407 equivalent pairs from 65.4% to 81.1% under the best prompt. Separating the generation of equivalence proofs from their acceptance lets a stronger prover raise recall without changing what an acceptance means. We release code and data at: https://github.com/hlxr89/SemantEq
Authors:Rahul Golder, Bimol Nath Roy, MM Faruque Hasan
Abstract:
Physics‑constrained machine learning (PCML) integrates domain knowledge, first‑principles, and physical constraints into AI/ML models. In this work, we present a step‑by‑step tutorial of a modular PCML framework, KKT‑HardNet, and illustrate how it could be used for hybrid modeling, process simulation, optimization and model predictive control in a single platform. KKT‑HardNet combines fast inference capabilities of neural networks with guaranteed satisfaction of algebraic constraints through differential projection. It can handle heterogeneous data and enforce both linear and nonlinear equality and inequality constraints. The modular design supports custom neural‑network backbones and user‑defined projection mechanisms for PCML. To enable efficient end‑to‑end training, KKT‑HardNet exploits implicit function theorem to backpropagate through the projection layer. This avoids computationally expensive automatic differentiation. This work also introduces an exact reduced‑order Fischer‑Burmeister reformulation and a primal‑dual interior‑point method as an alternative projection solve strategy. We also provide a diverse set of benchmarking problems for PCML in engineering applications. KKT‑HardNet achieves orders of magnitude better performance in terms of both constraint satisfaction and inference time. The KKT‑HardNet code and the benchmark datasets are available at https://github.com/souls‑tamu/kkt‑hardnet .
Authors:ZhiHong Liu, Shuyao Gao
Abstract:
Pipelines that combine large language models (LLMs) with static analysis let the model consume the analyser's output as evidence, but whether the model distinguishes true from false evidence, and whether its scrutiny depends on what the evidence claims, has not been measured. We built an intraprocedural analyser for C/C++ that produces proof obligations and audited vulnerable and patched functions from PrimeVul under six evidence conditions that differ only in the notes, including fabricated assurances that a missing check exists and fabricated alarms that an added check is missing. The study covers 916 pairs in four disjoint sets, eleven model configurations from four vendors and 16,572 audits; no model is trained or fine‑tuned. True evidence barely improved accuracy (+0.02). False evidence acted asymmetrically: relative to true notes about the same operations, false assurances lowered the detection of real vulnerabilities by up to 19 points, whereas false alarms never significantly lowered accuracy and were disputed far more often. An internally registered replication on seven new models confirmed that false assurances move verdicts towards their claim more than false alarms do (mean asymmetry +0.16, 95% CI +0.10 to +0.24). Instructions in the audit prompt did not reliably reduce deference; the best one failed to replicate. Verifying each claim in a separate call and showing the refuted claims raised accuracy under false assurances by 12 points in a pre‑specified test (p=0.002) without loss under true evidence; for a second model the gain was smaller and not significant. Pipelines should therefore verify a tool's guarantees before the model relies on them, and studies should measure deference in a direction‑aware way.
Authors:Marek Masiak, Andrzej Szablewski, Nicola Cancedda, Constantin Venhoff
Abstract:
Fine‑tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples. Difficult to localise and remove, backdoors can persist through safety training. While parameter‑efficient fine‑tuning methods capture the full parameter changes introduced during training, they are not easily interpretable. We present the full method of TopKLoRA, extending the preliminary proposal of Masiak et al. (2025), an interpretable‑by‑design LoRA variant in which only the k largest adapter latents are active at each token. Unlike SAEs, TopKLoRA is trained purely on downstream loss, where optimisation pressure incentivises learning functionally causal latents. We demonstrate this sparse LoRA adapter in a sleeper‑agent backdoor case study where we create model organisms by fine‑tuning gemma‑2‑2b, Qwen2.5‑1.5B, and Qwen2.5‑7B on a poisoned instruction‑following dataset. For each organism, we identify and ablate a circuit that is both necessary and sufficient to implement the backdoor. Furthermore, our new fine‑tuning method produces sparse circuits that are up to an order of magnitude smaller than a dense LoRA baseline, allowing surgical ablation of the malicious capability without sacrificing instruction‑following performance. Finally, we propose a technique to construct model organisms with localised circuits using TopKLoRA latents and gradient‑routing, which we use to validate the findings of our circuit search algorithm. We consider these findings as a step toward interpretable‑by‑design architectures that provide faithful units of analysis without compromising model performance. We share our code: https://github.com/interpretable‑finetuning/TopKLoRA.
Authors:Joao P. A. Dantas, Andre N. Costa, Diego Geraldo, Marcos R. O. A. Maximo, Takashi Yoneyama
Abstract:
This work introduces PoKER, a novel probabilistic model engineered to optimize missile launch effectiveness in air‑to‑air scenarios, specifically within Beyond Visual Range (BVR) air combat. Unlike conventional Weapon Engagement Zone (WEZ) models that delineate zones based on static distances such as maximum, minimum, and no‑escape ranges, PoKER applies machine learning to predict kill probabilities more accurately by integrating the stochastic behaviors of targets and missile miss distances. This model dynamically evaluates target behavior, greatly expanding the predictive capabilities of engagement analysis. By factoring in elements such as warhead lethality, target and shooter orientations, and the specific conditions of engagement, PoKER provides important insights into engagement dynamics and quantifies success probabilities. Consequently, it can potentially be an important tool for BVR air combat pilots, improving operational decision‑making within this specialized combat domain.
Authors:Yichun Peng, Yansong Li, Jin Hou, Qi Song, Xi Guo, Ning Zhang, Degang Wei, Huang Yang, Yu Guan, Jianping Ge, Lei Bao, Tianming Wang
Abstract:
Non‑invasive camera‑trap monitoring of endangered wildlife requires open‑set individual identification‑‑‑grouping field images into identities without predefined classes or a known population size‑‑‑yet pose deformation and cluttered backgrounds complicate image comparison. Here, we present OpenStripe, a multi‑agent framework that reconstructs canonical two‑dimensional (2D) tiger torso patterns and encodes four interpretable local stripe structures: breakpoints, bifurcations, irregular structures, and round structures. We evaluated OpenStripe on 100 camera‑trap records from 28 wild Amur tigers (Panthera tigris altaica), representing nearly one‑third of the documented population in the Northeast China Tiger and Leopard National Park. Combining all four complementary structures enabled automatic identity grouping with 0.90 matched accuracy and a 0.85 adjusted Rand index (ARI), outperforming 49 baseline configurations under the same backend (up to 0.43 accuracy and 0.14 ARI). In addition, results from OpenStripe indicate that stripe similarity in Amur tigers declines with geographic distance and suggest that traits such as fractal dimension and inter‑stripe spacing are associated with geographic proximity. OpenStripe thus provides an interpretable representation that connects automated individual grouping with spatial phenotypic profiling in wild populations.
Authors:Dong-hwi Kim, Chaewon Moon, Hojun Song, Dongbeom Kim, Junyeong Jang, Aro Kim, Gahyeon Kim, Heejung Choi, Jehee Kim, Gianella Cravioto, Sohyun Lee, Gyeongjin Choi, EunHye Jeong, Soo Ye Kim, Jaehyup Lee, Sang-hyo Park
Abstract:
Real‑world 3D reconstruction pipelines often rely on multi‑view images degraded by lossy compression. While compression artifacts have been extensively studied in single‑image or video restoration, their impact on multi‑view geometry remains largely overlooked. In this work, we identify compression‑induced cross‑view geometric inconsistency as a critical failure mode, where view‑dependent artifacts disrupt feature matching, dense correspondence, camera parameter estimation, and downstream 3D reconstruction tasks such as novel view synthesis. To address this problem, we propose CompMVR, a compression‑aware multi‑view diffusion restoration framework that jointly refines compressed views while preserving cross‑view consistency, and construct compressed datasets for systematic evaluation. Unlike conventional restoration methods that optimize each image independently for 2D perceptual quality, CompMVR leverages learned compression priors and cross‑view correspondence to recover visual details while improving geometric reliability across views. Experiments across diverse datasets, codecs, and compression levels demonstrate gains in restoration quality and downstream 3D tasks, including novel‑view synthesis, camera pose estimation, and view matching. These results establish compressed multi‑view restoration as a distinct problem and highlight its importance for 3D reconstruction under lossy compression. Code and data for this paper are available at https://mchaewon.github.io/CompMVR‑project‑page/.
Authors:Yonghui Niu, Yuhao Ji, Chang Nie, Jiao Zhang, Yilong Li, Junlan Feng, Ran He, Caifeng Shan, Chaoyou Fu
Abstract:
The rapid growth of video data has created an urgent need for efficient video understanding systems that can handle long and continuous visual inputs, posing significant challenges for Multimodal Large Language Models (MLLMs) due to severe memory bottlenecks. Moreover, existing compression methods often overlook the explosive accumulation of approximation errors. To address this issue, we propose Video‑MemComp, a training‑free and sensitivity‑aware context management framework for memory‑constrained streaming video understanding. Our method theoretically decomposes compression error into orthogonal local approximation error and layer‑wise structural sensitivity via a first‑order Taylor expansion. To minimize local approximation error, we perform token aggregation in a pre‑RoPE safe zone with an attention‑conserving logit bias, mitigating geometric distortions caused by RoPE rotations. To manage layer‑wise structural sensitivity, we dynamically allocate the KV cache budget based on offline gradient priors and online context entropy, prioritizing high‑sensitivity layers. Video‑MemComp outperforms or achieves performance comparable to full‑cache and state‑of‑the‑art compression methods on benchmarks including MLVU, VideoMME, and EgoSchema. Notably, it improves the overall accuracy of Qwen2.5‑VL‑3B from 62.3% to 64.1% while reducing the KV cache to an equivalent of 6,000 tokens per layer, saving over 90% of KV memory.
Authors:Juan Felipe Duran, Arthur Tobler, Martin Vallières
Abstract:
Hospitals regularly buy software that clinicians and patients depend on. Because these purchases are costly, hard to reverse and can affect patient care, buyers look for evidence of which peer institutions already use a product and which systems it connects to. This evidence is public but scattered across vendor websites, press releases and third‑party pages. Language‑model agents can now collect it at low cost, but the correctness of the extracted claims is difficult to establish, and a wrongly listed customer can mislead buyers and misrepresent an organisation. We present a pipeline that builds a knowledge graph of customer and integration relations from the open web under one rule: every edge must be supported by a verbatim quote from its source page. Pages come from a deterministic crawl of each vendor's site and a budgeted ReAct agent that searches the rest of the web. A permissive extractor proposes relations with quotes, and a layered grounding stage decides which to keep, using a referent step that reads the whole page when a quote names only one side, deterministic checks, and a panel of five model calls voting on nine criteria. On 61 vendors, the pipeline reaches 0.94 precision and 0.71 recall against a page‑level key covering the 6,428 candidate relations proposed by its extractor, whose verdicts were drafted by one model and re‑checked with a second. Against the golden reference, the marketplace's own curated list, it scores lower, but 89% of the relations it keeps that the reference lacks are stated on a page. A confidence score built from the panel's votes keeps more correct relations than the fixed rule at the same precision, and reaches 0.90 precision at 0.90 recall. We also discuss how narrow, bounded model calls, string‑matched evidence and voting address long‑context degradation and run‑to‑run variance.
Authors:Luis Bauer, Annalena Kofler, Andrés Asensio Ramos, Aparna Venkataramanasastry, Valentín Martínez Pillet, Bernhard Schölkopf
Abstract:
Predicting solar flares is a challenging space‑weather forecasting problem with potentially significant impacts on Earth's upper atmosphere, including disruptions to radio communications. We propose Active Region attENtion Aggregation (ARENA), a neural network architecture that predicts solar flares from a variable number of active‑region cutouts while learning to identify the region most relevant to the prediction. ARENA is trained using multiple‑instance learning under weak supervision, requiring only a flare/no‑flare label for the observation and no labels specifying the flare origin. The learned attention mechanism therefore provides a means of localizing the active region driving the prediction without explicit origin annotations. ARENA achieves a heidke skill score of 0.45 which is comparable to other flare prediction approaches. These results demonstrate that active‑region‑based aggregation can provide an efficient and interpretable approach to solar flare prediction. The code is available at https://github.com/LuisBauer22/ARENA.
Authors:Yubo Ye, Maryam Toloubidokhti, Sumeet Vadhavkar, Xiajun Jiang, Huafeng Liu, Linwei Wang
Abstract:
The interest in leveraging physics‑based inductive bias in deep learning has resulted in recent development of hybrid deep generative models (hybrid‑DGMs) that integrate known physics‑based mathematical expressions in neural generative models. To identify these hybrid‑DGMs requires inferring parameters of the physics‑based component along with their neural component. The identifiability of these hybrid‑DGMs, however, has not yet been theoretically probed or established. How does the existing theory of the un‑identifiability of general DGMs apply to hybrid‑DGMs? What may be an effective approach to construct a hybrid‑DGM with theoretically‑proven identifiability? This paper provides the first theoretical probe into the identifiability of hybrid‑DGMs, and presents meta‑learning as a novel solution to construct identifiable hybrid‑DGMs. On synthetic and real‑data benchmarks, we provide strong empirical evidence for the un‑identifiability of existing hybrid‑DGMs using unconditional priors, and strong identifiability results of the presented meta‑formulations of hybrid‑DGMs. Code is available at https://github.com/GoForIt‑YYB/Meta‑Hybrid‑VAE.
Authors:Haoran Gong, Jie Feng, Chenxu Wang
Abstract:
Generating editable 3D urban scenes from a single image requires detailed building assets with a coherent spatial arrangement. Occlusion and perspective ambiguity make it difficult to infer complete building geometry together with consistent relative scales and positions. We present CityLift, a framework that generates individual textured building meshes from an urban image by coupling scene‑level spatial modeling with building‑level synthesis. CityLift first learns building shape and placement jointly through coupled scene and instance streams, producing coarse sparse structures in a common coordinate system. We then introduce generative spatial inheritance, which maps each building's coarse occupancy and scene‑conditioned 3D feature field into a local volume to guide a flow model that generates finer structure. The same coordinate mapping places the generated meshes back into the scene, preserving their spatial relationship to the scaffold. To address the scarcity of urban data pairing scene images with individual 3D buildings, we construct CityLift‑5K, comprising 4,997 synthetic scenes with instance masks, building meshes, and placement annotations. Experiments on CityLift‑5K and scene blocks from UrbanScene3D and InstanceBuilding demonstrate improved building geometry and scene composition. The project page is available at https://ranhaogong.github.io/CityLift/.
Authors:Aayush Acharya, Qi Zhang, Eduard Dragut, Longin Jan Latecki
Abstract:
The rapid expansion of scientific literature far outpaces the speed of manual curation, creating a fundamental bottleneck in extracting structured knowledge from scientific texts. To address this, we introduce an agentic, ontology‑guided framework for knowledge graph (KG) construction. Our framework utilizes specialized agents for document understanding, information extraction, ontology conformance, and entity linking. First, an LLM extracts factual knowledge in the form of triples with explicit textual evidence from research papers. Then the triplets undergo a multistage verification process to ground the triplets in the ontology. By enforcing schema constraints and requiring evidence in the form of text fragments, we ensure the data is both semantically consistent and traceable. In an agreement study over 1,000 extracted triplets, Gemini‑2.5‑flash achieves Cohen's κ scores of 0.640‑‑0.700 against three human annotators, matching the human‑‑human agreement range of 0.616‑‑0.689. We applied this framework to 2,000 scientific publications and construct a novel, ontology‑grounded KG for climate science with 133,990 nodes and 755,308 relations; of the 23,327 triplets that the publications contribute after validation, 98.96% are judged correct. The pipeline code and KG is publicly available https://github.com/aayushacharya/climatekg.
Authors:Roy Amoyal, Shira Ifergane, Oren Freifeld
Abstract:
We introduce SemGeo‑Gen, an unsupervised method for generating approximate cross‑instance semantic‑geometric supervision from weakly aligned 3D object collections. Our goal is not to recover exact point‑level correspondences, which are often ambiguous across different instances and would require expensive, impractical manual supervision at scale. Instead, we automatically produce approximate correspondences that are semantically meaningful, geometrically consistent, and diverse enough to train modern correspondence models. Given only coarse category‑level rotation alignment, SemGeo‑Gen lifts multi‑view DINOv2 features to 3D, aligns object instances using continuous piecewise‑affine registration, and prunes candidate matches using semantic consistency. The resulting 3D correspondences can be projected into rendered views, yielding scalable 3D‑3D, 3D‑2D, and 2D‑2D supervision without manual annotation. We validate the generated correspondences against sparse human annotations from KeypointNet, obtaining 93% PCK@0.10 and 97% PCK@0.15. More importantly, we show that using the generated correspondences for synthetic pretraining improves recent state‑of‑the‑art semantic correspondence models on SPair‑71k after fine‑tuning on real data, with gains of up to 5 PCK@0.10 points. Together, these results indicate that approximate semantic‑geometric supervision generated from 3D assets can improve real‑image correspondence learning and serve as a scalable alternative to costly manual annotation. Project page: https://bgu‑cs‑vil.github.io/SemGeo‑Gen/.
Authors:Chunchun Yang, Yang Cao, Zhitang Chen
Abstract:
Continual Wi‑Fi sensing must balance plasticity, learning new rooms or users, against stability, keeping earlier ones, yet nodes often cannot retain channel state information (CSI) for privacy and storage reasons. We propose WiMoT (Wi‑Fi Moment Transport), which stores the per‑class feature mean and covariance of every earlier domain instead of samples. When the representation changes, stored statistics become stale. WiMoT restricts the change to an affine map on a frozen feature basis and applies the same map to the statistics, so they stay valid without old samples. Supervised contrastive learning learns the new domain, a closed‑form loss on the stored statistics keeps earlier classes separable, and a Gaussian discriminant classifier, refitted from the transported statistics with equal weight per domain, predicts without domain identity. On 3DO, WiMANS, and Widar3.0, WiMoT's average final accuracies are 94.02%, 89.61%, and 95.16% with at most 0.58 percentage points (pp) of forgetting. On 3DO and Widar3.0, it also has the highest balance of new‑ and earlier‑domain accuracy and is 3.77 and 13.11 pp ahead of the strongest of 21 baselines, which uses replay, staying ahead across domain orders and label budgets. On WiMANS, it matches the strongest replay baseline in accuracy (+0.41 pp) while reducing forgetting from 2.78 to 0.00 pp, and has the highest six‑order average accuracy. Because full covariances dominate the memory, storing them as rank‑32 factors shrinks the moment memory to 6.4%, losing at most 0.49 pp of accuracy, within edge‑node storage budgets.
Authors:Iakovos Evdaimon, Giannis Nikolentzos, Achilleas Tsimichodimos, Christos Xypolopoulos, Michalis Vazirgiannis
Abstract:
In this paper, we introduce WL‑SeqVAE, a sequence‑based variational autoencoder for generating graphs. The key observation behind WL‑SeqVAE is that node colors induced by the Weisfeiler‑Leman (WL) algorithm provide compact discrete summaries of local graph structure and can therefore serve as structural tokens for generation. Rather than learning a distribution over adjacency matrices directly, our method maps each graph to a sequence of WL colors and employs a variational autoencoder that encodes this sequence into a Gaussian latent distribution, from which an autoregressive decoder reconstructs the graph. During generation, compatibility constraints derived from the WL colors guide node and edge predictions, combining global information from the latent representation with local structural consistency. This formulation turns graph generation into a sequence generation problem and avoids directly parameterizing a quadratic adjacency matrix. We evaluate WL‑SeqVAE on molecular and general graph generation tasks. Across the evaluated datasets, WL‑SeqVAE is competitive with established baselines on several local structural and molecular quality metrics. Code is available at https://github.com/iakovosevdaimon/WL‑SeqVAE
Authors:Jiyeong Chae, Hyunkyo Seo, Heonjae Lee, Kyung-Joon Park
Abstract:
Autonomous mobile robots support material transport in warehouses and industrial facilities, where navigation must adapt to changing operating conditions. Sensor observations reveal physical obstacles, but do not by themselves specify temporary access rules, route preferences, or exceptions known to operators. This paper presents a natural‑language policy interface that brings this knowledge into ROS 2 Nav2. A local large language model converts instructions into JSON policies specifying destinations, spatial constraints, and temporal or event conditions. Deterministic checks govern mission acceptance, and three costmap adapters project accepted policies into the existing navigation stack without changing its planner or controller. On 150 researcher‑authored English commands, four local models produce 107‑142 correct accepted policies, compared with 44 for a rule‑based parser, with 3‑20 incorrect acceptances per model. Fault injection records rejection of eight endpoint/JSON faults and 29 validator‑stage faults. Navigation experiments use correct task policies to evaluate keepouts, zone costs, person‑dependent regions, and event‑conditioned passage in simulation and on a TurtleBot3. For Tasks 1‑3, mean travel times are 18.4‑24.5% lower in simulation and 23.1‑35.0% lower on the physical testbed than with the baseline lacking those policies. A native KeepoutFilter comparison and two‑robot demonstrations examine basic keepout behavior and shared policy projection. The interface connects operator instructions to explicit navigation policies while retaining Nav2's planning and control components.
Authors:Hawar Dizayee
Abstract:
VICReg prevents collapse with a hinge on the standard deviation of each embedding dimension, and its authors argue from the gradient that a hinge on the variance would fail. To our knowledge, this claim has not been tested experimentally. We test it in a controlled setting (ResNet‑18, CIFAR‑10 and CIFAR‑100), changing only the hinge and sweeping its weight μ over five orders of magnitude. At the default weight (μ= 1), the two hinges reach the same linear‑probe accuracy on CIFAR‑10 (about 89.5%). Both collapse below a threshold weight, but the standard‑deviation hinge's threshold is 35‑130× lower. CIFAR‑100 shows a consistent gap. In training, the variance hinge's gradient fades as the feature variance shrinks, while the standard‑deviation hinge's does not until that variance is very small. The standard‑deviation hinge therefore buys a safety margin: it keeps training at weights far below the point where the variance hinge collapses.
Authors:Zenghui Zhou, Xiaoke Fang, Zheng Zheng
Abstract:
Large Language Models (LLMs) increasingly serve as reasoning components in modern software systems, yet a model that answers a logical query correctly can contradict itself when the same problem is restated in an equivalent form. Static benchmarks evaluate queries in isolation and therefore fail to expose such inconsistencies. To address this limitation, we present LGMT, an open‑source tool that detects reasoning inconsistencies through metamorphic relations grounded in first‑order logic. Given a source test case in premise‑conclusion form, LGMT generates logically equivalent follow‑up cases and checks whether the model's entailment predictions agree across the pair, without ground‑truth labels. The tool provides 20 built‑in metamorphic relations, supports custom ones verified by the Z3 theorem prover, and runs from a CLI or a web studio, including an offline mock mode requiring no API key. Across six widely used LLMs on a bundled benchmark of 3,091 metamorphic groups, LGMT reveals inconsistency rates ranging from 20.84% to 29.44%. LGMT is publicly available on GitHub at https://github.com/MaxwelsDonc/lgmt‑tool, with a demonstration video at https://youtu.be/gou4opptu1U.
Authors:Zhiran Yan, Savankumar Bhanderi, Sourabh Lolge, Rui Song, Girdharbhai Paneliya, Shiva Agrawal, Gordon Elger
Abstract:
Recent advances in 3D semantic segmentation have significantly improved scene understanding in autonomous driving. However, existing methods are typically developed under a closed‑set assumption and evaluated under standard driving conditions, where all categories are known in advance. In real‑world driving, autonomous vehicles inevitably encounter previously unseen objects, making closed‑set perception insufficient for safe deployment. To address this challenge, we introduce DriveOSS, which, to the best of our knowledge, is the first real‑world multimodal dataset and benchmark specifically designed for 3D anomaly segmentation and open‑set semantic segmentation. It contains 12,661 synchronized and annotated frames captured using six surround‑view RGB cameras, a 128‑beam LiDAR, and a GPS/IMU unit. The dataset comprises 52 real‑world driving scenes collected under clear, nighttime, cloudy, rainy, and snowy conditions and is organized into ID (in‑distribution) and OOD (out‑of‑distribution) subsets. The ID scenes contain only known classes and are used for training, whereas the OOD scenes additionally contain unknown objects annotated as the outlier class and are used for validation and testing. Extensive experiments show that unseen objects remain a major challenge for current 3D perception systems. The dataset, benchmark protocols, and code are available at https://github.com/chrisyan/DriveOSS.
Authors:Gibran Benitez-Garcia, Hiroki Takahashi
Abstract:
Hand landmarks serve as input for gesture recognition, sign language understanding, and touchless human‑computer interaction. Yet hand pose estimators are typically evaluated with frame‑wise localization metrics such as PCK or MPJPE, which are computed over matched predictions and overlook missed or spurious hands, handedness errors, and temporal instability. To address these limitations, we present VERA‑Hand, a benchmark that evaluates estimators using manually verified, frame‑level 2D annotations for 10,690 hands across four complementary dimensions (detection, localization, identification, and temporal stability), as well as a joint Pose‑F1 metric that requires both correct detection and accurate localization. Evaluating six estimators spanning hand‑specific, whole‑body, reconstruction‑based, and foundation‑model paradigms, we find that no estimator dominates: rankings reorder sharply across dimensions. VERA‑Hand also exposes failures that localization‑only and downstream evaluations may not reveal, including cross‑hand overlap in over 85% of frames for a recent foundation model. A post‑hoc cleanup case study demonstrates how the benchmark can validate corrective interventions, while stratified analyses show that robustness varies across motion‑blur classes and hand regions. We release the verified annotations, protocol, and code at https://gibranbenitez.github.io/VERA_Hand/
Authors:Ashish Kumar Meena, Chandra Sekhar Seelamantula
Abstract:
Self‑supervised learning (SSL) offers a promising approach to medical image representation learning, yet augmentation strategies of SSL frameworks validated on natural images do not reliably transfer to specialized medical imaging domains. We analyze DINO, a knowledge‑distillation based SSL framework, and empirically demonstrate that its default augmentation strategy produces unreliable representations for retinal fundus images, degrading downstream glaucoma classification performance. Guided by the principle that effective cropped views should share minimal redundant information while retaining clinically relevant signal, we redesign DINO's augmentation strategy. Our approach biases the learned representations toward diagnostically critical features rather than low‑level image statistics. We evaluate our method for glaucoma classification by training a linear classifier on the frozen representations, achieving strong generalization across seven benchmark datasets. URL for code and weights: https://github.com/ashish‑256/task_dino
Authors:Qing Liu, Xinrui Chen, Yi Du, Ou Wu, Weiyao Zhu
Abstract:
Single‑stage fine‑tuning is pervasive, yet catastrophic forgetting persists when the source pre‑training data is inaccessible. Data weighting affords a natural lever to modulate sample influence. Yet existing weighting methods lack a principled mechanism to explicitly arbitrate knowledge retention against new‑task adaptation, and provide no unified differentiable objective to accommodate auxiliary signals such as sample diversity and sample characteristics. This is because current schemes privilege one goal over the other, and no explicit gradient‑compatible encoding exists for such signals. To address this, we propose DRIVE, a principled and efficient weight assignment framework. It derives per‑sample weights via a unified differentiable objective where three coupled signals co‑evolve: Shapley additivity splits contribution into anti‑forgetting and adaptation components, which are reshaped by a covariance‑based diversity term and stabilized by a sample‑characteristic prior. Far from a mere combination, this yields an end‑to‑end gradient‑based scheme balancing stability against plasticity. Theoretically, we prove that DRIVE enjoys superior stability‑plasticity guarantees over existing weighting schemes and enhances parameter estimation robustness. Empirically, it attains superior overall performance, faster convergence, and stronger robustness. Code at https://github.com/luciabcd/DRIVE.
Authors:Khaled Waleed Dawoud, Muhammad Haris Khan
Abstract:
Deploying Vision‑Language Models for video anomaly detection confronts a fundamental constraint. Their semantic reasoning demands computational budgets incompatible with the scale of real‑world video streams. We present Signal‑Aware Adaptive Fusion (SAAF), a framework that achieves efficient zero‑shot anomaly detection through three key innovations. First, we introduce a dual‑signal adaptive sampler operating in semantic embedding space: a delta signal captures sudden events via frame‑to‑frame cosine distance, while a drift signal detects static and gradual anomalies through deviation from rolling temporal context. Their OR‑gated combination with adaptive EMA‑based thresholding provides comprehensive anomaly coverage without training data. Second, we propose a threshold‑gated cascaded architecture where lightweight frame‑level scoring precedes expensive video‑level temporal analysis, invoking it only when confidence warrants additional context. Third, we design a signal‑aware weighted fusion mechanism that dynamically adapts score combination based on which signal triggered frame selection. Across four benchmarks, SAAF delivers competitive accuracy at substantially lower cost. Among training‑free zero‑shot methods, SAAF sets a new state of the art on XD‑Violence (92.07%), MSAD (86.70%), and ShanghaiTech (80.54%) AUC‑ROC, with XD‑Violence and MSAD also surpassing supervised baselines. SAAF sustains 44.95, 48.61, and 41.06 FPS on XD‑Violence, UCF‑Crime, and MSAD, a 2.43×, 2.63×, and 2.22× throughput improvement over the prior best zero‑shot method. Code: https://github.com/kdawoud91/SAAF
Authors:Shengchao Chen, Ting Shu, Sufen Ren
Abstract:
Photonic crystal fibers (PCFs) underpin critical applications in telecommunications, sensing, and high‑power laser delivery, yet their design remains a manual, expert‑intensive process that couples geometric intuition with deep waveguide physics. Vision‑language models (VLMs) offer a promising path toward automating PCF design, but can they truly reason about photonic physics from microscopy images, or do they merely recognize visual patterns? We introduce PCFBench, the first comprehensive benchmark for evaluating VLMs on the perception, reasoning, and decision steps of the photonic inverse‑design loop. PCFBench spans 26 tasks organized into six categories of increasing cognitive demand (Geometry Perception, Physics Understanding, Text Generation, Multi‑modal Reasoning, Inverse‑Design Decision, and Code Generation), built from over 220K FDTD simulation samples across 18 structurally diverse fiber families. We evaluate both individual VLMs on isolated tasks and multi‑agent systems (MAS) on the full sequential pipeline from perception to design. Our results reveal a striking perception‑design gap: current models can identify fiber geometry with moderate success, yet consistently fail to translate that visual understanding into physics‑aware design actions. Moreover, cascading agents through the pipeline exposes significant error propagation, where upstream perception mistakes compound into downstream design failures. These findings suggest that neither model scaling nor naive agent chaining suffices, and that closing the loop demands explicit physical grounding. PCFBench provides the community with a rigorous, multi‑dimensional testbed to measure progress toward AI‑assisted photonic engineering. The project is available at https://pcfbench.github.io/.
Authors:Cristian Tommasino, Antonio M. Rinaldi
Abstract:
Vision‑language models used for decisions in histopathology rarely return calibrated probabilities over the candidates a user cares about. Decision‑native models write a question and a candidate set chosen at run time into the input and return a normalised distribution from a small decision head in one forward pass. We build PathJev‑P, a decision‑native model for histopathology: MedGemma‑1.5‑4B with LoRA adapters and a decision head, trained on 40,000 patch states from three public collections whose expert labels are turned into typed questions with organ‑specific candidate sets and a "none of these" option. In a comparison over 3 seeds with an analysis plan fixed before testing, on test sets without documented overlap with the training data of the backbone, PathJev‑P is more accurate than a generative model trained with the same backbone, data and selection rule, the zero‑shot pathology model CONCH and a general medical decision model (balanced accuracy 0.541 against 0.492, 0.345 and 0.213); its advantage over the generative model comes from two organs absent from training. The calibration advantage depends on how the generative model's probabilities are read out. With the summed candidate log‑likelihood used in previous work, PathJev‑P has the lower calibration error (ECE 0.204 against 0.279); with a length‑normalised read‑out of the same generative model, examined post hoc, the two have similar mean calibration error (0.201) and the generative model is better calibrated on the unseen organs, where PathJev‑P remains over‑confident. A supervised linear probe on a pathology foundation model is better calibrated in every setting. In a qualitative whole‑slide demonstration, PathJev‑P labels unsupervised tissue clusters with abstention and separates carcinoma from other tissue while running the model on a fraction of the tiles.
Authors:Moritz Schillinger, Maximilian Bader, Adrian Buehler, Felix Wachter, Ferdinand Knieling, Vasilis Ntziachristos, Katharina Breininger
Abstract:
Combined optoacoustic (OptA) and ultrasound imaging is a promising modality across a wide range of pathologies. Its use in clinical research, however, is limited by a lack of analysis software accessible to users without programming expertise. Existing tools are either script‑based engineering frameworks or proprietary vendor applications. We present OPTARI, an open‑source graphical software application to facilitate large‑scale clinical optoacoustic‑ultrasound (OPUS) studies. OPTARI builds on the napari image viewer and the PATATO toolkit for OptA data processing. It supports native vendor and open HDF5 data formats, aligns with the IPASC standard, and is distributed as a standalone executable. Features include region‑of‑interest annotation, reconstruction, chromophore unmixing, deep‑learning‑based tissue segmentation, and temporal and spectral analysis modules. OPTARI offers a combination of features for clinical study analysis that none of the compared tools provide. The automated tissue segmentation produced results comparable to expert labeling for muscle tissue, and batch processing reduced analysis time from minutes to under one second per scan. The tool is used in ongoing clinical OPUS studies. OPTARI connects engineering frameworks with medical research and provides an open‑source basis for reproducible and standardized analysis in clinical OptA imaging studies.
Authors:Tianyun Zhong, Wei Wang, Jialin Li, Jianhong Tu, Liyao Li, Chuqiao Kuang, Tianshi Zheng, Songchi Zhou, Weiqi Zhai, Qile Jin, Jiajia Li, Jiasheng Zheng, Boxi Cao, Yaojie Lu, Hongyu Lin, Shiwei Ye, Bing Zhao, Hu Wei, Lin Qu, Xianpei Han
Abstract:
Large language model (LLM) agents are increasingly capable of supporting scientific workflows, but their capabilities across diverse research activities remain difficult to assess systematically. Existing benchmarks often focus on selected scientific capabilities or task types, limiting a comprehensive characterization of scientific agents' strengths and weaknesses in realistic research workflows. We introduce SCI‑AGENT‑EXAM, a workflow‑oriented benchmark comprising 120 expert‑curated tasks across 11 scientific fields and 7 research task types defined by the Scientific Workflow Task Taxonomy (SWTT). Each task requires both executable artifacts (e.g., code and scripts) and textual artifacts (e.g., reports and explanations). To evaluate the heterogeneous task outputs, we develop a hybrid evaluation protocol combining deterministic programmatic verification for executable artifacts with a calibrated, expert‑validated LLM‑based judge for textual artifacts. Beyond outcome assessment, we analyze progress and solution stability along execution trajectories to characterize how agents reach their final results. Experiments with 10 frontier LLM agents reveal substantial performance gaps: GPT‑6 Astra achieves the highest overall score of 80.9, yet only 26.0% of its runs exceed the 0.95 success threshold. Further analyses uncover distinct capability profiles across scientific task types and substantial variation in execution efficiency and stability. Our code and data are available at https://github.com/SKYLENAGE‑AI/Sci‑Agent‑Exam.
Authors:Jingmin Zhu, Yanyan Zou, Xiuyu Li, James Bailey, Dinh Phung, Tao Yu, Yunyun Hou, Jun Liu, Jun Zhou, Sirui Wang, Nan Duan, Jiachen Liu, Qiuhong Ke
Abstract:
Video reasoning requires understanding how events unfold over time, yet videos with accurate temporal annotations are expensive to collect. Self‑play offers an alternative by letting a model generate and solve its own tasks without manual annotation. However, existing video self‑play methods train on fixed sets of recorded videos, so they cannot alter the underlying event dynamics and rely on noisy pseudo‑labels derived from the model's own outputs, such as majority voting, that risk reinforcing shared model errors. To overcome these limitations, we introduce Tempo‑Zero, a self‑improvement framework that closes a loop of executable environments, execution‑verified self‑play, and environment evolution, where executing a program renders a video with the ground‑truth option and interval of every question. A single policy plays both Questioner and Solver, proposing questions and answering them with timestamped evidence lines, and is trained on execution‑grounded rewards instead of model consensus. The Solver's performance steers a curriculum over tasks and difficulty levels, and the policy selects rule changes that enrich the event dynamics of later rounds. Despite being trained solely on simple programs, without supervised fine‑tuning or external video data which prior methods typically require, Tempo‑Zero outperforms the base models and recent self‑play baselines on average across twelve real‑world video benchmarks at different model scales, with particularly strong gains on temporal grounding tasks. Project page: https://alchemist‑jo.github.io/tempo‑zero/
Authors:Detao Zeng, Guilin Ou, Nuo Chen, Yunhao Feng, Hongliang Wang, Qinli Yang, Junming Shao, Zhong Zhang
Abstract:
Large language model (LLM) agents increasingly execute tasks asynchronously: a main agent continues working while its subagents run, and the task state may change before their results arrive. The main agent must decide which work to update or preserve and which checks to repeat, while avoiding prohibited changes. Final task success alone does not reveal whether these results were handled correctly, and natural asynchronous arrivals are hard to reproduce. We introduce Async‑RBench, a benchmark for asynchronous task execution in LLM agents, with 200 executable tasks covering eight scenarios. A scenario controller regulates result delivery using predefined task‑state conditions, while an Event Response Contract (ERC) specifies what each response must change, preserve, avoid, and re‑verify. The Dynamic Replanning Score (DRS) weights the response process and outcome equally and is reported separately from task success. We evaluate nine agents and find DRS scores of 19.2 to 45.1, with the lowest scores on duplicate reports and subagent failures. Under our execution protocol, all nine agents obtain lower task success with asynchronous than with batched delivery (26.0% vs. 29.7% on average). For three agents, prompting the main agent to rebuild its plan after every result improves DRS by only 2.6 to 3.5 points. Together, these findings highlight the challenges of reliable asynchronous task execution and the need to evaluate how agents handle incoming results alongside end‑to‑end task success.
Authors:Xiaopeng Ke, Zhixin Wang, Xuebo Liu, Xinyu Ma, Xing Zheng, Bowen Song, Weiqiang Wang, Min Zhang
Abstract:
LLM‑based annotation has advanced rapidly, yet existing benchmarks remain misaligned with real‑world practice. Actual annotation often involves lengthy, task‑specific guidelines, and different annotators may produce divergent but reasonable judgments, especially on boundary cases, where single‑labeled dataset fail to reflect this disagreement. This gap also limits existing single‑agent and workflow‑based annotation systems, which often lack adaptability to complex, evolving guidelines. To bridge this gap, we introduce ANT‑A Benchmark, a real‑world multi‑annotator benchmark spanning 7 datasets across domains and modalities, with task‑specific guidelines and 5 independent human labels per instance. ANT‑A enables a distribution‑aware evaluation, using Fleiss kappa to assess how closely model annotations align with human labeling distributions. We further propose ANT‑OR, a multi‑agent annotation framework that refines prompts by distilling contrastive decision rules from positive and negative feedback. Experiments show that ANT‑OR improves agreement@>=2 accuracy by 4.5 points over the strongest baseline and achieves the highest alignment with human annotator distributions as measured by Fleiss kappa within 2 points. Code and benchmark are available at https://github.com/Krueske/ANT‑OR.
Authors:Sujung Hong, Tae Eun Choi, Youngjun Jun, Chanyong Yoon, Seong Jae Hwang
Abstract:
Text‑to‑image (T2I) diffusion models have advanced notably, but still fail to satisfy prompt conditions, resulting in attribute mismatches and missing objects. These errors normally fall into two categories: concept loss, when an object or attribute (i.e., concept) goes missing, and concept confusion, when an attribute is assigned to the wrong object or multiple objects blend together. Recent studies suggest that these challenges are caused by the limited capacity of CLIP text encoder to capture fine‑grained semantic details. Although several methods have been proposed, concept loss and concept confusion persist in recent T2I models when handling multi‑object, multi‑attribute prompts. In this paper, we conduct an embedding‑level analysis to develop a training‑free, inference‑time solution for concept loss and concept confusion across CLIP‑based T2I models. We find that (1) concepts mentioned later exhibit higher embedding entropy, reflecting less concentrated representations that are more often associated with concept loss, and (2) the first CLIP attention layer captures the strength of binding between each object and its attribute. Guided by our findings, we introduce TIE, a method that improves semantic alignment through a single text‑embedding update. TIE addresses concept loss via entropy‑aware singular value amplification and resolves concept confusion through interpolation‑extrapolation binding based on CLIP attention scores, all in a training‑free manner. Extensive experiments demonstrate that TIE enhances semantic fidelity in multi‑concept scenarios with minimal sampling overhead. The project page is available at https://hong‑sujung.github.io/TIE/.
Authors:Soheil Kooklan
Abstract:
The 3rd WEAR Dataset Challenge asks for an activity label for every one‑second window of four unseen participants. Each window has 15 VideoMAEv2 frame embeddings, the accelerometer signal of one random limb, and the windows are shuffled. This report is an empirical analysis of one system built after the challenge deadline. Window classifiers are stacked with features averaged over per‑participant graphs of video neighbours and scored successor candidates, and a mean‑field style decoder with a soft count constraint per activity labels all windows of a participant together. The setting is offline and transductive, and the count constraint relies on each activity being performed once. In a participant‑wise simulation with participant‑level bootstrap intervals, stacking added 0.051 macro F1 over stage‑1 fusion and the count constraint 0.011 (95% interval 0.006 to 0.017). A quadratic seam distance ranks the true same‑limb successor first among about 720 candidates in 83.9% of cases, but adding it to the link model changed macro F1 by ‑0.0001 (‑0.0006 to 0.0005). With the decoder settings and the mixing weight chosen inside the folds and the mislabelled spans left out of training, the system scores 0.936 in simulation (0.940 for participants who performed each activity once); the submitted configuration scores 0.938 (0.943). Per‑participant scores range from 0.80 to 0.97. These results come from a non‑nested stacking protocol that uses unlabelled features transductively, so they are not a fully inductive estimate. An oracle that averages the final unary scores over the true segments reaches 0.973, and 79% of the remaining errors involve the null class. On the private leaderboard, which no choice used, the selected Kaggle submission scored 0.9231 (public 0.9254, with bounds partly tuned there), and a later one with bounds chosen in simulation scored 0.9247 (public 0.9253).
Authors:Fanxing Ning, Jianguo Zhu
Abstract:
Large language models (LLMs) can turn market notices and news into features for electricity price forecasting. Such features also record when documents arrive and whether their extraction failed, so a gain measured against a model without text can reflect document activity as well as content. We add the counts of the same documents and of their failed extractions to a no‑text LightGBM model and measure each LLM representation against this activity reference, which needs no additional LLM requests. With gpt‑6‑luna as the extractor, we forecast Australian electricity prices from operator notices (5‑30 minutes ahead) and WattClarity news (day‑ahead) after the extractor's documented knowledge cutoff, averaging forecasts over three training seeds. For news, the counts alone lower mean absolute error (MAE) by 3.0%, most of the 3.6% reduction that the better LLM representation achieves relative to the no‑text model; its increment beyond the counts is not significant. For notices, two of the three LLM representations lower MAE beyond the counts, by 0.14‑0.21% and 0.45‑0.52%, whether the counts are cumulative or cover recent windows, whereas the third shows no significant reduction. Time‑shifted placebos give mixed evidence on whether these gains require text aligned with the target. Benchmarks using the operator's pre‑dispatch forecasts (notices) or price history alone (news) are more accurate than all these text models. Gains against the no‑text model and the activity reference differ in sign at individual seeds and, for notices with cumulative counts, in seed‑averaged forecasts. Text gains should be reported against both references.
Authors:Karyamsetty Martin
Abstract:
The asymptotic competitive ratio of online bin packing has been bracketed since 2018 by the lower bound 1.54278 of Balogh, Bekesi, Dosa, Epstein and Levin and the upper bound 1.57829 of their Advanced Harmonic algorithm. We revisit both sides with exact rational computation and prove what the computations say. On the upper bound, we rebuild the certification of Advanced Harmonic as an exact optimization over its dual parameters; with the published parameters this already certifies a ratio strictly below the published one, and re‑tuning the algorithm's proportion parameters gives a certified ratio of 1.5782147437 (an exact rational), independently re‑verified on all 402 scenarios. To our knowledge this is the first movement of the upper bound since 2018. On the lower bound, we prove that the 2018 adversary construction is rigid: extending it by any finite number of new batches of ordinary size (outside the window of sizes s with 2s > 1 and rational part at most 0.64819), placed anywhere, with counts depending on the observed statistics, adaptive or not, never improves the bound at any truncation level. The load‑bearing fact is that the construction is volume‑tight at every stop. A single window‑sized batch added to the truncation‑3 tree is shown by machine‑verified certificates to leave its value below the published limit. A computational map covers the remaining natural extensions: unions of structural variants, two‑statistic multi‑batch adversaries, and three families of adaptive conditions that the paper's separation theorem allows but never uses; none improves the bound. On the algorithm side we show that the marking, bonus‑item and red‑sand mechanisms of Extreme Harmonic bind nowhere in Advanced Harmonic, whose bottleneck is a two‑medium‑item bin that no known mechanism targets. We record two corrections to commonly cited numbers.
Authors:Tianyi Sun, Pengfei Yao, Yanze Wang, Hongda Chen, Yunyun Han, Jun Sun, Chuanrui Hu, Yafeng Deng
Abstract:
Retrieving the right skill from a large library is essential for LLM agents, yet requests and skills are written for different purposes and in different words. Existing retrievers narrow this request‑skill gap with one or two representation choices each, so it is unknown whether their gains hold in combination. No prior work measures the gap itself, so it is also unclear which part of it each choice narrows. We define a representation design space by five choices ‑‑ encoder fine‑tuning, query‑side and document‑side expansion, background projection and a lexical channel ‑‑ and evaluate them together for the first time, as modules of one retriever, SkillRep. With a 0.6B encoder and LLM expansion, SkillRep improves MRR@10 on SRA‑Bench over the strongest same‑size retriever by 7.02 points with the official skill pool and 8.79 with UniPool, a 111,247‑skill pool we build from public registries, and is best on SkillsBench. The value of each choice depends on the others. Fine‑tuning, for instance, reverses the ranking of the four training‑free modules and turns them from complementary into overlapping. To understand why, we measure the gap with three quantities: background, the embedding component that does not separate skills; unsaid vocabulary, the gold skill's content a request leaves unsaid; and dilution, the information lost when a skill is encoded into a single vector. They show which part each module narrows and by what mechanism. This reveals that modules narrowing the same part compete for it, while those narrowing different parts complement each other.
Authors:Haocheng Jiang, Herun Wan, Yongyuan Peng, Minnan Luo
Abstract:
When LLM‑powered agents interact, information is repeatedly shared, expanded, and forgotten, yet multi‑agent evaluation typically focuses on final task outcomes. We study such interaction as claim‑level information dynamics and propose RIVER, which decomposes agent outputs into atomic claims and evaluates three complementary dimensions: convergence, exploration, and retention. Convergence captures cross‑agent claim correspondence and stance alignment; exploration captures semantic breadth and extension beyond the source; and retention tracks source preservation and the persistence of new claims across stages and lineages. We evaluate five interaction strategies on 40 controlled news‑generation cases. Repeatedly pooled interaction promotes convergence but narrows the information space, whereas distributed relay yields broader, less shared information. Further comparisons show that method‑specific constraints increase exploration while weakening convergence and retention. Our findings suggest that collective information dynamics are better explained by underlying information‑flow mechanisms than by simple downstream performance.
Authors:Meng Li, Lijiang Chen, Qi Zhao, Jun Jiang, Longhao Zou, Shuchang Lyu, Guangliang Cheng
Abstract:
Robot manipulation involves localized changes: picking up a cup moves the cup while leaving the surrounding scene largely unchanged. Yet existing 3D visual pretraining methods, including recent dynamics‑aware variants, model state transitions at the scene level, compressing the motion of everything in a scene into a single global latent action. This conflates semantically distinct transitions in scenes with multiple moving parts, obscures dependencies between changing regions, and leaves the learned representation vulnerable to task‑irrelevant scene content. We introduce DynaSlots, an action‑free, label‑free pretraining framework that decomposes each point cloud frame into a small set of temporally consistent slots and models dynamics at this per‑slot level rather than the scene level. A temporal slot propagation module promotes consistent slot assignments across frames; an inverse dynamics model then extracts one latent action per slot from feature differences, and a diffusion‑based forward dynamics model, paired with a self‑supervised interaction head, predicts each slot's future state while learning pairwise proximity‑and‑motion relations. Experiments on Meta‑World, Adroit, and a RealMan RM65‑B robot demonstrate that DynaSlots outperforms prior scene‑level baselines. It achieves superior success rates in simulation and exhibits strong robustness to scene clutter and unseen objects in real‑world manipulation. Code is available at \hrefhttps://zhucezhanghao12345‑source.github.io/DynaSlots‑Anonymous/https://zhucezhanghao12345‑source.github.io/DynaSlots‑Anonymous/.
Authors:Yanze Wang, Pengfei Yao, Tianyi Sun, Hongda Chen, Yunyun Han, Jun Sun, Chuanrui Hu, Yafeng Deng
Abstract:
LLM agents retrieve skills from public hubs that hold up to about 100,000 entries and serve many models. As models improve, a hub must find the skills a given model has internalized, meaning it can do without them, and retire them for that model, yet internalization has not been measured with known reliability. Published rules that compare runs with and without a skill retire on the gain alone. The gain is near zero both for an internalized skill and when neither the model nor the skill delivers the capability, a state we call uncovered. We instead withhold the skill, score the model on a task synthesized from it, and trust a high unaided score only when another model gains from the skill, since nearly half of high scores come from tasks no model benefits from. Under four models, a published gain rule applied with one task per skill removes over half of the uncovered skills. On SkillsBench, 28 of its 31 retirements rest on probes no run passed in full, while retiring on the unaided score costs no detectable reward. Unaided verdicts reproduce across tasks more often than the gain rule's (Cohen's kappa 0.42 against 0.23) but differ across models, so a hub must read them per model. We release 1,251 skills with tasks and item scores under four models.
Authors:Yifei Wang, Botian Wang, YiFei Wang, Jiangtao Feng, Xinbo Zhang, Yawen Ouyang, Yixin Wu, Lihao Wang, Xiaoqing Zheng, Xuanjing Huang, Hao Zhou
Abstract:
Multi‑step retrosynthesis requires recursively transforming a target molecule through successive reaction steps into purchasable building blocks, and recent work has begun to draw on large language model (LLM) reasoning to tackle the task. In its latest evolution, LLM reasoning has entered the long‑horizon regime: models pursue objectives across interdependent steps, interact with environments, and adapt to feedback. This shift has markedly advanced LLM capabilities, as illustrated by breakthroughs in AI‑driven mathematical research. At this frontier, we introduce Solvent, an 8B LLM that successfully extends long‑horizon reasoning to multi‑step retrosynthesis. Through a two‑round post‑training curriculum, Solvent learns to interleave iterative synthesis planning, external availability verification, and adaptive route revision. Experiments show that, following a 2.1× performance boost from RL, Solvent scores 61.86% across four benchmarks, surpassing mid‑scale language models and prior search‑based retrosynthesis systems, and rivaling frontier trillion‑parameter LLMs.
Authors:Shan Peng, Shuaiqi Cheng, Miao Peng, Chenyi Zi, Jia Li
Abstract:
Multi‑stage analytical tasks require LLM agents to use intermediate evidence to decide whether to continue, redirect or stop. Self‑reflection, in which an agent critiques its output, is widely studied, yet a critique does not show whether evidence changed the next action. We call that capability self‑restraint. Existing agent benchmarks largely measure task completion or output quality and leave it untested. DMAIC‑Bench evaluates it with 97 seeded tasks across 15 industrial cases in four sectors, built around Lean Six Sigma's explicit statistical prerequisites. Paired counterfactual variants require opposite actions at the same workflow position, distinguishing warranted restraint from indiscriminate stopping. We assess execution traces with a private scorer and final claims through independent expert review. Across 13 models from six providers, execution‑level routing accuracy averages 92% on matched GO decisions and 19% on NO‑GO decisions, counting any forbidden downstream computation as a route error. Execution traces show agents reporting correct diagnostics before performing analyses that those results disallow. Separately, blinded expert review of the 11 audited agents estimates that reports act on invalid evidence at 47% of reached NO‑GO decisions, with an interval from 31% to 61%. Explicit stopping permission helps some models without consistently restoring correct decisions at both GO and NO‑GO forks. Sixteen life‑sciences tasks in bulk ribonucleic acid sequencing (RNA‑seq) and case‑control genome‑wide association studies (GWAS) show that the same framework can evaluate evidence‑dependent decisions beyond industrial workflows. Project code and data are available at: https://github.com/psptpx/DMAIC‑Bench‑public.
Authors:Tianyi Xu, Huazheng Wang
Abstract:
Lifelong LLM agents must learn from an expanding history of interactions while making each decision within a bounded context window. Memory makes past experience available, but the agent may still need to infer how to solve each new task from the retrieved records. Skills provide reusable procedures, yet may omit the task‑specific facts needed to apply them. Because retrieved memory and skills shape the agent's behavior, they also influence the experience available for future learning. We introduce MemCryst, a framework for lifelong context adaptation through bidirectional learning of memory and skills with a frozen language model. Recurring episodes stored in memory support skill creation and revision, and the interactions guided by those skills supply new memory for subsequent learning. Facts retrieved from memory and learned skills jointly form the agent's context, while task outcomes adjust the future retrieval influence of the records actually served without rewriting their content. Across ten settings spanning embodied control, long‑document reading, conversation, mathematics, and code, MemCryst leads seven reported comparisons. On ALFWorld, joint serving achieves 63.17% success, compared with 59.83% for memory‑only and 50.50% for skill‑only serving. Code is available at https://github.com/tianyi0216/Memcryst.git
Authors:Julia Simon, Daniel Steininger, Andreas Trondl
Abstract:
Developing robust vision models requires large amounts of annotated data, especially for tasks involving fine‑grained class distinctions under variable environmental conditions. Even in domains with numerous existing datasets, differences in annotation formats, label definitions and targeted learning tasks hinder data reuse and integration. To address this challenge, we introduce ReLEAP, a scalable Python framework for unifying heterogeneous image datasets through data‑source abstraction, systematic reuse of annotations across learning tasks, and mapping of labels to a shared hierarchical taxonomy. We demonstrate its applicability in precision agriculture by integrating fifteen diverse datasets covering a broad range of plant species and annotation types. Beyond unification, ReLEAP enables systematic recycling of existing data and provides a robust foundation for learning‑based annotation upcycling. Leveraging this foundation, we conduct extensive cross‑dataset and fine‑tuning experiments to investigate how domain‑specific models trained on unified data and foundation models can be used to automatically generate higher‑level annotations under unsupervised, weakly supervised and semi‑supervised settings, and how dataset characteristics influence their effectiveness. Our results show that unified data access and annotation upcycling substantially expand the pool of reusable training data and annotations while improving data transparency, enabling more sustainable and resource‑efficient use of existing datasets. We release the framework and trained models for academic use to support accessible and collaborative research, particularly in data‑scarce and heterogeneous domains.
Authors:Chi Zhang, Yi Sun, Weichuang Zhang, Quan Chen, Jieru Zhao
Abstract:
Large‑scale Transformers have achieved remarkable success across NLP, computer vision, and robotics, but their growing computational and memory demands challenge efficient inference. Spatial accelerators offer advantages in on‑chip bandwidth and energy efficiency, yet designing them for large models remains costly and time‑consuming. Existing accelerator generators typically focus on single‑operation support, overlooking the benefits of multi‑operation pipelining, which limits throughput and energy efficiency. We propose TranSAGE, a compiler‑based framework that jointly optimizes scheduling for large‑scale Transformers under multi‑operation pipelining and generates corresponding accelerator implementations. Our approach integrates a design space exploration (DSE) engine with multi‑operation dataflow optimization and a dynamic programming‑based graph partitioning algorithm to efficiently identify pipeline‑friendly operation clusters and their optimal dataflow parameters. We further design spatial architecture primitives and lowering passes to automatically generate efficient RTL implementations. Evaluations on four representative LLMs demonstrate that TranSAGE achieves up to 2.41x higher throughput and 13.04x higher energy efficiency than commercial GPU/NPU baselines. Compared with state‑of‑the‑art accelerator generators, TranSAGE achieves up to 11.33x higher throughput and 6.07x higher energy efficiency. Our code is available at https://github.com/sjtu‑zhao‑lab/TranSAGE.
Authors:Gerardus Croonen, Andreas Trondl, Julia Simon, Daniel Steininger
Abstract:
Quality inspection of sugar beets after harvest and storage is critical to preserve sugar content and ensure optimal processing efficiency. The transfer of harvested beets to field clamps or trucks provides an opportunity to visually inspect individual beets automatically on loader conveyors. However, this is a challenging task as the large number of beets causes partial overlap and occlusion, complicating both annotation and detection. In this work, we propose an end‑to‑end image‑based pipeline for detecting partially occluded sugar beets in loading scenarios as a first step toward automated quality inspection. We introduce a dedicated benchmark and present an automatic annotation strategy that leverages the SAM 3 foundation model to generate bounding boxes. We compare models trained on manually annotated and automatically generated labels. Additionally, we investigate deployment considerations by benchmarking multiple inference configurations, including execution on an edge device, under realistic operational constraints regarding precision, inference time and power consumption. Our results show that, despite substantial occlusion and inherent annotation ambiguity, foundation‑model‑based labels can substitute for a substantial share of manual annotation, trading labeling effort for data volume at a modest accuracy cost. Furthermore, we demonstrate that a Raspberry Pi 5 equipped with a Hailo accelerator enables high‑performance, low‑power inference, making it a practical solution for mobile outdoor deployment.
Authors:Yue Feng, Boshen Xu, Zihao Yue, Xiaokun Liu, Miao Yang, Xin Tao, Pengfei Wan, Qin Jin
Abstract:
Existing temporal video grounding (TVG) benchmarks primarily focus on localizing target video segments according to descriptive text queries of the visual content. However, real‑world queries are more diverse and often require the model to first understand the user's intent before localization, such as finding visual evidence, identifying the cause of an event, or recalling a remembered moment, yet such queries remain largely overlooked. To mitigate this gap, we introduce RVG‑Bench, a benchmark for reasoning‑driven audio‑visual temporal grounding, where every query is constructed such that the target segment can only be identified through reasoning about the query's underlying intent. We carefully construct the benchmark and rigorously verify its data quality to ensure models truly comprehend the audio‑visual content. RVG‑Bench comprises 1,255 videos and 1,890 query‑interval pairs across eight reasoning types. We evaluate 22 models and find that the strongest, Qwen3.8‑Omni‑Flash, achieves only 49.6 mIoU. Through a series of controlled analyses, including oracle‑grounding probes and test‑time scaling experiments, we systematically examine where and why current models struggle, providing insights for future model development.
Authors:Weihong Li, Hao Jiang, Yaohua Zha, Tao Dai, Shu-Tao Xia
Abstract:
3D MLLMs recognize objects via pattern matching, which fails when geometrically similar objects require functional reasoning to disambiguate (e.g., dresser vs. wardrobe). Our per‑sample Geometric Confusion Score (GCS) exposes this failure: baseline accuracy is 33.7% on ModelNet40's most ambiguous 20% versus 84.0% on its least ambiguous 20%. We identify a function‑as‑annotation pattern, in which functional language decorates a known identity instead of supporting inference, and hypothesize that reversing this information flow should concentrate improvements on the most ambiguous samples. PFIR (Part‑level Functional Inference for Recognition) reverses that flow. Built on PartNeXt part segmentation, its data pipeline produces 507K QA pairs with functional CoTs (geometry \to function \to identity), and its progressive curriculum culminates in dual‑objective GRPO: strict functional reasoning and descriptive grounding for Objaverse's diverse distribution. The two RL objectives yield nearly orthogonal task vectors (\cos = 0.056), enabling composition via Task Arithmetic. Under a strict dual OOD protocol (Objaverse \to ModelNet40; XYZRGB \to XYZ‑only), PFIR raises zero‑shot accuracy from 60.5% to 66.7%, with gains concentrated 2.6× more on geometrically ambiguous samples‑‑‑supporting part‑level functional reasoning as an effective route to more generalizable 3D recognition. Code: https://github.com/this233/pfir‑3d.
Authors:Linye Lyu, Jiawei Zhou, Daojing He, Yu Li
Abstract:
Modern autonomous driving systems widely adopt multi‑view Bird's‑Eye‑View (BEV) based 3D perception models due to their superior performance. Despite their success, the robustness of these models against adversarial camouflage attacks remains largely unexplored. Existing camouflage attacks primarily target single‑view 2D detectors, limiting their effectiveness in multi‑view 3D perception settings. To address this gap, we propose BEVCA, a novel adversarial camouflage generation framework tailored for multi‑view BEV‑based 3D perception. Our framework integrates a BEV‑feature‑based adversarial loss with a multi‑view neural rendering module, enabling effective and transferable camouflage attacks across different models and tasks. Extensive experiments in both digital and physical settings demonstrate that BEVCA outperforms state‑of‑the‑art baselines and exhibits strong robustness under real‑world conditions. Our code is available at https://github.com/WAVE‑AT‑IDEA/BEVCA.
Authors:Jiaxing Sun, Junyuan Gao, Ruikang Zhang, Hao Zheng, Lingli Ge, Xiao An, Jiang Wu, Huazheng Zeng, Jiahe Song, Chen Zhu, Haote Yang, Jingchao Wang, Lu Chen, Yuanyuan Cao, Ruijie Zhang, Shengxin Lu, Weijia Li, Lijun Wu, Conghui He
Abstract:
Screen Tables and Substrate Scopes are information‑dense figures widely used in synthetic chemistry literature, yet automated extraction of structured reaction data from them remains challenging. We introduce RxnExBench, a multimodal benchmark for fine‑grained reaction‑level evaluation. Built from high‑impact chemistry journals, it contains 3,565 expert‑annotated reactions and 2,361 molecular structures. The dataset adopts a unified schema with 15 reaction roles and 26 core fields, supporting downstream AI‑for‑Chemistry tasks such as retrosynthetic planning, yield prediction, and reaction condition recommendation. RxnExBench further provides a reasoning‑driven difficulty taxonomy, a modular annotation pipeline, and the Oracle‑MolRec protocol, which disentangles molecular recognition from higher‑level reasoning. Evaluation of 9 state‑of‑the‑art VLMs shows that performance degrades substantially as schema complexity and reasoning demands increase. Under the Oracle‑MolRec setting, results further reveal that accurate molecular recognition alone is insufficient, with multimodal integration and chemistry‑aware reasoning remaining the primary bottlenecks. We release all data and code to facilitate future research in chemistry‑aware multimodal understanding and AI‑for‑Chemistry dataset construction. The dataset and code will be released at https://github.com/opendatalab/RxnExBench.
Authors:Jiawei Dong, Zilong Bai, Pengtian zhu, Peng Zhou
Abstract:
Large Language Models (LLMs) have gained widespread adoption in various natural language processing tasks, but suffer from hallucination issues where they generate unfaithful or inconsistent content. While Reinforcement Learning with Verifiable Rewards (RLVR) has shown promising applications in reasoning tasks, its application to span‑level hallucination detection is challenged by localization ambiguity, reward sparsity on hard examples, and uncontrolled length inflation. To address these issues, we introduce Socratic Guided Policy Optimization (SOPO), a multi‑turn RL framework for robust hallucination detection. The key innovation of SOPO is a Socratic feedback mechanism, where a teacher model guides the reasoning trajectories during rollout by providing indirect hints inferred from the ground truth. This teacher‑guided approach enables the model to generate more effective reasoning trajectories for hard examples, thus alleviating reward sparsity. Furthermore, we incorporate Turn‑based Advantage Scaling to penalize verbosity bias, encouraging the model to develop efficient and autonomous reasoning capabilities. Empirical results on RAGTruth demonstrate that SOPO (4B) achieves state‑of‑the‑art performance, surpassing leading proprietary models (e.g., GPT‑5) and standard RL baselines, while obtaining higher reported mean scores than GRPO on four out‑of‑domain benchmarks, with mixed evidence against CAPO. We release our training and evaluation code alongside the 1.7B and 4B SOPO models at https://github.com/patsnap/SOPO.
Authors:Hyojun Kim, Hyeseon An, Yo-Sub Han
Abstract:
Green‑red watermarks favor tokens from a green list that a secret key derives from the preceding tokens. Diffusion language models, however, often choose a token before its preceding tokens, and the token later shapes its neighbors' green lists. Prior work therefore tilts each choice to raise the expected number of green tokens at its own and its neighbors' positions. Building on this objective, we first learn two components that prior work keeps fixed: vocabulary bands that serve as green lists, separate competing candidates and keep the null distribution exact, and a foresight predictor that estimates how each candidate shifts its neighbors' predictions. Second, we count green tokens as the detector does, once per distinct context‑token pair, so repetition earns nothing. On WaterBench with LLaDA‑8B‑Instruct, each learned component raises the z‑score at equal distortion, and their gains are nearly additive. With a strong watermark, ours detects at least 96% of responses at a 1% false‑positive rate while matching the LLM‑judge score of unwatermarked text. Ours also detects more responses on Dream and as many or more on code and question answering without a significant loss in task accuracy.
Authors:Victor Shea-Jay Huang, Rongqing Li, Zhangyue Yin, Kelong Mao, Yaqi Zhao, Xinyuan Wang, Yao Yin, Lirui Luo, Renrui Zhang, Hongsheng Li
Abstract:
Repository‑level coding benchmarks largely assume that tasks and environments remain unchanged throughout execution. In software development, code, dependencies, runtime state, and requirements can change while an agent is still working. We introduce SWE‑Shift, a benchmark for evaluating coding agents under such mid‑execution changes. From the same saved history and workspace, a Clean continuation proceeds without intervention, while a matched Shifted continuation receives a task‑relevant update. Success requires completing the current task, preserving still‑valid work, and satisfying the applicable state and evidence checks. The benchmark contains 120 scenarios across 12 change families, each evaluated at nine scheduled intervention positions. We evaluate 17 coding‑agent systems, all of which have lower Shifted than Clean success. GPT‑6 Astra drops from 67.35% to 24.79%, with 20.34% paired success. Tool/session changes and stale evidence yield zero paired success across all systems. Late‑position paired success is lower for some systems and higher for others. These results show that success on the unchanged task can coexist with failures to preserve work, recover state, or refresh evidence after an update.
Authors:Dongki Kim, Sangwoo Park, Jinheon Baek, Soyeong Jeong, Sung Ju Hwang
Abstract:
Scientific discovery requires identifying the scientific principles underlying observed phenomena to guide further investigation. As these empirical principles are often unknown in complex discovery processes, they are constructed via a hypotheses‑to‑principle pipeline to obtain recurring causalities from progressively accumulated observations during test‑time discovery. However, constructing this knowledge requires substantial domain expertise, time, and effort, making this translation stage a bottleneck for scientific discovery. To address this limitation, we propose Principia, an LLM‑based framework that constructs and continually refines scientific principles at test time. Drawing on how human researchers develop principles, Principia forms comparative causal hypotheses from related inputs and their observed outcomes, then synthesizes recurring relationships across these hypotheses into inductive principles. By applying the constructed hypotheses and inductive principles to the task, it obtains new observations, revises the hypotheses used based on these new observations, and reconstructs principles from the updated hypotheses, allowing scientific knowledge to continuously evolve throughout test‑time discovery. Across benchmarks spanning molecular optimization, protein engineering, gene perturbation, and molecular property prediction, Principia outperforms baselines relying on internal knowledge, external sources, or experience summaries, while generating principles that are plausible and consistent with the scientific literature, demonstrating its effectiveness in constructing and refining task‑specific hypotheses and principles at test time for scientific discovery. Our code is available at https://github.com/DongkiKim95/Principia.
Authors:Wenyu Liang, Hongzhi Yin, Tongtong Wu, Fan Liu, Kexin Dai, Weiqing Wang
Abstract:
Existing personalized information retrieval methods mainly model differences in user needs through user preferences and query intent, while paying little attention to the personalized expression differences that arise when different users express the same retrieval need. Existing real query data typically lack stable user identities, long‑term expression histories, and intent‑matched counterparts, making it difficult to systematically evaluate the impact of personalized expression differences on retrieval outcomes. To this end, we propose PersonalQuery‑Benchmark (PQB). PQB models personalized syntactic expression and personalized writing errors from users' historical texts, constructs personalized syntactic expression queries and personalized writing‑error queries, and validates benchmark quality through LLM assessment and independent human evaluation. PQB covers three Amazon product domains with 6,516 query instances in total, and is evaluated on five retrieval models representing lexical, learned sparse, dense bi‑encoder, late‑interaction, and hybrid architectures, together with two LLM‑based ranking models. Results show that even when the information need is held fixed, personalized expression differences still induce measurable ranking variation; different retrieval models exhibit clearly different sensitivities to syntactic variation and writing‑error variation, while higher overall retrieval effectiveness does not necessarily correspond to stronger robustness against expression variation. PQB provides a new benchmark and evaluation framework for systematically studying personalized expression robustness. The associated code and dataset are available at the following address: https://github.com/cpxjazz123/PersonalQuery‑Benchmark.
Authors:Yanshuo Chen, Chenxi Liu, Reza Shirkavand, Heng Huang
Abstract:
Post‑training quantization (PTQ) is a key technique for efficiently deploying large language models, as it compresses weights into low‑bit representations without retraining. Existing PTQ solvers, including GPTQ, LDLQ, and block coordinate descent, commit each weight block to a single codeword before its neighbors are considered, leaving near‑optimal alternatives unexplored. In this work, we cast row‑wise PTQ as posterior inference over discrete codebooks, in which existing solvers correspond to a zero‑temperature, hard‑decision limit retaining only the top codeword per block. We propose Message‑Passing Quantization (MPQ), which preserves soft posterior information across blocks during inference and commits to hard codewords only at the output stage. MPQ first uses vector approximate message passing to compute approximate blockwise posteriors, then refines a hard‑decision initialization by exact block coordinate descent. Across 2‑bit lattice and 3‑ and 4‑bit scalar quantization on dense language models from Llama, Qwen, and Mistral families, MPQ improves perplexity and zero‑shot accuracy over strong hard‑decision baselines, and composes favorably with preprocessing techniques such as activation‑aware scaling and incoherence processing. Our implementation is available at https://github.com/poseidonchan/MPQ.
Authors:Mahdi Naser Moghadasi
Abstract:
We report a systematic dissociation in modern text embedders: sentiment is almost perfectly decodable from their embeddings, yet almost entirely absent from their metric. Across eleven frozen encoders spanning 2021 to 2026, including 2026‑generation models and the 4B‑parameter LLM embedder Qwen3‑Embedding‑4B, a linear probe recovers sentence polarity at up to 91.3% accuracy while cosine similarity separates polarity by a margin of only 0.003‑0.064. Qwen3‑Embedding‑4B, the strongest probe in the panel, reaches 91.3% probe accuracy but only 64.3 retrieval MAP. Because retrieval, clustering, deduplication, and retrieval‑augmented generation operate on the metric rather than on a trained probe, these systems are effectively blind to information their encoder demonstrably contains. A task‑conditioning control suggests this reflects the pretraining objective rather than a prompting artifact: conditioning Jina‑v5 on its classification task rather than its retrieval task recovers only about half of the MAP gain that injection achieves on the same weights (63.2 to 73.8, versus 85.1), and leaves the alignment gap essentially unchanged. Sentiment‑Injected Sentence Embeddings (SISE) train a small (0.13M‑0.69M‑parameter) supervised‑contrastive head on a frozen encoder in seconds on a CPU, using the labelled training set. The head improves every encoder released since 2022 on full‑data probe accuracy (up to +5.8 points), retrieval MAP (up to +26.0) and polarity margin (7‑61x). This is a specialization, not a free gain: the compact 128‑d head loses 38‑41 Spearman points on STS‑B, while a fused variant that concatenates it with the base embedding loses 2.4‑12.0. With the head, a 0.6B encoder at 128 dimensions matches a 4B encoder given the same head (91.8 vs. 90.3 MAP). We release a CPU‑reproducible pipeline and a reusable diagnostic, the Attribute Alignment Gap.
Authors:William Jeffries
Abstract:
Agents and verifiers often rely on the text a tool prints, and that text can misrepresent what the tool actually did, even when the tool's computation is correct. We saw this happen in a production RAG research system used by the Pilsudski Institute in New York, where tool output formatters misled an agent into giving wrong or incomplete information to users. Each tool did its job correctly, but its output omitted search limits, result caps, or identifier namespaces. We also recreated an unlabeled estimate caught in code review before release. We studied whether these formatters misled agents and verifiers and how much the repaired wording helps. We fed tool results to agents using two formatter versions: one with the key omissions and one with all missing details clearly stated. That difference alone changed what the agents claimed and what the verifiers accepted. Unsupported agent claims dropped substantially for three of the four failure types. In one test, verifiers went from accepting incorrect answers 90 out of 90 times to rejecting them 90 out of 90 times. A control showed the verifiers could catch the errors once they were given the missing facts and asked to judge correctness. We include the audit prompt we used with Codex and Claude Code in an exploratory demonstration, written with prior knowledge of our own system's defects. When pointed at an old version of our own system, it rediscovered the defects that were still live and surfaced twelve more we had not catalogued. When pointed at OpenClaw and Hermes, two of the best‑known open‑source agentic systems, it found warped formatters in both. We opened PRs to fix both, and the OpenClaw fix has already been merged. Practitioners can run the same prompt on their own systems today.
Authors:Jiabing Yang, Yuan Xu, Yixiang Chen, Qisen Ma, Peiyan Li, Kai Wang, Chenhang Cui, Tao Yu, Zichen Wen, Wen Ye, Bowen Fang, Ziheng He, Xiangnan Wu, Yingda Li, Yan Huang, Liang Wang
Abstract:
Flow‑matching Vision‑Language‑Action (VLA) models generate expressive continuous trajectories for robotic manipulation, yet balancing real‑time responsiveness with execution stability remains a persistent challenge. Standard deployment workflows typically treat numerical solver depth and execution horizon as decoupled choices. The horizon amortizes inference latency, while solver depth is kept high to preserve integration fidelity. In this paper, we reveal a fundamental compute‑execution coupling in flow‑matching VLAs governed by a systematic sign reversal. While coarse numerical integration degrades precision under rapid replanning, it markedly improves task performance under extended open‑loop commitment. Geometrically, numerical truncation contracts lateral action dispersion around an invariant trajectory centroid, serving as an implicit regularizer that suppresses compounding open‑loop drift. Across LIBERO, RoboTwin 2.0, and real‑world Franka manipulation on representative VLA backbones including π_0, π_0.5, and GR00T, exploiting this joint coupling consistently achieves superior task success while drastically lowering inference latency, entirely training‑free and without architectural modifications. These findings establish numerical sampling depth as an operational variance regulator, shifting its role from a static precision budget to an active deployment knob in robot learning. Project page: https://jiabingyang01.github.io/Horizon‑Depth‑VLA/
Authors:Minho Park, Kinam Kim, Donghu Kim, Byungkun Lee, Dongyoon Hwang, Yongjae Shin, Junha Hyung, Hojoon Lee, Jaegul Choo
Abstract:
Behavior cloning for robotic manipulation is highly sensitive to the coverage of its training data, yet collecting diverse successful demonstrations remains costly. We study reinforcement learning (RL) not as a deployable controller, but as a data generator for downstream visuomotor imitation learning. Our key premise is that multiple successful solution modes cover a broader range of successful trajectories than repeatedly collecting a single narrow solution. We introduce DiscoDemo, which trains a privileged latent‑conditioned RL generator with a parallel reverse curriculum and a success‑gated diversity objective to discover diverse solutions while preserving task success. The trained generator produces multiple successful closed‑loop behaviors, which are safety‑filtered and exported as a static dataset for downstream imitation learning. Across four manipulation tasks, DiscoDemo produces broader successful‑state coverage and improves downstream visuomotor policies in both simulation and the real world. Averaged across tasks, downstream success increases to 62% from 40% for the strongest baseline in simulation, and to 78% from 39% on the real robot, respectively. These results suggest that successful solution diversity improves imitation learning for a pretrained flow‑matching VLA and remains complementary to corrective data collection. Project page: https://davian‑robotics.github.io/DiscoDemo
Authors:Mulugeta Weldezgina Asres, Lei Jiao, Christian Walter Omlin
Abstract:
Despite the promising development of deep learning for video anomaly detection (VAD), explainability (XAI) of these models remains a challenge. Explainable and privacy‑protecting VAD builds transparency and trust that facilitates adoption of computer vision systems, including safety and security monitoring. However, anomaly explainability in privacy‑preserving VAD systems is underexplored. In this study, we present the XAVAD approach, a unified descriptive anomaly XAI for an anonymized VAD, that integrates flexible zero‑shot anonymization, anomaly type categorization, visual localization, and scene context description. We provide empirical and qualitative assessments of multifaceted explanations integrated with full‑body visual anonymization for privacy protection. We adopt a multimodal CLIP embedding for videos, using improved LLM‑assisted prompting and pool‑scoring mechanisms for anomaly‑type identification. We enhance Grad‑ECLIP for spatial anomaly heatmap localization, leveraging computation and accuracy through our novel prompt‑guided patching autofocus mechanism. We employ VLMs to generate context and reasoning descriptions of detected anomalies through curated prompt engineering. We have evaluated the accuracy and computational cost of XAVAD on several publicly available CCTV and video datasets. Our experiments demonstrate substantial gains of our XAVAD over state‑of‑the‑art benchmark approaches and provide insights into privacy protection trade‑offs in XAI for VAD. We further present ablation studies and assessments across diverse configurations and modeling alternatives to demonstrate the efficacy of our proposed methods. Code: https://github.com/muleina/XAVADhttps://github.com/muleina/XAVAD .
Authors:Yiran Xiahou, Siliang Qin, Kai Chen, Shengzhi Zhang, Renyang Liu
Abstract:
Diffusion‑based image generation models produce high‑quality images but may generate harmful or policy‑violating content, motivating machine unlearning (MU) to selectively remove target concepts while preserving general capabilities. However, existing MU approaches often struggle with the distributed and entangled nature of concept representations, leading to inefficient optimization and imprecise edits when applied through large‑scale fine‑tuning over full diffusion trajectories. To bridge this gap, we propose Concept Acupuncturist (ConceptACP), a novel MU framework that integrates interpretability‑guided analysis with inference‑free fine‑tuning for efficient and precise concept removal. By establishing mappings between semantic concepts and cross‑attention blocks in the U‑Net denoiser and localizing the most concept‑relevant block for targeted optimization, ConceptACP avoids full diffusion trajectory optimization, thereby improving fine‑tuning efficiency. To mitigate concept entanglement, we propose a feature‑level correction mechanism based on Sparse Autoencoders (SAEs), where SAE‑derived sparse, semantically meaningful feature activations are manipulated to construct targeted correction signals, which are then projected back to the output space of the corresponding block and used as supervision for precise concept removal. Consequently, ConceptACP eliminates the need for external inference‑time modules while achieving efficient and fine‑grained unlearning. Instantiated on SDXL‑Turbo, ConceptACP improves the forgetting success rate by 36.34 percentage points in the standard one‑step generation setting on average over the best‑performing baseline, while preserving image quality and prompt fidelity, and generalizes effectively across other Stable Diffusion backbones. Our code is available at https://github.com/Eran‑Xia/ConceptACP.
Authors:Carson Cary, Yukai Jin, Yihao Huang, Zhankai Ye, Fangyi Li, Ting Hua, Xin Liu, Shangqian Gao
Abstract:
Mixture‑of‑Experts (MoE) models reduce computation through sparse activation, but storing their expert parameters remains a major memory bottleneck. We introduce GS‑MoE, a training‑free compression framework that addresses redundancy across and within experts through two compression axes: shared‑subspace rank and expert intermediate width. Groupwise low‑rank representations capture shared structure across experts while retaining expert‑specific projections, and Nyström‑based neuron compression reduces intermediate width within each expert. For fixed shared subspaces, we derive per‑expert reconstruction‑error reductions from the reconstruction objective and use them to guide expert grouping. Budgets are allocated hierarchically using a gradient‑free layer‑importance score to guide the allocation of rank and width budgets across layers, which eigenvalues and leverage then distribute as non‑uniform ranks across groups and intermediate widths across experts. Experiments demonstrate the complementary benefits of compression along both axes, particularly at high compression rates on fine‑grained MoE models. On DeepSeek‑MoE‑16B‑Base, at a 60% reduction in total model parameters, GS‑MoE retains approximately 90% of the uncompressed model's average reasoning performance, compared with 73% for D^2‑MoE, while achieving 20% lower out‑of‑domain perplexity. We provide a code implementation at: https://github.com/cbacary/GS‑MoE
Authors:Jakub Poćwiardowski, Mateusz Modrzejewski
Abstract:
Sparse autoencoders (SAEs) can expose features of a generative model without predefined labels, but in language models the evidence used to interpret these features has proven an unreliable guide to which of them steer the output. Whether interpretations of SAE features predict control in music generators has not, to our knowledge, been tested. We develop an end‑to‑end protocol for discovering and evaluating SAE features in text‑to‑MIDI generation and apply it to text2midi and MIDI‑LLM. The protocol characterizes features through correlations with musical metrics, shared instrument programs, and automated descriptions, each with its own control: note‑density adjustment, matched inactive excerpts, and permutation calibration. It then tests candidates by intervention, measured directly in the generated MIDI against unmodified generation and a matched random direction, ranks them in screens, and confirms selected features on new prompts. Evidence that survives these controls points to features that steer the output. Descriptions exceed permutation controls for instrumentation, and a feature whose description names a synth‑lead program introduces that program in a prespecified confirmation and in a replication on disjoint prompts and reappears in independently trained dictionaries. Features selected by density‑adjusted correlations show a small positive shift of their target metric as a family, whereas features selected by raw correlations show none. An atlas provides MIDI, piano‑roll, and audio examples for every feature of the targeted screen.
Authors:Monojit Banerjee, Zhujun Lan, Defu Cao
Abstract:
Deploying language agents in real applications demands more than the right answer: an agent must execute a multi‑step business process correctly, follow domain‑specific rules, and interact with a user across a dialogue, where the gap between saying and doing is the gap between a satisfied customer and a real loss. We argue that a deployable‑agent benchmark should jointly assess four facets: the outcome an agent reaches, the trajectory it takes, its operational viability, and its policy compliance. Customer relationship management (CRM), where mutable records meet operational rules and consequential effects, exercises all four. We introduce CRM‑Agent‑Bench, an execution‑grounded benchmark of 555 stateful, multi‑turn tasks across nine business domains and 36 capability tags. Each task supplies a task‑specific executable contract drawn from checks over actions, arguments, order, response, target state, and protected state; success is the strict conjunction of the checks declared for that task. We evaluate the complete agent system through these contracts and report scored coverage, latency, and cost. Across sixteen systems evaluated on the audited final release, strict success spans 64.32% to 83.60%; the closely grouped leading systems nonetheless diverge sharply in cost and per‑domain reliability. CRM‑Agent‑Bench offers an auditable framework for measuring contract‑grounded business‑transaction integrity and adherence to task‑declared benchmark policies, with a portable structure designed for extension as models improve. Github repository: https://github.com/salesforce‑misc/crm‑agent‑bench
Authors:Eric Yocam, Varghese Vaidyan, Paris Kalathas, Patrick Tague, Bruce DeBruhl, Yong Wang
Abstract:
Agentic AI systems using the Model Context Protocol (MCP) are vulnerable to indirect prompt injection through untrusted tool outputs. We demonstrate cross‑modal trust laundering, in which instructions concealed in an email‑signature image using least‑significant‑bit (LSB), discrete cosine transform (DCT), or metadata steganography bypass text scanners and enter a multi‑agent workflow when decoded by a tool. To counter this threat, we propose decode‑time provenance binding, which combines HMAC‑authenticated trust labels with an MCP‑maintained provenance ledger and context‑taint tracking. The mechanism labels decoded content as untrusted and blocks critical tool calls influenced by that content, even when a compromised agent substitutes a valid trusted label. We evaluate the approach using a real MCP testbed with three steganographic carriers and 24 adaptive payloads. The undefended pipeline exhibits 100% attack success, while keyword and regex defenses each permit 62.5% of attacks. Our mechanism blocks all attacks (0% success), including adversarial label substitution across mediated agent boundaries. A four‑component ablation validates the roles of decode marking, cryptographic authentication, lineage tracking, and sink enforcement. An audited trusted‑promotion mechanism increases legitimate‑task completion from 67% to 100% without compromising protection. The approach provides carrier‑independent containment within fully mediated MCP execution environments. Code and data are publicly available.
Authors:Muhammad Azam Afridi, Rajab Ali, Marco Passarotti
Abstract:
Demonstrations for in‑context learning are often retrieved by semantic similarity, which may be a poor proxy for relevance in structured prediction. We study demonstration selection for Universal Dependencies parsing with a POS‑signature policy that represents sentences with weighted part‑of‑speech statistics and retrieves three length‑compatible, gold‑parsed examples from the labeled training split. We evaluate two open‑weight language models without fine‑tuning on all 2,077 sentences of the UD English Web Treebank test set, comparing the policy with shot‑ and format‑matched fixed and semantic three‑shot baselines, as well as zero‑shot, chain‑of‑thought, and critique‑refinement systems. POS‑signature retrieval yields the highest labeled attachment score among the tested prompting systems for both models (72.4 and 73.2), outperforming semantic retrieval by 1.9 and 2.7 points; paired‑bootstrap 95% confidence intervals exclude zero for both contrasts. Chain‑of‑thought and refinement comparisons vary across models. These exploratory results favor the tested POS‑signature retrieval approach on English‑EWT. We release the prompts, archived outputs, and evaluation code.
Authors:Brianne Lee
Abstract:
Three students answered the same exam prompt by hand: "What is happiness? Are you happy?" Their ages, recorded at collection, are 17, 28, and 45. The 17‑year‑old's answer is hedged and formally phrased. Asked to match each answer to an age and gender label, most frontier language models confidently assign it to an adult. This paper reports a structured case study of that failure across six models: Claude Fable 5, Gemini 3.1 Pro, GPT‑5.6 Sol, Grok 4.5, GPT‑4o (2024), and DeepSeek V4. We ran 1,081 API sessions and scored 3,225 responses across six experimental conditions that manipulate framing, disclosure format, and audience‑awareness instruction. Three of six models perform at or near chance (1/3) on the matching task while reporting 73‑88% mean confidence. The youngest writer's answer is misattributed to an adult in up to 90% of trials. Removing the multiple‑choice scaffold collapses accuracy further: three models never place within three years of the true age once open‑ended estimation is required. Institutional framing ("a community‑college biology final") significantly degrades the best‑performing model's accuracy; a neutral label does not. A single audience‑awareness instruction significantly improves one model (+21 points) and moves another to ceiling, with no detectable effect on the three weakest models. Stated confidence is uncorrelated or inversely correlated with correctness for five of six models. After ground truth is revealed, one model names its own stereotyping in 68 of 114 explanations, yet shows the lowest accuracy and highest confidence of any model under blind presentation: articulate retrospective diagnosis that does not track the quality of its judgments. This is a three‑item, single‑topic, single‑language case study meant to isolate a mechanism, not to estimate a population bias rate. Data, code, and raw model outputs are publicly released.
Authors:Victor Morand, Yuxuan Zong, Mathias Vast, Josiane Mothe, Benjamin Piwowarski
Abstract:
Late‑interaction models offer state‑of‑the‑art efficiency‑effectiveness trade‑offs for neural retrieval by enabling document representations to be pre‑computed and efficiently indexed. Yet, their relatively simple interaction mechanisms, typically MaxSim, can limit their effectiveness compared to cross‑encoders. MICE, a recently introduced re‑ranking architecture, aims to combine cross‑encoder‑like effectiveness through expressive transformer‑based scoring with reduced inference‑time computation through document‑side pre‑computation. However, no practical way to store these document representations was explored. We address this limitation here and introduce MICE‑P, the first end‑to‑end retrieval pipeline with MICE, based on PLAID to index the contextualized document token representations. We conduct a controlled comparison of MICE, ColBERT and cross‑encoders by training comparable models using the Ettin Family under matched experimental conditions. Our results show that MICE‑P defines a new class of retrieval architecture with an efficiency‑effectiveness trade‑off of its own: it matches late‑interaction retrieval throughput while retaining the effectiveness of cross‑encoders.
Authors:Ranran Huang, Wenjing Bian, Weixun Luo, Ye Mao, Krystian Mikolajczyk
Abstract:
We present PixGR, an efficient framework that integrates lightweight pixel‑space generative modeling into feed‑forward 3D Gaussian reconstruction to model scene uncertainty. Context observations are encoded once into a scene latent representation, which conditions generative decoding from noisy target views to produce a shared 3D Gaussian representation. We formulate this process using pixel‑space flow matching, enabling end‑to‑end optimization of 3D reconstruction with direct pixel‑space generative supervision through differentiable rendering. This avoids VAE‑based latent compression and error propagation inherent in previous two‑stage pipelines. To preserve feed‑forward efficiency, PixGR caches the scene representation and performs iterative inference using a lightweight DiT decoder. Experiments demonstrate that PixGR outperforms regression‑based approaches in challenging sparse‑view settings, particularly with widely spaced observations and view extrapolation. Meanwhile, it maintains sub‑second inference and remains competitive with substantially heavier pretrained generative models. Our project page is available at https://ranrhuang.github.io/pixgr/.
Authors:Tianxiang Xia, Kai Lion, Niao He, Michael Muehlebach, Melanie Zeilinger, Hao Ma
Abstract:
Scaling laws continue to reward increases in model scale, while the resulting computational and memory demands drive greater reliance on cloud infrastructure for inference. Yet unreliable network connectivity, communication delays, and privacy requirements can make such reliance impractical, motivating local execution on edge devices. Local deployment requires models to fit within limited memory and execute efficiently with the available computational resources. Recent work has shown that learning sparse‑plus‑low‑rank representations enables elastic compression of a single checkpoint to multiple memory budgets. However, fewer parameters do not necessarily translate into faster inference. Indexing overhead and irregular memory access in unstructured sparse components can offset computational savings. To address this critical deployment barrier, we introduce S‑SALAAD, a unified framework that jointly learns the model weights and their low‑rank and structured sparse components during training. The framework supports a family of hardware‑compatible structured sparsity patterns. It couples this structure learning with budget‑aware compression and dedicated CPU and GPU kernels that exploit the learned structure for acceleration, enabling efficient inference from a single checkpoint across memory budgets. Experiments on Llama2 and Qwen3 models, covering both pretraining and distillation, show that S‑SALAAD advances the empirical Pareto frontier in perplexity, memory, and inference latency across multiple hardware platforms relative to the evaluated baselines. These results show that structured learning can translate elastic compressibility into reduced memory requirements and lower inference latency, enabling practical deployment of large models on edge devices.
Authors:Damian Hattler, Qiang Liu, Nils Thuerey
Abstract:
Partial differential equations (PDEs) are indispensable tools for modeling complex phenomena in physics and engineering. Physics‑Informed Neural Networks (PINNs) have emerged as a powerful alternative to traditional numerical solvers by directly incorporating physical constraints into neural network loss functions. Despite their advantages, PINN training requires multi‑objective optimization, which often leads to significant gradient conflicts among boundary, initial, and residual loss terms. The Conflict‑Free Inverse Gradient (ConFIG) method addresses these conflicts; however, integrating ConFIG with adaptive optimizers such as Adam introduces a critical vulnerability: the conflict‑free direction generated by ConFIG is further processed by the optimizer, and the resulting weight update may no longer be conflict‑free with respect to the task gradients. To address this issue, Cautious ConFIG is proposed as a concise three‑line extension to the ConFIG algorithm that empirically restores the conflict‑freeness guarantee prior to the weight update. The method is evaluated on three PINN benchmarks: the 1D Viscous Burgers' equation, the 1D Nonlinear Schrödinger equation, and the 2D Kovasznay flow. Experimental results demonstrate consistent performance improvements over the standard ConFIG baseline. Additionally, the stabilizing effect of cautious masking and its reaction to conflict severity, task number, and optimizer lag are investigated using a simple toy system.
Authors:Jialei Li, Dingxian Wang, Xiaoyan Zhao, Yimeng Bai, Xuanyi Zhu, Yang Zhang, Chao Yin, Wentao Shi, Ganesh Prabhu, Karthik Gomadam Rajagopal, Brett Levert, Darvin Yi, Andrew Rabinovich, Zhao Chen
Abstract:
Greenfield software development in freelance settings requires coding agents to turn incomplete client requests into working software. This makes requirement clarification essential, yet clients may be unable to provide continuous feedback over long development cycles, and receiving their answers does not guarantee correct understanding of complex requirements. These challenges motivate two benchmark design choices: clarification precedes code implementation, and its effectiveness is assessed through explicit tests of requirement understanding. Guided by these choices, we introduce SWE‑Clar, a benchmark grounded in real, high‑value client commissions to evaluate requirement clarification and code implementation in this setting. It comprises 100 tasks based on real software development jobs on Upwork, each with at least US\4,000 in total payments. Each task begins with multi‑round interaction between a coding agent and a client agent, followed by separate runs for code implementation with acceptance testing and clarification assessment. For the latter, we construct 512 questions paired with reference answers from real client‑freelancer interactions, covering both explicitly stated requirements and those inferred from the conversational context. Both runs begin from the same clarification context but remain isolated, preventing assessment questions from guiding code implementation. Experiments with nine frontier models show that even GPT‑6 Astra, which has the highest observed implementation score, achieves a task pass rate of only 67%, underscoring the challenge of independent software delivery in this setting. The dataset and evaluation code are available at https://github.com/upwork/SWE‑Clar.
Authors:Youyuan Long, Gokhan Solak, Arash Ajoudani
Abstract:
Dynamic manipulation of deformable linear objects (DLOs) remains challenging due to underactuated, high‑dimensional dynamics, delayed deformation responses, and the presence of multiple feasible motions under obstacle constraints. This paper presents BM2PC, a Bayesian Multimodal MPC framework for agile manipulation of DLOs. Building on the control‑as‑inference posterior, BM2PC formulates multimodal planning as a set optimization problem that balances control quality and behavioral diversity. An efficient incremental update maintains multiple promising control modes with low additional computational overhead. For real‑world deployment, BM2PC incorporates an execution dynamics model that predicts actual end‑effector motion and compensates for execution mismatch during fast manipulation. We evaluate BM2PC on rope tracking and obstacle avoidance in simulation and hardware, and on simulated disturbance rejection. In trajectory tracking, BM2PC generalizes better to unseen trajectories than a learning‑based baseline without task‑specific policy training or fine‑tuning. Real‑world experiments demonstrate accurate long‑horizon tracking, reducing RMSE by 44.0% on the longest task. In obstacle avoidance, BM2PC achieves comparable performance to existing multimodal MPC methods while reducing planning time by 29% to 66%. These results demonstrate accurate dynamic control and efficient multimodal planning within a unified MPC framework. The core implementation of BM2PC and supplementary videos are available at https://github.com/Longyouyuan/BM2PC.
Authors:Xiao Tan, Yixuan Wang, Zhexi Wang, Wenzhen Yuan
Abstract:
Coding agents can translate language instructions into robot programs. For contact‑rich manipulation, a key challenge is to generate and refine control logic that responds to physical interaction. We present FORCA, a coding agent that takes visual observations and wrist force/torque (F/T) measurements as input to write Python programs for contact probing and force‑responsive motion. Between executions, the agent inspects force records alongside visual outcomes to diagnose failures and revise both motion and force‑feedback control logic. FORCA requires no demonstrations, task‑specific training, or simulation. Across five real‑robot tasks, we demonstrate its effectiveness in three forms of contact‑rich manipulation: improving manipulation accuracy, adapting to changes in object or surface properties, and regulating contact under surface variation and object motion. Compared with a vision‑only variant of the same agent, FORCA improves grasp accuracy and task success and reduces contact loss in wiping and peeling.
Authors:Jiaxi Liu, Keshu Wu, Weizhe Tang, Bofeng Cao, Sikai Chen, Xiaopeng Li, Steven T. Parker, Bin Ran
Abstract:
Temporary traffic control redirects vehicles through work zones, requiring automated vehicles to connect an interpretation of the visible devices to a continuous passing path. Geometric predictors can provide several plausible paths, yet selecting the one consistent with the temporary control remains difficult. We propose SAGE (Semantic‑to‑Action Grounded Execution), a read‑decide‑execute‑check framework that separates path construction, path preference, and control compliance within a shared candidate space. The central idea is to use a vision‑language model's (VLM's) trajectory as a preference over intact geometric candidates. A role‑decoupled generator constructs the candidates, and rank fusion combines each candidate's geometric score with its agreement with the VLM trajectory. Supervision structured around the Manual on Uniform Traffic Control Devices (MUTCD) trains SAGE‑VLM to state the lane status and hazard side before its passage decision and trajectory. The predicted hazard side enters image‑space predicates derived from MUTCD provisions, which support a final intervention bounded in score and displacement. On 2,306 real‑world ROADWork validation episodes, the candidates include a path within 10 px of the human demonstration in 87% of episodes, yet selection regret accounts for 53% of the geometric ranker's error. SAGE reduces average and final displacement errors by 8.3% and 12.0% relative to the ranker, mostly in the far field, and hard‑constraint violations by 37%. The trajectory opinion improves all four tested generator‑selector combinations. SAGE is comparable to state‑of‑the‑art VLM modules in accuracy, with a faster‑responding VLM, and outperforms four other adapted driving VLMs in displacement error and safety measures. Project page: https://hhjj233.github.io/SAGE‑page/.
Authors:Peter Potash
Abstract:
LawZero's Scientist AI trains a Predictor toward the Bayesian posterior, with an objective that penalizes inconsistent probabilities among latent explanations. We ask whether current frontier language models, with no such training, are already consistent in this sense zero‑shot: do their probability judgments satisfy the law of total probability, P(X)=\sum_z P(z)P(X|z), when the latent z must be inferred from records in the context? In separate single‑turn calls, models are asked for P(X), for P(z) over the latent's states, and for P(X|z); a causal family applies the same test to P(y|do(x)) and the back‑door adjustment. Every query has an exact ground‑truth probability, and a subset of items is asked in three wordings to separate persistent incoherence from rewording noise. Across seven models, GPT‑6.1 Sol and Claude Opus 5.5 are the most coherent (mean absolute residual 0.08 and 0.12 log‑odds, about 2 percentage points). Claude Haiku 5.5, Gemini 3.8 Flash and DeepSeek V4‑Pro show persistent incoherence (0.27‑0.35 log‑odds) that is mostly item‑specific rather than a shared offset. Within every model, larger residuals go with larger errors against the truth (Spearman 0.40‑0.62), so the residual, which needs no ground truth, flags unreliable answers. Reasoning length does not predict coherence, across models or within a model across rewordings of the same question. Simple consistency checks pass almost everywhere; the failures are specific to reasoning through a latent variable. Code and data: https://github.com/ppotash/latent‑coherence
Authors:Omar Farouk Zouak, Selma Mani, Larbi Said Chikh, Raid Abderrezak Ouahioune
Abstract:
How well a poetry archive can be searched depends on what the user still remembers, not only on what the model understands. We show this for Classical Arabic poetry, benchmarking nine dense encoders and BM25 over two corpora, three indexing modes and two evaluation protocols. Which retriever wins is governed by how much of a poem's wording the query still carries. On tip‑of‑the‑tongue queries BM25 leads every dense encoder, including an 8B state‑of‑the‑art model (0.746 vs. 0.717 Recall@5); on abstract queries it falls to eighth out of ten (0.392 vs. 0.748). Editing the wording of the same queries shows that this ranking largely reflects query composition: adding one remembered half‑verse raises BM25 by 38 points, rewriting the quoted words by hand leaves BM25 level with the 8B encoder, and BM25's sensitivity to remembered wording is significantly steeper than that of the strongest dense encoder. The same axis controls explanation augmentation, whose effect swings from +37 to ‑39 points. The mechanism splits by system type: lexical containment predicts BM25's behaviour, while embedding proximity predicts dense encoders. Finally, the field's two standard protocols, known‑item retrieval and thematic relevance, dissociate on the same systems, a result reproduced by two blind human annotators and by two open‑weights judges, one of them under a rubric with no literal‑match criterion. A benchmark that reports one protocol, or one query register, can give a materially wrong picture. Code and data are available at https://github.com/Ellzo/what‑the‑query‑remembers.
Authors:Tianzhe Chu, Druv Pai, Yi Ma
Abstract:
Mixture‑of‑experts (MoE) transformers, the standard backbone for modern large‑scale deep learning, employ conditional computation: only a sparse subset of model weights is used in each forward pass. In MoE transformers, this sparsity arises in the MoE operator, which generalizes the multi‑layer perceptron in the classical transformer. In this work, we explore an alternative mechanism for conditional computation that analogously generalizes the attention operator. The sparsity induced by this mechanism naturally nests (i.e., hierarchically composes) with the sparsity induced by the MoE operator. We show that a transformer employing this nested sparsity mechanism, which we call Nestor, performs better than a conventional MoE transformer at equal compute and equal sparsity in language model pretraining. We also show that Nestor is well‑suited for continual pretraining: it can be adapted to longer contexts more effectively than conventional MoEs, as well as adapt more efficiently to new domains while forgetting less on old domains. Furthermore, we demonstrate that these benefits persist as model size, dataset size, and context length grow. Our results demonstrate the potential of alternative conditional computation mechanisms for deep neural network architectures. Code and checkpoints are available on GitHub at https://github.com/Ma‑Lab‑Berkeley/Nestor .
Authors:Jiyuan Tan, Vasilis Syrgkanis, Jose Blanchet
Abstract:
Even a carefully proofread paper can leave small gaps in its proofs or its stated assumptions. We introduce PaperFormalizer, a Lean‑based agentic pipeline that helps authors find and close such gaps in their own papers and turn each paper into a Lean‑verified version. Starting from the LaTeX source, PaperFormalizer transcribes the paper into a dependency graph of statements, audits each statement in natural language for faithfulness, ambiguity and correctness, and then formalizes and proves the statements in Lean against Mathlib, optional domain libraries and the author's personal toolkit. Only the author knows what a statement is meant to say and how a formalization gap in a result is best fixed. The pipeline therefore stops and asks the author whenever a statement admits more than one reading, presents every repair, in natural language and TeX symbols, as a proposal that the author can accept, decline or redirect, and leaves to the author every decision that could potentially change what the paper claims. The output is an annotated and revised version of the paper, a report of the changes, and Lean code in which each declaration is linked to the statement it formalizes. The author's toolkit grows as more papers are formalized, and authors can share their verified papers so that researchers who build on them can reuse their Lean code. By making verified papers practical to produce, PaperFormalizer aims to make research that rests on mathematical proofs more robust. PaperFormalizer runs inside a coding agent (Claude Code or Codex) and is available at https://github.com/Jiyuan‑Tan/PaperFormalizer. Papers with Lean code shared by users are available at https://paperformalizer.org.
Authors:Xiaopeng Zhang, Luzhe Sun, Yiqing Xu, Matthew R. Walter
Abstract:
Generating physically stable 3D structures from visual specifications requires resolving geometric ambiguity while accounting for physical interactions among objects. We present StackGen3D, a framework for generating stable block assemblies conditioned on a given block inventory and three orthographic silhouettes. Our approach consists of two stages. First, we develop a physics‑informed graph neural network (PI‑GNN) that models support relations between blocks and serves as a fast stability predictor for curating a diverse dataset of stable structures. Second, we train a conditional flow‑matching model on this dataset to capture the multimodal distribution of block configurations consistent with the target silhouettes. At inference time, lightweight geometric selection and refinement reduce inter‑block penetration without physics simulation. Preliminary experiments show that the PI‑GNN generalizes from simple stacks to complex structures substantially better than a parameter‑matched multilayer perceptron. For structure generation, Best‑of‑10 selection improves physical stability from 45.3% to 72.4% while achieving a silhouette IoU of 0.970. These results highlight the potential of combining relational physical reasoning, scalable data curation, and conditional generative modeling for creative 3D assembly. Project website: [https://ripl.github.io/StackGen3D‑web/](https://ripl.github.io/StackGen3D‑web/)
Authors:Shuangyu Xie, Kaiyuan Chen, Muhammad Zubair Irshad, Richard Cheng, Sergey Zakharov, Ken Goldberg
Abstract:
Segmenting robot demonstration data into disjoint temporal and semantic subsegments is valuable for learning, cleaning, retrieval, and analysis. Segmentation is often performed manually in a tedious and error prone process. Multimodal Large Language Models (MLLMs) have the potential for automating this process, but directly ingesting raw multi camera video streams into MLLMs can be costly and can quickly exhaust context windows. In this paper, we introduce KiRoSe (Kinematics for Robot Segmentation), a novel framework that focuses on the companion stream of robot kinematic data. Inspired by the principle of classical entropy coding, such as Huffman coding, where frequently occurring symbols are represented more compactly, we propose Sparse Motion Tokens (SMTs), a discrete kinematic vocabulary designed to compactly represent recurring, temporally localized motion patterns, which we refer to as kinematic events. SMTs are learned by encoding kinematic trajectories with temporal segment supervision and quantizing the resulting features. We use increasingly available robot demonstration subtask annotations to supervise an open language model to interpret the resulting token sequences. Evaluation against human labeled segmentation shows that KiRoSe can substantially outperform baseline segmentation methods in F1 quality and transfers zero shot to unseen robot embodiments and egocentric human motion, while preserving the language models open vocabulary knowledge. Beyond segmentation, KiRoSe tracks task progress on an unseen robot and detects when an action is being reversed.(website: https://berkeleyautomation.github.io/kirose/)
Authors:Farhad G. Zanjani, Animesh Karnewar, Fatih Porikli, Amirhossein Habibian
Abstract:
Generating multiple videos of a dynamic scene from a single reference video requires synthesizing both observed and unobserved content consistently across viewpoints. Existing camera‑control methods for video diffusion typically rely on architectural mechanisms, including camera embeddings, cross‑view attention, and geometric conditioning. Although effective for static scene content, these approaches often struggle with the fine‑grained, dynamic geometry that must remain coherent across views and over time. We synchronize the diffusion process itself by sampling the Gaussian noise prior in a canonical 3D space derived from the source video and rasterizing it into target cameras, rather than sampling independently in each 2D image plane. Our method, SyncVid, lifts a monocular video into a canonical 4D representation, associates noise with scene geometry and unseen space, and rasterizes the resulting structured noise into arbitrary views. Because rendering can introduce local correlations, we further propose a cross‑view‑consistent whitening procedure that decorrelates neighboring pixels while preserving correspondence across views. The structured noise serves as the diffusion prior for a LoRA‑adapted pretrained video diffusion model, requiring no architectural modifications. By enforcing spatiotemporal and cross‑view coherence at the source of stochasticity, SyncVid generates synchronized videos and improves both perceptual quality and cross‑view consistency in our experiments. Project page: https://qualcomm‑ai‑research.github.io/SyncVid‑Webpage.
Authors:Jiawei Chen, Yinxi Wang, Zihan Yang, Meijun Gao, Yaowei Liu, Shuming Zhang, Zecheng Li, Xiangju Mi, Yuxuan Liu, Runzhong Huang, Tianlin Li, Xun Zhang, Zhengyang Wang, Xu Miao, Tong Yang
Abstract:
What visual representation should a navigation policy act on? Navigation policies built on vision foundation models largely leave representation design to the encoders' pretraining objectives. We present RepresentNav, a navigation foundation model built on a self‑constructed representation space: a visual representation constructed by the policy's own objectives from vision foundation models. RepresentNav builds this space with Representation Residuals, a compact three‑layer transformer that attends over features spanning the full depth of multiple complementary vision foundation models, so that the policy learns to see what it needs for navigation. In the same space it dreams what it wants: before every action a latent chain‑of‑thought predicts the representation of the scene a moment ahead. The dream is trained end‑to‑end with the policy as a joint‑embedding predictive architecture (JEPA), with a next‑representation prediction loss and a regularizer on its targets. RepresentNav sets a new state of the art on VLNVerse, VLN‑CE R2R, EVT‑Bench, and two point‑goal benchmarks, and its gains carry over to a physical robot amid clutter and moving people. On VLNVerse fine‑grained, RepresentNav achieves 83.80% success, outperforming the previous state of the art by 20.05 percentage points, and on EVT‑Bench distracted tracking it reaches 85.5% with a single camera (+10.1). With the same encoders, the self‑constructed space clearly outperforms concatenating their outputs and learns to weight their layers unevenly, and a dream in it cuts collisions by a quarter. A single dreamed latent decodes into the depth, semantics, and 3D occupancy of the scene a moment ahead, more accurately than pretrained encoders applied to the real future frame, although occupancy is never supervised, and the rollout tracks the robot's own motion over the next second: the dream emerges as a compact latent world model.
Authors:Zhuoyuan Jiang, Weijie Lyu, Xiaoyuan Wang, Tiancheng Shen, Ming-Hsuan Yang
Abstract:
Existing 3D talking avatar methods mainly rely on parametric face models including 3DMM or FLAME, whose compact motion representations limit their ability to synthesize extreme expressions and to model accessories such as headwear. We introduce AudioLift, a two‑stage framework that uses generated multi‑view videos as the intermediate representation for audio‑driven avatar generation without parametric face models or per‑instance optimization. Our framework combines a synchronized multi‑view video generation stage with an inconsistency‑robust Gaussian reconstruction stage to recover coherent full‑head geometry from a single portrait image. Extensive experiments demonstrate that AudioLift produces 3D talking avatars capable of capturing more extreme facial expressions than existing methods, with improved geometric completeness and strong multi‑view consistency across the full 360^\circ viewing range.
Authors:Mingxiu Cai, Haoyang Xian, Xinyu Ma, Yutong Wang, Zhenxi Song, Jing Li, Xuebo Liu
Abstract:
Large language model (LLM)‑based multi‑agent systems (MAS) rely on role‑specific prompts and execution models to solve complex tasks collaboratively. However, prompt optimization typically keeps model assignments fixed, leaving differences in role requirements and their associated inference costs outside the search. We present UniMASO, a Unified framework for heterogeneous Multi‑Agent System Optimization through the joint adaptation of prompts and model assignments. UniMASO evaluates candidate updates through local, downstream, and global feedback, using cost‑aware Pareto search to retain useful quality‑cost trade‑offs. Topology‑aware co‑evolution then coordinates these updates according to agent roles and dependencies. Cross‑level trajectory memory further supplies historical successes and failures to guide subsequent optimization. Experiments on five benchmarks and five execution topologies show that UniMASO achieves competitive task performance at substantially lower deployment cost than the strongest homogeneous configurations. Under centralized execution, it improves the average score from 69.79 to 70.40 while reducing token‑weighted deployment cost by 85.2% relative to DeepSeek‑V4‑Pro with MASPO. These results support joint prompt‑model adaptation as an effective approach to efficient multi‑agent collaboration. The code is available at https://github.com/Achilles008/UniMASO_main.
Authors:Qiang Fu, Xinyue Li, Andre Wibisono, Arman Cohan
Abstract:
Chain‑of‑thought (CoT) monitoring aims to detect potentially harmful behaviors in large language models (LLMs) from their CoTs. Intervention‑based evaluations measure this property by asking whether a monitor can detect the role of an intervention from the CoT when the targeted behavior occurs, typically using minimal‑criterion sensitivity (\mathsfTPR_mc) proposed by~\citepguan2025monitoring. However, a target behavior observed under intervention is not necessarily caused by it, and we show that this metric can overestimate how often a monitor detects target behaviors actually caused by the intervention. Specifically, we establish \mathsfTPR_mc as an upper bound on this causal detection rate and derive a complementary lower bound, which provides a tighter evaluation framework for CoT monitorability. With this framework, we study whether LLMs can be trained to make their CoTs easier to monitor. We first construct a training dataset of paired control‑‑intervention tasks spanning multiple intervention types. Using the dataset, we design and apply three intervention‑specific training methods based on off‑policy self‑distillation, reinforcement learning, and on‑policy self‑distillation to train separate variants of each base model. Across various model‑monitor configurations, all three methods generally improve CoT monitorability over the corresponding base models under our extended evaluation framework. Our code is available at https://github.com/xinyue‑lxy/learning‑to‑be‑monitored.
Authors:Zhe Zhang
Abstract:
Given a fixed straight‑line drawing of a complete graph, we seek a two‑colouring of its edges that minimizes the number of monochromatic crossing pairs. We prove that the threshold decision problem is NP‑complete, even when the vertices are in general position and have integer coordinates of polynomial binary length. Thus exact minimization is NP‑hard, answering the complexity question posed at the Crossing Number Workshop in 2019. The reduction is from MaxCut on permutation graphs. Its main difficulty is to control the edges required by completeness, whose colours must remain free. We construct a large comparison grid with two stable colour phases and use additional points to assign costs to the edges that encode the source graph. A geometric separation eliminates one large class of unwanted crossings. The costs of all remaining edges are included in a computable baseline and bounded errors, with each unit of the source objective amplified beyond the total error. The construction is deterministic and has polynomial size and coordinate bit length.
Authors:Dennis Heijmans, H. Çağrı Bilgi, Rui Wang, Kubilay Atasu
Abstract:
Financial transaction networks are naturally represented as multigraphs, where repeated transactions between the same pair of accounts are modeled as distinct edges. Recently proposed multigraph neural network architectures leverage this structure explicitly to achieve a strong performance on anti‑money laundering (AML) tasks. However, their robustness to adversaries that manipulate transaction structures to evade detection remains largely unexplored. We present the first scalable framework for structural attacks and adversarial training of multigraph neural networks using node‑edge incidence matrices, adapting projected gradient descent and projected randomized block coordinate descent methods to optimize structural perturbations over such matrices. We further introduce a cycle‑based regularizer that penalizes changes in the global counts of directed 2‑ and 3‑cycles, providing a structural measure of attack plausibility. Experiments on synthetic pattern‑detection and realistic AML datasets show that our learned attacks substantially outperform non‑learnable perturbation baselines in reducing detection performance (PR‑AUC), while adversarial training consistently improves robustness against these attacks. These results demonstrate that multigraph neural networks are vulnerable to learned structural attacks and that adversarial training provides an effective defense. Our code is available at https://github.com/denheijmans/robust_multignn.
Authors:Ruopei Sun, Jianfeng Cai, Kangwen Zhao, Wengang Zhou, Houqiang Li
Abstract:
Context compaction lets long‑horizon coding agents continue beyond their context windows, but can erase the state needed to finish a task. Prior evaluations measure efficiency, retained information, or end‑to‑end success, leaving the effect of a specific history replacement difficult to isolate. Compact‑Bench pairs full‑history and summary‑only continuations from a shared repository state, and evaluates recurring details and successive production compactions. It covers 35 SWE‑bench repairs, 146 agentic sessions, and 106 production chains. Across 14 summarizer backbones, stable damage ranges from 5‑64% of 22 full‑history successes: Claude Opus 4.8 preserves the most repairs, while DeepSeek‑Flash damages the most. With Kimi‑K3 as summarizer, the Kimi Code prompt preserves the most task continuations (19/22), whereas Claude Code retains the most recurring details (15.0%). LLM summaries retain at most 16.0% of recurring details, compared with 79.1% for mechanical extraction; production summaries retain 15.7% at the first boundary. Failures recur across methods: one repair is damaged by 13/14 backbones, and another by all 5 prompts. These results show that reliable compaction requires preserving the progress needed to complete a task, and that retaining relevant information alone does not ensure successful continuation. Code is available at https://github.com/cai‑jianfeng/compact‑bench and data is available at https://huggingface.co/datasets/caijanfeng/compact‑bench.
Authors:Utkarsh Kanwat
Abstract:
Safety guards check content before an LLM application acts on it, and most paste that content into a plain‑text prompt. Content that imitates the prompt can append a second task, and the guard answers that one instead. We call this target rebinding. Our explanation is that the guard grades the last complete task in its input, here a harmless decoy, so the harmful request goes through unchanged. In a pre‑registered test, an appended forged query and decoy clear all 75 held‑out AdvBench requests that Shieldstral‑1.0‑3B flags. The guard is not persuaded: a directive to ignore the text above clears none of them, and in a follow‑up run a harmful decoy stops the attack. On our GuardRebind benchmark, each of five guards has an attack family, chosen per guard, that clears at least 93% of its flagged items. Rebinding explains this best on Shieldstral and gpt‑oss‑safeguard; on ShieldGemma and Llama Guard 3, generic evasion and trailing text respectively do part of the work, and Qwen3Guard falls mainly to a forgery of its whole task. Training the guard to score content no lower than its most harmful part cuts attack success from 75% to 0.8‑1.6% on attack families, decoys and forged queries held out of training. All of these attacks append the forged structure after the request, and we added the held‑out‑decoy evaluation after a first round of test runs. Clean AUROC falls by at most 0.01, but our test of non‑inferiority is inconclusive. A score‑guided search still beats this trained guard on 47 of 50 items, usually by placing forged structure before the request. On about half of the items where the search wins, the same search also finds benign text, with no forged structure, that clears the guard.
Authors:Tianda Sun, Dimitar Kazakov
Abstract:
Activation interventions can make a language model produce a target answer, but can the model use the intended condition in other computations? We study this question in Qwen3.5‑27B on tool‑use tasks where time or permissions determine eligibility and selection. Across 32 fresh worlds, two case‑specific residual‑writing variants raise familiar target‑selection correctness from 35/128 to 119/128 and from 33/128 to 114/128. Yet these gains disappear when familiar facts and identifiers are recombined within the same selection task, while selection of previously rewarded identifiers increases. The gains persist across output formats but yield little improvement in independent judgments about whether a candidate is eligible or should be selected. Internal edits can nevertheless change how a source fact affects a judgment: in a prespecified secondary comparison, an early‑layer source patch changes 50/64 judgments when that source is used, with one adverse change among 64 cases where it is not. This selective effect does not ensure correct complete actions. Together, these results separate target‑selection control, condition‑sensitive judgments, and complete‑action correctness. Evaluating how an edited state is used, rather than only its trained answer, exposes these differences.
Authors:Qiwen Gu, Xin Zhou, Xiwu Chen, Zhongyang Zhu, Zhuo Zhang, Junqiao Zhao
Abstract:
Controllable 3D part generation is pivotal for digital asset creation, yet existing methods lack the flexibility to accommodate diverse user workflows requiring varying levels of control. In this paper, we introduce FlexPart, a unified image‑to‑3D part generation framework that supports variable‑granularity geometric conditions within a single model. By seamlessly integrating heterogeneous inputs, ranging from sparse points to dense bounding boxes and masks, via gated adaptive modulation, FlexPart unlocks a powerful cross‑modal synergy. We also propose an asymmetric geometric guidance mechanism that leverages strong spatial priors to align representations across granularities, improving geometric consistency and facilitating inter‑task synergy. Under mask conditioning on PartVerseXL, FlexPart reduces Part CD by 16.6% relative to TripoSG + SAM3D and by 10.5% relative to OmniPart, while maintaining competitive whole‑mesh quality. The code can be found at https://github.com/Gucci233/FlexPart
Authors:Yunfei Xie, Howard H. Qian, Shiyi Lan, Kaiyu Hang, Chen Wei
Abstract:
Robot foundation models are typically evaluated through costly closed‑loop rollouts, motivating low‑cost offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To test this, we propose Sensorimotor Alignment (S^2), a training‑free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that S^2 correlates positively with downstream robot task success across model families, making it a low‑cost offline proxy for prioritizing models before closed‑loop evaluation. S^2 can also identify promising vision‑language model (VLM) backbones before costly fine‑tuning into vision‑language‑action models (VLAs). Beyond this correlation, S^2 offers representation‑level explanations for phenomena observed in VLAs and world action models (WAMs). With S^2, we extend PRH to robotics and make sensorimotor alignment measurable, offering a low‑cost, representation‑level view of robot foundation models that closed‑loop success rates alone cannot reveal.
Authors:Shengqiong Wu, Meng Luo, Bobo Li, Ziwei Liu, Shuicheng Yan, Ming-Hsuan Yang, Philip Torr, Hao Fei
Abstract:
Video foundation models have advanced rapidly in understanding and generating visual sequences, while video world models seek to capture the underlying dynamics of the world they depict. Despite their different objectives, both concern the relationship between video observations and an evolving world. This survey adopts world modeling as a unifying lens for examining these developments. Viewing an observed video as a partial, lossy, and temporally ordered projection of a latent temporal world, we organize the literature around four complementary operations: reading infers world properties from observed video; writing synthesizes video from specified or learned world conditions; sharing integrates understanding and generation through common modeling mechanisms; and interacting predicts and updates world states under actions and evolving observations. This perspective distinguishes temporal coherence in observed or generated sequences from consistency in the underlying world state. Across tasks, representations, models, datasets, and benchmarks, we trace a shift in emphasis from clip‑level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and evaluation grounded in interaction and downstream decisions, and we distill the open challenges that follow.
Authors:Ava Pun, Kevin Dai, Sally Chen, Maxwell Jones, Ruixuan Liu, Kangle Deng, Changliu Liu, Maneesh Agrawala, Deva Ramanan, Jun-Yan Zhu
Abstract:
Frontier coding agents are becoming increasingly adept at 3D modeling. Yet, they still face challenges when building objects meant to be assembled in the physical world, often creating invalid assemblies with floating, colliding, or disconnected parts. We argue that providing agents with tools for assembling builds and verifying their physical feasibility is sufficient for them to produce physically valid assemblies that still adhere to input prompts. Focusing on the domain of LEGO brick builds, we present BrickWright, a framework for generating physically feasible brick builds from text or image prompts using pre‑trained agents, where a build is physically feasible if it uses only real parts, has no colliding or floating parts, and forms a connected structure if possible. BrickWright guides agents to create builds by writing code with BrickLayer, a Python toolkit with two main components: an ergonomic brick assembly API that halves inference cost compared to working in raw LDraw, and a validator that provides feedback on physical feasibility. Whereas baseline agents achieve physical validity rates ranging from only 9% (GPT‑6 Luna) to 69% (GPT‑6 Astra), BrickWright‑guided agents achieve 100% validity on a sample of 512 prompts, while improving prompt adherence for lightweight agents and maintaining it for stronger agents. Additionally, they outperform specialized models on the text‑to‑brick task. Code is available on our project page: https://avalovelace1.github.io/brickwright/
Authors:Xiaolin Li, Zihao Huang, Fanpeng Meng, Chen Yang, Guanjun Wu, Kai Zhang, Jiemin Fang, Qi Tian
Abstract:
Single‑image 3D generation has achieved high fidelity on curated benchmarks, yet degrades sharply on real‑world images with uneven illumination, blur, and occlusions. Purifying such inputs requires deciding when to intervene and preserving the visual cues needed for 3D generation, while leaving clean inputs unchanged. We introduce Gate3DGen, a selective pre‑processing model that gates image inputs to existing 3D generators without retraining them. A degradation‑aware router bypasses inputs classified as clean and extracts features to guide purification of the remaining inputs. Conditioned on these features, a 3D‑oriented one‑step purifier purifies degraded inputs with spatial losses that encourage preservation of object identity, contours, and textural cues. To provide paired supervision, we construct a synthetic data engine that renders clean views of 3D assets and synthesises composite degradations. On the synthetic benchmark, the purifier improves image fidelity and downstream 3D geometry in only 2.3s on average, reducing TRELLIS.2 mesh distance by 61.5% relative to degraded inputs. VLM evaluation and human preferences on real photographs further support identity preservation and degradation correction. More demos and code are available at https://supra‑3d.github.io/gate3dgen.
Authors:Namitha Guruprasad, Abhay Yadav, Cheng Peng, Rama Chellappa, Anand Bhattad
Abstract:
Monocular models recover 3D geometry from a single image in one pass, but most assume a pinhole camera and produce distorted geometry on the wide‑angle imagery produced by fisheye and panoramic lenses. Camera‑aware models remove this assumption by predicting a per‑pixel ray field. However, they represent different camera geometries using a single fixed basis to capture both the overall camera projection and its finer variations. We argue that camera geometry can be viewed as having a hierarchy of complexity for representing rays. We present OmniRay, a feed‑forward model whose Ray Module is composed of three components: a monotone radial base that captures radially symmetric structure, a spherical‑harmonic residual for smooth deviations from it, and a free‑form residual for remaining local variation. These components are composed as successive rotations on the sphere, allowing the representation to accommodate diverse camera geometries. Given a single image, OmniRay predicts a ray direction and an up‑to‑scale radial distance for every pixel, from which a 3D point cloud is recovered without needing camera information at inference time. Extensive experiments across different camera configurations demonstrate that OmniRay establishes a new state‑of‑the‑art in camera‑agnostic monocular 3D reconstruction.
Authors:Matteo Fusconi, Alex Costanzino, Pierluigi Zama Ramirez, Luigi Di Stefano
Abstract:
Learnable Representations (LRs) can fit parametric families of functions to diverse kinds of signals. These emergent neural‑native data formats are gathering interest in the scientific community due to their compactness, resolution‑agnosticism and amenability to direct neural processing. On the other hand, gigapixel images are becoming increasingly widespread across a variety of scientific, cultural, and industrial domains. In this paper, we present GigaBench, the first benchmark and experimental evaluation focused on deploying LRs with gigapixel imagery. GigaBench includes images collected, acquired, and synthetically rendered across diverse domains, and addresses image compression and super‑resolution. We adapt a representative set of LRs, spanning Implicit, Hybrid, and Explicit methods, so as to operate reliably at the gigapixel scale on consumer hardware. Through systematic evaluation across reconstruction quality, model size, compression and inference times, we identify consistent trends across methods and suggest actionable guidance for future investigation.
Authors:Zunnan Xu, Shangchen Zhou, Yihang Luo, Chen Change Loy
Abstract:
Feed‑forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks. This division produces independent local reconstructions with inconsistent coordinate frames and duplicate estimates of shared cameras. Achieving global consistency requires propagating constraints from adjacent overlaps and distant revisits across the sequence. This requirement naturally motivates pose graph optimization. However, the choice of graph level creates a trade‑off. Chunk‑level graphs efficiently resolve scale drift and close large loops. Yet, their rigid transformations cannot correct internal errors or conflicting frame predictions. Conversely, frame‑level optimization offers local flexibility, but scaling it globally is computationally expensive and can break metric scale. HiP3R resolves this trade‑off using a hierarchical pose graph. HiP3R first optimizes a robust chunk‑level \mathrmSim(3) graph to establish the global layout and scale. It then holds this layout fixed and estimates one shared \mathrmSE(3) camera per frame, turning overlapping window predictions into a single trajectory. Experiments across multiple benchmarks demonstrate the effectiveness of HiP3R in improving camera trajectories and 3D reconstruction. Our method applies to both chunk‑based and streaming backends without additional training or fine‑tuning. The codes are available at https://kkakkkka.github.io/HiP3R.
Authors:Matteo Frascarelli, Lucia Pallottino, Alberto Landi
Abstract:
We consider the offline generation of coordinated paths for N interchangeable robots and N goals in planar environments with polygonal obstacles. After a conformal map to a circular domain, starts and goals become the poles and zeros of a rational function R, and the paths are the branches of its Generalized Root Locus (GRL), i.e. the streamlines of the logarithmic potential log R. We show that, for every phase outside a finite set of critical phases and a set of boundary phases, the induced gradient motion drives each robot to a distinct goal along pairwise disjoint paths that never reach the obstacles; the robot‑goal assignment is produced by the construction, and with the gain used as a clock all robots move as the closed‑loop roots of the GRL and arrive together at a prescribed time. Obstacles become streamlines through successive Milne‑Thomson reflections, and for a finite number of reflections a finite test certifies safety for the potential that is actually computed. Reciprocal root‑locus planning is recovered as a special case. The method is illustrated on two polygonal workspaces, 100 random instances compared with independent harmonic planning and artificial potentials, and a scene with ten robots and ten obstacles.
Authors:Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, Yanwei Fu
Abstract:
This paper studies category‑level RGB‑D object pose estimation, which recovers an object's 3D rotation and translation without instance‑specific CAD models or reference views. This task is highly ambiguous due to intra‑class variations, symmetries, and partial observations. Current solutions struggle to balance reliability and efficiency: methods based on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency and complex post‑hoc candidate extraction. To address this, we present F2G‑Pose, a real‑time direct, single‑pass framework that predicts category‑level pose, metric size, and camera‑frame completed shape from a single RGB‑D observation in one feed‑forward pass. F2G‑Pose lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object‑level structure with a Mixture‑of‑Experts Transformer. Joint size prediction and camera‑frame shape completion encourage metric consistency through dense geometric supervision. With its task‑specific network trained only on pose‑annotated synthetic SOPE data and its frozen RADIO backbone externally pretrained, F2G‑Pose achieves state‑of‑the‑art accuracy on SOPE and ROPE while running at 28 FPS for pose‑only inference, and further demonstrates strong camera‑frame geometry on HouseCat6D and real‑object transfer on HANDAL. Analyses of latency, robustness, shape completion, and GenPose++ candidate behavior further demonstrate the efficiency and reliability of F2G‑Pose, with real‑world manipulation experiments demonstrating deployment potential under segmented RGB‑D inputs. The project page, code, and pretrained models are made available at https://jrryzh.github.io/F2G‑Pose/.
Authors:Andrea Giuseppe Di Francesco, Rishabh Jain, Franco Maria Nardini, Nicola Tonellotto, Pietro Liò, Fabrizio Silvestri
Abstract:
Recent methods for injecting graph structure into LLM prompts include graph soft prompting, in which a graph neural network (GNN) learns graph‑specific tokens that capture relational information among entities in textual prompts. G‑Retriever combines graph soft prompts with a verbalized description of the same graph to strengthen the textual prompt signal. We conduct an in‑depth analysis of G‑Retriever and uncover a pathological behavior we term token collapse. Across three benchmarks, three graph encoders, two LLM families, and both frozen and LoRA‑adapted models, G‑Retriever's graph tokens become almost indistinguishable across inputs. We provide evidence that the same failure is inherited by subsequent methods, even with multi‑token pooling. Under the standard setup, swapping a token with the dataset mean, a token from another example, or one derived from a corrupted graph produces no resolved answer‑level effect in accuracy. Moreover, a learned constant prompt that never accesses the graph matches or surpasses G‑Retriever on all three benchmarks. These findings indicate that, rather than conveying graph content as intended, the collapsed token functions primarily as a generic soft prompt. We also examine conditions linked to this collapse and propose a two‑stage procedure that yields non‑collapsed tokens while maintaining competitive performance in most cases. However, token variability alone does not prove that the model uses graph information: via interventions and decoding probes, we show that depending on the configuration, a token can reflect the question, an answer prior, or graph‑derived content that the LLM may or may not use. Overall, our results offer a practical approach to evaluating graph‑token alignment rather than inferring it from task accuracy alone. Code to reproduce our experiments is available at https://github.com/difra100/Token_Collapse_KGQA.
Authors:Yufeng Liu, Bohao Fan, Juntu Zhao, Xinyi Wang, Hang Yu, Wenzhuan Luo, Yingzhi Wang, Qinghai Li, Yingdong Hu, Junliang Guo, Richard Wang, Dequan Wang, Yang Gao, Cheng Ma
Abstract:
Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine‑grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame‑level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine‑grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal‑guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real‑world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.
Authors:Ilan Theodoro, Jordão Bragantini, Loïc A. Royer
Abstract:
Large language models have shown outstanding performance on text‑first tasks such as coding, math, and writing. Recent efforts are shifting models' training to include scientific problems that often live in highly specialized domains beyond text, such as protein structure and biological images. However, in these scenarios, their general‑purpose accuracy compared with specialized algorithms remains in question. One such domain is cell tracking, which requires spatiotemporal awareness of cell dynamics. To evaluate this, we created a new benchmark, CellTrack‑Bench, which assesses Vision‑Language Models' (VLMs) cell‑tracking‑related performance against specialized trackers. The full benchmark draws from 15 standard cell‑tracking datasets across four imaging modalities and comprises 1,537 problems corresponding to 9,041 cell‑linking decisions. We selected a hard subset requiring greater reasoning effort by identifying failures shared across distinct algorithmic trackers. With this proposed dataset, we: (i) evaluated different environment harnesses for VLMs; (ii) benchmarked 9 frontier closed‑source models and 13 open‑weight models; and (iii) provided a detailed analysis of their failure modes. Code, the stored model outputs, and a pipeline that regenerates the benchmark from the source datasets are available at https://github.com/royerlab/celltrack‑bench.
Authors:Zhengyi Liu, Li Liu, Songtao Liu, Xianglong Liu, Xiangzhen Zhou, Mingyu Huang, Dingchao Gao, Shenggang Ying, Bolong Chen, Ji Guan
Abstract:
Tensor‑network simulation of quantum circuits depends on contraction orders with low computational cost and small intermediate tensors, found without excessive planning time. We present ArcTN, a contraction‑order optimization framework with two search presets. Light keeps planning overhead low, and Heavy spends more effort on refinement. Both combine randomized‑greedy construction, subtree reconfiguration, and parallel tempering with stage‑specific stopping rules. ArcTN also provides Fixed and Dynamic slicing to meet tensor‑size limits and executes the returned orders. On the 30 QC32 quantum‑circuit networks completed by all eight compared configurations, Light reduces FLOPs by 61.32% and plans 1.62× faster than Cotengra Auto/CMA‑ES, and Heavy reduces FLOPs by 43.83% and plans 7.21× faster than Cotengra AutoHQ/Optuna. Heavy also needs 11.85% fewer FLOPs than OMECO TreeSA‑8 and plans 1.33× faster. Dynamic slicing meets the size limits in all 90 runs on 18 scalar‑output networks and reduces total sliced FLOPs by 31.99% relative to Fixed slicing, at the cost of longer planning. With a common NumPy executor, Heavy has the lowest combined planning and execution time among five configurations on the runs that all of them completed. On 672 networks built from 320 MQT Bench circuits, Heavy obtains the lowest geometric‑mean FLOPs on the 142 large networks while planning 3.1× and 31× faster than Cotengra AutoHQ and Hyper128.
Authors:Jihoon Hong, Alireza Kheirandish, Rohan Sanda, Sara Fridovich-Keil
Abstract:
Local anomaly detection is the task of identifying spatially confined anomalous or out‑of‑distribution (OOD) regions within an otherwise in‑distribution (ID) image. Most existing methods are designed for a specific context, with a strong focus on industrial defect detection, where they achieve excellent performance on well‑established benchmarks. However, we observe that these methods can exhibit significant performance drops in other contexts, especially when the inlier distribution is multimodal. To account for this inlier complexity, we propose LACToS, a principled local OOD detection method based on statistical typicality and made tractable through two innovations: (i) grouping pixels into superpixels and (ii) using a computationally efficient proxy to estimate typicality. We evaluate LACToS across three diverse applications spanning medical imaging, human faces, and X‑ray security screening. LACToS performs more robustly than prior methods across these benchmarks. The code is released here: https://github.com/voilalab/LACToS.
Authors:Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu
Abstract:
Vision‑language models (VLMs) can perform a broad range of tasks across diverse settings, yet their performance in agentic contexts remains poorly understood. Although online evaluations in simulators and real‑world environments are essential for measuring interactive agent performance, they are costly and challenging to scale. Complementary offline VQA benchmarks can enable faster and more reliable comparisons across agentic domains, yet existing benchmarks remain limited and fragmented. To address this gap, we introduce AgentVQA, an offline benchmark for evaluating VLMs on realistic agentic scenarios across five agent domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. We build AgentVQA by manually selecting 1000 challenging, real‑world agent examples from 23 source datasets and adapting them for reliable offline evaluation. The resulting benchmark preserves realistic scenarios from interactive environments across diverse domains, and its model rankings track those from corresponding online evaluations. Across the 22 evaluated models, the best achieves only 58.5% accuracy. Our cross‑domain analysis shows that errors are driven primarily by failures to ground and maintain accurate representations of the current and evolving visual state, including localizing relevant UI elements, interpreting motion, tracking state changes over time, and inferring 3D spatial relationships, rather than by task misinterpretation or downstream logical reasoning.
Authors:Khaled Waleed Dawoud, Muhammad Zaigham Zaheer, Khan Muhammad, Muhammad Haris Khan
Abstract:
Vision‑Language Models (VLMs) have shown strong potential for video anomaly detection, but dense frame‑level inference makes them impractical for real‑time deployment. Existing acceleration commonly relies on uniform sampling, which ignores content dynamics and can either waste computation on static segments or miss localized anomalies. We propose MoG‑VLM a motion‑adaptive anomaly detection framework that leverages motion purely as a lightweight control signal to decide when VLM inference is necessary. To this end, we introduce Temporal Gradient Magnitude (TGM) which performs content‑aware sampling to concentrate computation on informative regions while avoiding redundant processing. We further introduce a hierarchical dual‑stream design in which dense multi‑frame VLM reasoning is activated only around frames with elevated anomaly likelihood, via anchor‑triggered context‑window expansion with early termination. A fusion module then reconciles these instantaneous semantic cues with temporal action dynamics. Across four benchmarks, our method achieves 91.91%, 84.30%, 87.88%, and 78.96% AUC‑ROC on XD‑Violence, UCF‑Crime, MSAD, and ShanghaiTech, respectively, establishing new state of the art on three benchmarks under the zero‑shot, training‑free setting. On XD‑Violence, we improve over the previous best result by +0.57% while delivering a 1.31x inference speedup. These results demonstrate that content‑aware allocation of VLM capacity, processing fewer frames more intelligently rather than all frames uniformly, is a more effective axis of progress than uniformly densifying VLM inference across frames. Code is available on GitHub: https://github.com/kdawoud91/MoG‑VLM.
Authors:Qi Lv, Jiayi Lin, Yuxin Jiang, Jianzong Wang, Xiang Deng, Mike Shou Zheng
Abstract:
Robot world models simulate how environments evolve in response to robot actions. Latent action models enable learning dynamics from videos by encoding visual transitions as proxy actions. However, reconstruction‑driven latent actions mix robot motion with object responses and other scene changes, complicating their correspondence with robot controls. We propose SkeLAM, a skeleton‑guided latent action model for controllable robot world modeling. SkeLAM decomposes latent actions into a kinematic component grounded by robot skeleton supervision and a residual component retaining complementary visual dynamics. Cross‑modal swap reconstruction encourages RGB‑derived and skeleton‑derived kinematic representations to be interchangeable during video reconstruction. The world model incorporates both components through separate conditioning pathways, while branch dropout reduces its dependence on future‑derived residual information. A lightweight adapter maps robot actions to the kinematic space, and subsequent world‑model fine‑tuning adapts generation to these predicted conditions. At inference, the model generates future observations from the current observation and robot actions. On DROID, SkeLAM reduces end‑effector ADE by 19.5% and LPIPS by 11.1% compared with DreamDojo. Motion probing supports kinematic specialization, while qualitative results demonstrate zero‑shot human‑to‑robot motion transfer in generated rollouts. Policy evaluation shows agreement between predicted success rates and those measured in real robot executions.
Authors:Jian Hou, Xin Zhou, Adam J. Spiers
Abstract:
Tactile sensing provides robots with contact information difficult to obtain from vision alone. Shear force is particularly important for understanding tangential contact, guiding corrective motions, and maintaining grasp stability, yet flexible and low‑cost tactile arrays often measure only normal force. We introduce VecTile, a low‑cost, 6 mm thin, flexible tactile array capable of three‑axis force sensing. VecTile combines a piezoresistive layer with passive silicone nubs that convert normal and shear forces into distinguishable pressure patterns. The sensor can be fabricated in‑house without specialized equipment. The complete system costs less than 35 USD for a 4x4 array. We validate the sensing mechanism through finite element analysis and hardware experiments. In controlled single‑taxel tests over 1‑5 N normal force and up to 1.5 N shear force, VecTile achieves root‑mean‑square errors of 0.39 N and 0.16 N for normal and shear forces. A VecTile calibrated while flat retains accuracy when curved, while a five‑minute manual calibration is sufficient for practical use. Two applications demonstrate its utility. First, shear readings are sensitive enough to stop a sharp scalpel from puncturing a balloon after contact. Second, the sensor substantially improves Diffusion Policy‑guided power‑plug insertion. All designs and code will be released upon paper acceptance. Project website: https://jiannnh.github.io/VecTile/.
Authors:Minkyun Seo, Hyungtae Lim, Jaesik Park
Abstract:
Point cloud transformers aim to capture detailed local geometry and relationships across a scene at a manageable computational cost. Two common approaches are window attention and global attention. They differ in how they enable scene‑wide interaction, which refers to information exchange among tokens across a scene. Window attention reduces computational cost by restricting self‑attention to groups of tokens called windows; in serialized designs, each window contains a fixed number of consecutive tokens from a spatially ordered sequence. Because each block only directly connects tokens within the same window, information exchange across windows relies on changes in window membership across blocks and hierarchical pooling. Global attention, in contrast, directly connects all tokens within a scene, but computing attention over all token pairs can be costly when the number of tokens is large. To balance computational cost and scene‑wide interaction, we introduce MixPT, a point transformer that uses a mix of window and global attention blocks in an interleaved schedule, while retaining a single fixed serialization pattern for its window blocks. For scenes with a large number of tokens, MixPT applies global attention to spatially pooled tokens and returns residual updates to the original‑resolution stream. Across indoor and outdoor benchmarks, MixPT achieves competitive accuracy with lower latency than representative window and global attention baselines. We further demonstrate the feasibility of scaling the attention stage in depth by repeating the interleaved pattern of window and global attention blocks.
Authors:Taekoan Yoo, Wonkyung Jung, Kyunghun Kim, Kyeongbo Kong
Abstract:
World action models increasingly rely on pretrained video models assumed to capture physical dynamics for prediction, planning, and action generation. Existing evaluations assess plausibility, probe representations, or measure predictive surprise, but rarely identify which history‑conditioned physical expectations conflict with observations. We introduce PhysTrace, a training‑free framework aligning MLLM‑generated, history‑conditioned expectations with observed video windows to temporally localize physical violations and provide interpretable descriptions without task‑specific probes or classifiers. We also introduce IntPhys2‑Timing, adding violation intervals to IntPhys2. PhysTrace achieves a Hit Rate of up to 70.7% across physical principles on IntPhys2‑Timing and outperforms the compared methods in overall possible‑impossible discrimination accuracy on IntPhys2. Applied to reconstructed videos along the trajectories of pretrained diffusion/flow‑based video models, PhysTrace reveals attribute‑ and noise‑dependent patterns in physical violation localization, enabling interpretable physical consistency analysis across timesteps.
Authors:Tao Lin, Yuxin Du, Haoxiao Wang, Zile Zhong, Bo Zhao
Abstract:
Agentic robot systems combine language‑model reasoning with robot policies to monitor execution and recover from failures. However, reviewing every action chunk incurs substantial overhead, even in familiar situations. We introduce FastHarness, an experience‑driven execution harness that adaptively schedules GPT‑6 Astra supervision of a π0.5 policy. A fast path uses stage memory and reviewed outcomes to reduce review frequency as experience accumulates, while a complementary recovery path focuses supervision on correcting errors. Once recovery is verified, routine execution resumes, with failure experience retained to guide future interventions. The policy remains frozen and generates fresh actions from current observations. Across four simulated manipulation tasks, reviewer token use decreases by an average of 57.2% per task from the first evaluated episode to the mean of the final five episodes. Mean episode time decreases from 85.5 to 52.0 minutes over the same interval. On the same tasks, mean success increases from 80% for GPT‑as‑Policy (π0.5 + GPT‑6 Astra) to 97.19% with FastHarness. These results highlight adaptive supervision as a practical route to more efficient agentic robotics without additional model training.
Authors:Ze-Yu Peng, Qing-Yu Lan, Zhan-He Wang, Jun Zhang, Yun-Song Piao
Abstract:
Evolutionary program search with large language models keeps the structure of an evolutionary algorithm and uses the model as its variation operator. Typically, the model generates each candidate program in a single request from parent programs and their scores. We present AgenticEvolve, an evolutionary program search framework in which each island is run by a persistent large language model agent. Between evaluations, the agent designs and runs experiments on the candidate program. To measure the contribution of experimentation, we compare AgenticEvolve with No‑analysis, an otherwise identical configuration with experiments disabled. On two heuristic design tasks with three large language models, AgenticEvolve produces better programs than No‑analysis and OpenEvolve on both hidden and unseen instances. It reaches the baselines' final quality with far fewer evaluations and comparable cost. On twelve equation discovery problems, it recovers the ground‑truth structure of more equations than No‑analysis and existing language‑model and genetic programming baselines.
Authors:Xinyang Lu, Shengzhe Wei, Sheng Tang, Weiqi Zhang, Juntong Fang, Zesong Li
Abstract:
Local feature matching is typically formulated for image pairs, although structure‑from‑motion and visual localization commonly match one reference image against many retrieved or covisible neighbours. Applying a pairwise matcher independently to these centre‑neighbour edges duplicates reference‑side computation and prevents the matcher from exploiting complementary multi‑view geometric evidence from the remaining observations. To address these limitations, we introduce MVLoMa, a sparse star‑graph matcher that extends LoMa from image pairs to multiple views. MVLoMa first uses a frozen multi‑view geometry model to jointly describe all images in a star, then injects the resulting cross‑view geometric context into local descriptors. It then matches the centre image to all neighbours in a single forward pass through per‑view self‑attention and bidirectional centre‑neighbour cross‑attention. For supervision, we derive positive correspondences only from depth‑valid, mutually visible dense warps and explicitly penalize high‑confidence matches that violate epipolar geometry. On 25‑view star graphs from HyperSim, ScanNet++, Mip‑NeRF 360, and Tanks and Temples, MVLoMa obtains the best macro‑average relative‑pose AUC among the compared methods at all three thresholds (58.65/85.41/90.99). Its matcher is 2.20 times faster than sequential evaluation of the 24 centre‑neighbour edges. The same design supports four tested frozen geometry encoders ‑‑ DA3, π^3, VGGT, and VGGT‑Ω ‑‑ with DA3‑Giant performing best on all reported encoder‑ablation metrics. Controlled ablations further isolate the effects of geometry‑aware descriptors, visibility filtering, geometric negative supervision, and joint multi‑view inference.
Authors:Yifan Yuan, Yancheng Zhu, Zuoyu Zhang, Xiaogang Peng
Abstract:
Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure. We present SurveyGraph, a framework built around a Methods‑Entity Knowledge Graph (MEKG) encoding method‑level follow‑on, competition, and shared‑resource relations. The graph supports core‑paper discovery, retrieval, hybrid outlining, and evidence‑gated narrative synthesis. We evaluate retrieval, semantic edges, citation grounding, collection coverage, and generation‑stage components. In a controlled seven‑method equal‑budget benchmark, MEKG‑anchored fusion balances recall and structural density, with an independent co‑citation check. In a separate archived graph‑native diagnostic, graph‑aware retrieval nearly doubles recall over flat lexical baselines across eight runs (p = 0.004). A semantic evidence gate achieves AUC = 0.75. Audits of 160 generated sentences find 80.6% anchored‑or‑partial citation support. Human adjudication of 108 stratified semantic edges finds 68.5% supported‑or‑partial relations (binary‑support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%). A SurveyBench‑facing collection probe reaches mean reference coverage of 0.521 across four topics with an eight‑paper budget. Same‑backend Qwen diagnostics on a 30‑paper/74‑relation graph show reduced organization signals when hybrid outlining is removed. Concurrent‑system comparisons provide bounded context rather than matched‑backend end‑to‑end evidence. The results position MEKG as an auditable structural layer for typed diagnostics, relation‑specific filtering, and controlled survey generation. Code and artifacts: https://github.com/xzzvsxd/SurveyGraph.
Authors:Yancheng Zhu, Yifan Yuan
Abstract:
Large language models can repeat an instruction while failing to keep obeying it. We study this recall‑compliance dissociation in long‑horizon dialogue and define State Integrity Collapse: a model preserves a stated constraint in memory but stops applying it to later turns. We introduce StateTrack‑Bench, a controlled suite of conversations covering numerical, format, role/persona, factual‑memory, and logical‑rule constraints. Across seven provider‑pinned model snapshots and five dialogue‑depth probes, strict joint retention ranges from 62.4% to 100.0%, with degradation concentrated in symbolic constraints and in one especially brittle model. A forked recall audit shows that 88.6% of observed failures are behavioral‑drift failures rather than ordinary forgetting; excluding the brittle outlier preserves this diagnosis (84.4% of failures). Extended evaluations at depths up to 50 turns confirm that the dissociation persists beyond the core benchmark, while Structured State Anchoring recovers up to 24.4 points on immutable constraints. The results show that reliability cannot be inferred from single‑turn instruction following alone; evaluation must track whether remembered constraints remain operational as dialogue state grows. Code and artifacts are available at: https://github.com/xzzvsxd/StateTrack.
Authors:Jiacheng Liu, Yaxin Luo, Xiaohan Zhao, Peng Sun, Xinyi Shang, Jiacheng Cui, Yonina C. Eldar, Zhiqiang Shen
Abstract:
Scaling embedding capacity alongside Transformer computation is an emerging trend in language models. This trend makes high compression rates increasingly useful. Hence, we study extreme compression of Per‑Layer Embeddings (PLE), which follow this trend and account for 35.2% of the parameters of Gemma 4 E4B. At such rates, conventional quantizers give each token‑layer vector its own code of only a few bytes. We instead develop OPTIC (Output‑Path‑fitted, Token‑shared Indexed Codes), which stores one quantized code tuple per token and decodes it at every layer with a layer‑specific affine decoder. Codes and decoders are fitted to the PLE output path rather than to the stored vectors. At about 100× PLE compression of bfloat16 (BF16) E4B, OPTIC retains an aggregate 97.8% of the uncompressed model's performance across seven benchmarks, whereas product quantization, the strongest of six conventional baselines, retains 66.3%. At 594× and 1,002× PLE compression of BF16 E4B, OPTIC retains 90.5% and 84.1%. On the official E4B release with 4‑bit linear weights and 16‑bit activations (W4A16), OPTIC retains 96.9% at about 100× PLE compression. During inference at about 100× PLE compression, BF16 E4B uses 29.7% less GPU memory than with uncompressed PLE at the same key‑value cache size, while running at most 0.40% slower.
Authors:Changseon Yu, Hyeokjun Kweon
Abstract:
Training‑free personalization adapts a frozen vision‑language model (VLM) to recognize user‑defined concepts from only a few exemplars by retrieving concept entries at inference time. While recent retriever‑based methods enrich each concept with descriptive evidence, they largely treat concepts in isolation, so the retrieved information can be plausible yet non‑discriminative, especially when the personalized concept set contains highly confusable, fine‑grained categories. We argue that personalization is inherently relational: correct recognition depends on the specific distinctions between a concept and its nearest alternatives. To this end, we propose ReCoG, a Relational Concept Graph for retrieval‑augmented personalization. ReCoG represents concepts as nodes and augments them with directed edges that explicitly capture discriminative cues between concept pairs, elicited by a frozen VLM during database construction. At inference, ReCoG retrieves a shortlist and resolves ambiguity via explicit pairwise comparisons conditioned on the corresponding relational edges, selecting the concept that consistently wins against competing candidates. We also introduce FINGER benchmark to test personalized recognition on confusable concept sets. Across FINGER, ReCoG substantially outperforms prior methods, with the largest gains in the most confusable regimes. ReCoG also achieves consistently best performance on established personalization benchmarks including captioning and VQA, confirming that relational evidence is broadly beneficial beyond fine‑grained settings. Project Page: https://csy‑98.github.io/ReCoG/
Authors:Valentino Sacco, Saurabh Pandey, Alexander Florian Jercher, Maximilian Schilling
Abstract:
Multimodal Large Language Models have shown promising performance in computer‑use tasks, from planning user workflows to grounding natural‑language actions to precise UI targets. Existing end‑to‑end benchmarks primarily emphasize long‑horizon task completion, while GUI grounding benchmarks remain focused on conventional controls and a limited set of professional interfaces. We introduce POINTERBENCH, a diagnostic suite of 1,500 screenshot‑instruction pairs probing three complementary capabilities: thin sub‑element identification and geometric grounding in spreadsheets (POINTERBENCH‑Sheets), character‑level grounding in long‑text documents (POINTERBENCH‑Text), and mixed GUI grounding across 100 professional applications (POINTERBENCH‑Pro). We carefully curate each task with pixel‑exact target geometry and evaluate both general‑purpose multimodal models and 39 specialist GUI grounding configurations. We find text grounding to be the most challenging setting, with Claude Fable 5, GPT‑5.6 Sol, and the leading native specialist UI‑Venus‑2‑9B achieving 69.6%, 53.4%, and 25.8% success on POINTERBENCH‑Text despite respective 83.2%, 77.7%, and 54.5% average success across POINTERBENCH. These results reveal substantial headroom in fine‑grained GUI grounding across frontier and specialist models. We publicly release the benchmark and evaluation code at https://github.com/warmwindOS/pointerbench.
Authors:Tirana Noor Fatyanosa, Ummu Ditya Erliana, Mahendra Data, Muhammad Nurwegiono, Eleazar Tadeo Eman, Alham Fikri Aji
Abstract:
Clinical Question Answering (QA) underpins Clinical Intelligent Decision Support Systems, but progress is severely limited for low‑resource languages such as Indonesian by the absence of public domain‑specific data. We introduce IndoNutriQA, an expert‑curated Indonesian clinical QA benchmark of 911 question‑answer pairs built from real clinical‑nutrition reports. Rather than a large‑scale training corpus, it is positioned as a high‑quality evaluation resource: every pair is grounded in a clinically relevant context, enriched with structured metadata (anthropometric assessment, medical diagnosis, document provenance, chunk segmentation). Eight standardised question types following the Nutrition Care Process (NCP), including the Problem, Etiology, Sign and Symptom (PES) diagnosis statement, are answered for every report, which was written by a graduate‑level dietetic intern, by one of three nutritionists. Answers are free‑form clinical prose rather than extractive spans, and an expert validation study finds 6‑10% of reference answers contestable. The dataset supports generative, extractive and retrieval‑augmented QA as well as zero‑ and few‑shot LLM evaluation. We report fine‑tuned baselines (IndoBERT variants, mT5, RAG) with 5‑fold cross‑validation, learning curves and bootstrap 95% confidence intervals, and benchmark eight contemporary LLMs zero‑ and five‑shot. We further calibrate our LLM‑as‑judge rubric against three expert raters (one RD, two nutritionists), finding it human‑comparable on factuality and completeness but unreliable on clinical safety, and a human error analysis shows it misses unsupported content that the raters identify. IndoNutriQA is released to foster research in Indonesian clinical NLP and nutritional decision support.
Authors:Yulong Tian, Lei Wang, Jincan Yang, Hao Wu, Hao Han, Fengyuan Xu, Sheng Zhong
Abstract:
Multi‑exit networks improve inference efficiency by allowing inputs to terminate early at intermediate exits. We identify a privacy vulnerability in cloud deployments of these networks: a malicious Cloud Service Provider (CSP) can infer sensitive properties of individual user inputs solely from inference latency, without accessing inputs, intermediate states, or model outputs. The attack exploits the CSP's control over the model supply chain. Prior to deployment, the CSP supplies a specially crafted multi‑exit network manipulated so that inputs carrying a target sensitive property are routed to a designated exit with high probability, while other inputs exit elsewhere. At serving time, the CSP infers the exit index from per‑query latency, thereby determining whether each input possesses the target property without accessing inputs, outputs, or internal model states. We propose a training framework combining selective label smoothing and entropy‑covariance optimization to control confidence distributions across exits and reduce premature termination. We conduct experiments on four datasets across ten target properties. The results demonstrate strong attack effectiveness and stealthiness. The attack achieves an average F1 score of 0.73, compared to 0.25 for the strongest baseline, while main task accuracy drops by less than 1.3% and inference speedup degradation remains below 0.1 in most settings. Our code is available at https://github.com/kazefjj/Weaponizing_the_Exit.
Authors:Miha Ožbot, Seiichi Ozawa, Igor Škrjanc
Abstract:
In this paper, we present a novel one‑shot federated clustering approach based on evolving fuzzy clustering and shuffler‑based anonymization. In federated clustering, each round reveals additional model information to the server and can increase the opportunity to infer properties of local datasets, even when raw samples are never shared. Consequently, fewer client‑server exchanges and unlinkability are important, so transmitted information cannot be attributed to any single client. Data‑stream clustering algorithms, which learn in a single pass over the data, align well with the one‑shot federated learning paradigm, where a single communication round can be sufficient. The proposed Evolving Federated Density Peak Clustering (eFDPC) method combines evolving fuzzy data‑stream clustering and density peak clustering in a federated setting. Each client independently evolves micro‑clusters from private data mini‑batches through cluster addition and merging. To limit communication, density peak clustering is applied to the learned micro‑clusters and only the selected peaks are transmitted, rather than the full local models. A shuffler‑based layer is introduced between the clients and the server to anonymize contributions prior to reconstruction and global peak selection. We consider a peak‑only variant, eFDPC‑P, as well as eFDPC‑G with server‑side label propagation. Experiments on synthetic and real‑world benchmark datasets show that eFDPC outperforms recent federated clustering baselines in one‑shot settings and scales to higher‑dimensional, non‑IID data.
Authors:Wenhua Jin, Tiancheng Chen, Sheng Tang, Jiahao Li, Yupeng Liu, Weiqi Zhang, Juntong Fang, Zesong Li
Abstract:
Moving object segmentation is conventionally formulated at the sequence and object level, marking an entire instance as moving once it moves anywhere in a video. Recent instantaneous formulations distinguish moving from stationary intervals, but still generate masks at the object level through frozen segmentation foundation models. Because the mask is derived from an instance‑level proposal, motion is typically attributed to the entire instance rather than localized to the regions that actually move. Moreover, such instance‑level predictions can produce masks even when no part of the instance is actually moving, revealing a more fundamental issue: motion is not an intrinsic property of an instance; it can only be determined by comparing at least two frames. Building on this premise, we introduce MoRSe, which jointly interprets two frames to predict a moving‑region mask, distinguishing actual scene motion from apparent motion induced by the camera. A single RGB encoder replaces multi‑stage foundation stacks, so no optical flow, point tracks, or separate mask generator is needed at inference. We release MoRSe‑5K, a benchmark of 42 sequences with Static, Dynamic‑Static, and Dynamic settings designed to evaluate the model's ability to distinguish and localize moving and stationary regions under different camera and scene motion conditions. MoRSe outperforms both specialized moving object segmentation methods and dense 4D reconstruction methods with motion‑thresholded predictions, while producing the fewest false positives on sequences where nothing moves.
Authors:Jiahao Meng, Qize Yang, Yuxuan Wang, Yunfei Chu, Wensi Dai, Jin Xu, Yunhai Tong
Abstract:
Deep research agents synthesize open‑web evidence into long reports, but current benchmarks do not evaluate research centered on a source video. We introduce Video‑Centered Deep Research Report Generation. Given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments. This task has direct practical value because it helps users locate relevant moments, verify claims, reproduce demonstrated procedures, and make informed decisions from video content. We present a skill‑based workflow and OmniDR‑Bench for this task. For evaluation, we propose Omni Atomic Rubric Trees (Omni‑ARTs). Each tree contains three dimensions, 23 shared directions, and task‑specific atomic binary criteria. The directions test audiovisual grounding, multimodal evidence use, and interactive presentation. Audited Rubric Construction (ARC) builds these trees through independent evidence collection, automated checks, behavioral calibration, and iterative human review. OmniDR‑Bench contains 50 Chinese and English tasks across 19 realworld subcategories. It includes 6,414 criteria. Across 18 agent configurations, the best system scores 63.3 out of 100. Current agents still need broader content coverage, stronger causal and logical analysis across modalities, and more accurate citation support. We further analyze models, harnesses, skill configurations, video categories, and research trajectories. The code and data are released at https://github.com/marinero4972/OmniDR‑Bench.
Authors:Yudhistira Arditya Pratama, Yi-Zeng Hsieh
Abstract:
Recognizing facial expressions in unconstrained video suffers from two coupled problems: self‑attention grows quadratically in cost with clip length, and crowdsourced labels are noisy. We present GM‑GReFEL, a spatiotemporal state‑space network with a geometry‑aware decision stage. Its Modulated GroupMamba backbone replaces a spatiotemporal Vision Transformer. Six channel groups are scanned separately, four over the frame plane and two over time, so the cost grows linearly with sequence length (\mathcalO(N)); after each layer, a Channel Affinity Modulation gate mixes the groups again. On top of the backbone, a GReFEL reliability module turns the normalized entropy of each prediction into a mixing weight between the classifier output and a vote from trainable geometric anchors, so that facial geometry counts for more when the network is unsure. The model is initialized by static‑to‑dynamic weight transfer and trained with a Triple Loss. In five‑fold cross‑validation, the best fold scores 66.57%/76.46% UAR/WAR on DFEW, 0.3 points or less below the strongest compared method, and 45.66%/62.27% on MAFW, ahead of all compared methods; on FERV39k the mean over five splits, 42.30%/52.66%, is competitive. The five‑fold means are 64.07%/74.54% on DFEW, on par with recent large pre‑trained models, and 38.90%/54.37% on MAFW, below the best of them. A paired t‑test on DFEW indicates a modest benefit of temporal modeling (+1.61 UAR points, one‑sided p=0.031). Attention stays on plausible facial regions, and latency is close to a ViT‑B/16 baseline (47.4 vs.\ 46.8 ms per clip). Code: https://github.com/ardiytama/GroupMamba‑DFER.
Authors:Wenjie Xuan, Yapeng Li, Hang Su, Chen Cheng, Juhua Liu, Sheng Zhao, Ziming Huang, Bo Du
Abstract:
The scarcity of positive cases remains a major bottleneck in developing automated diagnostic systems for fetal Congenital Heart Defects~(CHDs), particularly for rare subtypes. Although normal cases are accessible with effort, positive cases are hard to collect due to low incidence rates and privacy concerns, causing severe class imbalance or absence of positive training samples. Existing works resort to synthesizing positives with conditional generative models, but they are task‑specific and require positive training samples, hindering application for fetal CHDs. To this end, we propose a positive‑free mask‑guided counterfactual generation framework for fetal echocardiography, namely Pos4CHD, which enables realistic, high‑quality synthesis of pseudo‑positive samples from semantic masks, with only normal samples at hand. Technically, we first develop FE‑Gen, a mask‑guided generative model tailored for fetal heart ultrasound synthesis that mitigates histogram drift and enhances image‑mask spatial alignment. Building on FE‑Gen, Pos4CHD employs a mask‑to‑image generation paradigm for CHD synthesis. It first manipulates semantic masks to simulate fetal‑heart structural anomalies and then renders anatomically consistent images. Extensive experiments on in‑house datasets, together with expert evaluations, demonstrate the visual fidelity~(‑12.97 FID) and anatomical consistency~(+3.96% DSC) of synthesized images. Further validations across diverse real‑world CHD diagnosis tasks exhibit promising results and consistent increments, including zero‑shot classification~(achieving 93.25% ACC on EbA and 72.92% ACC on TAPVC), lesion detection~(+6.95% mAP50 on VSD), and anatomy segmentation~(+10.49% DSC on fetal heart) compared with baselines. Codes and models are publicly available on \hrefhttps://github.com/DREAMXFAR/Pos4CHDGitHub.
Authors:Aman Behera, Namit Solanki, Mehul Anand
Abstract:
Long‑context models often use answer‑bearing evidence less effectively when it appears away from a prompt boundary. We compare this position bias across Transformer, state‑space, and hybrid Mamba architectures by moving one gold document through a fixed set of nine distractors. The evaluated Transformers show a beginning advantage that the evaluated state‑space models lack, while both families favor evidence near the end. A matched pure‑versus‑hybrid Mamba comparison points in the same direction but remains statistically uncertain. The family difference appears among models that can answer the task. Transformer primacy reappears on synthetic retrieval, where state‑space ceiling prevents a family comparison. A corpus swap within one state‑space architecture changes overall accuracy without a detectable change in curve shape. Prompt changes can create or nearly remove a beginning advantage. Position remains decodable across architectures even when response curves differ, and attention concentration near initial tokens covaries with beginning sensitivity in the tested Transformer family. These exploratory results show that effective context use depends on the evaluated architecture family, capability, task, and prompt, without isolating the sequence mixer as the sole cause.
Authors:Junyu Lin, Bosong Qi, Pengcheng Shi, Jiayuan Li
Abstract:
Real‑world deployment of personalized federated learning (PFL) requires adapting to heterogeneous clients while preserving recognition performance beyond local distributions. However, high local accuracy can conceal failures on locally rare or absent classes. Effective personalization should therefore strengthen global recognition while raising the personalization floor. We propose an evaluation protocol that complements local accuracy with global, rare‑class, and unseen‑class evaluations to reveal generalization obscured by local scores, and assesses personalization floors among the worst‑performing clients to expose weaknesses hidden by aggregate performance. To meet these deployment challenges, we propose FedGRC, a unified PFL framework integrating generalization and personalization through (i) Consensus‑Guided Representation Learning (CGRL), which learns discriminative representations through shared directional and magnitude guidance; (ii) Multilayer Metric Synthesis (MMS), which constructs a shared covariance metric from these multilayer representations for statistical classification; and (iii) Support‑Adaptive Shrinkage Personalization (SASP), which adapts class means within this metric by shrinking local estimates toward shared means according to local sample support, enabling adaptation while retaining shared knowledge. Extensive experiments, including benchmarking against 21 methods across 5 heterogeneity settings on 3 datasets and comprehensive ablation studies, validate the effectiveness of FedGRC. On CIFAR‑10 and CIFAR‑100, FedGRC surpasses state‑of‑the‑art (SOTA) baselines in global recognition and personalization floors by at least 2.17% and 2.13%, respectively. On the real‑world heterogeneous FedRS dataset, it outperforms the respective best baselines by 2.15%, 3.30%, and 4.61% in local adaptation, global recognition, and the personalization floor.
Authors:Aman Behera, Payal Payal
Abstract:
Offline tests can fail to reveal whether an agent update will remain safe after deployment. We operationalize this post‑audit mirage using matched safe and harmful lifecycles that expose identical verifier‑visible evidence while an isolated scorer records opposite deployment truth. Our theorem applies standard observational‑equivalence and indistinguishability ideas to this executable audit boundary and states the resulting error‑abstention tradeoff. The Post‑Audit Mirage benchmark makes that boundary executable across authorization, scheduling, and shared‑capacity environments. IdentifiedRangeMonitor uses fresh group‑labelled harm outcomes to return deploy, hold, cannot determine, or unsupported. Experiments show that monitoring can recover the missing distinction in favorable settings but may remain undecided for rare groups, small effects, or missing observations. Verifier‑agent evaluation must therefore test both calculation validity and whether the available evidence supports the deployment decision.
Authors:Hao Wu
Abstract:
Our previous analysis found that a model's native hashed N‑gram table is where continual updates disturb the base least, but its writes needed backpropagation and changed the pretrained table. We present Delta N‑gram, a forward‑only write path into that table. The pretrained table T^0 is never modified. Evidence is written into a sparse patch Δon the rows it addresses: a 4.1M‑parameter writer emits a target and a gate per (position, hash head) event, and a block‑averaged delta rule keeps each row a convex combination of its old value and the gated targets. The writer is trained offline on raw text with the pretraining objective; at use time writing takes forward passes only, and clearing Δrestores the original model. A 0.6B prototype trained to answer questions keeps 16/16 facts after a second write; online gradient writes keep 2‑‑4/16. We study two writers: a plain writer that encodes evidence with the T^0‑only model, and a delta writer that reads its current memory while encoding. On Qwen3.8‑Flash‑Next, the plain writer's memory is document‑specific: on 64 held‑out news events it lowers continuation NLL by 0.068 nats/token against empty memory and by 0.024 against another document's memory, and 0.063 and 0.013 after 16 intervening articles. The delta writer's memory is more document‑specific and keeps more of it under distraction (70% vs 54%). Over 35,000 writes on our previous seven‑task chain both writers stay stable and the delta writer drifts less (layer‑2 math drift 176% vs 188%); later accuracy changes by ‑0.26/‑0.52pp (‑7.29pp for gradient updates) and small‑k_i drift is 17.9/17.5% (39.2%). Acquisition is negative, and general‑input drift reaches 32.8% (20.75%): format rows read by every example grow to 4‑‑12 times their pretrained norm. Zeroing eight such rows removes over 98% of layer‑2 drift on math inputs, together with their NLL gain.
Authors:Aleksandr Lazarev, Sofia Zimina, Roman Kuznetsov, Tatiana Zemskova, Iaroslav Koshelev, Dmitry Yudin
Abstract:
Generated robot videos can look convincing while depicting the wrong interaction: a different object is moved, a grasp disappears, or a placement ends outside its destination. We introduce RGraph‑Eval, a reference‑based framework that makes these discrepancies explicit through task‑grounded temporal scene graphs. Unlike approaches that construct separate evidence representations for different evaluation dimensions, we organize task entities, interactions, and motion within a unified, interpretable scene graph representation. A foundation‑model pipeline identifies manipulation roles, grounds persistent object masks, infers and verifies state transitions and action intervals, and reconstructs manipulated‑object motion in three dimensions. Applying this representation to a generated video and a reference demonstration supports an interpretable RGraph‑Eval score with visible‑object, relation‑change, and 3D path‑shape components. We also introduce ShareRobot‑SG, a collection of 1,000 manipulation scenes from ten ShareRobot sources annotated with the multimodal representation. Our evaluation separates graph‑generation quality from agreement with human video judgments and includes comparisons with adapted SVG2 and SG‑Ego, as well as a benchmark of three video generators. The framework connects robot‑video assessment to the observable entities and interactions underlying a task while making the graph annotations reusable beyond evaluation. Code will be publicly available at https://github.com/SashaDance/rgraph‑eval.
Authors:Yuannuo Feng, Yubing Ye, Wenshuai Yao, Yizhe Chen, Ngai Wong, Wenyong Zhou, Wang Kang
Abstract:
Diffusion language models generate multiple token positions in parallel, but few‑step decoding makes parallel commitment difficult: tokens that appear reliable individually may be jointly inconsistent, while premature commitments can prevent later context‑dependent refinement. In this paper, we introduce SetCommit, a training‑free method for coordinated parallel token commitment. Instead of selecting tokens independently by confidence, SetCommit selects a commitment set that balances token reliability against local joint commitment risk, preserving conditioning opportunities for subsequent denoising steps. It further performs budget‑neutral local exchanges, replacing vulnerable earlier commitments when newly revealed context supports more reliable alternatives. SetCommit requires no retraining or architectural modification and is compatible with existing diffusion decoding strategies. Experiments with LLaDA‑8B and Dream‑7B on GSM8K, MATH‑500, MBPP, and HumanEval show accuracy/pass@1 gains of up to 7.4 percentage points over native decoding at 32 model evaluations. At common quality targets, geometric‑mean incremental speedups range from 1.44 to 1.74×. Code is available at https://github.com/Kissmetothemoon/SetCommit.
Authors:Yun Li, Ehsan Javanmardi, Simon Thompson, Jiachen Gong, Qunli Zhang, Zifan Zeng, Shiming Liu, Peng Wang, Yidu Zhang, Manabu Tsukada
Abstract:
Diffusion trajectory planners with domain‑specific experts face a practical question in unseen regions: can they rely on selecting among existing experts, or must the planner itself adapt? We study this question using a planner with a shared adapter and per‑domain expert adapters. Across three real‑world held‑out maps, none of eight test‑time routing schemes improves the fleet‑mean lateral offset over the static default, which uses the same expert for every scene. Even an oracle that selects the best expert for each scene reduces the mean offset by only 0.9 m. In contrast, reward‑based fine‑tuning on just 20 target‑region scenes yields substantially larger gains. On matched evaluation slices, joint replay achieves a fleet‑mean offset of 6.9 m, compared with 12.4 m for the static default. Comparisons under the same target‑region training budget show that target‑region training drives the improvement, while replaying training scenes limits forgetting on the original ODDs. On nuPlan, fine‑tuning across training cities also improves open‑loop offsets, although closed‑loop performance deteriorates. Further diagnostics reveal that two test‑time adaptation baselines achieve low offsets by producing degenerate trajectories. These results show the limited headroom for selecting among existing experts and the greater open‑loop gains available from adapting the planner.
Authors:Yuhang Zhang, Yang Li, Qiang Wu, Mingyuan Li
Abstract:
In standard acoustic anomaly detection (ASD), models are usually trained separately for each machine or domain using explicit metadata. However, in realistic deployments, machine identifiers are incomplete, unreliable, or simply unavailable. Thus, metadata‑free machinery ASD asks a single model to monitor mixed machine populations without machine IDs. In this paper, we argue that the main failure mode is not a weak detector family, but representation entanglement. The heterogeneous normal modes overlap in frozen pre‑trained feature spaces, making anomaly scoring and approximate retrieval unreliable. We study this problem directly and propose HARMONY (Hierarchical Anchor Retrieval on Manifold for Oblivious‑source acoustic aNomalY detection), an anchor‑guided geometric repartitioning framework that reorganizes mixed‑source features into compact Voronoi regions and reuses this structure for hierarchical retrieval. On DCASE 2020, HARMONY improves mean AUC from 75.40% to 90.00% (14.60 percentage points) over the strongest unified baseline and from 83.90% to 90.00% over a strong foundation‑feature baseline. On a single CPU, its hierarchical retrieval achieves a 7.8x speedup over exhaustive search with only a 0.20 percentage‑point AUC drop. These results suggest that incorporating explicit geometric partitioning can improve both detection accuracy and retrieval efficiency for ASD. Code is available at https://github.com/Zzyh0208/HARMONY.
Authors:Fabian Isensee, Constantin Ulrich Harsy, Maximilian Rokuss, Benjamin Hamm, Philipp Vollmuth, Klaus H. Maier-Hein
Abstract:
Interactive 3D segmentation models such as nnInteractive turn a few spatial prompts (points, scribbles, boxes) into accurate volumetric masks, but a human still does the annotation work: finding each structure, placing prompts, checking the result and correcting it. General‑purpose AI agents can now interpret images, call tools and write code. We show that such agents can take over this work without additional training: the user makes a request in plain language, and the agent works out what has to be segmented, locates the targets, prompts nnInteractive, inspects the 3D result, corrects it and responds to feedback. Asked for 'the muscles used for chewing', an agent identifies and segments the masseters and pterygoids; asked to count liver lesions, it writes its own code to find candidates and splits a merged lesion on request. Our open‑source nninteractive‑agent provides nnInteractive as tools over the Model Context Protocol (MCP), together with a skill that encodes how experienced users prompt it. On a benchmark spanning CT, MRI, PET and micro‑CT of humans, a mouse and an ant (14 images; 51 targets from organs and tumours to muscles, foot bones, vessel trees and an insect brain), Claude Code and Codex reach a mean Dice of 0.81 and 0.84 from the request alone; when told where each target is, both reach 0.85 and prompt nearly as effectively as an oracle that places prompts using the reference segmentation. A text‑prompted 3D model (VoxTell) is more accurate on most standard radiological targets but less robust: it returns an empty mask for one of them and fails on the leg muscles, the foot bones, the mouse and the ant, where the agents remain accurate. Code: https://github.com/MIC‑DKFZ/nninteractive‑agent
Authors:Zizhe Wang, Jia Deng, Xin Lu, Zehao Jia, Yi Huang, Yao Zhu
Abstract:
Capturing a loose object inside a spacecraft requires a robot to approach a target that can translate and rotate, and whose motion changes when contact is made. These conditions motivate evaluation beyond manipulation of supported objects. We introduce AstroBench, an open benchmark for free‑floating capture and space robotic manipulation with a fixed‑base arm. The task suite combines four initial‑motion tracks with sample‑storage operations based on MELFI hardware and cap and captive‑screw servicing based on the Robotic Refueling Mission. The two workcells provide nine task definitions and 29 task‑target combinations. A metadata‑filtered Objaverse inventory supplies 8,414 object instances for asset expansion, alongside five cabin visual contexts and versioned task instructions. We report two acquisition‑focused experiments using existing checkpoints. A frozen π_0.5 DROID joint‑position policy captures 2 of 200 static targets and 4 of 200 moving targets across twenty object identities. A cube‑specific ACT policy captures 8 of 10 static targets and 6 of 10 moving targets in a separate experiment. Each experiment retains its recorded observation, action and capture configuration. The results document difficult direct transfer for the tested generalist checkpoint and successful task‑specific capture behavior, while identifying concrete settings for studying motion‑aware perception and contact‑sensitive control. Code, task definitions and editable workcell assets are available at https://github.com/AstroUpLab/AstroBench.
Authors:Noah Gießing, Ankit Satpute, Hamid Rahkooy, André Greiner-Petter, Moritz Schubotz
Abstract:
Formalized mathematics libraries such as Lean's \textttmathlib provide machine‑verifiable representations of mathematical knowledge, including definitions, theorems, and their relationships. If effectively connected to informal mathematical resources, formal mathematics could help uncover previously implicit links between mathematical concepts and improve access to existing formalizations. However, connections between informal and formal mathematics libraries are currently sparse and unsystematic. In this paper, we make first strides towards better formal integration. We construct a dataset linking informal statements from the Stacks Project to their formalizations in \textttmathlib by exploiting expert annotations. We evaluate retrieval, LLM‑based reranking, and LLM‑assisted verification to discover new unannotated informal‑to‑formal links. To facilitate exploration and reuse of formalized knowledge, we integrated the newly discovered alignments into the MaRDI knowledge graph, enabling mathematicians to find formal counterparts to informal mathematical content. We discuss the difficulties and challenges in using LLMs for linking formal and informal mathematical content. All code and data necessary for reproducing this paper can be found at: https://github.com/AnkitSatpute/Formal‑Informal‑Mathematical‑Alignment
Authors:Qiuyu Kong, Zanxi Ruan, Marco Cristani, Yiming Wang
Abstract:
Long‑text image‑text alignment is essential for fine‑grained scene understanding, yet existing methods mainly focus on extending context length or improving local alignment. We identify an overlooked failure mode, saliency collapse, where models over‑rely on text that features visual prominence while underutilizing non‑salient context. Consequently, retrieval performance degrades sharply when salient content is reduced, despite that the non‑salient context remains informative. To address this, we propose RoSA, a saliency‑aware finetuning framework that decomposes image‑text pairs into complementary salient and contextual views, enabling joint learning of prominent and contextual semantics. We propose a modality‑independent decomposition strategy to combine object‑level visual saliency estimation with LLM‑based text decomposition to bridge vision and language. We employ auxiliary objectives to explicitly align these views, ensuring robust representation with both salient and contextual features. RoSA greatly improves long‑text retrieval and demonstrates strong robustness to saliency collapse across benchmarks. Moreover, our auxiliary objectives are plug‑and‑play, bringing consistent gains to existing tuning methods. Code and checkpoints are available at: https://github.com/intelligolabs/RoSA.
Authors:Yifan Wang, Dongliang Xu, Xiaoyong Li, Jing Li, Peter S. Pan, Yue Yao
Abstract:
Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM‑based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single‑source evidence; and (3) have only binary or multi‑class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog‑Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack‑stage labels. Second, we use multi‑source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step‑by‑step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post‑detection attack‑progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack‑progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro‑F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy.
Authors:Abhishek Barali, Aayush Paudel
Abstract:
Extraction pipelines ask language models for JSON that follows a schema, and the schema can set a minimum number of list entries. We ask what models do when the document holds fewer items than that minimum. Under a preregistered plan, we test eight open‑weight models on 558 real documents from nine annotated datasets, with six more models on hosted services as a reference. Three findings stand out. First, models treat the count as a target: a minimum shown in the schema, but not enforced, raises the share of lists longer than the true count from 34.5% to 55.4%. Second, the padding is hard to catch, because most added entries are real text taken from the document; about half of them pass a check that each entry occurs in the source. Third, common fixes help only partly: a line saying fewer items are fine removes the effect on average but misses more true items, a count in prose ("up to k") also raises over‑counting, an "N/A" option under an enforced minimum still leaves half the lists padded, and re‑asking with the validation error makes padding far more common. We release the code, answer lists and run records at https://github.com/AbhishekBarali/padding‑to‑k
Authors:Gregor Cerar, Denis Sodin, Matjaž Škrlec, Urban Rudež, Mihael Mohorčič, Miha Smolnikar
Abstract:
Power‑grid transient stability is assessed by simulating credible faults, but repeating simulations as operating conditions change is costly, motivating faster critical clearing time estimators. Within the AI‑ASSIST project, we developed a retrieval‑based estimator around three adopted requirements: estimates retain their supporting states, retrieval abstains without compatible support, and support statistics are tested against observed error. Abstention turns estimator uncertainty into a demand on the archive. Exact switching‑configuration equality is used. The dense synthetic BUS39 archive covers 99.9996% of 1,044,797 records; the operational ELES archive covers 73.47% of 322,823 records. At query time, 59.37% of ELES states have an earlier same‑topology candidate. Under causal availability, retrieval covers 56.34% of records; on records covered under both protocols, mean absolute error increases by 10.0%. No tested support diagnostic ranks first on both grids. Most tested supervised regressors have lower error than retrieval on BUS39, all higher on the matched retrieval‑covered ELES population. On ELES, gradient boosting becomes more accurate than retrieval when temporally adjacent states are excluded, so the accuracy trade depends on the archive. Under the adopted compatibility key, archive availability constrains deployment more than conditional accuracy on ELES. Compatible coverage and chronological availability are auditable before an estimator is fitted.
Authors:Jialin Tian, Zhuofan Li, Xinkai Wang, Bokai Lin, Chenyi Wang, Xinyu Zhan, Cewu Lu, Lixin Yang
Abstract:
Robot demonstrations record what the robot saw and what it did. They do not specify how the 3D world changed in between. Scene‑level 3D flow describes it by tracking 3D scene points, providing the world representation with 4D geometry. Prior work has used flow to improve robot policies. Flow served as an intermediate prediction that a separate model converts into actions, as a prior injected at inference time, or as an extra output learned together with the action. In all of these designs, flow stays outside the policy, as a separate module or as an extra prediction head. We introduce UniFlowWAM, a Unified World‑Action Model that makes 3D scene flow a third modality on par with vision and action. Its world representation therefore covers both visual semantics, through future visual features, and 4D geometry, through 3D flow, and flow receives its own supervision. UniFlowWAM is a diffusion transformer with one expert per modality, and the three experts exchange information through joint attention. We train UniFlowWAM under a ten‑objective role grid, where each objective assigns every modality exactly one of three roles, namely target, condition, or absent. The same model thus serves as a policy, as forward and inverse dynamics, and as a predictor of future visual features or 3D flow. We evaluate UniFlowWAM on RoboCasa‑GR1, LIBERO, and LIBERO‑Plus in simulation, and on four real‑world bimanual manipulation tasks. On RoboCasa‑GR1, UniFlowWAM raises the success rate of its base model without flow from 52.1% to 56.4%. It preserves in‑distribution success on LIBERO at 98.1% and reaches 87.7% zero‑shot on LIBERO‑Plus. On the real robot, it achieves 87.0% average success, compared with 68.5% for the strongest baseline.
Authors:Ignacy Kolton, Wenxuan Li, Zongwei Zhou, Jarosław B. Ćwikła, Arkadiusz Sitek, Pedro R. A. S. Bassi, Szymon Płotka
Abstract:
Automated radiology report generation requires models to do more than produce fluent clinical narratives: they must explicitly state supported diagnoses while avoiding unsupported ones. However, existing medical vision‑language models (MedVLMs) are typically trained with token‑level maximum likelihood objectives, which prioritize text agreement over diagnostically critical terms. As a result, models can miss key diagnoses and introduce wrong ones. We identify this as an objective misalignment between conventional language modeling and the asymmetric diagnostic requirements of radiology reporting. To encode these requirements, we introduce Diagnostic Loss (DLoss), a differentiable, architecture‑agnostic objective that operates directly on decoder token probabilities. DLoss separately aggregates probability mass over vocabularies associated with diagnoses supported by the reference report and with unsupported alternatives. Using asymmetric pooling across token positions in the report, it encourages supported diagnoses to receive sufficient probability in at least one position while suppressing spurious probability spikes for unsupported diagnoses across all positions during training. The loss can be applied to existing 3D MedVLMs without modifying their vision backbones, language decoders, or inference procedures, and introduces no additional inference‑time computation. We train and evaluate DLoss on 50,000 abdominal CT scans across several 3D MedVLM architectures on both in‑distribution and out‑of‑distribution report‑generation benchmarks. Across these settings, DLoss improves diagnostic faithfulness while maintaining competitive performance on conventional language‑generation metrics, showing the value of asymmetric sequence‑level supervision for radiology report generation. The code for DLoss is available at https://github.com/gmum/DLoss.
Authors:Tardelli Ronan Coelho Stekel
Abstract:
Many language‑model pipelines end in a closed‑set decision, such as selecting a tool. Prompting a large language model for it is slow and order‑sensitive. We ask how small an encoder for such decisions can be and whether it should read the options jointly or separately. In a three‑seed study at 32 million parameters, per‑option scoring beats joint scoring on a held‑out sample of a 37‑benchmark decision suite by 5.8 points of chance‑corrected skill at matched optimizer steps and by 2.8 at approximately matched training compute; two external benchmarks agree, though not significantly on their unseen tasks. Crossing reader with initialisation shows that the advantage requires a reranker initialisation, whose query‑candidate pair input matches per‑option reading; from the raw encoder the two readers are indistinguishable. Per‑option scoring needs 5 to 13 times more encoder passes: at most twice the median latency, but more than four times the mean. Of six released joint encoders (17 million to 1 billion parameters), the 68‑ and 150‑million‑parameter models exceed a 435‑million‑parameter DeBERTa‑v3 zero‑shot classifier by 4.9 and 5.9 accuracy points on 209 overlap‑audited System1‑Arena tasks and are at least on par with it on the 145 tasks unseen in training. The smallest answers a median request in 14 milliseconds on 16 processor threads, against 728 for the classifier. Reordering options changes 12 to 23% of the joint models' answers (per‑option scoring: none). Up to 68 million parameters, they lead their size range on a public leaderboard's private, mainly in‑distribution tests.
Authors:Hyunwoo Yu, Jongbeom Lee, Hyunbin Kim, Sungjun Park, Jaemin Choi, Jincheol Yang, Seunghun Moon, Bongjoon Hyun, Kyomin Sohn, Kyeongbo Kong, Suk-Ju Kang
Abstract:
Sparse attention reads only a small subset of the key‑value cache, and how to treat the unread tail remains a key challenge. Discarding the tail biases the output, and sampling it as previous approaches do fails at reads of 5% and below, where the tail likely holds heavy hitters the selector missed. Our key observation is that the simplest possible proxy, one token that folds the entire tail into its count‑aware mean, already yields lower output error than a sample at every read behind a practical selector. We further show in closed form that at low reads sampling nearly doubles the weight error of discarding while a single proxy stays below it. Building on these findings, we explore approximating the dense attention distribution with a few proxies that target the tail. We propose OpenAttention, a pipeline of clustering, ErrorGate and top‑k selection followed by a count‑aware mean proxy. To implement clustering and selection both efficiently and effectively, we design training‑free Query‑Geometry Clustering (QGC) and a Query‑Geometry Selector (QGS), which compress the keys while respecting the query directions. Additionally, unlike prior methods that rank clusters by centroid alone, ErrorGate allocates the exact read by the estimated weight error of each cluster. On RULER‑32K HARD with Llama‑3.1‑8B, OpenAttention retains 99% of dense accuracy at a 1% read. It also achieves state‑of‑the‑art accuracy across models, context lengths and datasets. Our customized attention kernel for OpenAttention runs the decode attention stage 10× faster than dense attention at a 1% read.
Authors:Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu
Abstract:
Ophthalmic clinical decision support requires fast, accurate, and privacy‑preserving inference, yet autoregressive large language models (LLMs) incur substantial latency and computational overhead for decisions among predefined actions. We therefore investigate a decision‑oriented alternative to generative reasoning. Specifically, we first introduce RetiJev‑Data, a source‑grounded ophthalmology corpus comprising 78,158 clinical‑decision and diagnosis questions derived from articles in PubMed Central and a MeSH vocabulary covering 531 eye diseases, and 50,074 guideline‑grading questions spanning 11 grading systems, along with a 24,662‑question evaluation benchmark. Based on this, we develop RetiJev, a family of 0.8B‑9B decision models that directly score candidate answers without autoregressive decoding. Across all model sizes, RetiJev outperforms both a 27B LLM and the commercial Jev 1.13 decision model on guideline grading by 10.4‑23.0 points. RetiJev‑9B further achieves the highest accuracy on clinical decisions, improving over both baselines by 5.4‑6.5 points while matching them on diagnosis. It answers a case in 158 ms on one GPU, 3.5x faster than LLM generation and 470x faster than reasoning. Models are released at https://github.com/Awenbocc/RetiJev.
Authors:Pengfeng Song, Shaozhong Zhang, Pengfei Shao
Abstract:
Fine‑grained image‑text retrieval requires a model not only to assess global semantic consistency between an image and a text, but also to model visual evidence for local textual semantics, including objects, attributes, quantities, actions, and relations, so as to distinguish semantically similar candidates. Existing methods commonly use fixed image patches, detected regions, or text tokens as the basic matching units and then aggregate local interactions through max pooling, cross attention, or similarity accumulation. Such strategies remain limited in semantic unit construction and global matching consistency. This paper proposes a hierarchical fine‑grained cross‑modal alignment framework, termed HFCMA. The framework progressively transforms low‑level patch and token representations into adaptive visual regions and textual phrases, and then performs structured region‑phrase matching within a global retrieval scoring process. Specifically, HFCMA first adopts dynamic granularity discovery (DGD) to aggregate raw patches and tokens into semantically compact visual regions and textual phrases. It then formulates region‑phrase matching as an entropy‑regularized optimal transport problem, thereby estimating a structured alignment matrix under global marginal constraints. To reduce background noise and local semantic drift, HFCMA further introduces background debiasing, textual importance weighting, and prototype consistency regularization. Experiments on Flickr30K and MS‑COCO show that HFCMA effectively combines dynamic semantic unit construction with OT‑based structured matching under a dual‑encoder retrieval framework. The resulting fine‑grained score provides structured local alignment information for candidate ranking and improves the discrimination of subtle semantic differences.
Authors:Xuyang Zhong, Jiuding Yang, Yiwen Guo, Chen Liu
Abstract:
Mix‑policy optimization incorporates off‑policy expert data into the reinforcement learning (RL) loop to transcend the capability boundary of on‑policy RL. However, existing methods either provide static full‑trace guidance that suppresses self‑exploration, or adopt expert prefix guidance without correctness guarantees, leading to suboptimal performance or even training collapse. In this paper, we propose MOER, a mix‑policy optimization framework that harmonizes expert guidance with self‑exploration. MOER injects a decaying expert chain‑of‑thought prefix into the prompt and lets the model complete the remaining reasoning, with the prefix length progressively annealed to encourage self‑exploration. A selective rollback mechanism replaces failed completions with the original full expert trace, guaranteeing the correctness of every off‑policy trajectory. Furthermore, a harmonized mix‑policy objective optimizes on‑policy tokens with a GRPO loss while supervising both expert prefix and completion tokens with a trust‑region SFT loss, substantially stabilizing training. Extensive experiments show that MOER achieves the highest average scores in mathematical and general domain reasoning tasks, outperforming all basic and mix‑policy baselines, and generalizes consistently across diverse tasks, model scales, and base model categories. Codes are available at https://github.com/CityU‑MLO/MOER.
Authors:Jing Wei, Guanyi Chen, Qi Chen, Chuyu Zhang, Zheng Zhou, Chutian Wang, Jian Zhang
Abstract:
Data‑intensive GPU applications increasingly rely on NVMe SSDs to extend GPU memory. NVIDIA GPU Direct Storage (GDS) enables direct SSD‑GPU transfers via PCIe peer‑to‑peer DMA, eliminating host staging. However, GDS retains three key limitations: (1) its control path traverses the kernel for every I/O request, introducing software overhead; (2) its stream‑ordered path imposes strict total ordering even when workloads only require batch‑level ordering; and (3) its closed‑source user‑space engine hinders diagnosis and custom scheduling. We present uGDS, https://github.com/ScaleX‑IO/uGDS, an open‑source user‑space GPU Direct Storage library built on four principles: user‑space control, unified cross‑vendor abstraction, workload‑aware scheduling, and end‑to‑end transparency. The host CPU manages the entire NVMe command lifecycle in user space, while payloads move via PCIe P2P DMA without consuming GPU compute. A minimal kernel module is used only for one‑time privileged setup. uGDS partitions hardware queues between synchronous and batch traffic, enforces DMA‑safe buffer lifetimes, and provides cuFile‑compatible synchronous, batched, and stream‑ordered APIs with polling or interrupt completion on both NVIDIA CUDA and AMD HIP. Evaluated on an A100 with a Samsung 990 PRO, uGDS achieves up to 12.12x GDS's batch‑read bandwidth and 30.69x its batch‑write bandwidth (8‑ and 4‑KiB requests, batch size 128), and reduces 4‑KiB synchronous write latency by 84.1% with two threads. Its stream‑ordered path sustains 5.49 GiB/s on a single stream, where GDS retains only 40.8% of its synchronous bandwidth. In LMCache, uGDS reduces median time‑to‑first‑token (TTFT) by up to 62.7% and raises serving goodput by 2.63x. On AMD, uGDS achieves up to 1.19x hipFile's synchronous‑write bandwidth.
Authors:Firoj Paudel
Abstract:
Nepali‑English code‑switched conversational speech presents severe challenges for automatic speech recognition (ASR), chiefly through script confusion (inappropriate mixing of Devanagari and Latin orthography) and elevated error rates at language switch points. Supervised fine‑tuning (SFT) can adapt multilingual foundation models, but human code‑switched audio is expensive to collect and annotate. We curate a new benchmark of 13,044 spontaneous podcast utterances (15.45 h total; 10,214 test, 2,830 dev), audited and verified to eliminate machine translation leakages and acoustic loops. We investigate a two‑stage synthetic‑to‑real adaptation framework: Stage 1 applies SFT on 0.95 h of speech synthesized via a multi‑speaker, dual‑engine TTS pipeline; Stage 2 explores Group Relative Policy Optimization (GRPO) with transcription and script rewards. We find that synthetic SFT provides the primary cross‑domain transfer to spontaneous human speech, reducing Mixed Error Rate (MER) from 23.40% to 21.52% (‑1.88 pp) and switch‑point error rate (SPER) from 35.89% to 28.77% (‑7.12 pp). Subsequent GRPO yields small, statistically significant adjustments on 173 test utterances (1.69%), driving English script confusion (SCR_en) from 6.45% to 6.36% (p <= 0.001) and SPER to 28.70% (p = 0.022 clustered). Crucially, ablation reveals that in this SFT‑converged encoder‑decoder setting, an auxiliary script reward term yields performance identical to MER‑only optimization because rollout candidate groups exhibit near‑zero script variance on synthetic data. On monolingual benchmarks, SFT improves Nepali MER by 2.71 pp (stabilizing at 18.42% after GRPO, a 2.38 pp gain over baseline) while preserving English lexical WER (3.38% vs. 3.33%). Verified transcripts, timestamps, episode metadata, and reconstruction scripts are publicly released.
Authors:Yeon Su Park, Chanyoung Park, Tae Soo Kim, Seungju Kim, Hyunwoo Kim, Daho Jung, Jinsung Yoon, Juho Kim
Abstract:
LLMs are increasingly used as AI tutors, yet existing benchmarks overlook the challenging behaviors common among students with learning and behavioral difficulties. We introduce TutorCaseBench, a benchmark that grounds AI‑tutor evaluation in qualitative case studies from pedagogy, educational psychology, and special education. Our pipeline converts documented student categories into case‑specific scenarios, tutor‑student dialogues, and rubrics grounded in source evidence. The benchmark contains 923 dialogues across five challenge types, evaluated through open‑ended response generation (1,560 items) and multiple‑choice selection (2,537 items). Across 11 state‑of‑the‑art LLMs, even the strongest model satisfies a rubric criterion in only 55.8% of open‑ended cases, while models are better at selecting appropriate strategies than generating situated responses. These results reveal a gap between intervention recognition and context‑specific adaptation, especially for open‑weight models. All data are publicly available at https://github.com/kixlab/TutorCaseBench.
Authors:Chenyu Zhu, Ruoyu Zhao, Zhichao Lu
Abstract:
Predicting relative trained performance from architectural specifications can guide design when training costs limit experimentation. Neural Spectral Capacity (NSC) provides such a heuristic, but scores projections independently and assigns identical scores to compatible reorderings of the same layer configurations. We introduce Residual‑Aware Neural Spectral Capacity (ResNSC) for Transformers, using a specification‑derived statistical reference for accumulated residual updates to determine the input and residual‑state statistics in each projection's capacity calculation, while retaining additive evaluation without data or instantiated weights. When residual widths vary, earlier choices affect later scores, so we augment NSC's allocator (NSC‑DP) with residual history beyond its layer‑and‑budget state. ResNSC‑DP jointly allocates residual and feed‑forward widths under a resource budget, with certified bounds on the best numerically evaluated ResNSC score within a specified finite candidate set. At the 121M and 374M pretraining scales, selected architectures achieve lower mean perplexity than uniform, variable‑width, and tapered allocations on FineWeb‑Edu, WikiText‑2, and LAMBADA at comparable parameter budgets. In inherited‑weight LLaMA‑7B pruning using the LoNAS supernet, ResNSC‑DP improves eight‑task average accuracy over NSC by 3.28‑‑5.46 percentage points across eight matched parameter budgets. Paired reversals preserve the projection shapes and NSC scores yet worsen performance in both settings, exposing performance differences hidden by position‑independent scoring. The code for this work is available at https://github.com/Optima‑CityU/neural‑spectral‑capacity.
Authors:Taehyun Byun, Taemoon Jeong, Yoonbyung Chai, Woojin Wie, Insu Park, Kyungjae Lee, Sungjoon Choi
Abstract:
Humanoid motion‑tracking policies can reproduce diverse behaviors, yet connecting independently acquired motions into a continuous sequence remains challenging. Interpolation connects boundary poses without considering whether their relative position, heading, and timing provide suitable conditions for a transition. A generative motion prior can produce richer motion, but poorly arranged boundaries may cause it to generate excessive locomotion, turning, or another unintended action. We present Spatio‑Temporal Alignment with a Motion Prior (STAMP), a framework for connecting independently acquired humanoid motions through a Robot‑to‑Human‑to‑Robot process. STAMP uses human motion space to draw on rich motion priors and reuse transition knowledge across robot embodiments. Its key component is a learned alignment model that determines the boundary conditions for motion inbetweening. Given the two motions retargeted to a human rig, the alignment model ranks candidate positions, headings, and transition durations for the following motion. A human motion prior generates the transition under the selected alignment. The completed sequence is retargeted to the robot and used as a reference to train a motion‑tracking policy. We evaluate the tracking success of the generated transitions on Unitree~G1 and~ROBOTIS K1. Physical robot demonstrations further show that STAMP connects independently acquired motions into long‑horizon sequences. Project page: https://childtoy.github.io/STAMP‑project
Authors:Haoran Lv, Chengjin Ni, Manqin Wang, Shenli He
Abstract:
We give a self‑contained proof that, for a finite detailed balanced mass‑action system with positive rate constants, each positive stoichiometric compatibility class contains exactly one positive equilibrium and every positive trajectory converges to it, together with a Lean 4 formalization of the complete argument from persistence to convergence. Boundedness and persistence come from the barrier argument of OpenAI (2026) for weakly reversible systems: an arbitrary‑tolerance affine approximation yields a finite‑minimum barrier whose superlevel sets confine trajectories and give one logarithmic bound for all solutions from a given initial state. The Horn‑Jackson dissipation identity identifies the positive equilibria as an exponential family with parameters in the orthogonal complement of the stoichiometric subspace, which gives uniqueness in each class. A bounded‑speed descent argument replaces LaSalle's invariance principle; it uses neither uniqueness of solutions nor continuous dependence on initial data, and it places every cluster point in that family. The route given here is direct and elementary, and its main theorem is checked by the Lean kernel against explicit definitions. The proofs, the Lean development, and the manuscript were produced in ClawsGO Science, an agent harness that packages a complete research environment: Claude Opus 5.5 made the research decisions, GPT 6 Astra carried them out, and the Lean kernel checked every formal step.
Authors:Zhiyang Li, Ruijiang Jin, Thibaut Klenke, Yusuke Sekikawa, Nakamasa Inoue
Abstract:
Semi‑dense local feature matching commonly aggregates contextual information before coarse‑to‑fine correspondence estimation. However, intra‑image and inter‑image aggregation serve different roles: the former should propagate spatial evidence within each image while retaining structured two‑dimensional dependencies, whereas the latter should gather information from plausible corresponding regions while limiting noise from unrelated locations. We propose WovenAnchor Matcher (WAM), a semi‑dense matching framework based on role‑specialized context aggregation. WAM introduces two complementary operators. WovenMamba performs intra‑image aggregation through horizontal‑then‑vertical state‑space propagation, allowing vertical updates to operate on horizontally contextualized features and thereby encouraging a structured two‑dimensional receptive‑field bias. AnchorCrossAttention performs inter‑image aggregation by using cross‑attention to gather context from plausible corresponding regions across the image pair. It performs this exchange over local representative anchors that aggregate feature evidence within each window. On MegaDepth, WAM achieves 66.1 pose AUC@5^\circ, improving on the JamMa baseline by 2.0 AUC points under the same evaluation setup. Ablations show that removing or replacing either specialized component reduces accuracy, providing evidence that intra‑image propagation and inter‑image retrieval benefit from role‑specific aggregation designs. Code and pretrained weights are available at https://github.com/mmaiLab/WovenAnchorMatcher.
Authors:Shijie Li, Yilin Gao, Tieyuan Chen, Yun Luo, Weiyao Lin, Hang Yu
Abstract:
Training large language models for complex reasoning tasks relies on two complementary signals: sparse, outcome‑grounded reinforcement learning (RL) for correctness, and dense, token‑level distillation for guidance. However, complementarity in supervision does not guarantee compatibility in optimization; imitation‑focused distillation can conflict with the outcome‑driven RL objective, and existing hybrids lack a principled resolution. We propose GIFT (Gradient‑Informed FilTering for Distillation‑Augmented Reasoning RL), a novel method that directly addresses the allocation problem of which teacher updates to apply. GIFT performs a token‑level geometric analysis, decomposing the distillation gradient into a component parallel to the RL gradient, which measures first‑order conflict and provides a safety filter, and a component orthogonal to it, which quantifies novel directional guidance and provides a measure of value. This decomposition yields a principled scoring rule to rank token‑level guidance. We combine this with a progressive coverage annealing schedule that shrinks the budget of tokens receiving distillation, ensuring that the most conflicting updates are discarded first while the most complementary ones are retained at full strength. This facilitates a smooth transition from dense, geometrically‑filtered guidance to a pure, outcome‑grounded objective at convergence. Extensive experiments on mathematical and logical reasoning benchmarks show that GIFT consistently outperforms strong baselines across multiple model scales, achieving relative gains of up to 9.5% on difficult mathematical reasoning tasks. Code: https://github.com/lilifree/GIFT
Authors:Seon Gyeom Kim, Hannah Kim
Abstract:
Agents can create rich representations, yet these representations are rarely used as a medium for expressing user intent. We present Agent‑Bound UI, an interaction model that separates reusable interaction structures from task‑specific meanings. Agents bind task‑specific meanings to interactions in reusable views and publish the resulting bindings as shared state, enabling humans and agents to understand and act on shared surfaces whose meanings vary across tasks. We implement ABRA, a framework‑agnostic prototype of Agent‑Bound UI, in which users and agents collaborate through chat and shared interactive canvases. We demonstrate ABRA through a representative use case and evaluate four LLM‑backed agents on canvas authoring, owner‑side consequence resolution, and cross‑agent invocation. The code and demo video are available at https://github.com/megagonlabs/abra.
Authors:Jiaqi Chen, Xianglong Yan, Chengzhu Bao, Xie Chen, Yulun Zhang
Abstract:
Mixture‑of‑Experts (MoE) large language models reduce computation through sparse activation but still incur substantial storage costs. Mixed‑precision post‑training quantization (PTQ) reduces these costs by quantizing different model components at different bit widths, but existing methods often struggle to preserve generation quality under tight bit budgets. We identify two key challenges: (1) nonlinear quantization errors at extremely low precision are underestimated when cross‑layer propagation is ignored; (2) direct router fine‑tuning may cause routing drift, and calibration gains may not generalize to downstream tasks. To address these issues, we propose ProQuant, a weight‑only PTQ framework for MoE models. Prune‑or‑Prioritize Mixed‑Precision Allocation (PoP) jointly optimizes expert pruning and bit‑width assignment. It allows higher‑precision experts to substitute for severely degraded ones while improving bit‑budget utilization. Routing‑Consistent RMSNorm Finetune (RoCo) adapts expert input representations through mathematically equivalent transformations. It compensates for quantization errors to reduce routing drift. Extensive experiments demonstrate that ProQuant outperforms existing methods across diverse generative tasks. Code is available at https://github.com/ThrowBug/ProQuant.git.
Authors:Angel Raychev, Nathan Lile, Rafael Rafailov
Abstract:
We study generative reward models (GenRMs) as online preference providers for policy optimization in creative writing. Across Qwen3 reward‑model sizes from 0.6B to 14B, Thinking judges trained with GRPO have higher static Gold‑label accuracy than Answer‑Only judges trained with SFT. This ranking often reverses when the judges train policies through online DPO: Answer‑Only rewarders yield higher peak and top‑five‑average policy Elo in seven of eight matched settings under a fixed, learned 32B Gold evaluator. Intermediate Thinking‑judge checkpoints can also outperform later, more statically accurate checkpoints as rewarders. Sweeps over rewarder size, policy size, training duration and DPO regularization show substantial benefits from reward‑model capacity, including judges larger than the policy. We analyze trajectories with evaluator‑specific Elo arenas, conditional match‑resampling intervals and descriptive scaling fits. These results demonstrate an evaluator‑rewarder gap in the studied pipelines: fixed‑distribution judging accuracy does not determine downstream optimization quality. The comparison changes judge‑training objective as well as response format, and the main ranking is conditional on its learned reference rather than a universal human‑quality result. A limited follow‑up using a different learned judge is inconclusive overall.
Authors:Cristian Espinal Maya, Santiago Jiménez Londoño
Abstract:
35.2% of task time in world employment is at least partly delegable to AI agents in 2026, and only 2.5% can be handed over in full. Existing measures score exposure to generative AI, mostly for the United States. We build the Agent Delegability Index (ADI): two language models rate all 18,796 ONET 30.2 task statements on a frozen, preregistered rubric of delegation levels and barriers; tasks are mapped to 433 ISCO‑08 unit groups and weighted by ILOSTAT employment in 146 countries. Four findings. First, delegable employment rises with income, from 22.5% in low‑income to 48.7% in high‑income countries, a slope of 7.23 percentage points per log point of GDP per capita. This delegation divide is compositional: the physical barrier dominates the jobs poorer countries employ most. Second, at the occupation level ADI tracks exposure closely (r = 0.93 with Eloundou et al. and 0.89 with Felten et al.), above the preregistered bound of 0.80; ADI adds delegation levels and barriers on top of exposure, for the world, not a new ranking of occupations. Third, full delegation is rare and peaks for tasks that take a human about twelve minutes (11.8 minutes). Fourth, the preregistered adoption gap is not supported: across 101 countries, delegability and observed AI usage correlate at r = 0.86, and the gap between them is not wider in middle‑income countries (p = 0.27). Model agreement is kappa = 0.77; a blind human rating of 300 tasks agrees with the model consensus at kappa = 0.85, and an agentic panel with web search at kappa = 0.87. We release the index, the atlas, the prompts and an MCP server at https://github.com/Cespial/delegation‑divide.
Authors:Jie Ying, Xutao Zhang, Ningyuan Chen, Mengce Zheng, Jun Li, Zhongjie Zhu, Tiantian Zhu
Abstract:
When tool‑use agents write files, execute code, or send data, their final responses do not explain the resulting security events, and heterogeneous runtimes expose the relevant evidence through incompatible interfaces. We present AgentTriage, an executable contract for replayable security diagnosis across heterogeneous runtimes. The contract admits conclusions whose claims cite canonical runtime events and requires abstention when mandatory evidence is missing. Runtime overlays normalize native observations, while a deterministic pipeline joins invocation and result lifecycles, constructs a typed provenance graph, selects a policy‑decisive slice, and binds claims to recorded evidence. Given the same normalized trace, overlay version, and pre‑run policy oracle, the diagnosis can be recomputed and its claims re‑checked without language‑model judgment. Across five runtimes, one contract captures all 159 applicable executions, with an audit of native and synthesized identifiers. On 120 adjudicated cases, AgentTriage achieves 0.858 grounded joint diagnosis, 0.966 required‑evidence coverage, and a 0.036 unsupported‑claim rate, improving joint diagnosis by 0.175 (95% CI [0.092, 0.258]) over the strongest same‑schema baseline. These results quantify the precision of replayable diagnosis and the observation boundaries that require abstention. We release an open‑source implementation and selected sanitized evidence at https://github.com/AsnGs/agenttriage‑open‑source.
Authors:Shiyang Zhang, Miao Yang, Zhengda Li, Yang Shao, Ruifang Zhai, Hui Peng, Peng Song
Abstract:
Vision‑Language‑Action (VLA) models have shown promising results in multi‑task robotic manipulation by leveraging visual and language understanding for action generation. However, existing lightweight VLA models employ a single dense action expert to handle all tasks, which limits their multi‑task capability under constrained computational budgets. To address this issue, we propose SmolVLA‑MoE, a novel framework that introduces sparse Mixture‑of‑Experts (MoE) into the FFN layers of the action expert in VLA models. SmolVLA‑MoE replaces dense FFN layers with multiple parallel expert FFNs and a token‑level router, enabling each token to adaptively select the most suitable expert for action generation. We further introduce a shared expert with a tunable weight parameter to balance the inheritance of pre‑trained capabilities and task‑specific expert specialization. Through sparse activation, SmolVLA‑MoE activates only a single expert FFN during inference, with a total activated parameter count of approximately 500M including the frozen VLM backbone, while benefiting from approximately 800M total stored parameters. We evaluate our method on four task suites of the LIBERO benchmark. SmolVLA‑MoE achieves an average success rate of 84.25%, outperforming SmolVLA (82.75%) and OpenVLA‑7B (76.5%), while maintaining comparable inference efficiency. Our code is publicly available at https://github.com/dudu435/smolvla‑moe.
Authors:Andrés Daniel Godoy Ortiz
Abstract:
This paper studies the local geometry of semantic meaning in large language models. Current representation analyses ask whether meanings are close in embedding space, or whether linear probes separate semantic classes, but they do not characterize the within‑concept local geometry: what is invariant across realizations of a single semantic factor, what varies, and what changes when meaning is refined. This paper characterizes that geometry through two controlled probes evaluated on four architecturally diverse decoder‑only LLMs (Gemma 4, Llama 3.2, Qwen 2.5, Pythia). A cross‑lingual probe uses language as a perturbation to separate exact meaning matches, same‑factor realizations, nearby concepts, and out‑of‑domain contrasts. A refinement probe separates specificity‑bearing semantic narrowing from high‑probability attributive elaboration and adversarial controls; both probes are validated by blind human judgments. Four findings recur across all models: under mean pooling, once dominant variance directions are removed, sentence pairs order by semantic relatedness (same factor, then same domain, then different domain) without overlap over a band of layers; semantic factors organize around an invariant core surrounded by structured realization‑specific variation; refinement produces directed displacement into tighter neighborhoods, including against controls built on the same base noun; and sentence‑level and token‑local readouts expose complementary geometry (volume contraction at the sentence level, sharper anisotropic endpoints at token‑local readout). We interpret these results as evidence that LLM semantics are organized as local anisotropic neighborhoods in which meaning contracts under refinement.
Authors:Jianrong Lu, Zhiyu Zhu, Hui Liu, Junhui Hou
Abstract:
Training‑free few‑step samplers are often described as nearly quality‑lossless under conventional FID/CLIP‑style evaluation. It remains unclear whether that conclusion transfers to more realistic and complex generation‑quality benchmarks. We first systematize the quality metrics used to evaluate training‑free sampling acceleration for diffusion and flow‑matching models. We find that representative training‑free step‑shortening methods often remain nearly lossless under FID/CLIP‑class metrics, yet can suffer substantial quality degradation when evaluated with semantic‑composition, dense‑instruction, and long‑text‑rendering benchmarks such as GenEval, DPG‑Bench, and LongTextBench. Parallel sampling is a notable exception: by retaining the full‑step discretization and removing only the solver's temporal dependency, it can preserve near‑lossless quality while accelerating sampling. Building on this observation, we propose ParaLin++, a step‑parity parallel sampler that preserves the full‑step discretization while accelerating the solver iterations. Experiments across Stable Diffusion‑v2, LSUN Church, FLUX.1‑dev, and Ming‑Flash‑Omni 2.0 show that ParaLin++ preserves the full‑step Euler quality profile at 2.6×‑‑4.8×, 1.68× on FLUX, and 1.21× on Ming. Project page: https://paralinplus.github.io/.
Authors:Shiming Chen, Shuhuang Chen, Salman Khan, Fahad Shahbaz Khan
Abstract:
Domain generalization (DG) aims to improve the generalization of the model to any unseen domains. Targeting this goal, existing DG methods mainly focus on learning domain‑invariant features or prompts from the given source domains. However, these methods rely on source‑domain data, and collecting and annotating large‑scale single‑ or multi‑source datasets is often infeasible, limiting the practical applicability of DG. To tackle this challenge, we propose \underlineDomain \underlinePrompt Learning (DoP), a method built on a large vision‑language model (e.g., CLIP) for zero‑shot domain generalization, which recognizes unseen classes in unseen target domains without access to source‑domain data. DoP essentially learns the target domain knowledge via prompt both in text and visual modalities to overcome the domain shift in text‑visual modalities under the test‑time adaptation formulation. Specifically, DoP first takes text domain learning (TDL) to simulate various distribution shifts by learning diverse style tokens implicitly in the text space with inter‑style diversity loss and intra‑style consistency loss. Subsequently, DoP employs visual domain learning (VDL) to further refine the target‑domain‑relevant style token generated by TDL by making use of the test visual data with self‑supervised learning. Extensive experiments on popular DG benchmark datasets (e.g., PACS, VLCS, OfficeHome, and DomainNet) and ImageNet show that our DoP consistently outperforms the state‑of‑the‑art DG approaches. Codes are available at: https://github.com/shiming‑chen/DoP.
Authors:Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah
Abstract:
Federated unsupervised domain adaptation (FUDA) addresses the challenge of adapting models to an unlabeled target domain using decentralized source domains while preserving data privacy. However, existing FUDA methods typically require multiple communication rounds, rely on complex aggregation strategies, and often struggle with noisy pseudo labels and inconsistent source knowledge. To address these challenges, we propose SKD‑ETR, a novel one‑shot FUDA framework that combines Smoothed Knowledge Distillation (SKD) and Exponential moving average‑based Teacher Refinement (ETR). SKD trains a student model on the target domain using smoothed soft pseudo labels generated by the ensemble of source models. This reduces overconfidence, mitigates noise, and improves robustness. ETR further refines each source model by interpolating its parameters toward the student via exponential moving average (EMA), thereby transferring target‑domain knowledge back to the teachers. This bidirectional refinement enhances pseudo‑label quality and student generalization without additional communication overhead. SKD‑ETR eliminates complex aggregation by initializing the student model randomly and performing a single‑round distillation process. Extensive experiments on OfficeHome, Office‑Caltech, and DomainNet demonstrate that SKD‑ETR achieves competitive performance while being communication‑ and computation‑efficient, and robust under noisy supervision. The code for this paper is available at https://github.com/A‑Abedi/SKD_ETR.
Authors:Ruixuan Deng, Zhengfeng Ji, Jiayu Zhang, Li Zhou
Abstract:
Quantum key distribution (QKD), whose security is derived from quantum mechanics rather than computational assumptions, is moving into standardization and commercial deployment. Yet its security guarantees rest on technically delicate proofs checked by hand. Formal verification closes this gap, establishing a two‑factor assurance: unconditional security from physical principles, and machine‑checked proofs against reasoning errors. We present FVQKD, a Rocq library for machine‑checking QKD security proofs. Its first result is a complete, end‑to‑end machine‑checked proof of composable, finite‑key security for BB84. The mechanization reconstructs the complete proof infrastructure from first principles, including the smooth entropy framework, the entropic uncertainty relation, and the Quantum Leftover Hash Lemma. It uncovers necessary side conditions that the published argument leaves unstated, one of which reveals a missing proof branch, and it grounds the information‑theoretic bound in a formal semantics of protocol execution, which the pen‑and‑paper analysis lacks. The proof is entirely admit‑free, relying only on the Rocq kernel and standard foundational axioms. To our knowledge, this is the first end‑to‑end formal verification of a QKD security proof. Built on MathComp, MathComp‑Analysis, and CoqQ, the development provides quantum‑information infrastructure designed for reuse across broader QKD variants and quantum cryptography applications.
Authors:Siran Dai, Qianqian Xu, Peisong Wen, Yang Liu, Xiaochun Cao, Qingming Huang
Abstract:
Self‑supervised Vision Transformers learn strong image‑level representations, but their patch‑level representations can be unreliable for dense prediction. We study this mismatch and find that the class token concentrates global semantics on a small number of representative patches, while dense prediction errors often occur on low‑attention regions. We provide a local gradient analysis showing that, along the [CLS] attention readout, image‑level self‑distillation gradients are weighted by [CLS]‑to‑patch attention mass. To provide masked patches with global semantics without changing the architecture, we propose Cross Distillation (CODI), which mixes a small amount of the teacher class‑token distribution into each teacher patch‑token target. CODI gives every masked patch an attention‑independent semantic correction while preserving the standard self‑distillation architecture. Experiments on dense and image‑level benchmarks show that CODI improves both dense and image‑level performance under aligned pretraining settings. Code is available at https://github.com/EldercatSAM/CODI.
Authors:Zhao Tan, Xianglong Bao, Zhipeng Xi, Ming Jin
Abstract:
Natural language querying for time series databases aims to let users find what they need in long histories of data by asking in plain language. Shape queries ask how a series moves. They are the main challenge of this task. Recent work makes them feasible by using symbolization as a bridge between the query and the long history. However, existing symbolizations were designed for compression, similarity indexing, or classification. What a symbolization should provide for natural language querying has not been systematically studied. We therefore examine the definitions of six classical symbolizations against this task and identify three limitations. (1) Direction is missing from the string, (2) direction is measured at too small a scale, and (3) the letters are arbitrary. Three design principles follow. We follow these principles and make a minimal redesign of SAX called TT‑SAX (two‑track SAX). The level letters stay. Each segment gains a trend letter. It is measured across neighboring segments rather than inside one, and graded to match the adjectives that queries use. We conduct extensive experiments on the public NLQTSBench benchmark, comparing TT‑SAX with the six classical symbolizations across six LLMs. TT‑SAX outperforms all six on shape queries, and ablations trace its gains back to the three limitations one by one. This paper is the first systematic study of time series symbolization with natural language querying as the design goal. It provides a foundation for future research on this task. Our code is available at https://github.com/Atlamtiz/tt‑sax.
Authors:Victor Li, Owen Tucker, Michael S. Hughes, Temiloluwa Prioleau, Shengpu Tang
Abstract:
A digital twin (DT) for type 1 diabetes (T1D) can predict how an individual's blood glucose will respond to different insulin treatments without actually trying them. For each treatment, downstream statistics of predicted glucose trajectories, such as glycemic risk, can be computed to guide clinical decisions. However, the quality of DTs is usually measured by the accuracy of their predicted glucose trajectories, which may not reflect the quality of the downstream treatment decisions they inform. In this work, we directly evaluate DTs by this decision quality on downstream treatment ranking, i.e., whether a DT predicts glucose trajectories that can be used to rank candidate treatments as the real patient would. We present an evaluation protocol over 30 virtual patients from the FDA‑accepted UVA/Padova model, each simulated under 21 different basal‑bolus treatments. For each patient, a DT is fitted under one treatment and then produces glucose trajectories for the remaining 20 treatments. Our results show that prediction accuracy and decision quality disagree. A baseline that models the average patient, never seeing the individual's glucose data, has the lowest prediction accuracy of all six methods (RMSE: 63.6 mg/dL) but ranks treatments with the second‑highest mean Spearman (0.766), while a black‑box neural network that predicts glucose nearly twice as accurately (RMSE: 32.9 mg/dL) ranks them no better than a random ordering of the 20 treatments (0.049 Spearman). Therefore, selecting a DT based on prediction accuracy alone can prefer one that recommends treatments at chance, and decision quality must be evaluated directly rather than assumed to follow from prediction accuracy.
Authors:Zhixin Lin, Dongliang Xu, Jungang Li, Shidong Pan, Jun Feng, Yanbin Sun, Jing Li, Liang Liu, Yue Yao
Abstract:
Unlike conventional GUI agents, personalized GUI agents require user preferences to guide sequential execution over changing GUI states. This raises two challenges: determining HOW each preference should alter execution strategy and WHEN it should take effect along the interaction trajectory. Existing benchmarks primarily evaluate overall task success, offering limited insight into these questions. We introduce PrefDiag‑Bench with two components: (1) a categorization of preferences into five types, characterizing HOW they affect execution; and (2) a stage‑wise diagnostic framework with 1,664 probes, tracing WHEN preferences should take effect and where failures occur. We evaluate eleven GUI agent systems, including end‑to‑end agents and agentic workflows, under two forms of user context: explicit user profiles and behavioral memory logs. Although existing agents achieve relatively high completion rates on non‑personalized tasks, performance is substantially lower on preference‑personalized tasks. Even the best‑performing model, Seed‑2.0‑Pro, achieves only 47.17% on Process preferences under explicit user profiles. Agents struggle particularly with Ranking and Process preferences. Further analysis reveals two key capability bottlenecks: selecting relevant evidence from user history and recognizing when preferences should take effect during execution. Controlled interventions that explicitly provide relevant evidence improve end‑to‑end task success by up to 7.06%. PrefDiag‑Bench offers a more systematic evaluation of GUI agents' personalization capabilities beyond overall task success. Code and data are available at https://github.com/Zhixin‑L/PrefDiag‑Bench.
Authors:Xiaonan Xu, Wenjing Wu
Abstract:
Purpose: Coding agents follow a project's instruction file, such as CLAUDE.md or AGENTS.md, even when it disagrees with the existing code and even when they notice the disagreement. We measure this behaviour and test whether one sentence in the user's own instruction file makes agents follow the code instead. Methods: We built 40 Python projects in which the instruction file asks for one convention while the existing code uses another, for example print instead of logging. Claude Code and Codex CLI added a feature to each project, with and without a sentence in the user‑level instruction file that gives the existing code precedence, in 320 runs. Results: Without the sentence, 90.1% of the agents' choices followed the stale file. Claude Code pointed out the conflict in 79 of 80 runs and still followed the file. Codex CLI rewrote the existing code to match the file in 20 of 80 runs, and the project's own tests then failed in 17 of them. The sentence raised the share of choices that follow the code from 4.2% to 91.4%, stopped the rewrites and raised the test pass rate from 89.4% to 100%. Conclusion: We attribute the behaviour to how agents receive the two sources: the instruction file reaches the model as an instruction and the code as tool output, which models are trained to rank lower. Users can add the sentence today, and makers of agents and models can build the same rule into their products.
Authors:Yebo Feng, Jingyang Dai, Liangrui Pan, Nelson Lau, Jiahua Xu, Yang Liu
Abstract:
LLM agents that act on live Web data must combine heterogeneous evidence, remember what they have done and improve from outcomes without weakening their own safeguards. We study this problem in autonomous investing, where every action is irreversible and based on untrusted Web content. We present Steward, a multi‑agent LLM system that draws the boundary of self‑evolution in its type system. First, nine role‑specialised agents decide jointly: four analysts read news feeds, SEC filings, macroeconomic series and market data and emit typed reports, from which a Portfolio Manager sets target positions that a protected Risk Manager audits. Second, a seven‑layer shared memory, assembled by code from the system's own record, gives each decision its position context, the market narrative, calibration and the agent's past decisions. Third, the system improves itself by review: code grades every trade and forecast nightly and writes the grades into memory, and each quarter a reflector proposes prompt learnings that a deterministic gate applies reversibly and replay of recorded inputs verifies. All three operate around an invariant core of order arithmetic, risk rules and discipline‑agent prompts that no self‑edit can reach. In a five‑month live deployment (102 trading days, 1,632 LLM calls), the system contained a 24.6% sector drawdown to a 0.3% loss; in replay of its recorded inputs, a single model given the same raw data traded more and lost 1.35 points against the market over five days where the joint design showed no significant loss, and no injected instruction was followed; and a 2015‑2026 backtest shows the core alone lowering the maximum drawdown under all 50 random entry policies. The reflector traced the dominant failure to its root cause, and the gate applied a learning that the agent adopted on trigger evenings and on no control evening. Code: https://github.com/yebof/quant‑agent
Authors:Isaia Nisoli
Abstract:
We consider a bounded operator on a separable complex Hilbert space, reached only through its action on vectors and that of its adjoint, and split it along a subspace of finite dimension into a compression, a far block acting on the complement, and the two couplings between them. Given upper bounds on the norms of the couplings and of the far block, we show that outside a disc a pseudospectrum of the operator lies in a pseudospectrum of the compression, with an explicit loss in the level; this reverses the classical inclusion of the Ritz values in a pseudospectrum of the operator. The same bounds, with the smallest singular value of the shifted compression, bound the resolvent of the operator, count its eigenvalues with algebraic multiplicity inside every contour outside the disc on which a computed inequality holds, and enclose its spectral projector; no normality, self‑adjointness or positivity is used.
Authors:Zhipeng Cao, Peiyang Song, Nikolaus Vertovec, Weiru Cui, Zhihao Yang, Mingkang Liu, Luxia Lin, Bai Xue, Alessandro Abate
Abstract:
Lyapunov functions and barrier certificates give state‑space guarantees of stability and safety, but synthesizing and verifying them‑‑‑especially for nonlinear, high‑dimensional systems‑‑‑remains an open challenge. Existing pipelines verify candidates with numerical solvers (sum‑of‑squares programs), interval methods, or satisfiability‑modulo‑theories (SMT) solvers, which often require relaxations, are conservative, or scale poorly with model dimension and nonlinearity. These limitations have also shaped benchmark design: standard test suites are dominated by low‑dimensional polynomial systems chosen to suit what solvers can handle rather than what engineering applications require. We address both sides of this challenge. First, we introduce DynaCertBench, a benchmark of 105 instances: 80 non‑redundant systems curated from the literature, extended with high‑dimensional and control‑constrained variants and new nonlinearly coupled systems, comprehensively spanning stability, region‑of‑attraction, forward‑invariance, and safety tasks. Second, we present LLM2Cert, an LLM‑driven framework that synthesizes candidate certificates and verifies them against their proof obligations: each obligation is routed to an SMT solver or directly to the Lean Proof‑Repair agent. On DynaCertBench, LLM2Cert verifies 81 of 105 instances (77.1%), a 17.1 percentage point improvement over one‑shot proposal without feedback. On the 81 fixed certificates, LLM2Cert verifies 22 that All‑SMT leaves unresolved. Code and data are available at https://github.com/FormalCert‑AI/llm2cert.
Authors:Louise Yorke
Abstract:
LLM judges are used to determine which of two answers is better. Trusting these verdicts requires knowing how repeatable they are and how much they depend on the judge's instructions. We show that similar aggregate scores can conceal substantial changes in individual verdicts, both when the same setup is repeated and when the instructions change. Using 500 Arena‑Hard v2.0 prompts with two models' answers held fixed, we run GPT‑4.1 as the judge under all eight combinations of criteria, output format and task framing, and repeat every setup unchanged at temperature zero on a pinned model version. Rerunning the default setup moves the aggregate score by about one point but changes the win/tie/loss outcome on 21.2% of prompts. Instruction changes introduce further sensitivity: in one setup, which asks for the verdict without reasoning and also drops several other instructions, outcomes change on 28.2 percentage points more prompts than the two setups' unchanged reruns do on average, and the aggregate score falls by 4.09 points. In another, where the criteria prioritise accuracy, outcomes change on 6.7 percentage points more prompts by the same comparison, while the aggregate score barely moves. A stable aggregate score therefore does not establish that individual verdicts are repeatable. This distinction matters when those verdicts are used as preference labels or to assess individual responses. Evaluations that rely on individual verdicts should repeat each setup they use and report both aggregate scores and prompt‑level agreement.
Authors:Hiroyasu Usami
Abstract:
An online tracker retains one state and may reset only to the current observation, although its complete observed history is available. Offline knows the future and obeys the same reset rule. Under a hard discrepancy radius, for every horizon T\ge5 and every feasible randomized causal policy, a fixed path on a known five‑point metric requires at least (T+1)/2 expected refreshes, against exactly two offline refreshes with a strict radius margin. Missing the useful prefix center forces a refresh at every suffix observation. The path visits four values, with resolution fixed as T grows. Write J_λ for cumulative discrepancy plus a charge λ>0 per refresh, and V_T for the sum of successive observation distances. A candidate prefix adapts Bienkowski and Schmid's randomized lower bound to this objective under current resets, even with a known horizon and V_T<3λ. This construction needs finer grids; a fixed finite metric with positive minimum separation instead admits a competitive policy for penalized tracking. An anytime policy satisfies J_λ\leλ+4\sqrtλTV_T without knowing the horizon or final variation. Clipping for a known horizon gives J_λ\leλT and a factor‑three comparison with the minimum of three analytical bounds, without knowing final variation. Current‑value diameter and readout sensitivity give same‑input output bounds for mixtures, including squared‑distance softmax.
Authors:Yifeng He, Jiachen Liu, Hao Chen
Abstract:
AI agents now run experiments and report findings, so research produces claims faster than researchers can check them. Each empirical claim implies a property that must hold over its cases, but it states that property only in prose. Without an explicit property, no test can decide which evidence would falsify a claim, so validation cannot keep pace with the claims agents produce. To make claims testable, we approximate a claim's semantics by abstracting its property skeleton, a parameterized relation that recurs across claims. We present ARATest, a property‑based testing library that checks these skeletons as executable tests. A typed declaration binds a claim to a skeleton and fixes the claim's compared quantities, cases, scope, and numerical policy. A shared case type makes every generator supply cases in the claim's statistical unit. A three‑valued semantics assigns each property one of three verdicts: holds, violated, or inconclusive. Every violation carries a replayable witness. A default generator reads the evidence that agent‑native research artifacts (ARA) link to each claim, and auto‑research generators dispatch an agent to run released code for fresh cases. Our DissClaimer plugin supplies that agent and runs these generators as continuous testing. Of 243 claims in 33 ARA artifacts, 54 received faithful declarations bound to evidence: 32 held, eight were violated, and 14 remained inconclusive. Two violations exposed errors in published papers that their authors confirmed. ARATest detected all 553 seeded faults without a false alarm on 19 valid controls, outperforming two LLM judges. Fresh runs exposed behaviors that reported evidence could not show. These results suggest that one fixed test can check explicit claim properties against both reported and fresh evidence.
Authors:Cheng-Yin Tu
Abstract:
Should an embodied agent's experience be baked into its policy weights, or kept external and retrieved at inference? A memory‑augmented teacher (a VLM that retrieves lessons, with their episode images, from its own past attempts) produces trajectories on a vision‑conditioned reach task; from the same trajectories we distil LoRA student policies and evaluate them under matched protocols, including one that re‑attaches the teacher's frozen retrieval buffer to a fixed set of student weights. We pre‑registered the hypotheses, arms, tests, and analysis rules before training and before any p‑value. Distilling the memory‑augmented behaviour into weights beat an action‑only control by +1 pp (n.s.); the registered test proves underpowered for its own minimum effect, so we withdraw ‑ rather than refute ‑ the memory‑in‑weights claim. Attaching retrieval to a fixed set of student weights added +34 pp (z=5.15), reaching the pooled memory‑augmented teacher reference (49.3%); the pre‑registered protocol contrast agrees (+23 pp, z=3.36); under a pre‑registered auto‑reset sensitivity count the two are +36 and +29 pp. An exploratory behavioural probe with pre‑registered per‑arm predictions is consistent with behavioural distillation transferring the marginal correction but not the conditional, situation‑specific correction that retrieval supplies. The recipe that follows is to externalise the memory: in follow‑up controls registered before they ran, the same frozen buffer lifts the base model with no adapter to 48% (sensitivity 48%) and the other three adapters from 19‑26% to 53‑55% (sensitivity 17‑21% to 50‑55%), and the adapter adds nothing detectable once the buffer is attached. On a second, harder task, the retrieval advantage points the same direction but falls short of significance (+6 pp, 95% CI [‑4.5, +16.4]); exploratory diagnostics tie the attenuation to memory coverage.
Authors:Riteng Zhang, Xiaoqian Wang
Abstract:
Sparse autoencoders (SAEs) were originally motivated by discovering monosemantic features, a property that characterizes individual features but does not specify what the learned feature set should look like as a whole. As SAEs have increasingly been adopted for attribution, circuit discovery, and intervention, researchers have begun evaluating metrics that describe global properties of the feature set, such as sparsity, dead feature rates, reconstruction fidelity, and disentanglement, a diverse set that extends far beyond monosemanticity. However, this implicit shift beyond monosemanticity has not been formalized as a clear, well‑defined objective that comes before these evaluation metrics. Without such an objective, existing metrics remain ad hoc, with their necessity unjustified. In this work, we formalize this ultimate objective by introducing a new paradigm for SAE evaluation: requiring that SAE features, and more generally dictionary‑based decompositions of model representations, form an atomic interpretable unit basis, a coordinate system for semantically meaningful explanation. Under this paradigm, we organize existing and new measurements such as DAME (Dominance‑Aware Monosemanticity Evaluation) as proxy metrics and introduce a comprehensive benchmarking framework. Evaluating hundreds of SAEs across multiple architectures and model families, we uncover failure modes in which some SAEs appear monosemantic under existing evaluation metrics while remaining functionally polysemantic: the most dominant features, those contributing most to model computation, are systematically less interpretable and more redundant. These failure modes warrant attention from SAE researchers as targets for improved training and architectures, and motivate dominance‑aware evaluation as a standard check for practitioners before SAE features are deployed for attribution, circuit tracing, or intervention.
Authors:Alankrita Bhatt, Mukur Gupta, Andrea Montanari, Pulkit Tandon
Abstract:
Conditional generation is a key capability to be supported by Large Tabular Models, enabling a number of applications. Accurate conditional generation has to contend with two key challenges: (i) Data comprise a mixture of numerical and categorical entries, the latter often with high cardinality and heavily unbalanced proportions; (ii) Conditional generation becomes increasingly difficult for rare sub‑populations. The best models in this class use a hybrid of continuous and discrete diffusions. In this paper, we design a continuous diffusion model with strong generation quality. An important element is the use of cross‑entropy loss for categoricals, for which we provide a first theoretical justification. We introduce metrics that focus on rare subpopulations. Further, we generalize conditional generation techniques to enable new types of conditioning. Finally, we show that our model (trained on labor data) can capture classical insights from econometrics. Code is available at https://github.com/granica‑ai/DiffLTM
Authors:Seemandhar Jain, Manmohan Chandraker
Abstract:
Reproducing a published 3DGS paper takes weeks of expert engineering before any extension can be proposed. Prior autonomous‑research systems read papers, draft hypotheses, and write code, but are limited by short fixed templates, low executability, and weak traceability, and none has been demonstrated end‑to‑end in a live computer‑vision subfield with intense activity. SplatLab is an autonomous research system for 3D Gaussian Splatting that addresses these limitations. Given a research prompt, a target‑paper URL, or no input at all, it reproduces a published baseline, proposes an extension, trains and ablates across Mip‑NeRF 360 and LLFF, writes the manuscript, and binds every reported number to a logged training run. Five design choices distinguish it: (i) candidates mutate plugins from a verified‑reproduction backbone matched to published baselines; (ii) a cascade‑weighted Elo tournament across eight islands scores candidates on measured GPU PSNR with verbal‑gradient feedback; (iii) a standing falsifier subtracts Elo on failure, integrating stress‑test survival into selection; (iv) every reported number is bound to a logged training run, and the run cannot terminate while any claim is unbacked; (v) a seven‑category 3DGS taxonomy wraps EVOLVE‑BLOCK markers in domain‑aligned scaffolds transferable across 3DGS repositories. On PSNR‑targeted runs, SplatLab's manuscripts beat their reproduced targets by +0.18 to +3.78 dB, including +3.78 dB on astrophotographic nebula rendering, for which no prior 3DGS method exists. The research cycle compresses from months of graduate‑student time to days of mostly autonomous compute. Code, all manuscripts, experiment ledgers, and the cross‑run paper store are available on the project page.
Authors:Zi Yang, Ying Li, Alejandro Lancho, Darío C. Larese, Pablo M. Olmos, Michael Minyi Zhang
Abstract:
Kernelized attention provides a useful perspective for understanding self‑attention as a similarity‑based aggregation mechanism, but existing approaches often rely on fixed or stationary kernels, limiting their ability to adapt the similarity structure to complex token interactions. In this paper, we propose flow‑based spectral kernel attention (FSKA), a framework for learning expressive nonstationary attention kernels. FSKA formulates kernelized attention through spectral density learning and parameterizes the resulting bivariate spectral density with normalizing flows. This design enables flexible and regularizable kernel learning while remaining compatible with scalable linear‑attention computation. We further show that expressive kernel learning enables a simplified attention design with removed query‑‑key projections and fixed orthogonal value projections, resulting in a more parameter‑efficient architecture. Controlled experiments on classification, ablation, and scalability profiling show that FSKA achieves competitive or improved performance over existing attention mechanisms, while reducing projection‑related trainable parameters by approximately 40%‑‑47% compared with most baselines and maintaining favorable linear‑attention memory scaling. Code Available: https://github.com/ziyang18/FSKA.
Authors:Ronaldo Cristiano Prati
Abstract:
aggmetrics is a Python package that extends any binary performance measure expressed in the four confusion‑matrix counts to multiclass, multilabel, ordinal, cost‑sensitive, soft‑label and cross‑validation settings, by substituting one of four aggregation operators for what libraries usually expose as an average switch. The package also stores, for every measure, the structural properties that decide which identities apply to it, can infer those properties from a user‑supplied formula, and uses them to warn when a measure is used where it does not behave as commonly assumed. Its test suite encodes the results of the companion theory as executable tests. The package is BSD‑3 licensed, depends only on NumPy, and is available at https://github.com/rcprati/aggmetrics; documentation at https://aggmetrics.readthedocs.io.
Authors:Ahmed Mehdi Inane, Mahdi Haghifam, Gintare Karolina Dziugaite, Ioannis Mitliagkas
Abstract:
What does it mean to have an optimal data selection algorithm? Optimality is conventionally framed as finding a subset that minimizes the population error of the classifier trained only on that subset. Our starting point in this work is first to show that this goal is information‑theoretically impossible unless the data selection algorithm has access to a validation set order‑wise larger than the training set. This impossibility result motivates shifting from unverifiable population risk minimization to a one‑sided guarantee that we refer to as faithfulness: for any candidate subset, we provide an estimable proxy that upper‑bounds the difference between training on the full training data and on only the subset. We model training with the Unadjusted Langevin Algorithm and bound the Kullback‑‑Leibler divergence between subset‑trained and full‑data‑trained parameter distributions by the trajectory‑accumulated gradient discrepancy. Minimizing this discrepancy gives a principled way to select a subset. Even though the resulting problem is NP‑hard, we propose a swap‑based local search that decreases the bound at every accepted step. It can therefore refine the output of any selector without worsening its bound. Finally, we provide experimental results on CIFAR10 and CIFAR100 showing that it matches existing data selection baselines. Implementation of our algorithm and the code for reproducing the results are available at: https://github.com/Mehdi‑Inane/data_selection_swap.
Authors:Muhammad Danial, Muhammad Taimoor Khan
Abstract:
Recent reports, such as "Who is using AI to code?", indicate that approximately 30% of Python code contributions to GitHub are AI‑generated. Given the inherent unreliability of this code, formal verification emerges as a promising approach to establish trust in software correctness. Although leveraging large language models (LLMs) for formal verification has garnered increasing attention, practical adoption remains limited due to the challenges in writing formal specifications and verifying program correctness against them. Existing tools predominantly support verification for languages such as Java and C, with minimal support for Python. We present TrustedCode, an automated framework for verifying AI‑generated Python code. TrustedCode uses LLMs to (i) type‑check Python code, (ii) translate it into WhyML via the Why3 framework, and (iii) verify it against formal specifications using Why3 back‑end automated theorem provers. Our method integrates chain‑of‑thought prompting through LangGraph to identify and address LLM limitations in specification generation and proving the specified programs. To evaluate TrustedCode, we conducted experiments on ten Python programs of varying complexity. Our analysis reveals that 50% of the observed errors stem from domain knowledge gaps, 30% from syntactic inaccuracies, and the remaining 20% from semantic and type‑related issues. These findings underscore the importance of targeted improvements in LLM‑guided formal verification workflows for Python. TrustedCode is an open source tool available at: \hrefgithub.com/mtaimoorkhan/TrustedCodegithub.com/mtaimoorkhan/TrustedCode. The video demonstration is available at: \hrefyoutu.be/Nr5I2g8ec‑syoutu.be/Nr5I2g8ec‑s
Authors:Tengwei Song, Robert Hoehndorf
Abstract:
LLM agents are flexible in language interaction, but using them as long‑horizon planners is costly and prone to hallucinated or invalid actions. This paper targets a different role for LLMs: constrained bridges between language and symbolic planning. We propose MemPlan, a memory‑conditioned PDDL planning framework for partially observable interactive text environments. MemPlan keeps the PDDL domain fixed and reconstructs the PDDL problem at each step from parsed observations, candidate hypotheses, and interaction memory. Positive graph memory assigns relevance‑based action costs to candidate hypotheses for missing objects, locations, or conditions, guiding the planner toward more plausible checks. Negative memory records execution‑refuted assumptions and compiles them into exclusion constraints, preventing repeated attempts at candidates already contradicted by feedback. A unified schema‑conditioned language‑‑symbol interface, trained offline with supervised fine‑tuning, connects text environments to the planning loop through observation‑to‑fact parsing and symbolic‑action‑to‑command grounding. Experiments on TextWorld, ALFWorld, and Robotouille show that MemPlan improves task completion and step efficiency over language‑only and PDDL‑aware baselines, while substantially reducing online token cost compared with LLM‑planning baselines such as ReAct. The same fine‑tuned interface is also reused across TextWorld task types without task‑specific fine‑tuning. Our code and data are available at: https://github.com/tewiSong/MemPlan.
Authors:Nimun Bajwa, Massimo Poesio
Abstract:
Activation probes are computed directions in a language model's residual stream, used both to detect safety‑relevant behaviours and, by adding their direction back into the stream, to steer them. We explore how the main design choices behind such probes interact, within one pipeline, on one model (Llama‑3.1‑8B‑Instruct), and across three behaviours: deception, high‑stakes awareness and sycophancy. The pipeline crosses two estimators (difference‑of‑means and logistic regression) with three token‑pooling schemes (last, mean and learned attention), selects appropriate layers to detect or steer a behaviour, and evaluates every direction twice: as a detector of the behaviour, and as a steering vector to push outputs towards the behaviour. Prior work read together suggests a read/write dissociation, in which logistic regression reads (detects) best but difference‑of‑means directions write (steer) best. We find that it holds for deception and high‑stakes but not for sycophancy, where the pattern reverses. The reversal appears to follow partially from the data rather than from the estimators. The best pooling technique and the layer choice depend on the behaviour and data, and the layer that steers best can sit well before the layer that detects best. The right steering metric to use between changing a model's perception and eliciting behaviour depends on the task ‑ and the two can disagree outright. The exported probes and directions power an interactive interface through which a user can score and steer text directly.
Authors:Chenyi Tong, Ping Nie, Yu Zhang
Abstract:
Hallucinated citations make it harder to trace sources and verify the evidence behind scientific claims. Detecting them is difficult because bibliographic records can be incomplete or differ across source types and publication versions. A plausible match alone neither validates the citation nor establishes its underlying source. We introduce CiteResolve, a benchmark of 423 real‑world hallucinated citations annotated for source recoverability. We further propose CiteAnchor, a source‑aware grounding framework that adapts retrieval and field‑level verification to each source type and separately determines citation validity and whether a unique source can be recovered. On an existing detection benchmark, CiteAnchor achieves 0.841 F1 with GPT‑5.6 Luna and 0.784 with an open‑weight backbone, outperforming the baseline with the same backbones by 7.9 and 36.8 points. On CiteResolve, it achieves 74.7% and 70.7% end‑to‑end task success, outperforming the corresponding web search baselines by 16.3 and 65.3 percentage points. Our code and data are available at https://github.com/oldwizard7/CiteANCHOR_code.
Authors:Saurabh Kumar
Abstract:
Every implementation of Bitcoin's Lightning payment network disclosed security vulnerabilities in 2026, and every advisory told operators to upgrade. Whether they do has never been measured, because Lightning gossip carries no software version. Matching the feature bits a node announces against the source of every LND, Core Lightning, Eclair and LDK release, we show that gossip reveals a node's release line after all. Where Core Lightning nodes state their version, the inferred line is exact for 132 of 136 node‑line pairs. Across 814 daily snapshots of the public graph from 2023 to 2026, the network patches on release and platform schedules, not on disclosures. Half of LND nodes upgrade within 88 days of a release; small nodes and those that skipped releases lag far behind. The three largest upgrade waves came about a day after the Umbrel app store shipped a new LND version. Since June 2025, 17% of LND upgrades came in the 48 hours after a platform release, windows that cover 4.4% of the time. Disclosures, a median 256 days after the fix, moved few operators: of 12 we can test, only Core Lightning's of July 2026 sped up upgrades, and only among nodes one release line behind. Its later call to restart offline was followed mostly by nodes already patched. Two months after Lightning Labs' August 2026 batch, 71% of the LND nodes exposed at disclosure still ran an affected release. For maintainers, the lever on the nodes that lag is the platforms that package their releases.
Authors:Bangpu Chen, Yaxuan Li, Shirui Peng, Xiangtian Si, Liuxin Chu, Xitong Cao, Hongbo Jin, Jiayu Ding
Abstract:
Dynamic representations based on 4D Gaussian Splatting (4DGS) have shown strong capabilities in reconstruction and rendering, yet current methods remain limited in supporting complex natural‑language referring in dynamic scenes. Existing 4D Gaussian methods are primarily designed for rendering‑oriented dynamic modeling, while current semantic extensions often rely on scene‑specific semantic optimization or are evaluated on relatively simple query settings. In this paper, we study Referring 4D Gaussian Splatting (R4DGS), a task for grounding temporally varying, multi‑target or reasoning‑intensive, and zero‑target or distractor expressions in dynamic 4D scenes. To support this task, we introduce R4D‑Bench‑QA, a benchmark with structured query annotations. We further present ReferGaussian, a unified framework built upon 4DGS that converts low‑level dynamic Gaussians into persistent, language‑addressable entities for training‑free referring inference. ReferGaussian organizes mask‑supported Gaussian entities in an EntityBank, uses Multimodal Large Language Models (MLLMs) to decompose complex queries into trackable object phrases and structured referring constraints, and performs constraint‑aware grounding without scene‑specific semantic optimization. Experiments on R4D‑Bench‑QA and the 4D LangSplat protocol demonstrate improved temporal target selection and competitive spatiotemporal grounding without scene‑specific semantic training. Project resources are available at https://chenbangpu.github.io/R4DGS/.
Authors:Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim, Jinwoo Shin, Sihyun Yu
Abstract:
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre‑trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self‑Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self‑distillation to predict future representations from noise, using representations extracted from ground‑truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real‑world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long‑horizon and out‑of‑domain tasks under zero‑shot environment generalization, and substantially improves human‑to‑robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.
Authors:Ripon Chandra Malo, Zimeng Yu, Tong Qiu
Abstract:
AI coding agents could change whether organizations buy software or build the functionality they need. We focus on organizations and professional buyers whose domain knowledge can help them specify and evaluate tools for their work. We define the software substitution frontier as the functions customers could build and maintain at an acceptable standard for less than buying them. Using public data, we compare growing capacity to build with indicators of market change. Since 2023, the length of benchmark tasks that leading agents can complete at 50 percent success, measured in human working time, has doubled about every 129 days. A broad search identified 575 potential open‑source alternatives to selected commercial products, although some functional matches still require independent validation. The Craft suite produced 12 application repositories over 12.0 days of public development, but these remain early versions with unverified completeness and reliability. Market evidence is mixed: purchased software gained US investment share and tracked commercial installations grew. Yet 32 percent of respondents to an organizational survey reported forgoing at least one software purchase because their organization could build the functionality with coding agents. These self‑reports do not establish the spending displaced or the reliability of the resulting tools. The gap motivates a warning: customers' capacity to build could affect individual buying decisions before a broad purchasing shift becomes visible. We examine this possibility using benchmark projections and observable market tests. The projections are an extrapolation, not a forecast of customer switching.
Authors:Muheng Li, Wenlong Mou
Abstract:
Reliable feedback is essential for coding agents to develop programs for partial differential equations (PDEs). Yet evaluating these programs on new PDE problems often requires expensive reference computations, while fixed datasets restrict the range of experiments. We introduce Generator‑Agent‑Verifier (GAV), a scientific experimentation environment that makes new PDE problems verifiable by construction. The Generator learns a distribution of continuous reference solutions using data from numerical simulations or experimental measurements to represent a target physical regime. Using the method of manufactured solutions, it works backward from the generated reference solutions to construct matching PDE problems without a separate numerical solve. The Agent designs experiments, investigates failures, and iteratively improves its program. The Verifier evaluates candidate programs against the reference solutions and provides numerical feedback on accuracy and computational cost. This feedback can also guide the generation of new problems that probe the current program's weaknesses. We provide a theoretical analysis of evaluation reliability and guided development. Experiments on Navier‑Stokes and Vlasov‑Poisson‑type equations across different data sources and regimes demonstrate the effectiveness of GAV. In particular, on the numerical Navier‑Stokes task, GAV reduces the mean test error of three existing PDE agents by more than an order of magnitude relative to their original setups. Code is available at https://github.com/PotatoJnny/PDE‑GAV.
Authors:Kaijian Zou, Vaibhav Chennareddy, Yunxiang Zhang, Naihao Deng, Longju Bai, Jiaxin Pei, Rada Mihalcea, Lu Wang
Abstract:
Coding agents spend substantial computation generating actions and processing long interaction histories. Poor action choices can steer agents into low‑value states, wasting computation and increasing API costs. Sampling and verifying alternative actions can improve the agent's decisions, but applying this procedure at every step adds substantial overhead. We introduce SWE‑Verifier, a lightweight generative model that selectively triggers candidate sampling and verification. It scores the coding agent's proposed action to determine when additional candidate sampling is needed, then ranks the resulting actions before execution. We train SWE‑Verifier from Qwen3.5‑4B using preference data that groups contrasting actions with their resulting environment observations and rubric‑guided teacher scores. Using the mini‑swe‑agent harness, SWE‑Verifier improves resolution from 75.47% to 77.20% on MiniMax‑M2.7 at 1.09 times cost and generalizes to DeepSeek‑V4‑Flash without training on its actions, improving resolution from 72.60% to 74.27% at 1.13 times cost. Across both models, SWE‑Verifier verifies fewer than 4% of steps and achieves comparable or higher resolution than the trajectory and all‑step baselines at substantially lower cost. These results demonstrate that selective candidate sampling and lightweight verification provide complementary gains in coding‑agent performance and inference efficiency. Our model, code, and data are available at https://github.com/zkjzou/SWE‑Verifier.
Authors:Henry M. Haase, Andrew R. Winters, Jan Glaubitz
Abstract:
Summation‑by‑parts (SBP) operators are a cornerstone of provably stable high‑order methods for hyperbolic conservation laws. For nonlinear problems, the flux‑differencing framework of Fisher and Carpenter combines such SBP operators with entropy‑conservative two‑point fluxes to obtain semi‑discretizations that are conservative in the sense of Lax‑‑Wendroff and discretely entropy‑conservative or stable. So far, this framework has been developed exclusively for operators that are exact for polynomials. Function‑space SBP (FSBP) operators lift this restriction by being exact on a general finite‑dimensional function space that can be tailored to the problem at hand. In this work, we extend flux differencing to FSBP operators for general (potentially non‑polynomial) function spaces. We prove that, whenever the operator has a diagonal norm and its function space contains the constants, the resulting semi‑discretization is conservative in the sense of Lax‑‑Wendroff, semi‑discretely entropy‑conservative within a single element, and entropy‑stable across element interfaces. We also provide the first theoretical accuracy analysis of FSBP operators: We characterize their design order through an algebraic criterion on the underlying function space, prove that suitable non‑polynomial spaces achieve the same design order as polynomial spaces of equal dimension, and show that the flux‑differencing semi‑discretization retains this order. Notably, the entropy‑conservative two‑point flux is reused unchanged; only the underlying function space differs from the polynomial case. Numerical experiments for the Burgers' equation and the compressible Euler equations in one and two dimensions confirm these properties.
Authors:Saad B. Alarifi, Mohammed A. Alam
Abstract:
Large language models are increasingly used in competitive programming, and their outputs are sensitive to how a prompt is phrased. Prior work has compared prompt strategies for accuracy on competitive programming, but not for runtime speed, and has not situated the results against human performance. In an exploratory study of one prompt in each of nine styles on GPT‑5.5 across three accuracy tasks from the International Olympiad in Informatics (IOI) and two NP‑hard speed tasks, we found that no single prompt was best: the strongest prompt depended on the task type and on whether results were summarized by the mean or the median, suggesting that, at least for these tasks, prompt style should be matched to the objective rather than treated as universal. Many differences in mean scores came from a few outlier runs, showing that evaluating prompting by a single metric can obscure how much its effectiveness varies. Compared descriptively with human IOI contestants, who worked under different conditions, the AI was neither strictly better nor strictly worse, exceeding the average contestant on one task but falling short on the other two. By extending prompt comparison from accuracy to runtime speed and grounding it against human performance, this study gives a broader picture of how prompt style affects competitive programming performance.
Authors:Siavash Golkar, Jake Kovalic, Shirley Ho
Abstract:
LLMs can act as scientific agents using tools to consult specialists and run experiments. Yet specialist information must enter their context, generally as text, images, audio or video: a bottleneck between models with rich, high‑dimensional representations. We advocate direct representation exchange: a mindmeld conveying many properties, including information an LLM would not think to request. It should be extensible to new specialists, flexible across questions and tasks, and compositional across independently connected specialists without joint training. The Latent Interconnect Protocol (LIP) learns connectors mapping specialist representations into LLM internal states. Training only connectors keeps this practical as LLMs grow and evolve. We connect three protein and three small molecule specialists to Gemma 3 (4B and 12B parameters) and Kimi K2.6 (1T total, 32B active), freezing all pretrained models. Independently trained connectors provide useful information without changing the LLM or existing connectors, supporting familiar and new question types about unseen objects. We demonstrate zero‑shot transfer to cross‑domain tasks without joint training. Task performance and compositionality generally improve with LLM capability, with especially strong gains in zero‑shot task transfer. Limitations remain: current LLMs were not trained for specialist representations as native inputs; access does not ensure reliable interpretation, integration or use in reasoning. We call on specialist developers to provide consistent representation interfaces and, especially, on LLM developers to train models to receive, combine and reason with these inputs. These efforts could expand scientific agents' information while preserving conventional tools' flexibility. Data, pretrained connectors, and code for training, evaluation and interactive inference: https://github.com/PolymathicAI/AI‑Mindmeld
Authors:Esmail Gumaan
Abstract:
When an LLM agent's move fails, the harness writes the move and the failure message into the transcript and asks again. In tool calling, this record makes small models more likely to repeat the failed action. We test whether the same holds in games (Sokoban, Connect Four, Othello), where a move can fail by being illegal or by being legal but losing, both verified exactly. On fixed probe items built by perturbing a correct move, the record backfires: pooled over 8 instruction‑tuned models (135M‑1.7B) it makes the failed move more likely (corrective gain G = ‑1.38 nats per token), through the move's surface form, for illegal moves but not for losing ones, and not for every model. An exploratory analysis shows what sets the sign: after the record, the failed move's log‑probability barely depends on how likely it was before (slope 0.18). Moves the model would not have played become more likely, and greedy decoding starts to produce them; moves it was about to play become less likely, but greedy decoding still repeats 38% of them. A description of the failure that omits the move string keeps the model's preference instead. Along the OLMo 2 1B post‑training ladder the inversion is present in the base model and largely removed at the DPO stage. RAGEN's default models invert in RAGEN's own Sokoban format before any RL. A "do not repeat" instruction does nothing, a clean retry is the worst harness, and a decoder ban stops exact repeats but not illegal play. All measurements ran in float32 on a free Colab GPU, which first reproduced the original paper's number.
Authors:Vaidehi Hegde
Abstract:
When a language model is deployed, no reference solution exists, so the reasoning trace it writes is the only evidence for judging its answer. The reference‑free substitutes each fail differently: self‑assessment degrades the answer it was meant to protect, sampling agreement measures stability rather than correctness, and perturbation studies read the expected answer off a human‑written template, grading a model rather than a trace. We ask how far a single trace can be verified with no reference solution, no judge model and no weight update, and find, for a frozen LLaMA‑2‑7B‑chat on GSM8K, that detection is not what limits such a pipeline. Checking a trace against itself reaches an AUROC of 0.741 and stops there for a structural reason: two thirds of wrong traces state no false equation, having executed a mistaken plan correctly, and on wrong‑plan failures the detector flags a wrong trace less often than a right one. A candidate‑derived metamorphic probe exposes many of these failures: we abstract the trace into a program over the question's quantities, perturb one quantity, execute the program, and ask the model the rewritten question once, so every trace supplies its own test. Valid perturbations exist for 937 of 1,319 test questions (71.0%); among the 847 that also have a parseable first trace, supported traces are correct 60.6% of the time against a 31.6% base rate, although the probe also refutes 45.5% of correct traces. No tested policy yields a statistically significant improvement in accuracy. A flag earns its retry only when detector precision exceeds D/(R+D), where R and D are the rates at which a retry repairs a wrong trace and damages a correct one; our repair rate is so low that, at the same retry budget, even a perfect detector would recover an estimated 66 of 1,319 questions. Repair, not detection, is the open problem.
Authors:Jona Joy
Abstract:
Large language models (LLMs) are increasingly integrated into embodied robotic planning, yet standard benchmarks primarily evaluate end‑to‑end task completion in nominal environments, obscuring failure origins across the execution stack. We present Alpha, a systematic fault‑injection and robustness‑evaluation framework for LLM‑driven manipulation in simulation. Alpha injects calibrated faults across perception (sensor lag), actuation (grip slip), kinematics (unreachable targets), and planning (planner‑output corruption), alongside concurrent conditions. To separate physical vulnerabilities from planner sensitivities, we introduce a matched methodology pairing an LLM agent (Llama‑3 via Ollama) against a deterministic Scripted Oracle planner across a balanced 150‑episode matrix under identical seeds, geometries, and safety criteria. Across the matrix, the Oracle achieves 64.0% (48/75) success while the LLM achieves 44.0% (33/75) (paired McNemar p = 3.01 x 10^‑4). Crucially, excluding planner‑output corruption, performance across all physical and kinematic conditions is identical: 55.0% (33/60) for both planners with zero discordant pairs (McNemar p = 1.00). This demonstrates that physical actuator slip, kinematic reach boundaries, and perceptual staleness affect deterministic controllers and LLM planners equivalently. The performance gap is driven entirely by planner‑output corruption (100% Oracle vs. 0% LLM), isolating the prompt‑to‑JSON serialization and validation interface rather than a cognitive reasoning deficit. We also audit 169 historical pilot attempts for transparency.
Authors:Jeongbin Sohn, Jina Lee, Max Yen, Dowan Gwon, Jeahn Han, Floyd Hepburn-Dickins, Brian Coltin, Ryan Soussan, Pyojin Kim
Abstract:
We present SIGNN++, a learning‑based star identification (SID) framework for celestial navigation in GNSS‑denied nighttime environments. Although researchers have developed learning‑based SID methods to improve robustness to observation noise, prior evaluations have focused mainly on synthetic or non‑terrestrial environments, leaving their reliability under real terrestrial conditions unclear. To address these limitations, SIGNN++ extends SIGNN, a graph neural network‑based SID method, with three components: a more realistic noise model for training data, Fourier Transform (FT) of angular‑distance edge features, and RANSAC‑based Perspective Star Alignment for identification and attitude refinement. The results show that SIGNN++ is more robust to terrestrial observation disturbances and achieves higher star identification accuracy and solved rates than the original SIGNN and publicly available SID baselines, while maintaining comparable attitude accuracy and practical computation time for continuous operation. The implementation code of SIGNN++ and datasets will be available at https://signnpp.github.io/signnpp/.
Authors:Yatin Kumar, Shivani Saini, Ramesh Kumar Vats
Abstract:
Scientific machine learning provides a promising framework for solving complex problems in science and engineering. Notably, physics informed neural networks (PINNs) have shown considerable potential for solving differential equations and modelling engineering systems. Solving time fractional Navier Stokes equations (TFNSEs) on complex domains is challenging due to temporal nonlocality, nonlinear flow dynamics, geometric complexity, and incompressibility constraints. To address these challenges, we propose a geometry gated, divergence‑free Gaussian radial basis function Kolmogorov Arnold network (GeoDiv KAN). The proposed architecture uses geometry dependent feature gating and a streamfunction formulation to preserve incompressibility by construction. GeoDiv KAN is evaluated for the time fractional incompressible Navier‑Stokes equations on square, circular, and star‑shaped domains. Numerical experiments show that GeoDiv KAN achieves higher predictive accuracy than PINN, PIKAN, RBF KAN, and DF RBFKAN across the considered benchmark problems. To establish practical relevance, GeoDiv KAN is applied to two data driven CFD cases: unsteady flow past a circular cylinder using the \hrefhttps://doi.org/10.5281/zenodo.5039610Zenodo dataset and turbulent RANS flow through a complex U bend using the \hrefhttps://huggingface.co/datasets/JDecke/ubend‑cfdU‑bend dataset.
Authors:Vineeth Venugopal, Elsa Olivetti, Jonathan Welch
Abstract:
We introduce FormBench, a platform for testing the ability of models to retrieve precise chemical information from technical documents. FormBench contains 55,352 queries and two retrieval corpus variants each with ~1M passages: FormBench‑Random (C0), with randomly sampled distractors, and FormBench‑Structured (C1), with structurally proximate distractors. Each query is paired with an anchor passage, a hard negative, and a soft negative ‑ derived from a materials ontology through entity overlap scoring, providing a deeper graded signal on a model's discriminatory power. We evaluate 10 model configurations spanning commercial API encoders (OpenAI, Voyage), open‑source encoders (Nomic, GTE‑large, Qwen3‑Embed), and their task‑adaptive fine‑tuned (TAPT) variants. We find: (1) commercial APIs score very poorly across all configurations (OpenAI‑3‑large: 0.009 on C1, 0.213 on C0); (2) TAPT on FormBench triplets lifts Qwen3‑Embed to 0.4085 on C1, 0.4835 on C0; (3) BM25 remains competitive (0.375 on C1, 0.467 on C0), exceeded only by TAPT‑Qwen3; (4) corpus size has a significant impact on retrieval behavior, as demonstrated through a corpus‑size ablation; (5) the API failure traces to within‑domain geometric collapse; (6) APIs outperform even the TAPT variants on classification and clustering tasks. FormBench is released in BEIR‑compatible format at huggingface.co/datasets/vinven7/FormBench under CC‑BY‑4.0.
Authors:Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang
Abstract:
Model fusion aims to integrate the capabilities of multi‑expert models into a single model. Existing evaluations offer limited coverage of reasoning with long chains of thought and tool use, while strong average performance can mask substantial losses in individual expert capabilities. We introduce FusionEval, a benchmark that provides controlled expert pools, integrates fusion methods with and without additional training, and offers a unified evaluation suite. The pools contain domain experts independently trained from Qwen3 and Llama base models across math, code, science, instruction following, and tool use. The suite evaluates capability retention and computational cost on eight public benchmarks under comparable data access rules, reporting normalized average and worst performance, runtime, and peak GPU memory usage. Extensive experiments show that training‑free methods preserve reasoning performance but substantially degrade tool use at both model scales. In contrast, MOPD preserves tool use and achieves a more balanced capability profile, but requires additional data preparation, inference, and training. Jointly retaining reasoning, instruction following, and tool use at low computational cost remains a challenge. We hope FusionEval supports future research on efficient model fusion that jointly preserves expert capabilities. Our code is available at https://github.com/Baicaihaochi/FusionEval_Benchmark.
Authors:Krishanu Adhikary, Prashant Devadiga
Abstract:
Many Indian banks serve customers through FAQ pages and scripted chatbots. Assistants based on language models could respond faster and carry out requests such as blocking a card, but only if they give correct information and follow the bank's rules. These rules prescribe a procedure, a set of required steps, such as obtaining a closure quote before closing a deposit. Adapting models to this domain requires measuring how well they follow the procedure, before and after training. Benchmarks usually judge only the records left at the end of a conversation. Many required steps, however, leave the records unchanged, so an assistant that skips them can still be judged successful. We present TauIndianBankBench, an action‑oriented benchmark of 1,000 Indian retail‑banking tasks where an assistant helps a simulated customer. Besides the final records, it checks that the assistant took every required step and made no unrequested change. For every model, 6.5‑12.0% of sessions are judged successful when only the final records are checked, although the assistant skipped a required step or made an unrequested change. Scoring both procedure and outcome shows more accurately how well language models perform in Indian retail banking.
Authors:Pranav Varshney
Abstract:
Every interaction between AI agents begins with a handshake: the initiating agent must locate its counterpart, learn what it can do, and establish a session before the first useful message can be sent. Surveys of agent communication protocols name handshake overhead as an efficiency criterion, yet none report measurements. This paper presents the first empirical, phase‑decomposed benchmark of the handshake in three major agent protocols: MCP (in both its 2025 initialize and 2026 discover eras), A2A, and ACP. Using each protocol's official SDK, we instrument every request for round trips, per‑phase latency, wire bytes, and token cost across cold‑start, warm‑repeat, capability‑mismatch, and degraded‑metadata scenarios (N=50 per cell, on loopback and under an emulated 50 ms RTT). Round‑trip count dominates handshake cost. A legacy MCP handshake takes three round trips and 230 ms at 50 ms RTT; A2A needs one round trip and 61.0 ms. Round trips are not the whole bill: when an LLM reads the counterpart's metadata, handshake cost scales with the counterpart's capability inventory at 24‑107 tokens per capability, so a 200‑capability counterpart spends 11% of a 200k‑token context window before the agent reads its own task. MCP's own 2026 revision cuts its handshake from three round trips to two and, with era pinning, to zero on warm reconnects, protocol evolution that corroborates the cost we measure. We release the full harness, raw data, and analysis at https://github.com/pvarshh/handshake‑benchmark.
Authors:Zhennan Wan, Jianfei Chen
Abstract:
Reinforcement learning (RL) is an effective way to strengthen the long‑context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable. Human annotation of such data is expensive, naturally occurring data are too scarce to sustain RL, and existing synthesis methods depend on external corpora or stronger teacher models, which become harder to obtain as the trained models get stronger. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short‑context tasks. Therefore, we propose LongZero, a method that lets an LLM improve its own long‑context reasoning without any external data. Instead of writing questions and answers over existing documents, LongZero first has the model synthesize the question and answer with its supporting evidence in a compact structured form, and only then separately expands the evidence into a long multi‑document context. LongZero combines these self‑written multi‑hop reasoning tasks with self‑written multi‑round retrieval tasks and trains the model with GRPO. We evaluate LongZero on two LLMs across five long‑context reasoning benchmarks and compare it with RL on data synthesized from external corpora. Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long‑context reasoning of LLMs.
Authors:Zhanshuo Cao, Jiakai Tang, Yuqi Zhou, Sunhao Dai, Jun Xu
Abstract:
Existing sequential recommenders usually allocate a fixed computational budget to every item in a user's interaction history, regardless of the item's contribution to the final prediction. We challenge this de facto design with two diagnostic findings. (1) Non‑uniform computational need: items within the same sequence vary drastically in how much computation they require. Most converge after shallow processing while a critical minority only stabilizes at full depth. (2) Suboptimality of fixed allocation: given oracle access to the target item, a non‑uniform depth assignment consistently outranks the fixed one, indicating that existing uniform computation both wastes capacity on already‑converged items and sacrifices overall recommendation quality. These observations motivate a shift from fixed‑depth computation to item‑level adaptive reasoning. To this end, we propose LoopRec. LoopRec recursively applies a single weight‑shared block and introduces a lightweight gate at each recursion boundary that makes a binary exit‑or‑continue decision for every item independently. Items whose representations have sufficiently matured exit early, while those requiring deeper reasoning continue to subsequent recursions. This design converts the fixed global depth into a learned, per‑item property without auxiliary depth labels, and is readily adaptable to diverse backbone architectures. Extensive experiments on public benchmarks with four backbone families demonstrate that LoopRec consistently outperforms matched fixed‑depth counterparts. Further analysis reveals that LoopRec's learned allocation exhibits interpretable patterns aligned with our post‑hoc oracle analysis: lower‑frequency items and more recent interactions naturally receive deeper computation. Our code is available at https://github.com/Elpmis117/LoopRec.
Authors:Yanqing Liu, Letian Zhang, Chengzhi Liu, Zijun Wang, Jieru Mei, Hongliang Fei, Yuyin Zhou, Cihang Xie
Abstract:
World models turn visual representations into predictions and actions. We argue that their visual foundation should combine physical‑state information, native language alignment, and pixel reconstruction, so that a single representation can support prediction, instruction, and inspection. However, existing encoders typically specialize in only one or two of these capabilities; we therefore introduce ALPS, a pretrained visual state encoder that unifies all three in a single set of patch tokens. Using only still images and text, we pretrain a ViT‑B/16 from scratch with language, reconstruction, and spatial correspondence objectives, and then refine it via cross‑view self‑distillation. In a unified evaluation across fifteen public encoders, ALPS achieves the lowest aggregate LIBERO state‑probe error among ViT‑B models (matching the 3.5x larger V‑JEPA 2‑L) while maintaining strong language alignment and accurate pixel reconstruction. Downstream, frozen ALPS tokens reach the highest mean policy success on LIBERO (85.5%) when paired with our native text tower, match or exceed leading encoders on world‑model planning across PushT, Wall, and Granular, and allow predicted future tokens to be decoded directly into pixels for visual inspection. We hope this work helps establish visual pretraining as a shared foundation for understanding, prediction, and action in world models.
Authors:Shibo Jin, Zhiqiu Xu, Yiling Zou, Shangyu Gong, Mayur Naik
Abstract:
Through a tangle of real‑world interactions, from synthetic training pipelines to data shared on the open web, frontier LLMs absorb what their predecessors and competitors create. How well does each model play its role in this ecosystem, as a creator and as a solver? We introduce Environment Duels, a dynamic agentic benchmark that evaluates both capabilities on interactive environments. Each model writes environments, each with a privileged hint, aiming for tasks that capable solvers fail without the hint and pass with it. Across nine frontier LLMs, solving and authoring prove to be distinct skills: the strongest solvers are not the strongest authors, while smaller models learn more from how others played and write the hardest environments and the most useful hints. Beyond the scores, the pairwise interactions form a microcosm of how models affect one another. Head‑to‑head outcomes are transitive and so yield a stable ranking; independent authors converge on the same designs; and in a reinforcement‑learning study with Qwen3.8‑27B, the learner's hint lift on an author's environments tracks how much training on those environments improves it on held‑out reasoning and agentic benchmarks, whether the environments are its own or another model's. Each of these patterns has a counterpart in how models shape one another outside the duel. By staging these interactions under controlled conditions, Environment Duels offers a window onto how LLMs teach, challenge and learn from one another. Code is available at https://github.com/LLM‑Duels/EnvDuels.
Authors:Yanzhe Chen, Ziyi Yang, Jifeng Zhu, Qiming Huang, Ruihe An, Peiyao Xu, Hesen Yang, Runda Liu, Chang Gong, Zhijun Cao, Zechen Bai, Wenzheng Zeng, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, Kevin Qinghong Lin, Mike Zheng Shou
Abstract:
Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer‑use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal‑directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer‑use to robot‑use? We introduce PAPAV, a unified capability‑centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
Authors:Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi, Zabir Al Nazi, Mahbub E Sobhani, Shubhashis Roy Dipta
Abstract:
Many NP‑complete (NPC) problems have a polynomial‑time (P) twin that differs in a single structural detail, such as 3‑ versus 2‑colouring a graph. A model that reasons rather than recalls should notice which side of this boundary a problem falls on, yet existing benchmarks either fix the wording of each task or ignore its complexity class, so they cannot separate the two. We introduce P2NPCBench, a benchmark of 12 problem families in which every NPC problem is paired with a P twin posed over the same surface. To remove the textbook cues that enable recall, we build it with AgentGAN, an adversarial multi‑agent framework in which a generator rewrites each statement until a panel of frozen code models fails, while an executable validity gate keeps every rewrite answer‑ and class‑preserving. The result is 2,114 hardened statements with hand‑verified reference human solvers. Evaluating 12 proprietary and open‑source models, we find that every model scores lower on NPC statements than on their P twins (35.4% vs. 63.8% pass@1), and that 95.1% of model‑family pairs fail more often on the NPC side. A tenfold time limit raises NPC accuracy only from 28.5% to 31.9%, so failures stem from wrong or crashing programs rather than slow ones. Hardening also narrows the class gap from 24.8 to 14.6 points, showing that part of the apparent gap reflects familiarity with textbook phrasing rather than complexity class.
Authors:Adrien Deschenaux, Nadezhda Ilieva, Justin Deschenaux, Manuel Madeira, Dorina Thanou, Aurélien Lucchi, Antonio Orvieto, Caglar Gulcehre, Pascal Frossard
Abstract:
Recent Diffusion Language Models (DLMs) use Self‑Conditioning (SC), or Loopholing, to improve their performance. While SC is traditionally seen as an auxiliary heuristic, its empirical success raises the question: can a continuous generative model be based purely on Self‑Conditioning? We answer affirmatively with LAW (``Loopholing All the Way''), which propagates the Expected Embedding under the denoiser's prediction across steps, decoding the tokens once, at the end. On Sudoku, LAW outperforms previous Discrete and Continuous Diffusion Models, including Loopholing Discrete Diffusion. On Maze, LAW is the most accurate from 2 to 64 NFE. After training on TinyGSM and sampling at temperature 1 with 1024 NFE, LAW solves 42.2% of GSM8K problems against 34.5% for uniform Loopholing Discrete Diffusion. Additionally, fine‑tuning a pre‑trained Uniform Diffusion Model (Duo) with LAW increases its accuracy from 17.2% to 48.1%. Finally, on OpenWebText, LAW pushes the Pareto frontier and approaches Autoregressive Models (ARMs) with 64 NFE. Find our code at https://github.com/ilievanadezhda/law.
Authors:Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh, Thanh Danh Le, Khang Nhat Hoang Vo, Huynh Dai Nhan Tran
Abstract:
Vietnamese pre‑trained encoders have improved rapidly, but the strongest ones are large, multilingual‑vocabulary models or depend on external word segmentation, which makes them costly to deploy. We present GoiBERT, a family of two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. GoiBERT learns from two complementary teachers. A same‑tokenizer Vietnamese encoder first teaches general‑purpose representations. A much larger teacher then provides task‑level guidance through soft labels that are generated without ever labeling its own training data. Across eight benchmarks covering text classification, aspect‑based sentiment analysis, natural language inference, and named entity recognition, GoiBERT‑large achieves the highest average score among non‑teacher models. It does so with fewer than half the parameters of the strongest compact baseline and twice its throughput. GoiBERT‑base retains 99% of that baseline's performance at a quarter of its size. Our analysis shows that compression is nearly lossless for sentiment and classification tasks but remains costly for inference and entity recognition, which points to clear directions for future work. Our code and models are available at: https://github.com/Nhantran65/GoiBERT.git
Authors:Osher Azulay, Cheng-Lin Hsieh, Stella X. Yu
Abstract:
Humanoid fall recovery requires selecting a motion that suits both the surrounding scene and the task. The same fallen pose can admit several ways to stand, with different routes, support transitions, and final headings. Yet reliable recovery alone does not provide control over which of these alternatives the robot executes. Steering this choice requires anticipating the physical consequences of a requested movement. We present RISE, an approach to perceptive humanoid recovery that lets task objectives steer how the robot returns to standing at inference time. It builds on two insights: diverse recoveries can be composed from shared movements, and anticipating how these movements unfold allows recovery to be steered toward a task objective. We learn a recovery vocabulary that connects intended movements to motor actions through sensory feedback. A visual diffusion planner learns from executions of this vocabulary to predict motion intent as future body states. Task costs guide this prediction toward a desired heading, rise speed, or path. The planner then generates motion latents conditioned on the steered intent, allowing the requested movement to shape its execution. Simulation experiments demonstrate diverse recoveries, terrain adaptation, and test‑time steering without retraining, complemented by real‑world demonstrations on a Unitree G1. Project page: https://osheraz.github.io/RISE/
Authors:May Espinola, Qifan Zhang, Yu Xiang
Abstract:
Recent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general‑purpose AI models for robot manipulation without task‑specific training. However, their manipulation capabilities have not been systematically evaluated on standardized robotics benchmarks. In this report, we present a unified closed‑loop manipulation framework for evaluating frontier AI models on the LIBERO‑10 benchmark. Our framework enables models to control a robot directly through visual observations, robot proprioception, and structured tool calls, primarily by specifying target end‑effector poses and gripper commands, without task‑specific fine‑tuning or pretrained manipulation skills. We evaluate six frontier AI models, including GPT‑6 Astra, GPT‑6.1 Sol, GPT‑6 Luna, Claude Opus 5, Claude Opus 5.5, and Gemini‑4 Argon, across three reasoning effort levels, resulting in 18 configurations and 1,800 evaluation episodes. Our results show that GPT‑6 Astra achieves the highest success rate of 44%, followed by Gemini‑4 Argon at 43%. Increasing reasoning effort generally improves manipulation performance, but the benefits vary across models and come at additional inference cost. We also observe substantial differences across tasks, with models performing relatively well on object placement tasks but struggling with precise spatial arrangements and complex multi‑step interactions. These findings demonstrate both the potential and limitations of frontier AI models for robot manipulation, highlighting the challenges of translating reasoning capabilities into reliable physical actions. We release our evaluation framework and experimental results to support reproducible benchmarking and future research on reasoning‑based robot manipulation.
Authors:Lorenz Brehme, Benjamin Steenhoek, Ruth Breu
Abstract:
Coding agents are typically evaluated using one‑shot, well‑formulated prompts. Benchmarks such as SWE‑bench use existing GitHub issues as input for a coding agent and evaluate whether the agent successfully solves the task. However, in real‑world use cases, prompts are often not as well‑formulated or structured as existing GitHub issues. Additionally, a coding agent session is not limited to a single prompt; it typically involves a multi‑turn interaction in which the user iteratively adds new tasks and requirements that the agent must fulfill. To benchmark these abilities, we developed SWE‑DIVE, a dynamic, interactive validation and evaluation framework for benchmarking coding agents in a multi‑turn environment. To achieve this, we introduce a user simulator that can simulate different behavioral patterns and communicate with the coding agent. We also introduce a conversation validator that evaluates the user simulator's outputs to determine whether they fulfill the initial requirements and guide the agent toward successfully solving the task. The entire framework is embedded in a Harbor environment, which orchestrates the coding agent, user simulator, and conversation validator, and executes the test environment to determine whether the agent successfully solves the task. Our results and source code are available in our GitHub repository (https://github.com/lorenzbrehme/swe‑dive_usersim).
Authors:Zhaohui Yang, Quanbao Lin, Chengqi Shi, Zhenglang Weng, Jicheng Lin, Boyu Chen, Tengjie Zhu, Guanzhu Ren, Jiahao Xu, Yang Feng, Xiaokang Yang, Yao Mu
Abstract:
Humanoid loco‑manipulation requires an effective interface between high‑level task policies and whole‑body control. Existing interfaces typically rely on a single action representation for both task intent and whole‑body coordination. This may burden the high‑level policy or leave one aspect underspecified. We present SP‑HLM, a framework built on a sparse‑pose‑driven interface. The interface combines five task‑critical body poses with a learned latent motion prior to represent spatial intent and whole‑body coordination in complementary forms. SP‑VLA, a vision‑language‑action model, first predicts task‑relevant sparse poses and finger joint positions from multimodal observations and language instructions using asynchronous action‑chunk inference. Given these sparse targets, SP‑MGM, a motion generation model trained through two‑stage distillation, reconstructs coordinated full‑body motion in real time. A pretrained motion encoder maps this motion to a compact latent representation. SP‑GMT, a general motion tracker, then integrates the explicit sparse targets with this implicit motion prior for whole‑body control. Experiments in simulation and on a real humanoid robot demonstrate improved motion tracking, whole‑body coordination, and loco‑manipulation success over existing approaches. Additional results and videos are available on our \hrefhttps://sp‑hlm.github.io/\textcolorblueproject page.
Authors:Yicheng Xu, Zhikai Xu, Ziyuan Chen, Yuhang Dong, Tao Zhang, Zhucun Xue, Yong Liu, Jiangning Zhang
Abstract:
Robots must act under physical and social constraints while adapting to changes in their environment and human needs. Existing embodied‑agent systems often focus on specified goals or abstract execution into symbolic actions, limiting evaluation in an evolving world. We introduce TARSphere, a robot‑oriented full‑stack framework for physically and socially grounded long‑horizon embodied agents. TARSphere‑Infra runs robot policies in a physics simulator with embodied NPCs and constrained dialogue. An event‑driven finite‑state machine (FSM) uses completed actions and delivered information to update task states and evaluate outcomes. TARSphere‑Harness, its embodied brain harness, lets a frozen multimodal large language model (MLLM) act through multi‑level executable skills, using observations, hierarchical memory, and execution feedback to revise its decisions. On this foundation, agent‑in‑the‑loop curation aligns task semantics, causal dependencies, and scene geometry to construct TARSphere‑Bench: 360 episodes across 30 indoor scenes. Five paired conditions assess perceptual robustness, disruption adaptation, task update, human coordination, and intent understanding for navigate‑and‑report and fetch‑and‑deliver tasks. Our evaluation of 8 MLLMs shows that general‑purpose models can outperform embodied models, while task updates and human interaction remain difficult. Frequent collisions and falls further motivate evaluating execution safety alongside task success, even with a shared action interface and decision budget.
Authors:Chunlei Li, Siyuan Yu, Yang Gao, Shuai Li, Peng Yu, Aimin Hao, He Wang
Abstract:
Body deformation caused by muscle contraction plays a vital role in the visual realism of character animation, yet in most 3D animation pipelines artists have to manually specify muscle activations, which is time‑consuming and gives no guarantee of physical realism. To address this, we propose a new method in which a reinforcement learning (RL) controller discovers per‑muscle activations that track reference motions without manual tuning. Existing RL muscle controllers stop at 1D muscles; we go one step further, keeping the 1D model inside the RL control loop as a fast surrogate, and using a 3D volumetric simulation to produce the final deformation. Because both stages share the very same constitutive law, the muscle deformation is produced by the same activations that generated the motion. We further add a measured electromyography (EMG) prior that steers the redundant activation solution toward real human muscle recruitment. To achieve stable, efficient, and realistic simulation, we propose a novel geometric multigrid for Extended Position‑Based Dynamics (XPBD) that is built on a coarse tetrahedral cage with barycentric‑coordinate prolongation and closes a V‑cycle on tetrahedral meshes. Within this GPU‑parallelized solver, we inject the same Hill‑type force‑length law as an energy‑based fiber constraint that turns per‑fiber activation into realistic muscle contraction and bulging. We demonstrate volumetric muscle simulation on a 194‑muscle human body with a diverse set of motions (walk, kick, swim, overhead‑squat, etc.), simulating the full‑body model at 18 ms per frame on a single RTX 4090. Our work takes a step toward character animation that is not only visually convincing but also biomechanically grounded. The website with code and videos is at https://chunleili.github.io/project‑page‑rlmuscle/.
Authors:Chenxu Zhou, Zhiyuan Yu, Yudong Jin, Yinji Shentu, Yu Zhang, Jiaming Sun, Xiaowei Zhou
Abstract:
This paper addresses the problem of creating immersive volumetric videos from sparse and low‑resolution capture resources. Conventional dense multi‑view reconstruction methods can produce high‑quality volumetric videos, but their reliance on costly capture setups makes them impractical in real‑world scenarios. Recent approaches use generative models to relax capture requirements, but still suffer from limited resolution and 4D inconsistency. To overcome these constraints, we propose RealMe, a unified framework built around a high‑resolution video diffusion model that enhances limited captures into high‑resolution (2K), 4D‑consistent videos for volumetric video reconstruction. Specifically, RealMe constructs auxiliary data from capture videos and encodes them into latents along with layout‑aware positional embeddings. The video diffusion model adapts factorized spatio‑temporal attention to the multi‑view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation. This adaptation helps the model capture temporal dynamics and multi‑view relationships, leading to more 4D‑consistent generation. To reduce the data gap between generated videos and real captures, we apply a latent frequency loss that provides frequency‑domain supervision in latent space. Extensive experiments on the public DNA‑Rendering benchmark and our self‑captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state‑of‑the‑art methods in visual fidelity and 4D consistency.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Alarm root‑cause analysis is studied on data that cannot be shared. Operator logs are confidential, so what the field publishes is a causal graph between alarm types rather than per‑incident causes, a fixed scenario set, gated, or never released. Two recent benchmarks carry per‑incident root causes at scale; neither states a download location or a licence. We give a generator rather than a dataset. It runs on real backbone topologies from the Internet Topology Zoo and records the exact root cause and the propagation path of every alarm ‑‑ a by‑product of generation, not a labelling pass. Every emitted field binds to a term in NORIA‑O, RFC 8632, RFC 8345 or TMF642, and an unknown term fails the run. We ship fixed configurations, splits, a scorer and nine baselines. A one‑line rule ‑‑ rank the device that raised the earliest critical alarm ‑‑ takes top‑1 at 0.650 against 0.575 for our own temporal method, and wins 13 of 15 runs over three backbones and five seeds. An LLM given the same view reaches 0.600, the same model allowed to query the graph 0.625, and the rule wins on real operator alarms too, at 0.773: a property of alarm streams, not of our generator. One fault class separates the table: when a site loses power its PoP reports nothing, so every method that ranks alarmed devices scores 0.000 or near it, while the two that walk the topology to a silent candidate take 1.000 and 0.833.
Authors:Chengxi Li, Hanlin Wu, Zhicheng Cai, Huan-ang Gao, Wei-Ying Ma, Jingjing Liu, Hao Zhou
Abstract:
Reinforcement learning (RL) based on PPO‑style clipping has become an effective approach to improving the reasoning capabilities of large language models (LLMs). However, standard token‑level clipping measures policy change only through the conditional probability of the current token, while the prefix conditioning context is itself generated by the policy. This mismatch becomes especially consequential in long‑response and off‑policy training. We reformulate autoregressive generation as a finite‑horizon Markov decision process and derive its performance‑difference identity under behavior‑policy sampling. The resulting change of measure identifies the cumulative likelihood ratio of the generated prefix as the exact importance weight for each stepwise policy‑improvement term. Starting from the monotonic‑improvement principle of trust‑region policy optimization, we then construct a prefix‑level trust region using the exact reverse‑KL coordinate r‑1‑\log r, whose behavior‑policy expectation equals the corresponding prefix‑distribution KL divergence. This yields EPIC, a PPO‑style sequence‑clipping method that directly regulates prefix‑level policy drift without relying on a local quadratic approximation or expert‑routing replay. Experiments on off‑policy reinforcement learning with a sparse mixture‑of‑experts language model show that EPIC consistently outperforms standard and sequence‑level clipping baselines across mathematical‑reasoning benchmarks, while remaining competitive with routing‑replay‑based alternatives. Code is available at https://github.com/LiChengxi666/EPIC.
Authors:Ruopeng Gao, Yukun Huang, Jiarui Hu, Shuai Wang, Wenbo Hu, Ying Shan, Limin Wang
Abstract:
Autoregressive video diffusion models generate videos chunk by chunk, yet producing plausible continuations does not guarantee that they capture how scenes evolve. Under teacher forcing, strong temporal redundancy may allow local visual continuity to serve as a shortcut for current‑chunk denoising, reducing the pressure to model visual evolution. We introduce Future Chunk Prediction (FCP), an auxiliary objective jointly optimized with teacher‑forced denoising that predicts temporally separated chunks without access to the intervening visual context. Drawing supervision directly from training videos, FCP requires no additional pretrained visual encoder and leaves autoregressive inference unchanged. Our method achieves superior physical plausibility over standard teacher‑forced autoregressive training with a lower overall training compute budget, highlighting the effectiveness and efficiency of future‑chunk supervision for modeling visual evolution. Intriguingly, our visualizations suggest that FCP fosters a more structured representation space, which may be valuable for building autoregressive world models.
Authors:Davide Carnemolla, Dario Catalano, Mario Di Raimondo
Abstract:
Facial recognition has become central to modern authentication systems, security frameworks, and personalized services. However, its widespread adoption is hindered by severe privacy risks, particularly surrounding unauthorized access to sensitive biometric data. To address these challenges, this paper presents GhostFaceMPC, the first end‑to‑end, privacy‑preserving face recognition system to evaluate a deep convolutional neural network extractor and matching under secure multi‑party computation. GhostFaceMPC enables secure processing of facial data across all stages of a face recognition process, from feature extraction and storage to matching, without exposing raw images or biometric embeddings at any stage. We introduce a multi‑party computation‑friendly quantization of the GhostFaceNet model and circuit optimizations that make face recognition under multi‑party computation practical. The code is available at https://github.com/crypto‑unict/GhostFaceMPC.
Authors:My Chiffon Nguyen, Tianyi Hu, Vinod Anbalagan, Henry Gagnier, Pouya Sadeghi, Farah Abdou, Tidor-Vlad Pricope, Diego MMS Oliveira, Jocelyn K D'Arcy, Madeline Smith, Oluwaseun A Ajayi, Yusser Al Ghussin, Zena Al-Khalili, Micol Altomare, Shayan Amir, Ahanaf Aziz, Bryan Chen, Ruchira Dhar, Abdellah El Mekki, Rawan Mohammed Elframawy, Akhil Eppa, Silvia Fernandez, Alex Gimei, Arya Hariharan, Christian Hoang, RamMohanRao Kadiyala, Diana Kozachek, Peerat Limkonchotiwat, Ankita Maity, Di Mo, Ayush Munot, Aylin Naebzadeh, Anastasiia Nikitina, Dina Pisarevskaya, Divas Poudel, Jebish Purbey, Md Rabius Sany Apu, Ahmad Mufli Ramadhan, Ralitza Nikolova Rizova, Diogo M. Serrano-Miranda, Vivek Gopalkrishna Silimkhan, Luis Fernando Solis Navarro, Kaiser Sun, Arya Suneesh, Manaswitha Sunkara, Tejaswi Sunkara, Jonathan E Swindell, Hanna Yukhymenko, Jingyi Zheng, Esther Adenuga, Ilia Afanasev, Wafa Aissa, Lisa Alazraki, Lauren C Altomare, Priscilla Amondi Amuok, Emile Anand, Gabriel Assis, José Matías Avalos, Silvia Lorena Avila, Zakaria Baannou, Niyati Bafna, Yash Bagla, Sarah Bahrami, Bontu F Balcha, Vanya Bannihatti Kumar, Irene Baucells, Abdumutalib Begmuratov, Naman Bhatia, Souvik Bhattacharyya, Isabel Blanco Ferreiro, Felipe Bochini, Sayuru Bopitiya, Angana Borah, Tulika Bose, Olena Bugaiova, Juliana Carvalho, Nancy F Chen, Stephy Chen, Rochelle Choenni, Jayanth Krishna Chundru, Leandro A Cossio, Nebojša Damnjanović, Ona de Gibert, Marco De Santis, Pedro Delgado, Kaustubh Dholé, Adnan El Assadi, Wesam El-Sayed, Nadine M Elframawy, Oumayma Essarhi, Naome A Etori, Reuben Chagas Fernandes, Saanidhi Pravin Gade, Javier García Gilabert, Álvaro González-Escapa, Bin Hu, Kaiyu Huang, Abderrahmane Issam, Danka Jokić, Ulugbek Khalilov, Dominik Krzemiński, Roy Ka-Wei Lee, Haichang Li, Chen Cecilia Liu, Zhengyuan Liu, Cedric Lothritz, Houman Mehrafarin, Antonio Miranda-Escalada, Fengran Mo, Mehrnaz Mofakhami, Wafaa Mohammed, Felipe Moreno-Vera, Prahitha Movva, Jannatul Nayem, Van Ngo, Joy Olusanya, Marc Pàmies, Avinash Kuney Pandey, Yurii Paniv, Fred Philippy, Vladislav Poritski, Dikshya Poudel, Joanna Radoła, Sara Rajaee, Swati Rajwal, Prerana Rane, Muhammad Ravi Shulthan Habibi, Lucas Resck, Shubhashis Roy Dipta, Ananya Sahu, Eduardo Sánchez, Borja Sánchez Clemente, Aleix Sant, Nobin Sarwar, Istiak Shihab, Santhoshini Pranathi Singaraju, Jane Smith, David Stap, Emily Z Su, Ananth K Suresh, Sajag Swami, Ruiyang Tan, Nazia Tasnim, Khalid Tounoussi, Patteera Triamamornwooth, Muhammad Saad Umar, Shankar Venkitachalam, Oksana Volchek, Lingbo Wang, Debela Desalegn Yadeta, Rushikesh Zawar, Jinghan Zhang, Mike Zhang, Weihua Zheng, Marzieh Fadaee, Akhil Arora, Károly Boczka, Julia Kreutzer
Abstract:
As LLMs expand their language coverage, evaluating how well they understand and represent diverse cultures becomes increasingly important. However, many benchmarks remain English‑centric, with limited cultural coverage and a focus on factual knowledge over its application in real‑world cultural contexts. We propose CulturalRiddles, a benchmark of original, culturally grounded riddles created entirely by humans through a participatory community effort, spanning 62 cultures (45 countries, 51 languages). These riddles are deeply rooted in their respective cultures and require abductive and compositional reasoning, alongside detailed factual and social knowledge, language skills, and an understanding of culturally specific associations. We benchmark 28 LLM configurations using both automatic and large‑scale human evaluation. Even frontier models achieve at most 22% accuracy in some communities. We publicly release all artifacts alongside an open contribution process to expand the benchmark's linguistic and cultural coverage.
Authors:M. Mikail Demir
Abstract:
Given the enormous number of claims to be checked, verifiable legal AI requires scalable, cost‑effective models that answer the practical needs of law. System One decision models are built on large language models but, instead of generating text, return a calibrated probability for each option. This suits the verification of legal authority, where the question is closed: a given holding, or none of them. We test whether such models abstain when the correct authority is missing. We extend CaseHOLD with an abstention condition in which the correct holding is replaced in 1,670 of 5,314 items, and compare four decision models with six generative LLMs. When no option is correct, generative LLMs still pick one in 38‑86% of items; the most cautious decision model does so in 30%. On standard CaseHOLD, a decision model matches the best generative LLM (88.1% vs. 88.2%) at 1/55 of the cost. Accepting that LLM's answer only when a decision model agrees halves its wrong citations, and distilling a decision model's probabilities into Qwen3‑8B, without labels, yields a well‑calibrated local model. Legal AI should be judged by its wrong citations, not by accuracy alone.
Authors:Naru Suzuki, Satoshi Ikehata, Takehiko Ohkawa, Yoichi Sato, Shuhei Kurita
Abstract:
Recovering metric world‑space hand motion from egocentric video supports understanding human interactions and transferring them to embodied agents. Three challenges complicate this task: multi‑stage pipelines propagate errors from independently estimated hand and camera motion; monocular observations leave physical scale ambiguous; and occlusion and out‑of‑view hands admit multiple plausible motion trajectories. We present DewHand, a unified model for direct joint reconstruction of world‑space hand and camera motion. To avoid compounding errors from multi‑stage estimation, DewHand jointly predicts hand and camera motion in a shared world coordinate system within a single‑stage pipeline. To resolve metric‑scale ambiguity, it combines scene‑level geometric features from VGGT‑Ω with metric‑space supervision, while hand‑specific features preserve fine‑grained articulation. To handle occlusion and out‑of‑view hands, we formulate reconstruction as conditional motion generation with flow matching. Under the same architecture, this generative formulation outperforms direct regression in both accuracy and temporal smoothness with only two sampling steps. Experiments on HOT3D and ARCTIC demonstrate improved world‑space accuracy and temporal consistency over state‑of‑the‑art methods. DewHand reduces rigid‑aligned world‑space error relative to StableHand from 32.9 to 28.4 mm on HOT3D and from 76.2 to 36.2 mm on ARCTIC. It also recovers camera scale within 10% on two thirds of HOT3D clips, versus about a quarter for SLAM. Moreover, in zero‑shot evaluations on HoloAssist, H2O, and TACO, captured with different devices, DewHand achieves the lowest rigid‑aligned world‑space error (WA‑MPJPE) on each dataset.
Authors:Nithin Sivakumaran, Ali Payani, Hyunji Lee, Mohit Bansal, Elias Stengel-Eskin
Abstract:
Forming an accurate theory‑of‑mind (ToM) is key to successful multi‑agent collaboration and requires reasoning under uncertainty to capture multiple possible partner states. This is hard to do in the text representation space native to large language model (LLM) agents, which more easily describe a single partner state. To address this challenge, we introduce ToM2Vec, which learns to form abstract theories‑of‑mind in a rich latent space via a fixed budget of continuous reasoning tokens. During training, supervision for these tokens comes from the partner's actual thoughts, which the ToM tokens are trained to abstract and match. At test‑time, ToM tokens are generated by each agent and act as conditioners for that agent's thoughts and actions. Across several environments covering collaborative, competitive, and mixed‑motive settings (Overcooked‑AI, multiparty negotiation, Decrypto, Hanabi, and Collab Escape/Capture), we evaluate ToM2Vec's ability to support multi‑agent interactions and collaboration, finding consistent improvements over the base model and a variety of competitive trained and prompted baselines. We show that latent reasoning is critical, with performance dropping when reasoning is expressed in text. Moreover, we show that ToM2Vec training generalizes to partner model families not seen at train‑time, including a proprietary model (GPT‑5.4). In our analysis, we provide evidence that the latent tokens encode a ToM, rather than task‑specific information: ToM2Vec's tokens improve the agent's ability to directly answer questions about partner beliefs, desires, and next actions.
Authors:Qiuyang Mang, Wenhao Chai, Runyuan He, Dongruixuan Li, Zihan Zheng, Yujun He, Xuanyi Zhou, Hao Wang, Siddhant Ray, Bo Peng, Ruyi Ji, Shuo Yang, Haocheng Xi, Boru Chen, Xinyang David Han, Melissa Pan, Mayank Mishra, Boxi Yu, Sijia Liu, Kerem Akillioglu, Yi Pan, Zirui Wang, Mert Cemri, Diogo Antunes, Yichuan Wang, Aiwei Ivy Zhang, Zhenyang Chen, Qizheng Zhang, Kaiyuan Liu, Wenyuan Huang, Sherry Ruan, Shu Liu, Dacheng Li, Zerui Cheng, Zeyu Shen, Shang Zhou, Hanfei Chen, Yuzhong Zhang, Bohan Lyu, Qu Ao, Han Zheng, Chonghe Jiang, Jingzhuo Hu, Xiaokun Chen, Kunjun Li, Zhiqing Zhong, Haoye Tian, Tong Zhao, Yunzhong He, Hanchen Li, Ofir Press, Jiantao Jiao, Kunle Olukotun, Karthik R. Narasimhan, Jingbo Shang, Koushik Sen, Matei Zaharia, Alex Dimakis, Ion Stoica, Joseph E. Gonzalez, Alvin Cheung
Abstract:
Can LLM agents make contributions to frontier computer science R&D? We introduce FrontierCS 2, a benchmark of 48 open‑ended tasks. We take a paper and the code its authors released and delete the paper's contribution. The deleted code becomes the human reference that sets the bar. Earlier open‑ended benchmarks give each agent a problem statement and grade one program after the run. FrontierCS 2 places the agent inside a research environment. The agent inherits a working repository and may submit a patch at any point. For each patch, a judge returns a development‑suite score but keeps the evaluator, the held‑out final suite and the reference private, and grades the final submission on that suite against the reference. The tasks span 13 domains, from machine‑learning systems and high‑performance computing to databases, security and robotics. Beyond these 48 tasks, FrontierCS 2 keeps a private set of more than 100 tasks, to be announced in the future. We evaluate 12 agents. The best one is Claude Opus 5.5, passing 58% of its runs. No run passes on 16 tasks. A higher score is not the same as a pass: most runs that beat the reference on the combined final score still fall short on some metric or workload. On a shared capability scale for agents and tasks, the median task is roughly as difficult as what the strongest agent can handle.
Authors:Liu Cao, Botian Xu, Ruoqu Chen, Jiajun Liu, Mingzhi Pei, Zihao Wang, Zhongyu Li, Mengdi Xu
Abstract:
Real‑time whole‑body humanoid teleoperation remains challenging due to the computational latency introduced by optimization‑based retargeting pipelines and future‑frame conditioning. Existing motion tracking policies typically require dense reference targets including full joint positions, body‑part transformations, and even future frames, which tightly couples the controller to expensive upstream modules. In this work, we present HuMiT, a whole‑body teleoperation system built on a unified tracking policy that requires only a compact reference target, consisting of root height, root velocity, and the SE(3) poses of four sparse keypoints at the current frame, yet achieves competitive tracking fidelity compared to methods relying on more diverse reference states including complete joint positions, dense body‑part transformations, and even future frames. Experiments show that our policy achieves a communication‑to‑inference latency of ~20 ms measured between headset streaming and policy output and a video‑based optical flow latency of 65 ms (median) across different motion types. We validate the system on a Unitree G1 humanoid, showcasing split‑second reactive responses such as catching a tossed ball, as well as coordinated loco‑manipulation capabilities. Project page: https://xiaohu‑art.github.io/Track/.
Authors:Arshan Dabirsiaghi
Abstract:
Coding agents that run long enough must compact their context, summarizing the conversation, discarding the original, and carrying the summary forward as memory. We show that in popular open‑source coding agents this step is an exploitable trust boundary. Harnesses that flatten the typed conversation into a single plain string, marking each message with a text label such as [User]:, destroy message provenance in code, before the summarizer runs. An attacker who controls anything the agent reads can forge a user turn that survives compaction as a genuine instruction. Across five flattening harnesses (pi, opencode, Cline, Aider, OpenHands) the forged approval launders into the agent's memory 87‑100% of the time, up to a ~908,000‑token context window; across four role‑typed harnesses (Codex, Gemini CLI, Qwen Code, Continue) the identical attack succeeds 0% of the time. We follow the attack to executed consequences. A laundered standing approval drives real tool‑running agents to execute attacker‑controlled curl | sh in a sandbox, and end to end in the live pi TUI at ~50% against a 0% paired clean arm. The same channel reliably steers an agent's plan away from a file an attacker designates off‑limits, which can blind a code review to a planted backdoor. Because the provenance is destroyed deterministically in harness code, no prompt‑level defense can restore it; we measure a code‑level provenance check that catches 77% of laundering while demoting 2% of legitimate summary content. The defect also has a straightforward fix at the serializer, escaping the role labels before flattening; the role‑typed harnesses avoid it entirely by keeping messages role‑typed.
Authors:Samuel Foo Enze, Manthan Patel, Fan Yang, Linus Kramer, Pascal Roth, Marco Hutter, Cesar Cadena
Abstract:
Autonomous mobile robots operating in dense crowds and cluttered sites require 360^\circ perception to react to surrounding hazards. While panoramic LiDAR offers full‑surround coverage, learning representations within reinforcement learning (RL) is sample‑inefficient under sparse rewards, and existing 3D pretrained point‑cloud encoders are too memory‑heavy for massively parallel RL simulation. We present RangeNav, a compact (183k parameters) LiDAR range‑image encoder pretrained entirely on synthetic data. RangeNav uses anisotropic convolutions and circular padding to handle the extreme aspect ratio and azimuthal seam of range images. We pretrain the network using a navigation‑aligned objective combining range reconstruction, semantic segmentation, and bird's‑eye‑view (BEV) prediction, then freeze it for policy learning. In simulation, adding semantic and BEV supervision increases success rates over a reconstruction‑only baseline by an average of 32.03 and 23.73 percentage points (pp) in procedurally generated training and out‑of‑distribution realistic test environments, respectively. Zero‑shot real‑world deployments on a Unitree B2‑W quadruped demonstrate robust navigation through dense crowds, low‑light settings, and construction zones where limited‑field‑of‑view depth policies struggle. Code and models will be open‑sourced. Project website: https://rangenav.github.io/.
Authors:Yawei Li, Jiazheng Li, David Rügamer, Bernd Bischl, Yulan He, Mina Rezaei
Abstract:
Calibrating large language models (LLMs) in open‑ended generation is uniquely challenging because many distinct token sequences can express the same underlying meaning. Prior calibration techniques target either fixed‑set classification confidence or the binary correctness of individual strings, making them inherently ill‑suited for the dynamic, multi‑string nature of open‑ended semantic outcomes. To bridge this gap, we introduce S‑EDL, a novel evidential method that natively optimizes LLMs for semantic calibration. By eliciting evidence from the sequence likelihoods of the model itself, S‑EDL constructs a differentiable Dirichlet prior over prompt‑specific semantic classes. Optimizing the evidential loss under this prior yields a calibration‑aware training signal over variable semantic classes, completely bypassing the need for fixed multiple‑choice options. Evaluations across four open‑ended benchmarks and three models demonstrate that S‑EDL substantially reduces semantic calibration error while improving accuracy. By aligning generative likelihoods with semantic correctness, this work establishes a framework for deploying trustworthy LLMs in complex, open‑ended applications. Code is available at: https://github.com/sandylaker/s‑edl.
Authors:Taddeo D'Adamo, Simon Thivet, Catherine Pothier, Riccardo Simionato, Allan Fries, Jonathan Lemus, Costanza Bonadonna, Laure Tougne Rodet, Bertrand Kerautret
Abstract:
Quantitative analysis of environmental phenomena increasingly relies on image‑based measurements. In volcanology, the geometric characterization of volcanic plumes and clouds from RGB images remains hindered by the scarcity of dedicated segmentation solutions and partially handcrafted annotations remain widely prevalent. We present GuidedSAM‑Plume, an open‑source expert‑guided framework tailored for the interactive segmentation of volcanic plumes and clouds. It leverages the zero‑shot, spatial prompt‑based capabilities of SAM2. The framework was thoroughly validated on a challenging dataset through dedicated segmentation metrics and field‑relevant volcanological descriptors. From minimal user inputs, it is able to significantly increase performance compared to fully automatic dedicated solutions while preserving the reliability of derived volcanological parameters. The absence of retraining requirements lowers adoption barriers within the volcanological community, and the framework's demonstrated suitability for complex, diffuse plume‑shaped phenomena extends its applicability to broader environmental science domains. The re‑annotated dataset, produced under the guidance of expert volcanologists within the proposed framework, ultimately yields a more standardized release. The framework and accompanying dataset are made available at https://github.com/tad‑damo/GuidedSAM‑Plume.
Authors:Niclas Flehmig, Mary Ann Lundteigen, Shen Yin
Abstract:
Incorporating AI systems into safety systems requires monitoring to avoid silent faults and ensure safe and reliable operation. Single monitors often do not provide actionable information for human supervisors, who act as the system's adaptive controller. We present COMET, a coordinated multi‑layer runtime monitoring framework for human‑supervised AI‑enabled safety systems in which each monitor is anchored in a different fault mode. Monitor outputs are combined in a coordination layer and mapped by a communication layer to a traffic‑light display. In this work, COMET consists of three monitoring layers: signal integrity, physics consistency, and the AI system's competence envelope. The framework is evaluated on an open dataset of compressed‑air leaks, where an AI system detects leaks from acoustic emissions, with held‑out conditions simulating covariate shift, semantic shift, and sensor faults, events expected during runtime but unknown in their exact appearance. All monitors are frozen after development and tested zero‑shot, and the framework is compared against no‑monitoring and single‑monitor setups. The bare AI system fails silently and confidently under all three shift types, whereas COMET reduces the trusted‑wrong‑answer rate to zero on covariate‑shifted no‑leak data and on three of four sensor faults, at a nominal false‑alarm rate of \approx 1%. Single monitors come close in detection but cannot attribute the fault mode. COMET converts their anonymous detection flags into diagnoses. Residual silent misses under semantic shift define the boundary of passive monitoring. Runtime monitoring frameworks for AI‑enabled safety systems must therefore combine quantitative fault detection and isolation with qualitative, human‑oriented feedback.
Authors:Julian Silva, Jaliya Kodithuwakku, Navindu Sirimanna, Shaveen Herath, Wageesha N. Manamperi, Joshua P. Kulasingham
Abstract:
Driven by recent advancements in hearing assistive devices, target speaker extraction (TSE) in multi‑talker environments has drawn significant attention. To address this, hearing technologies identify attention using neural signals recorded via electroencephalography (EEG) to guide TSE, enabling neuro‑steered auditory attention decoding. However, most existing techniques rely on time‑frequency masking, which suffers from low fidelity in spectro‑temporally overlapping regions and is often restricted to narrowband audio. In contrast, we introduce NeuroCodec, which, to the best of our knowledge, is the first generative EEG‑guided TSE framework operating in the latent space of a neural audio codec. Specifically, NeuroCodec models spatial EEG dependencies via graph convolutions and fuses cross‑modal EEG and speech representations through attention mechanisms and Mamba‑based temporal modeling. We further introduce a six‑phase training curriculum that progressively incorporates latent reconstruction, multi‑scale spectral, and adversarial objectives to ensure stable, high‑fidelity speech generation. Experiments on the public Cocktail Party and KUL datasets show NeuroCodec outperforms baselines, across several perceptual metrics. A listening experiment with 20 participants validated that NeuroCodec offers greater perceptual accuracy, achieving a mean score of 82.24 compared with 49.63 for M3ANet. Additionally, the proposed method is seen to enable 44.1 kHz reconstruction with spectral content extending up to 20 kHz, further supporting the perceptual results. Finally, we extend NeuroCodec with a causal streaming formulation suited for real‑time processing, providing a basis for hearing assistive devices.
Authors:Mohammed Saqlain
Abstract:
Literature‑searching language agents are becoming a standard component of scientific assistants for the life sciences, yet there is little controlled evidence on which parts of their harness matter. We run a controlled ablation on biomedical claim verification (SciFact) with the live PubMed API as the only tool. Holding the backbone and output format fixed, we compare four harnesses: closed‑book answering, single‑shot retrieval using the claim as the query, a ReAct‑style agent that writes its own PubMed queries, and an oracle given the gold abstracts, each with two efficient frontier backbones. Naive single‑shot retrieval finds the gold paper for only 6% of claims and turns overconfident errors into abstentions without improving accuracy. Letting the model formulate its own queries triples gold recall and yields the best verdicts (62% accuracy vs. 49‑59% for the other realistic harnesses), although these differences are not significant at our sample size and agents use fewer than half of their search budget. With gold abstracts, the same models reach 79‑86% accuracy, a significant 17‑24 point gap showing that retrieval, not reasoning, is the main bottleneck. Finally, a citation audit shows that closed‑book models cite PubMed IDs that almost always exist (327/335) but are almost never relevant: a cross‑family LLM judge rates only 3 of 327 existing citations as relevant to the claim, versus 67% for citations produced by the search agent. Existence checks are therefore insufficient safeguards for scientific agents. We release code, prompts, and all cached tool calls at https://github.com/saqlain2204/real‑pmids‑wrong‑papers.
Authors:Puru Vaish, Meelad Dashti, Christoph Brune, Pascal Mettes, Tatiana Tommasi, Nicola Strisciuglio
Abstract:
Compositional zero‑shot learning (CZSL) aims to recognize novel combinations of previously observed objects and attributes. Prior work showed that large‑scale vision‑language models encode decomposable semantic structures that can be recomposed to enable generalization. The Geodesically Decomposable Embeddings (GDE) framework formalizes this by identifying directions in the tangent space at the Frechet mean of the embedding distribution, which encode primitive visual concepts. Unseen attribute‑object compositions can be synthesized through vector addition of these primitives. We revisit the assumption that compositional generalization requires explicit attribute‑object pair re‑composition. By means of new compositional scoring functions and extensive evaluations, we show that directly using the decomposed primitives suffices for recognizing unseen compositions. In that, we gain insights on how the geometry of the embedding manifold (Euclidean, spherical, or hyperbolic) and dataset structure affect compositional reasoning. Across models (CLIP, DINOv2, MERU, HyCoCLIP), we show that compositional structure consistently emerges as primitive representations, enabling inference on CZSL benchmarks (MIT‑States, VAW‑CZSL, UT‑Zappos50K, Clothing16K) and group robustness on Waterbirds and CelebA. Code: https://github.com/nis‑research/Compositionality‑with‑Primitives.
Authors:Xinze Li, Dingwen Zhang, Zhenghao Liu, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
Abstract:
Retrieval‑Augmented Generation (RAG) improves the performance of Large Language Models (LLMs) by retrieving external knowledge. Recently, some RAG methods interleave reasoning and retrieval, enabling LLMs to progressively generate sub‑queries and retrieve external knowledge. However, these methods may suffer from redundant or ineffective sub‑queries and insufficiently diverse query exploration. Moreover, due to the tightly interleaved nature of reasoning and retrieval, it remains unclear how the evolving reasoning process influences query intent. To characterize the evolution of query intent during reasoning, we introduce search momentum, which captures the accumulated retrieval tendency induced by historical changes in generated sub‑queries along the reasoning trajectory. Through momentum‑based analysis, we find that the query generation process tends to continuously evolve along existing retrieval trajectories. Based on this observation, we propose MAPS, a Macro‑Adaptive Parallel Reasoning framework for regulating search momentum and mitigating query‑intent drift. MAPS first retains the retrieved knowledge and a refined reasoning state, thereby reducing the influence of historical reasoning trajectories on subsequent sub‑query generation. Subsequently, MAPS extends the conventional sequential reasoning process into a parallel reasoning paradigm. At each iteration, MAPS identifies complementary retrieval directions, generates corresponding sub‑queries for parallel retrieval, and consolidates the retrieved evidence to update the knowledge and reasoning states. Experiments on six question‑answering benchmarks demonstrate that MAPS consistently outperforms representative RAG methods across different backbone models. All code and resources are available at https://github.com/NEUIR/MAPS.
Authors:Linfei Li, Ruining Hu, Lin Zhang, Zhong Wang, Fengyi Zhang, Ying Shen, Binqiang Wang, Xin Zhang
Abstract:
Vision‑language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high‑level semantics into part‑level physical interaction remains challenging. To address this gap, we propose UniRAP, a unified model for inferring part‑level physical affordances and mapping language instructions to actionable geometric representations. UniRAP formulates this problem as a conditional multimodal generation task, integrating visual inputs, textual instructions, and optional prompts into a shared spatiotemporal representation space through a unified interface token mechanism. To predict executable contact geometry, we introduce a Unified Affordance Decoder (UAD), which jointly performs object detection, part‑level affordance segmentation, and 4‑DoF interaction pose estimation by leveraging intermediate segmentation features. In addition, we propose a curriculum‑based transfer training strategy that progressively adapts the model from general visual parsing to interaction‑aware perception, improving data efficiency under limited textual annotations. Experiments show that UniRAP achieves state‑of‑the‑art performance on referring expression segmentation, affordance grounding, and interaction pose estimation, while maintaining strong spatiotemporal consistency in dynamic video scenarios. These results demonstrate the effectiveness of UniRAP as a unified perception framework for language‑guided physical manipulation.
Authors:Arslan Bisharat, Mujtaba Nazari, Khushboo Bhadauria, Brian Ortiz, Eric Spencer, Beatriz Santos, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad
Abstract:
Memorization inflates the benchmark scores of large language models (LLMs), since a model can reproduce answers it saw in training. Researchers test for it with consistent identifier renaming (CIR), which gives every identifier in a task a new name, and treat a lower score on the renamed task as evidence of recall. We test this method on the Temporal Logic of Actions Plus (TLA+) language, whose model checker acts as an oracle by confirming whether a renamed task preserves the exact behaviors of the original. Across 111 TLA+‑Bench tasks, renaming lowers the correctness of Claude Opus 4.8 from 28% to 18% and of GPT‑5 from 9% to 7%. Most of this decrease comes from how the test grades, which requires exact names that most descriptions never state. Grading by behavior shrinks the decrease to about 3 percentage points for Opus 4.8 and to 0 for GPT‑5. In two model families fine‑tuned on disjoint halves of the tasks, renaming removes about half of the training gain in likelihood and costs 4.9 percentage points (Qwen) and 1.3 percentage points (Gemma) on unseen tasks. A lower score after renaming therefore mixes memorization, name‑based grading, and the cost of new names.
Authors:Johannes Keller, Albrecht M. Bloße, Adrian Lindenmeyer, Thomas Neumuth, Daniel Schneider
Abstract:
Vision‑language‑action (VLA) policies need to recognize gaps in their training knowledge to request useful demonstrations or assistance in new situations, especially when failures are costly. Disagreement within an ensemble of independently trained policies can guide these decisions, but requires training and running multiple policies. We propose post‑hoc kernel‑variance heads that estimate epistemic uncertainty from the embeddings of a single policy. Each head computes the conditional variance of a Gaussian process conditioned on embeddings from the training demonstrations. Fitting requires no regression targets or gradient updates and leaves the policy unchanged. In experiments with SmolVLA on LIBERO‑10, the kernel variance heads perform competitively with two‑policy ensemble scores in task‑level correlation with task success, in demonstration acquisition during active learning, and in referral to a stronger supervisor policy as a proxy for human assistance. For SmolVLA, the head on the vision‑language backbone reduces uncertainty‑scoring time by up to approximately a factor of 50 compared with ensemble scores, while also substantially reducing setup costs. We also show that the established evaluation protocol on LIBERO‑10 primarily rewards recognition of coverage differences between tasks, while execution variability weakens assessment of within‑task rankings. Thus, controlled perturbations provide a complementary evaluation: modality‑specific heads identify unfamiliar camera, instruction, or state inputs, while fused heads can detect unfamiliar combinations of individually familiar inputs. With a supervisor proxy trained on the evaluated perturbation families, the decoder head improves within‑task referral over ensemble disagreement. These findings show that useful uncertainty estimates for adaptive robot learning can be obtained from a single policy.
Authors:Jungseob Lee, Seungyoon Lee, Hyeonseok Moon, Seongtae Hong, Suhyune Son, Sugyeong Eo, Minhyuk Kim, Chanjun Park, Jaehyung Seo
Abstract:
Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge. To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports. We show that checking answers against the passage removes this trade‑off. Based on this observation, we propose Accept‑or‑Repair (AoR), a training‑free procedure. AoR keeps a sampled answer if the passage supports it. Otherwise, it tries up to two replacements with contrastive decoding, returning one as soon as it passes this check. It rejects replacements that repeat the answer generated without the passage when the passage favors another answer. Our experiments show that AoR raises the score over sampling by 12.5 points on counterfactual passages and 2.3 on ordinary ones, surpassing eight contrastive and prompting baselines on both, including when each receives the same rejection rule. Replacing only initial answers that the passage does not support improves the average score of every decoder. These results suggest that models can follow passages that contradict their knowledge while preserving accuracy on passages that agree with it. Code will be released at https://github.com/js‑lee‑AI/AoR.
Authors:Ido Levy, Segev Shlomov, Leshem Choshen
Abstract:
Retrieval Augmented Generation (RAG) grounds large language models in external knowledge. Modern pipelines combine embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification, yet these techniques are typically evaluated one at a time. It remains unclear whether their isolated gains survive composition, whether retrieval quality and answer faithfulness favor the same design choices, and whether conclusions from small corpora hold at large scale. We pre‑declare three hypotheses and introduce LASAGNA, an open‑source framework for controlled RAG evaluation. We test 450 configurations across eight benchmarks and corpora ranging from 5K to 200K documents. Three findings emerge. First, individually strong techniques from different phases of a RAG pipeline do not necessarily compose: three retrieval techniques predict a +47% combined improvement but deliver only +17% when used together. Second, component gains stack differently for retrieval and faithfulness, and optimizing one can hurt the other. Third, the best composition changes as the corpus grows: the combination that wins on a small corpus may not remain best at large scale. These findings show that gains measured in isolation are not reliable predictors of realistic performance, and that new RAG techniques should be tested with the full surrounding stack. We call on researchers to do so and supply LASAGNA to simplify these experiments.
Authors:Yanze Zhang, Maggie Yongqi Guan, Shudong Liu, Qi Hao, Kanye Ye Wang, Meng Qu, Xiaobo Zhou
Abstract:
Coding agents are becoming an increasingly important paradigm for automating software engineering tasks. However, existing coding benchmarks primarily evaluate agents at the function or repository level, leaving their capabilities in more complex distributed software systems largely unexplored. In practice, many modern applications adopt microservice architectures, where end‑to‑end performance emerges from interactions among multiple independently running services, requiring agents to reason beyond local code changes. To study this capability, we introduce MicroPerf‑Bench, a benchmark for evaluating coding agents on end‑to‑end performance optimization in microservice systems. Constructing such a benchmark is challenging because reproducible system‑level performance fixes are sparse and difficult to attribute. We address this challenge by injecting empirically grounded performance anti‑patterns into real‑world open‑source systems while preserving functionality and ensuring measurable performance degradation. The resulting benchmark contains 65 optimization instances across seven systems and nine anti‑patterns, each with an executable environment, functional tests, and mixed‑workload evaluation. Across 1,755 runs covering nine agent‑model configurations, the best configuration recovers only 42.0% of the injected performance degradation. The dominant failure is not invalid code, but functionally correct and constraint‑compliant changes that fail to improve end‑to‑end performance, highlighting system‑level bottleneck localization as a key challenge for current coding agents. The source code and data are available at https://github.com/zyzcjnb/microperf.
Authors:Alessandro Canevaro, Julian Schmidt, Hang Yu, Ye Jin, Sajad Marvi, Georg Martius, Julian Jordan
Abstract:
Learning motion planning policies that remain reliable under closed‑loop execution is a central challenge in Autonomous Driving (AD). Imitation Learning (IL) scales well to large AD datasets but suffers from distribution shift, causal confusion, and compounding errors under closed‑loop execution. In contrast, Reinforcement Learning (RL) typically requires handcrafted rule‑based rewards and massive simulation. We study a simple closed‑loop IL framework that uses RL purely as an optimization tool, guided by a trajectory‑distance reward derived from logged expert data. We refer to this training strategy as Expert‑Guided RL (EGRL). Through a controlled empirical study we find evidence that the training procedure, rather than model size and complexity, accounts for a substantial part of the performance gains. We then introduce DriveMLP, a lightweight MLP policy trained with EGRL, as a simple instantiation of these findings. It achieves a score of 86.51 on the nuPlan Val14 Reactive benchmark, a +3.2 point improvement over the previous best IL planner (Flow Planner), with >15x faster inference. Code is available at: https://github.com/alessandro‑canevaro/DriveMLP .
Authors:Venkatesh Thirugnana Sambandham, Torsten Schon
Abstract:
Text‑to‑Image models reproduce societal stereotypes, and existing mitigation applies one demographic correction to the whole scene. In multi‑person scene generation this creates a trade‑off: global steering improves aggregate balance only by homogenizing the people within an image. Steering‑based methods like Embedding Arithmetic drive the intra‑image diversity below the unsteered baseline. Text‑ and latent‑projection approaches shift the aggregate skew with only marginal gains in intra‑image diversity. We present Spatially Gated Steering (SGS), which steers the demographics of each person independently by extracting per‑person masks from the Diffusion Transformer's (DiT) internal attention maps at a critical denoising timestep and applying a distinct demographic vector within each mask for the remaining steps. SGS needs no fine‑tuning, inversion or layout specifications, and uses a lightweight segmentor over frozen DiT attention features to localize subjects at early timesteps. On skewed occupational prompts, SGS reaches high intra‑image racial and gender diversity while keeping aggregate balance competitive and image quality near baseline, a result we validate on two architecturally distinct diffusion transformers, FLUX.1‑Dev and SD3.5‑Large. Codebase and model weights are available at https://github.com/cvims/spatially‑gated‑steering
Authors:Zhuoyu Wu, Wenhui Ou, Qiawei Zheng, Xuhong Qiu, Wenqi Fang, Raphaël C. -W. Phan
Abstract:
Edge medical segmentation must reconcile small model size, high accuracy, and fast inference. TernScope addresses this tradeoff through model‑‑hardware co‑design, learning a compact segmentation network whose low‑bit operators also simplify hardware execution. Split‑channel large‑kernel blocks preserve spatial context with limited computation, while branch fusion removes redundant inference paths. Ternary Progressive Quantization‑aware Training (TPQ) then adapts the fused network from deep to shallow layers, through the decoder, and finally to the input and prediction layers. All 25 convolutions use ternary weights and 8‑bit activations, enabling packed storage and multiplier‑free computation. The resulting model uses 0.2483 M parameters and 0.0949 GMACs while retaining competitive segmentation accuracy on in‑domain and external datasets. Under the same 45 nm technology, TernScope reduces compute‑array area by 88.46% compared with the original Gemmini, a systolic‑array accelerator with 8‑bit × 8‑bit multipliers. These results demonstrate a path from accurate lightweight medical segmentation to compact low‑bit hardware.
Authors:Matthias Heyrman, Chenhao Li, Ioannis Dadiotis, Marco Hutter
Abstract:
Perceptive locomotion policies typically observe a dense robot‑centric elevation map, although only the terrain beneath and around the next footsteps informs the control decision. Learned encoders compress the whole visual input, leaving the policy to work out which inputs matter while learning the baseline locomotion task. Learning both at once slows training and, without computationally expensive attention mechanisms, makes generalization to new terrain harder. We instead remove redundant inputs from the elevation map before the policy sees it. A lightweight foothold prediction model (FPM) predicts future footholds from proprioception and the velocity command alone, and we sample the elevation map only in small patches around these predicted footholds. We train the prediction model alongside the locomotion policy, periodically retraining it on new policy rollouts as the robot's gait matures. This allows the perception to anticipate where the policy will step, while the policy learns to leverage terrain information around those steps. With the proposed lightweight preprocessing, the policy reaches a given terrain difficulty in roughly a third of the training iterations required with the complete map. It outperforms policies that observe the complete map where footholds are sparse, including in narrow walled corridors, and on unseen terrain, and it matches an attention‑based encoder trained at several times the cost. The policy transfers zero‑shot to a Unitree G1 humanoid traversing various obstacles, stairs, gaps, and step‑downs beyond the training distribution in both indoor and outdoor environments.
Authors:Takumi Fujimoto, Hiroaki Nishi
Abstract:
A frozen multi‑horizon forecaster can be corrected online from its own residuals. However, under label delay the target of a block arrives only after the whole horizon has passed, and the corrector's retained state grows with the number of channels. Aiming to combine a small retained state with a large error reduction, we propose DRiC (DCT Ridge Corrector), which fits a ridge regression of the leading discrete cosine transform (DCT) coefficients of the next residual block on those of the frozen forecast and of the latest input block, with sufficient statistics updated recursively. Its local variant, DRiC‑L, keeps one such regression per channel; its global variant, DRiC‑G, pools all channels into one regression, whose statistics do not depend on the channel count. A guard tracks a running scale of the frozen forecast's residual for each channel; it skips updates on blocks far outside that scale and clips the issued correction. We evaluate on seven datasets (four ETT series, Weather with its missing‑value codes filled, Electricity, and Traffic) with DLinear, PatchTST, and iTransformer backbones, against 10 naive, gradient‑trained, and closed‑form methods. DRiC‑L with eight DCT coefficients (DRiC‑L8) reduces the mean squared error (MSE) by 14.55% on average over 126 conditions, against 12.98% for ELF, the most accurate comparator, while retaining 15‑17 times less state, although ELF is more accurate on Traffic. DRiC‑G retains 14.6 kB on the 321‑channel Electricity data and 18.9 kB on the 862‑channel Traffic data and is the most accurate method at or below its state on both. However, 5 to 7 of the 10 comparators, all with at least 10 times its state, are more accurate there. On the unfilled Weather file, DRiC‑L8 without the guard has on average 8.0 times the frozen forecast's MSE, whereas with the guard it reduces the MSE by 7.94%. Code: https://github.com/keiotakmin/dric.
Authors:Jingzhe Zhang, Peixuan Zhang, Genying Li, Bing Ma, Shuchen Weng, Si Li, Boxin Shi
Abstract:
Cinematic multi‑shot video generation requires both narrative planning and iterative refinement. Existing methods either lack narrative reasoning or rely on feed‑forward workflows without policy updates. In this paper, we propose CineEvolve, an agentic self‑evolution framework for cinematic multi‑shot video generation that unifies a Director and a Critic within a single vision‑language policy. Emulating real‑world filmmaking that couples iterative direction with critical feedback, we train CineEvolve via alternating reinforcement learning, where the Director surfaces flaws to sharpen the Critic, and the updated Critic guides the Director's targeted revisions. To stabilize co‑evolution, we introduce Evolutionary GRPO (E‑GRPO), leveraging historical anchoring and revision safeguarding to mitigate advantage inflation. Additionally, we contribute CineTraj, a 40K cinematic trajectory dataset spanning creation, critique, and revision. Extensive experiments demonstrate that CineEvolve outperforms existing approaches. Our code will be available at https://github.com/JingZhe‑8/CineEvolve.
Authors:Rafael Lopes Almeida, Ina Pöhner
Abstract:
Graph neural networks (GNNs) are among the standard tools for molecular property prediction, but the atom‑level explanations of their predictions are hard for medicinal chemists to act on. We propose a change of input representation rather than a new model: molecules are encoded as fragment‑level graphs from BRICS decomposition, where each node is a synthetically meaningful fragment whose features are the element‑wise sum of its atoms' features. Because only the representation changes, any message‑passing GNN and any post‑hoc attribution method produces explanations at the resolution of functional groups instead of single atoms. Across four architectures, four prediction tasks, and two explainers under scaffold splits, fragment‑level encoding matches atom‑level test accuracy on roughly five‑fold smaller graphs. Integrated‑Gradients attributions recover known pharmacophores above chance on two enzyme targets, namely BACE at 1.5 times the chance rate, and PTR1 at 3.2 times. A like‑for‑like comparison against an aggregated atom‑level baseline shows that post‑hoc aggregation is not a stable substitute: its BACE hit rate range both exceeds and falls below the fragment model's hit rate depending on two arbitrary aggregation choices, whereas fragment‑level training yields a single convention‑free attribution at no predictive cost. The code is available at: https://github.com/4D‑Lab/frame
Authors:Thorsten Altenkirch
Abstract:
We explain an attempt to define semi‑simplicial types (SST) in homotopy type theory by first defining a small type theory whose contexts are exactly the data of semi‑simplicial types, constructing \mathrmSST\,n uniformly in that syntax, and then interpreting the syntax in the universe. The syntax can be defined without any truncation and without K, and the construction of \mathrmSST\,n goes through. The naive interpretation does not: it transports along the semantic images of equations that hold only propositionally, and making these transports agree needs uniqueness of identity proofs (UIP) for the semantic types. Moving the problem into the syntax, by making composition a constructor of a higher inductive‑inductive type (HIIT), does not remove it either: it reappears as a coherence problem in the syntax. Fording the HIIT (storing equations instead of computed indices), with conditional rules, makes the syntax provably a set. This is UIP for the syntax, obtained by encode‑decode, and it does not by itself restrict the semantics; but the naive interpretation of the conditional rules again needs UIP for the semantic types. A semantics into groupoids that avoids UIP will be discussed in a separate note. The Agda development accompanying this note is in the repository https://github.com/txa/sst‑new.
Authors:Colin Doumont, Henry Moss, Philipp Hennig
Abstract:
Bayesian optimization (BO) is typically used only for problems where the underlying function is deemed "expensive" (e.g. in time or money), with the speed of the BO algorithm being of little importance. In the real world, however, problems span a continuous range of values, going from cheap to expensive. In this paper, we present BO algorithms through the lens of a speed‑performance Pareto frontier: we compare BO against faster algorithms like CMA‑ES, and highlight how certain regions of the Pareto frontier are currently empty. As a solution, we propose a new set of fast BO methods that exploit the linear structure in existing BO algorithms, for both continuous and combinatorial inputs (from ~10 to 6000+ dimensions). Our methods achieve performance similar to state‑of‑the‑art BO, while being more than 10^2× to 10^3× faster, opening the field of BO to an entirely new class of problems.
Authors:Ahmed Mahrous, Roberto Di Pietro
Abstract:
Many learning systems are performative: once deployed, their predictions change the data distribution on which future models are trained. In multi‑agent settings, this feedback is routed through a network of interacting learners, so stability depends not only on the strength of performativity but also on who affects whom. We study repeated retraining in multi‑agent performative prediction and identify a spectral threshold that separates stable learning from worst‑case learning obstruction. Prior work often collapses cross‑agent feedback into a scalar contraction constant, missing this network structure. We instead introduce the reflexivity matrix Γ, which records how strongly one agent's deployment can move another agent's best response, defined from per‑agent gradient sensitivities and curvature parameters. The threshold occurs when the spectral radius of this matrix crosses one. Below the threshold, repeated retraining converges geometrically to a unique performatively stable equilibrium, with cumulative excess loss bounded uniformly in the time horizon. Above the threshold, we construct noisy supercritical instances in which every local‑information learner incurs expected cumulative excess loss growing linearly for horizons up to order 1/(ρ(Γ)‑1). Together, these results characterize the threshold behavior through upper and lower bounds and recover the single‑agent scalar threshold from prior work as a special case.
Authors:Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, Dacheng Tao
Abstract:
Fine‑grained multimodal intelligence requires models to both understand a designated region and localize it from language. Existing methods learn these directions with separate objectives and task‑specific annotations. We present Generalized CycleGRPO, the first unified reinforcement learning framework coupling region understanding and localization across spatial, temporal, and spatio‑temporal domains. Given a region, a multimodal large language model generates candidate descriptions and then localizes each description to reconstruct the input region. Reconstruction overlap provides a verifiable reward for captioning, while individual localization rewards improve the inverse mapping. A common region‑space formulation extends this cycle from image masks and boxes to video intervals and timestamped spatial evidence, without reference captions or an external language‑model judge during reinforcement learning. In videos, the cycle encourages descriptions that distinguish a target event from similar recurrences. We introduce DOVE (Distinctive Occurrence‑level Video‑caption Evaluation), a human‑verified contrastive benchmark for this temporal distinctiveness. CycleGRPO improves SAMTok's overall gIoU from 57.5% to 68.2% on GroundingSuite and average gIoU from 71.2% to 78.7% on GRES. For temporal tasks, it achieves 51.9% tIoU on Charades and 33.3% Temporal Evidence Coverage on DOVE, compared with 45.6% and 14.8% for the Qwen3‑VL initialization. These results establish region‑text cycle consistency as a general self‑evaluation principle for multimodal understanding and localization. Models, code, and DOVE are available at https://devinxzhang.github.io/Generalized‑CycleGRPO‑Page/.
Authors:Bei Huang, Yixin Chen, Hongbin Zha, Yuru Pei, Siyuan Huang
Abstract:
High‑fidelity 4D physical simulation requires accurate physical parameter assignments that align with diverse material properties. Previous methods struggle with critical bottlenecks: they either rely on computationally intensive per‑scene optimization or predict parameters from purely visual inputs. The latter suffers from inherent ambiguity, e.g., the inability to distinguish the stiffness of rubber from its appearance alone. To bridge this gap, we propose AutoPhyX, a text‑conditioned framework for predicting spatially‑varying physical properties. Since training robust feed‑forward models requires fine‑grained supervision, we first introduce a novel part‑controllable data generation pipeline that decomposes complex 3D assets into semantically distinct components and pairs them with physically plausible, diverse parameters and corresponding text descriptions. Leveraging this dataset, AutoPhyX employs a cross‑modal modulation mechanism where text features dynamically modulate visual features for precise physical grounding. By formulating property prediction within a voxel field, AutoPhyX ensures compatibility with diverse 3D representations, including meshes, point clouds, Gaussian Splatting, and NeRF. Experiments demonstrate that our method enables physically plausible, text‑driven parameter assignment in a single forward pass. It achieves high accuracy and robust generalization to in‑the‑wild objects, paving the way for future physics‑intensive embodied and robotic applications.
Authors:Wenhai Wan, Teng Zhang, Shao-Yuan Li, Xinrui Wang, Qiang-Sheng Hua, Songcan Chen
Abstract:
Real‑world datasets often follow a long‑tailed distribution, making generalization to tail classes difficult. We revisit this problem through the lens of shortcut learning, where models prefer the easiest predictive cues (e.g., background or textures) over object‑centric semantics, especially under scarce and biased supervision. We find that this tendency is amplified for tail classes: limited examples often share similar contexts, making non‑semantic signals highly correlated and thus tempting shortcuts, whereas head classes with diverse appearances and environments encourage more stable object‑focused representations. Motivated by this observation, we propose Shortcut‑Resistant CAM Distillation (SRCD), a plug‑and‑play framework that transfers object‑focused explanations from head to tail classes. SRCD operates in the Class Activation Map (CAM) space, where a CAM provides a class‑specific spatial evidence map for a prediction. SRCD aggregates CAMs from a small set of head‑class candidates into a shortcut‑resistant teacher using an energy‑model weighting based on coherence and concentration, and distills it to the tail‑class CAM. We provide a theoretical analysis that quantifies shortcut reliance as shortcut‑region evidence mass in CAM space and shows that SRCD suppresses tail shortcuts. Extensive experiments on long‑tailed benchmarks consistently improve strong baselines. The code is available at https://github.com/Haifeng3/SRCD.
Authors:Julian Herold, Markus Götz, Alexander Schug
Abstract:
Approximate Bayesian computation (ABC) enables Bayesian inference for models with intractable likelihoods by estimating parameter posterior distributions from simulated data. Sequential ABC methods improve sampling efficiency by iteratively adapting the proposal distribution toward regions of high posterior probability. However, these methods typically operate in generations, where each generation must wait for its slowest simulation before the next can begin. On a parallel machine, this synchronization barrier leaves workers idle and reduces throughput, particularly when simulation runtimes vary substantially. We introduce a generation‑free ABC sampler that updates its proposal when simulation results arrive and sends a new candidate to each available worker. The proposal is based on the best particles evaluated so far. To estimate the posterior, we reuse the full history and reweight particles against a reconstructed mixture of sampling proposals. We show how errors in approximating the mixture of sampling proposals affect the estimated posterior and establish consistency for the smooth‑ABC posterior when the approximation becomes asymptotically exact. In our experiments, the method achieves well‑calibrated posterior estimates where a pyABC baseline with the same kernel and bandwidth schedule under‑covers. From the asynchronous run's timing alone, we predict how much a synchronization barrier would slow the sampler. These predictions agree with measured synchronization penalties ranging from 1.2x to 400x. On a Cellular Potts tissue simulator, the asynchronous sampler completes 2.1‑2.4x as many evaluations as pyABC in the same hour, and the discrepancy threshold for its best 100 evaluations is 4.1x smaller. With the starting bandwidth set from prior‑predictive discrepancies, it also produces a more concentrated two‑parameter posterior with a larger effective sample size than pyABC.
Authors:Jiaru Qian, Yihong Dong, Xue Jiang, Chun-Wah Hsu, Zhi Jin, Ge Li
Abstract:
Parallel thinking has emerged as a promising paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs) by exploring multiple reasoning paths simultaneously. However, we identify that parallel thinking via Reinforcement Learning (RL) typically suffers from the mode collapse of reasoning paths, where parallel thinking trajectories degenerate into similar ones. This homogeneity undermines the fundamental motivation of parallel thinking, that is, maintaining multiple hypotheses concurrently to prevent premature loss into a suboptimal solution. In this paper, we propose Diverse Parallel Thinking (DPT), a simple yet effective RL approach for parallel thinking designed to encourage path diversity. DPT rewards the model to develop distinct reasoning strategies by penalizing the prefix similarity of parallel thinking paths, thereby enhancing a more comprehensive exploration of the reasoning space. We conduct various experiments and analyses across multiple reasoning benchmarks to demonstrate the effectiveness of DPT. Results show that DPT successfully outperforms previous parallel thinking methods, achieving state‑of‑the‑art performance over baselines. We validate path diversity as a key element to improve parallel thinking. Our code is available at https://github.com/JiaruQian/DPT.
Authors:Jian Zhong, Zeyu Liu, Pingchuan Cao, Rongduo Han, Shunye Tang, Guohuan Xie, Mingyuan Qin, Ran Ji, Xiao Liang, Wei Wang, Haining Zhang
Abstract:
Long‑horizon dialogue agents need to keep user memories stable enough for personalization while adapting when the user genuinely changes. Existing memory systems often append every observation or overwrite old entries by heuristic rules, which makes transient noise hard to distinguish from real belief drift. We introduce PreCoMem, a memory consolidation framework inspired by predictive coding that represents memory as confidence‑weighted beliefs about the user's state. Its core signal, Effective Surprisal, combines semantic deviation and contradiction evidence, then downweights this signal when retrieval is diffuse and unreliable. According to effective surprisal, a two‑threshold gate maps each turn to one of three deterministic updates: MAINTAIN reinforces supported beliefs, PROFILE stores ambiguous cues as low‑confidence hypotheses, and CORRECT softly decays reliably contradicted facts. The PROFILE stage acts as an explicit waiting room, preventing premature commitment to noisy evidence. Experiments on LoCoMo, LongMemEval‑S, and PersonaMem‑v2 show consistent state‑of‑the‑art accuracy; our new DiMoBench confirms that PreCoMem adapts to non‑stationary belief drift without over‑reacting to noise. The code is available at https://github.com/ZhongWishing/PreCoMem.
Authors:Yuan Ge, Pei Zhang, Xize Cheng, Lingzhou Mu, Mingyang Wang, Jin Xu, Baosong Yang, Shengxiang Gao, Jingbo Zhu, Zhengtao Yu, Tong Xiao
Abstract:
Omni‑modal large language models (LLMs) have achieved remarkable progress in audio and video understanding. However, existing models struggle with long‑form, multi‑speaker videos, especially in understanding the entire video and recognizing who says what and when. To characterize the capability limit of current omni LLMs and guide future model development, we introduce Omnilingual‑MsASR, a multilingual, multimodal benchmark for long‑form Multi‑speaker Automatic Speech Recognition (MsASR) covering six languages, recordings up to 30 minutes, and scenarios with up to 20 speakers. Omnilingual‑MsASR provides parallel audio and video inputs to evaluate models' cross‑modal consistency and their ability to exploit complementary cues across modalities. Extensive evaluations of leading omni‑modal LLMs reveal substantial performance bottlenecks, while their tcpWERs frequently exceed 100%. These findings highlight fine‑grained temporal localization, speaker disambiguation, and cross‑modal synergy as key challenges for future omni‑modal foundation models. Our work is available at https://github.com/QwenLM/Omnilingua‑Bench/tree/main/Omnilingua‑MSpeaker.
Authors:Shujun Yao, Junhui Lin, Congying Han, Tiande Guo, Ziwen Liu, Bonan Li
Abstract:
Feed‑forward visual geometry models, exemplified by VGGT, have demonstrated strong multi‑view reconstruction capabilities through dense global attention. However, the resulting all‑to‑all cross‑view computation scales quadratically with the number of input views, making long‑sequence inference particularly expensive due to substantial frame‑level redundancy among highly overlapping observations. We uncover a topology‑first, geometry‑later evolution in VGGT camera tokens: shallow‑layer tokens reliably capture local view neighborhoods, whereas deeper‑layer tokens progressively encode globally organized inter‑view geometry. Motivated by this observation, we introduce BlazeVGGT, a training‑free framework for structure‑aware frame‑level key‑‑value sparsification. At each global‑attention block, BlazeVGGT constructs a view‑level similarity kernel from camera‑token relations and employs greedy determinantal point process inference to select a compact set of mutually complementary views. The selected views constitute the key‑‑value context, while all input frames are retained as queries. This asymmetric attention design preserves dense per‑frame predictions while eliminating redundant cross‑view computation, allowing our method to be directly integrated into pretrained VGGT as a plug‑and‑play module without parameter updates. Its advantages become increasingly pronounced with sequence length: at 2,000 frames, BlazeVGGT achieves approximately 13× faster inference than VGGT while reducing the Chamfer Distance from 0.41 to 0.32. Extensive experiments demonstrate a favorable accuracy‑‑efficiency trade‑off in point‑cloud reconstruction and camera‑pose estimation, compatibility with existing VGGT accelerators such as FastVGGT and LiteVGGT, and transferability to VGGT‑based models including StreamVGGT and PAGE‑4D. Codes are publicly available at https://github.com/MagiciannnY/BlazeVGGT.git.
Authors:Xuan Yang, Chunxiao Liu, Hongsheng Xin, Hao Xu, Ning Miao
Abstract:
Large language model (LLM) agents increasingly use tools to complete tasks in stateful environments. However, many training environments remain largely static apart from changes caused by the agent's own actions, whereas external events introduced by other users, services, or sudden contingencies in real environments can invalidate ongoing plans. To address this gap, we introduce DynaWeave, an automated synthesis framework that transforms existing static tasks into verifiable dynamic scenarios. The framework generates exogenous events relevant to each task, covering changes in task requirements, environment state, and external information. It uses directed acyclic graphs (DAGs) to represent dependencies between events and an execution engine to activate them during agent interaction. A dynamic verifier revises only the evaluation checks affected by events that actually occur and retains all other checks. Using DynaWeave, we collect 2,095 verified interaction trajectories across 36 environments and use them for supervised fine‑tuning of Qwen3.5 models. Experiments on Gaia2, CostBench, and ToolMaze show that DynaWeave achieves consistent improvements in overall performance across all three evaluated model sizes, increasing the task completion rate on CostBench by 67.34 percentage points and even exceeding the performance of Gemini‑3.1‑Pro‑Preview on ToolMaze. These results support dynamic scenario synthesis as a practical approach to training agents to adapt to changing environments. Our code is available at https://github.com/Miaow‑Lab/DynaWeave.
Authors:Jay Agarwal, Siddharth Khare, Dhruv Kumar
Abstract:
Jet tagging is one of the most important applications of machine learning at the Large Hadron Collider. Recent models build in more physics knowledge or rely on large‑scale pretraining, but the experiments place strict limits on their inference cost. Knowledge distillation can transfer their performance to small students that are cheap to run. We ask which properties of a teacher help the student, how small a student can get and how close it can come to the state of the art. On a jet tagging benchmark, we distil six teachers that differ in architecture, symmetry and pretraining, including two foundation models, into five small students. Every teacher improves every student, and the gain depends mainly on the strength of the teacher. We then shrink our two strongest distilled students, Transformer and L‑GATr‑slim, in width and in depth. With 20 thousand parameters, a distilled L‑GATr‑slim outperforms every published jet tagger trained from scratch at a fraction of the inference cost of every teacher. Even with two thousand parameters, it matches ParT, a model with a thousand times as many parameters. We also study what the students learn from their teachers. Our benchmark of inference cost shows that students of the same size can differ by almost an order of magnitude in speed. We release the code and the weights of all trained models.
Authors:Chihiro Arata, Shohei Taniguchi, Gouki Minegishi, Kan Hatakeyama-Sato
Abstract:
Neural speech codecs provide discrete tokens for speech generation. Lower frame rates shorten token sequences, but each token must then carry more linguistic and acoustic information. Variable‑frame‑rate codecs address this trade‑off by adapting token allocation to speech content, but threshold‑based allocation lets content determine the token count, so the sequence length a speech generator must produce varies across utterances and datasets at a fixed setting. We present RubatoCodec, a variable‑frame‑rate speech codec built on X‑Codec‑2 with an exact utterance‑level event budget and streaming decoding. Its Adaptive Firing mechanism uses changes in codec features and waveform spectra to divide speech into variable‑duration events without a learned boundary predictor. Discrete codes and an explicit duration represent each event, allowing a speech generator to predict and decode one event at a time. Controlled experiments show that adaptive allocation outperforms fixed‑rate coding in reconstruction fidelity and intelligibility at low event rates. A model trained on large‑scale multilingual speech achieves competitive reconstruction quality across English, Japanese, and Mandarin, with the best intelligibility preservation among the compared codecs on the evaluated English and Japanese benchmarks at comparable bitrates. In a matched text‑to‑speech comparison, the adaptive representation yields lower WER and higher UTMOS and UTMOSv2 with comparable speaker similarity. The speech generator also learns the codec's utterance‑level event budget without an explicit generation‑time rate constraint. These results demonstrate that budgeted adaptive speech representations support both high‑quality reconstruction and streaming text‑to‑speech generation. Code and weights are available at https://github.com/third‑intelligence/RubatoCodec and https://huggingface.co/third‑intelligence/RubatoCodec.
Authors:Xiujian Liang, Jiacheng Liu, Mingyang Sun, Qichen He, Anda Cheng, Cewu Lu, Jianhua Sun
Abstract:
Real‑world physical sensing exhibits complex, heterogeneous noise patterns that deviate significantly from idealized simulation, posing a fundamental bottleneck for sim‑to‑real transfer. Existing sensor modelings typically treat depth noise as a monolithic black‑box process, overlooking the distinct physical mechanisms that govern different error modalities. In this work, we introduce a physics‑grounded paradigm that disentangles monolithic noise into two complementary modalities: sensing invalidation and measurement inaccuracy, enabling a tailored treatment of noise sources based on their physical origins. Building on this insight, we propose PRISM, a tripartite framework that distills 3D Visual Foundation Model features as rich spatial‑semantic priors for physics‑based reasoning. To address the inherent sparsity and class imbalance of invalidation regions, we develop Hierarchical Positive‑Prioritized Supervision, integrating multi‑scale positive‑weighted objectives with a positive‑preserving dynamic hard mining strategy to enforce precise artifact delineation. Extensive benchmarks demonstrate that PRISM achieves state‑of‑the‑art fidelity in noisy depth synthesis. Furthermore, downstream robotic experiments show that PRISM facilitates a 92.1% average success rate in the real world, marking a significant improvement over monolithic baselines. Our project page is available at: \resizebox\linewidth!\hrefhttps://shawnleung‑sii.github.io/prism‑icml2026/https://shawnleung‑sii.github.io/prism‑icml2026/.
Authors:Chuangxin Zhao, Meimei Zhang, Canran Xiao, Jiachen Luo, Jiajun He, Kaijie Chen, Huy Phan, Tomoki Toda, Biörn W. Schuller
Abstract:
Multimodal multi‑label emotion recognition is important for affect‑aware systems, but models trained with complete text, visual, and acoustic streams often degrade sharply when some streams are unavailable at deployment. We address this train‑test gap by learning a compact representation that preserves emotion‑specific evidence from any available subset of modalities. We propose the Multi‑modal Emotion Recognition Perceiver (MERP), which maps the joint sequence into a fixed‑size latent bank, thereby keeping repeated reasoning independent of raw input length; it extracts label‑specific evidence before and after latent reasoning and aligns the two levels through reconstruction. MERP further uses positive and negative class prototypes, a noise‑gated feature enhancer with frequency‑shaped perturbations during training, and confidence‑aware class‑wise fusion to strengthen separability, reduce sensitivity to missing inputs, and combine complementary predictors. Under 20 percent test‑time random modality drop, MERP achieves micro‑F1 scores of 0.563 on CMU‑MOSEI and 0.780 on M3ED, exceeding the strongest evaluated baselines by 0.004 and 0.005, respectively; in the severe CMU‑MOSEI audio‑only setting, it retains 0.473 micro‑F1 versus 0.022 for LDDU. These results show that fixed latent reasoning provides a practical, testable route to robust multimodal emotion recognition without requiring missing‑modality examples during training. Code is available at https://github.com/Creative‑zcx/MERP.
Authors:Keith Kwok, Wanting Ning, Tej Deep Pala, Desmond C. Ong, S. Supraja, Renwen Zhang, Soujanya Poria
Abstract:
Recently, we see a rise in conversational agents as a form of social and emotional support, where disclosure will unfold over time. Yet, current evaluations of these agents typically examine isolated conversational states. In analyses of naturally occurring conversations, disclosure history and current disclosure are confounded which tends to make their effects difficult to isolate. We thus introduce Trajectory‑based Disclosure Evaluation (TraDE), an evaluation framework grounded in Social Penetration Theory that utilizes predefined disclosure trajectories and evaluates how preceding disclosure history shapes model behavior under matched current disclosure. Through the use of TraDE, we demonstrate across multiple models and relational behavior benchmarks that responses to disclosures matched on depth can differ depending on the preceding trajectory, particularly in whether models accept or challenge the user's framing. We also observed that these differences track accumulated exposure to deeper disclosure rather than any single moment of vulnerability or the order of disclosures. It also tends to persists for several turns after disclosure returns to shallow levels. For several behaviors, we also demonstrate that models diverge most consistently from human supporters in the Emotional Support Conversation corpus (ESConv) not when users open up but rather when they step back in their disclosure. TraDE thus provides a systematic way to expose history‑dependent relational behavior that evaluations of isolated conversational states would otherwise miss.
Authors:Yuta Horioka, Naoya Chiba, Yuki Uranishi
Abstract:
In implicit neural representations (INRs), 3D signals are encoded directly in network weights, making stable weight‑space operations essential. However, permutation symmetry causes independently trained yet functionally equivalent models to appear distant in weight space, often inducing large loss barriers under linear interpolation. Existing neuron alignment methods either commit early to discrete matches and become trapped in inconsistent local optima or rely on computationally expensive gradient optimization over continuous relaxations, both of which are unreliable in challenging cases such as NeRF. We adopt progressive alignment, a layer‑wise strategy that follows inter‑layer dependencies from input to output. At each layer, a correlation matrix is estimated and aligned with a discrete anchor obtained by the Hungarian algorithm, after which temperature‑controlled spectral annealing gradually sharpens the correlation structure toward a permutation matrix. This update is deterministic and lightweight, avoiding the inner‑loop gradient optimization required by continuous relaxation methods. A final refinement stage further optimizes the network layer‑by‑layer via weight matching, starting from the aligned solution. Experiments on MLPs, CNNs, and Neural Radiance Fields (NeRFs) show that the proposed method reliably recovers neuron correspondences and achieves stable alignment across architectures. In practical experiments aligning independently trained NeRF models, our method reduces the loss barriers under weight interpolation compared with existing approaches.
Authors:Rodion Shkokov, Tatiana Zemskova, Dmitry Yudin
Abstract:
Robots deployed in human‑centered environments need maps that support both geometric navigation and open‑vocabulary semantic queries beyond a fixed label set. A key challenge is enriching such maps with semantic information without relying on manually annotated segmentation masks or closed‑set semantic categories. Existing geometry‑driven active mapping maximizes coverage but treats semantics as a passive by‑product of the collected trajectory, whereas active semantic mapping typically depends on supervised semantic segmentation. We present AOV‑GS, an active 3D Gaussian mapping system that guides exploration using frozen foundation models. During coarse exploration, AOV‑GS re‑ranks geometrically promising candidate views according to CLIP mask‑embedding novelty computed from SAM instance segmentation, biasing exploration toward semantically informative regions. Frozen SAM and CLIP provide instance masks and language‑aligned embeddings throughout mapping, while a lightweight scene‑specific language field is fitted offline directly to the collected CLIP embeddings rather than manually annotated masks, enabling open‑vocabulary text‑query segmentation on novel views. In a controlled comparison using the same SAM‑CLIP perception stack, AOV‑GS improves average language‑field mIoU over geometry‑only active planning by 1.3 percentage points on Replica (34.6% on eight scenes) and by 1.8 points on Matterport3D (23.1% on five scenes). With CorrCLIP, AOV‑GS reaches 35.6% on Replica, 2.0 points above the matched Geometry‑based + CorrCLIP baseline. Novel‑view PSNR remains comparable to the geometry‑only baseline on both datasets while depth L1 improves on Replica. Code is available at https://github.com/OptimusFaber/AOV‑GS.
Authors:Ju-Chun Ko
Abstract:
"Abliterated" ("uncensored") builds of open‑weight language models remove the activation direction that mediates refusal, and are popular partly on the assumption that a model that no longer refuses is free of political censorship. We test this on two matched pairs of 2‑bit models: NVIDIA's Nemotron 3.5 Bolt 74B‑A9B and Qwen3.8‑Flash‑Next, each against a community abliteration. Each model judged 22 statement pairs (on the Republic of China (Taiwan), 1989 Tiananmen, control countries) in Traditional Chinese, Simplified Chinese (a character‑level conversion) and English, and answered 27 open questions per script. Script mattered far more than abliteration: the original Bolt scored +0.58 on Index S (‑1: People's Republic of China (PRC) position; +1: ROC position) in Traditional Chinese, +0.57 in English and +0.05 in Simplified Chinese, and affirmed "Taiwan is part of the PRC" with probability 0.91 in Simplified but 0.06 in Traditional characters; abliteration changed its scores by at most 0.06, mostly by raising "yes" on both sides. In Qwen, abliteration removed every refusal on 1989 and raised scores by 0.21 to 0.34, yet the abliterated Qwen still affirmed that Taiwan is part of the PRC in all three scripts. Naming the state the Republic of China (Taiwan) raised every model's score by 0.10 to 0.33 but left the script gap. Control pairs showed no script gap; both edits changed only their declared tensors. In 740,000 web documents, Simplified text on Taiwan used PRC‑framing terms 3.7 times as often as Traditional text. Steering the script gap away failed; a stance direction from the ROC wording raised scores on new statements by 0.04 to 0.06; negated, it lowered them more. A four‑sentence system prompt raised the abliterated Qwen to +0.93 to +0.97. Keyword scoring rules failed the models closest to the ROC position mostly for attributing or denying the PRC claim. Uncensored is not unbiased.
Authors:Changshun Wu, Mengyu Li, Mathis Marsault, Xiaowei Huang, Chih-Hong Cheng
Abstract:
How do LLM agents' strategies differ, and how much can instructions to adopt a persona change their behavior over a game? We study these questions with AgentMonopoly, an environment where agents communicate, buy and trade properties, and make decisions that affect later turns. We compare models from four families, each at two sizes, in games with identical models, different sizes of the same family, or different families. We also assign one agent a persona, a description of how it should behave, intended to change a tendency observed in games without that persona. The tested models differ in how often they trade, waive rent, and manage cash and properties. Larger models do not consistently win more games. Persona instructions change the words agents use and some of their actions, but the changes do not always match the role across all decisions. These results show why persona following should be assessed through actions and outcomes as well as language. As our findings are conditional on the tested checkpoints, prompts, and five seeds per configuration, the open‑source nature of AgentMonopoly provides the experimental infrastructure for further extending these comparisons. Our code is available at https://github.com/SuELS‑Group/Agent_Monopoly.
Authors:Qinghang Liu, Conglin Wang, Zongsheng Li, Boyong He, Han Gao, Pengcheng Wang, Tao Sun, Lige Liu, Chang Liu
Abstract:
Long‑horizon robot manipulation requires a robot to remember task‑relevant information beyond its short observation window. Existing memory‑augmented policies typically retrieve relevant information from history and reason over it at execution time, placing a heavy reasoning burden on the high‑level model. To address this challenge, we propose PAL (Plan first, Act Later), a hierarchical memory framework for vision‑language robot manipulation. It maintains a short‑term visual buffer, a semantic‑event memory, and an executable \textttsubtask\_list. A high‑level vision‑language model (VLM) converts streaming visual observations into the \textttsubtask\_list. At execution time, the VLM selects the currently relevant subtask from the \textttsubtask\_list and passes it to a low‑level vision‑language‑action (VLA) policy. To teach the model what to write to memory and when to update it, we introduce a two‑stage data generation procedure that first expands sparse update supervision and then uses self‑labeling with task‑grounded verification to generate training data that better match the state distribution encountered during execution. We evaluate our approach on memory‑dependent benchmarks and real‑world manipulation tasks. On RoboMME, PAL achieves a 56.05% success rate, outperforming the strongest baseline, FrameSamp‑Modul, at 44.51%. In real‑world experiments across four manipulation tasks, \pal reaches 60.83% overall success, surpassing MemER (45.00%) and FrameSamp‑Modul (28.33%). These results show that proactively writing future‑relevant subtasks into hierarchical memory improves long‑horizon visual reasoning and robot control. Code and videos can be found on our website: https://qinghangliu.github.io/pal‑webpage/
Authors:Adi Simhi, Yaniv Nikankin, Jonathan Herzig, Idan Szpektor, Yonatan Belinkov, Shay B. Cohen
Abstract:
As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi‑turn conversations where two LLMs discuss subjective matters, can agree or disagree, and may change their stances. Using a linear classifier on individual conversation activations, we identify an internal representation of agreement. We find that it varies substantially across turns and conversations and does not reliably track external agreement at the individual conversation level. We therefore move beyond the representation of agreement in individual conversations, by aggregating the agreement representation across conversations. Externally, this aggregate view reveals a simple structure: when pooled across conversations, agreement can be predicted well with a Markov process, with agreement at each turn being a direct function of agreement at the previous turn. Similarly, when internal agreement is aggregated, it couples with the external Markov agreement through a power‑law mapping, exposing Markovian dynamics in internal representations. Additionally, we find that after a change of stance, models express agreement externally before their internal representations reflect the same shift. These results reveal a joint structure between internal and external agreement that emerges only when internal features are aggregated across conversations. Code available at https://github.com/technion‑cs‑nlp/SeeingTheForestForTheTrees.
Authors:Yongjin Yi, Jinyong Park, Sejoong Kim
Abstract:
The Objective Structured Clinical Examination (OSCE) is widely used to assess clinical communication skills in medical education, requiring trainees to take histories and reason clinically with standardized patients (SPs). Large language model (LLM)‑based virtual standardized patients (VSPs), typically built by persona‑injected prompting, offer a practical alternative. However, existing evaluations rely on a single backbone LLM, small human rater panels, or LLM‑as‑a‑judge scoring without an utterance‑aligned human‑expert reference, and so cannot identify the specific shortcomings of individual models. We evaluated four LLMs (GPT‑4o, Claude 3.5 Sonnet, Llama‑3.1 70B, and HyperCLOVA X), spanning commercial, open‑source and Korean‑developed models, against a human‑expert reference across three Korean OSCE cases (hematuria, fever in pregnancy, and chronic cough). In a double‑blind, fully crossed design, 17 senior medical‑student raters scored every model utterance on three sentence‑level and six encounter‑level Likert metrics, yielding 7,480 data points. We analyzed the scores using a stratified linear mixed‑effects model based on an 11‑category history‑taking and 3‑level linguistic‑form taxonomy. At the encounter level, three of the four LLMs fell 0.08 to 0.47 Likert points below the human‑expert reference on Comprehensibility, Consistency and Hallucination‑freeness. The difference was not statistically significant for Hallucination‑freeness in Claude 3.5 Sonnet and GPT‑4o, Consistency in GPT‑4o, and Engagement in HyperCLOVA X. Llama‑3.1 70B fell significantly below the reference on all six metrics. We also introduce a web‑based OSCE platform that supports virtual practice across diverse clinical cases and continuously collects student‑VSP dialogues under research and privacy consent.
Authors:Boning Li, Longbo Huang
Abstract:
Heuristic solvers for general‑sum extensive‑form games produce a stream of iterates, and practice reports the NashGap of the returned profile. We ask what a solver can know about its own iterates from the regret accumulators and the terminal‑distribution accumulators its updates already maintain. In two‑player games the NashGap of a weighted average strategy equals, at the maximizing player, the weighted external regret plus a signed value correction. Bounding the regret by the running counterfactual‑regret accumulator gives an online certificate whose cost per checkpoint is linear in the number of terminal histories. The certificate stays sound under alternating updates and in games with three or more players. The mismatch between what alternation evaluates and what the averages record enters as one signed comparator term, maximized in a single backward pass per player whose comparator weight is nonzero. The bound is read off the regret state, with no best response against the average. Run a portfolio of training‑free regret‑matching solvers, output the iterate with the smallest certificate, and verify that iterate once. On the benchmark's biased Leduc family a pool of sixteen training‑free solvers, seven original and nine recent variants, solves 89% of the 64 instances at threshold 10^‑5 this way, matching the trajectory‑minimum union of the solvers. The same protocol solves 92% of the 64 pre‑registered held‑out instances and 89% again on a third set of 64 instances that no earlier stage used. The zero‑best‑response certificate certifies all 57 in‑sample solves and all 57 fresh ones, and 58 of the 59 held‑out ones. Its exact‑comparator refinement certifies every solve in the three sets. The certificate states both and attributes its bound. Code is available at https://github.com/lbn187/CertNE.
Authors:Ruqi Chen, Zilei Wang, Wei Wang
Abstract:
Analogical reasoning is widely considered a fundamental component of human intelligence. Few benchmarks are designed to explicitly test far analogical reasoning ‑ reasoning between domains that are superficially dissimilar yet share deep relational or structural similarities. In this work, we introduce an analogical reasoning benchmark called Abstraction and Reasoning Corpus of ANAlogy (ARCANA). The tasks of ARCANA require performing far analogical mappings from real‑world objects or physical phenomena to abstract grid‑based transformations, different from the well‑known ARC‑AGI benchmark which primarily emphasized innate human priors. We validate ARCANA through a human study, demonstrating that humans solve these tasks reliably. In contrast, evaluations across a range of large language models show a distinct performance gap compared to humans, revealing systematic failure modes that remain challenging for current models. To address this gap, we propose ConceptSIR, a reasoning framework based on concept slippage, introspection, and reflection. Experiments demonstrate that ConceptSIR enables standard LLMs to solve several challenging ARCANA tasks that were unsolved by direct prompting in our evaluation. Overall, our results suggest that far analogical reasoning remains an open problem for contemporary LLMs. All the materials are available at sites.google.com/view/arcanacogtest. The complete codebase, including benchmark, and ConceptSIR, is publicly available at https://github.com/density‑11/arcana‑test. All ARCANA data and task instances can be accessed and explored at https://arcanacogtest.org/.
Authors:Chun Gu, Xiaofei Wei, Yiming Chen, Bozhou Zhang, Juntong Zhang, Xiatian Zhu, Li Zhang
Abstract:
Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact‑rich, recovery, and precision‑critical states encountered during deployment. Existing methods mitigate this issue by augmenting offline demonstrations with human‑in‑the‑loop data or online reinforcement learning. Despite this, offline post‑training adapts slowly because it cannot immediately absorb deployment experience, while online RL typically relies on value functions or auxiliary actor‑critic models, increasing both training complexity and instability. We present TIDE (Training In DEployment), a human‑in‑the‑loop online post‑training system. TIDE introduces a provenance‑conditioned supervised learning framework that updates the policy online from human interventions and policy‑generated rollouts collected during deployment, without a learned critic, reward model, or action scorer. Across four real‑world bimanual manipulation tasks, TIDE improves milestone‑normalized task progress and binary full‑task completion over the warm‑start policy and matched additional offline demonstrations, and performs comparably to the fixed‑policy collection baseline while making updated checkpoints available during deployment. Each task‑specific online post‑training session, from the start of deployment rollouts until the final checkpoint is available, completes in less than one hour while learning and checkpoint handling run asynchronously.
Authors:Peize Li, Hanzhen Zhao, Xingguo Xu, Yilong Fan, Zhen Tao, Bo Yin, Zhifei Liu, Qingdong He, Xiaobin Hu
Abstract:
One‑step generative models enable fast text‑guided image editing, yet they often introduce unintended changes to regions that should remain unchanged. Temporal aggregation stabilizes editing fields across noise levels, but leaves their spatial allocation insensitive to local variations in reliability. Consequently, temporal stability alone does not ensure spatially consistent editing. To address this lim‑ itation, we propose ER‑Chord, a training‑free and inversion‑free framework for reliability‑guided spatial redistribution. Our key insight is that agreement between editing fields at neighboring noise levels provides a local reliability signal for allocating editing strength. We formulate this allocation as an equal‑energy op‑ timization that redistributes control energy according to local reliability while preserving the original editing directions and total control energy. This formu‑ lation admits a unique global solution that can be efficiently computed using existing field evaluations, requiring no additional denoiser calls. Experiments on two benchmarks, PIE‑Bench and TEdBench, demonstrate improvements over state‑of‑the‑art one‑step editing baselines, achieving superior source preservation alongside consistent gains across multiple generative backbones.The project page is at https://github.com/Peizeli1/ER‑Chord.
Authors:Xiujian Liang, Liuyuan Na, Zehao Du, Jiacheng Xu, Weixi Song, Mingyang Sun, Xingjun Ma, Cewu Lu, Jianhua Sun
Abstract:
Predictive representations for robot manipulation should connect scene geometry with the state changes induced by actions. Our intuition is that understanding what an action will change starts with understanding where things are: complementary RGB and depth observations ground a common scene representation from which to infer those changes. Guided by this intuition, we introduce DepState‑JEPA, a dual‑stream RGB‑‑Depth joint‑embedding predictive architecture that connects spatial understanding with action‑conditioned prediction through a shared latent state, DepState. Built on V‑JEPA~2.1, the framework uses depth both as a complementary observation stream and to ground both modalities in metric 3D space, aggregating shared information into DepState while retaining modality‑private features for decoding. Ray‑conditioned readouts predict local features without target depth; after an action‑conditioned transition, the same decoders predict future RGB and depth features using the predicted state and fixed initial private features. Robot‑state readouts provide recursive proprioceptive inputs; object‑trajectory supervision and calibrated forward kinematics shape state learning through motion targets and joint‑‑TCP consistency. Comparisons show stronger RGB‑encoder transfer after RGB‑D adaptation, feature completion, conditioned visual forecasting, and manipulation success. Within‑model ablations identify contributions from depth content, metric coordinates, and future‑state matching, with additional control gains from FK regularization. The project page is available at https://shawnleung‑sii.github.io/depstate‑jepa/.
Authors:Junjie Wang, Zhihao Yuan, Yunhan Tang, Sheng Xu, Jiaqi Mao, Xiaoyu Ye, Yueru Luo, Xu Yan, Chun-Mei Feng, Zhen Li
Abstract:
Vision‑Language Models (VLMs) excel at 2D recognition but often fail to maintain consistent 3D relations across sparse viewpoints, leading to spatial hallucinations under occlusion and perspective changes. Existing methods commonly fuse all frames with dense geometry, incurring substantial computation and context overhead while encouraging blind reliance on imperfect estimators. We present Think‑Capture, a query‑conditioned framework that first identifies task‑critical objects and then acquires their geometry on demand. A 3D toolkit lifts zero‑shot masks into a common coordinate frame, and a Symbolic 3D State Abstraction compresses object and camera states into structured text. Hierarchical Group Relative Policy Optimization (GRPO) trains the VLM on cases in which geometry either helps or misleads, teaching it when to use tool evidence and when to fall back on visual cues. Think‑Capture‑4B reaches 54.4% on MindCube‑Tiny (+21.5 points over its backbone), 39.1% on the MMSI‑Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2). It also remains robust under coordinate and mask perturbations. The full stack fits one 24 GB GPU and serves as an asynchronous high‑level reasoner on an Agibot G1. Our code is available at https://github.com/819346675/Think‑Capture.
Authors:Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan, ZhenXiang Yan, Huilin Xu, Haitao Yu, Lilin Wang, Rui Chen, Blaze Chen, Fengzong Lian, Renfen Hu
Abstract:
Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt‑level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensure a robust and realistic evaluation, we construct a bilingual (English‑Chinese) benchmark grounded in a comprehensive 84‑label safety taxonomy, consisting of two subsets: DiagBase, which covers single‑ and multi‑turn prompts with direct harmful queries and template‑based jailbreaks, and DiagAttack, which embeds harmful intents into complex instruction‑following tasks to form challenging instruction attacks. Extensive experiments demonstrate that DiagSafe significantly enhances the reliability of LLM‑as‑a‑judge evaluation, achieving 95.34% agreement with human annotations ‑‑ substantially higher than the 69%‑87% achieved by the baselines ‑‑ and high inter‑judge consistency (Cohen's κ = 0.864). Furthermore, applying our framework reveals that leading LLMs differ markedly in their unsafe rates, and each model also exhibits unique vulnerabilities along different dimensions. In addition, we observe that models are particularly susceptible in Chinese and multi‑turn contexts. Ultimately, DiagSafe provides a reliable and explainable tool for auditing and optimizing LLM alignment.
Authors:Rushab Munot
Abstract:
Decision models return normalized probability distributions over candidate choices, which downstream applications can use to select among choices. Although their providers emphasize the reliability of these probabilities, calibration on tasks with a single correct choice does not establish their reliability when multiple choices are correct. We evaluate three decision models (Jev, Liquid d1, and GPT‑6 Luna Decisions) on three diverse datasets (MetaTool, HotpotQA, and ECtHR) to study probability allocation and thresholding. Even when models rank every correct choice above every distractor, in 28% to 50% of those lists, at least one correct choice receives a probability below the uniform probability of 1/n, where n is the number of choices. We also present each model with five identical copies of a correct choice and observe probability disparities, with mean gaps ranging from 17 to 79 percentage points. We further show how these allocation disparities affect the recovery of multiple correct choices under probability thresholding. Decision‑model probabilities can rank correct choices, but a probability threshold does not identify the set of correct choices.
Authors:Jungseob Lee, Giyoon Jang, Chailin Nam, Jiwoo Kum, Sihyeon Lee, Kyojun Choo, Jihun Song, Eunseo Song, Chanjun Park
Abstract:
Speculative decoding uses a target model to verify a tree of draft tokens in parallel. We study whether the resulting next‑token distributions can help select the next tree. For unrestricted target models, we show that guaranteed exact reuse requires a distribution computed at the next round's complete context. We characterize when retaining this distribution requires sacrificing accepted tokens and establish conditions under which this loss persists across decoding rounds. Across six targets, we find that reranking first‑token branches with a separately computed distribution at the current context improves expected acceptance over draft ranking, especially with small tree budgets. Drawing the first token from this distribution before allocating the tree further improves expected acceptance over pruning a preselected tree, even at equal mean tree size. In our experiments, both selection rules also decode faster than draft ranking when the separate target pass overlaps standalone drafting. Our findings show that making target probabilities available before tree allocation can improve the efficiency of speculative decoding. Our code is available at https://github.com/js‑lee‑AI/TargetRows.
Authors:Md Ali Ashraf Yad, Nazifa Tabassum Nabila
Abstract:
Neural network models are increasingly constructed from synapse‑resolution connectomes, and the same circuit is now available in reconstructions from other animals and pipelines. We do not know whether such models are dependent on the reconstruction they are built from, and whether connectome differences are biological or measurement errors. We studied the Drosophila head‑direction circuit in nine versions: four flies (male CNS, hemibrain, Fly Wire, BANC) and five variants of the same fly varying only by thresholds, proofreading or synapse detection. Using wiring‑based correspondences validated by anatomical labels, we place all differences on one scale. A change of synapse detection within one fly changed the circuit more than halfway to the smallest inter‑fly difference on four distances; proofreading differences were smaller but depended on the metric, and left and right halves of one fly differed no more than a random half. Networks trained on each wiring to hold a cued heading developed near‑perfect ring attractors, but their error increased from below 1 degree to 10‑28 degrees when all weights changed by 1%. Training on each wiring did not transfer to any other connectome, including other versions of the same fly; refitting thirty cell‑type gains, we repaired filtering‑level differences but remaining error depended on wiring distance. Training with noise lowered precision and enhanced transfer. Imaging data indicate compasses drift smoothly and do not break; they function despite differences comparable to those of this study.Therefore, connectome‑constrained models need to reflect the reconstruction they stem from and the robustness normal circuits exhibit and models lack.
Authors:Yipeng Lu, Youqi Liao, Zhenyu Wang, Yuning Peng, Zhen Dong
Abstract:
The scarcity of high‑quality annotated visual data remains a critical bottleneck for autonomous‑driving perception. Simulators provide training data at scale, but the limited realism of synthetic videos compromises perception performance in real‑world applications. To address this limitation, we propose NeuralSRender, a sim‑to‑real video translation framework that enhances the photorealism of synthetic videos while maintaining geometric consistency with the source scenes. Built on a pretrained video diffusion transformer (DiT), NeuralSRender uses a lightweight projector to inject geometric and material priors from G‑buffers, and LoRA enables parameter‑efficient adaptation. The model is trained in two stages: it first learns G‑buffer conditioning from synthetic videos with accurate G‑buffers, then adapts to real‑world appearance using real videos with estimated G‑buffers. During the second stage, an additional LoRA module is trained while the projector and first‑stage LoRA remain frozen to reduce interference from estimation noise. NeuralSRender introduces only 1.45% parameter overhead and can be trained on consumer‑grade GPUs with 24 GB of memory. To support training and evaluation, we introduce GenReal, a large‑scale driving video dataset comprising synthetic and real‑world videos with corresponding G‑buffers. Experiments demonstrate improved photorealism with competitive geometric consistency. Training perception models on our translated videos instead of raw simulator renders improves detection \mathrmAP_50 on nuImages from 35.1 to 40.6 and segmentation mIoU on Cityscapes from 32.4 to 40.8. Our code, data, and model weights are available at: https://github.com/WHU‑USI3DV/NeuralSRender.
Authors:Eric Liang, Alan Luo, Kaiwen Yuan
Abstract:
Recent advancements in Vision‑Language Models (VLMs) enable end‑to‑end visual grounding via language prompts and generative decoding, demonstrating largely improved zero‑shot and open‑vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety‑critical real‑world systems. However, the physical adversarial robustness of edge VLMs has not been systematically studied, and the fundamental differences in output with modern object detectors create methodological challenges for fair evaluation. To compare the two families directly, we evaluate every selected representative model on one outcome: whether an adversarial patch worn on a person evades detection of the model. We report this by introducing a unified metric, conditional attack success rate (cASR), counted only on images the model still localizes under a matched gray occluder, and partition every failure into omission (no box), substitution (a box on the patch), and displacement (a box on neither). We benchmark the physical adversarial vulnerability of compact grounding VLMs against a range of object detectors, and find that the VLMs can be as vulnerable as the detectors. We disclose that failure mode is largely associated with the output format of a model rather than its architecture family. Of the three, only omission is visible to a downstream system; displacement and substitution return confident, well‑formed boxes, making them potentially more dangerous failures. Further, we find that, under distribution shift, VLMs that primarily failed by omission instead shift to failing more by displacement/substitution. Thus, a grounding VLM can inherit the detector's vulnerability while showing the possibility to adopt more dangerous failure modes. Our code and data are available at https://github.com/ericliang1/physical‑adversarial‑robustness.
Authors:Angela P. Schoellig, Matthias A. Müller, Sebastian Trimpe, Melanie N. Zeilinger, Mahathi Anand, Andrea Carron, Oliver Hausdörfer, Victor G. Lopez, Friedrich Solowjow, Haocheng Zhao, Siqi Zhou
Abstract:
Motivated by ten years of Learning‑Based Control Invited Sessions at the IEEE Conference on Decision and Control (CDC), this tutorial paper reviews key advances at the intersection of machine learning and control. We focus on methods that combine the flexibility of machine learning techniques with the structure and theoretical foundations of control to design autonomous systems that are capable, reliable, and safe. We first introduce core notions for establishing stability and safety guarantees, then review approaches that learn dynamics models to improve control under uncertainty, and finally discuss data‑driven methods that directly synthesize or learn policies, including reinforcement learning, imitation learning, and Bayesian optimization. We further discuss representative applications and open challenges. The goal is to provide an accessible entry point to the field and to support continued progress toward reliable learning‑based control in real‑world systems. To support learning and practical exploration, we share implementations and illustrative examples along with our discussion, together with additional resources on our website: https://learning‑based‑control.github.io/tutorial/.
Authors:Hoang Tran, Huong Ha, Tri Dang, Jeffrey Chan, Hongyu Zhang
Abstract:
Root cause analysis (RCA) and mitigation prediction are essential for incident management, enabling the identification of underlying causes of failures and ensuring the reliability of complex cloud‑based systems. With rising incident volume and system complexity, incident reports have become a valuable resource for these tasks. Recent studies have shown promising results using Large Language Models (LLMs) to predict root causes and mitigation actions from textual descriptions in incident reports. Despite these advances, further progress is hindered by the lack of publicly available datasets; existing work relies on private data, which limits transparency, reproducibility, and generalization. To address this gap, we introduce CloudIR‑Bench, the first publicly available benchmark providing a dataset and evaluation framework for incident report‑based RCA and mitigation prediction. Curated from incident reports across five leading cloud‑based systems, CloudIR‑Bench captures a diverse range of real‑world production scenarios representing authentic failures and engineers' reasoning. We also release an open‑source evaluation framework that includes eight recent LLM‑based approaches. Our comprehensive evaluation includes zero‑shot and in‑context learning experiments, along with a qualitative analysis to identify where and why LLMs fail. Benchmarking results reveal the capabilities and limitations of existing LLMs in this domain. The benchmark, including the dataset, framework and supplementary materials, is available at https://github.com/hoangtnm/CloudIR‑Bench.
Authors:Jiachen Guo, Xinshan Zhu, Boyun Wang, Yanyan Liang, Jun Wan, Feng Lin, Ajian Liu
Abstract:
Multimodal large language models (MLLMs) have made rapid progress in general visual understanding, yet they still struggle with fine‑grained instance‑level consistency verification, where the goal is to determine whether two images depict the exact same physical object under varying viewpoints, illumination, and backgrounds. This task requires both global semantic perception and precise examination of subtle local evidence, since semantically similar objects may differ only in small suspicious regions. To address this challenge, we propose LocalAgent, a collaborative agentic framework that integrates the complementary strengths of a generalist MLLM and lightweight specialist verifiers. The MLLM first reasons globally to identify suspicious local regions, which are then delegated to a Local Evidence Verifier (LEV) toolkit for fine‑grained appearance and geometric matching. LEV comprises Local Appearance Consistency (LAC), which estimates appearance similarity under illumination and color changes, and Local Geometric Correspondence (LGC), which estimates spatial correspondence under viewpoint changes. LEV returns structured scalar evidence that the MLLM integrates for the final decision. We further introduce an auxiliary‑capped reinforcement fine‑tuning (RFT) strategy, in which answer correctness dominates auxiliary tool‑use and format signals, enabling the policy to learn when verification is necessary. Experiments show that LocalAgent improves mean accuracy by 4.9% over the strongest CoT‑RFT baseline while invoking a verifier on only 12.13% of samples and preserving the general multimodal capabilities of the underlying MLLM. The open‑source repository is available at https://github.com/Firefly‑zoo/LocalAgent‑main.
Authors:Chun Gu, Juntong Zhang, Jingyu Li, Li Zhang
Abstract:
Retargeting human motion to humanoids requires preserving motion semantics while accounting for the robot's physical constraints. Kinematic pipelines prescribe a robot trajectory through geometric matching, without establishing whether it can be executed under the robot's dynamics. Prior physics‑supervised neural methods train bidirectional full‑sequence models on physics‑refined kinematic trajectories, remaining tied to prescribed kinematic solutions and requiring noncausal context. We propose SMR, a streaming neural motion retargeter learned from semantic‑conditioned physics experts. We convert human motion into robot‑aware task‑space targets for body landmark positions, orientations, and velocities. Physics experts track these targets directly in simulation under contact, dynamics, and actuator constraints. We train a single strictly causal Transformer on their executed state trajectories that complete the motion and pass semantic and physical validity checks. At each incoming frame, the student uses only current and past human observations to generate a robot motion reference for a downstream whole‑body controller. Experiments demonstrate that semantic‑conditioned physics experts generate motion with higher semantic fidelity and physical validity than direct optimization‑based retargeting, while the resulting streaming retargeter achieves improved closed‑loop performance over kinematic and physics‑refined baselines under a common whole‑body controller.
Authors:Zhangchi Hu, Zilong He, Yifan Zhao, Xiangyu Yu, Jie Chen, Peixi Wu, Yansong Peng, Hebei Li, Xiaoyan Sun
Abstract:
RiO‑DETR is the first real‑time end‑to‑end transformer for oriented object detection, yet further localization gains require a more expressive box representation. Although distributional regression has succeeded in horizontal DETRs, extending it to oriented detection introduces geometry‑misaligned boundary modeling and topology‑inconsistent orientation refinement. We present RiO‑DETRv2, a geometry‑aware uncertainty modeling framework for real‑time oriented object detection. To resolve the geometry mismatch, Object‑Aligned Boundary Distribution Refinement represents box boundaries as probability distributions in an object‑aligned coordinate system and progressively refines them across decoder layers. For better orientation topology consistency, we introduce Circular‑to‑Local Angular Residual Refinement that transitions from global periodic search to local fine‑grained estimation. Unified OBB Quality‑Aligned Learning further aligns classification, boundary and angular refinement with complete‑OBB localization quality. Together, these designs improve localization without compromising efficiency, achieving a better balance between speed and accuracy. Specifically, RiO‑DETRv2‑l/x achieves 53.7/54.1 mAP50‑95 on DOTA, outperforming RiO‑DETR‑l/x by 1.2/1.5 points with nearly unchanged computational cost. Code is available at https://github.com/RicePasteM/RiO‑DETRv2.
Authors:Austin Meek, Kyle Cox, Matthew Nguyen, Austin J. Brockmeier, Steven Basart
Abstract:
Reward hacking is a major problem in current production machine learning systems, resulting in costly and dangerous failures. Here we study how to monitor for and mitigate reward hacking in realistic settings, including tool calls and long contexts. We apply a dynamic intervention to steering directions obtained by three different methods: difference‑in‑means vectors from labeled rollouts, gradient optimization‑based vectors, and vectors extracted when models reference an anti‑reward‑hacking model spec. This dynamic intervention perturbs model representations less than uniform steering vector addition to the residual stream, allowing for more precise edits. We find that we are able to cheaply and significantly reduce reward hacking without correspondingly significant costs to model coherence or capability, and that steering vectors originally extracted from coding environments also generalize to reducing reward hacking in other domains. Our results pave the way towards latent space monitoring and control over realistic production systems.
Authors:Derong Yu, Qingsong Zhao, Yanbing Yue, Yikai Li, Zhi Zhang, Guoyan Zheng
Abstract:
One‑shot landmark localization in volume scans with varying fields of view (FOVs) is challenging. While existing self‑supervised learning methods have shown promising performance, they rely on contrastive learning built upon augmented views of the same image, which inherently limits their generalization in cross‑modality scenarios. This motivates research on cross‑modality embedding learning using paired multi‑modality images. However, existing methods suffer from not only limited training data, but also the requirement of multi‑stage iterative training. To address these limitations, we propose UNIVERS, an end‑to‑end trainable framework that learns UNIversal Voxel‑wise Embedding via Randomized Synthesis. Our framework leverages diverse pseudo‑data synthesized from organ shape templates, with randomization applied to semantic layout, visual appearance, FOV, and spatial pose. Coupled with cross‑domain discriminative voxel‑wise contrastive learning, this domain randomization strategy forces the network to disregard domain‑specific biases and learn features that are robust against variations in imaging contrast, resolution, and spatial context. Consequently, the trained model enables one‑shot landmark localization in multi‑modality volume scans with arbitrary FOVs. In this study, the efficacy of the proposed method is demonstrated by applying it to the task of localizing the adrenal glands, which are paired, small, and irregular endocrine organs. In particular, comprehensive experiments are conducted on three challenging adrenal gland datasets. The experimental results demonstrate that UNIVERS consistently outperforms the state‑of‑the‑art (SOTA) one‑shot landmark localization approaches, with particularly prominent advantages in cross‑modality scenarios. The source code is publicly available at https://github.com/Kendra9999/UNIVERS.
Authors:Truong-Son Hy
Abstract:
Accessible research software can connect published antimicrobial peptide (AMP) records to sequence analysis, molecular visualization, and interactive design without requiring a dedicated application server. We present AMP Atlas, an open‑source, browser‑only workbench that combines a DRAMP‑derived database with local physicochemical calculations, published‑neighbor retrieval, and sequence‑guided structural comparison. The reported snapshot contains 12,784 source records, 10,579 canonical records, 5,183 source SMILES annotations, and 433 PDB chain structures. An inverse helical‑wheel editor lets users specify peptide length, fill residues gradually, and immediately retrieve exact‑length database matches under positional and property constraints. Selected‑position residue frequencies support further editing while preserving the remaining constraints. A Web Worker performs searches and numerical calculations; a WebGL viewer provides interactive molecular representations. The static distribution occupies 138.8 MB and supports GitHub Pages hosting. Verification includes 23 Python tests, 30 browser‑core/worker checks, a designer DOM integration suite, and agreement against fixed numerical references. This report describes the implemented algorithms and assumptions; it does not establish predictive accuracy for antimicrobial activity or experimentally validate designed peptides. The website is available at https://hysonlab.github.io/amp‑atlas/ and the source code and database are available at https://github.com/HySonLab/amp‑atlas
Authors:Genpei Zhang
Abstract:
Amodal video segmentation asks for the complete mask of an object in every frame, including the parts hidden behind other objects. The best current methods generate these masks with fine‑tuned video diffusion models, which need many denoising steps and an extra depth network, and give a different answer for every sample. Self‑supervised video world models of the JEPA family are trained on a closely related task: predicting the representation of masked space‑time regions from the visible ones. We therefore treat occlusion as masking. The tokens that the object's visible mask does not cover are dropped, the pre‑trained V‑JEPA 2.1 predictor fills them in, and a light head decodes the observed and predicted features into amodal masks. The resulting model, OccJEPA (Occlusion‑as‑masking JEPA), is deterministic and 33.5x faster than the diffusion‑based state of the art, and more accurate on SAIL‑VOS, on TAO‑Amodal, including a held‑out test split (+4.00 AP50) and visible masks tracked by SAM2, and on Kubric. Controlled experiments trace the gain to the predictor's pre‑trained weights rather than to its capacity, for world models from 87M to 1.85B parameters. Code is available at https://github.com/zgp110/occjepa.
Authors:Oleksandr Kholodniak, Oleksandr Prokhorov
Abstract:
Dropout and zoneout are routinely added to long short‑term memory (LSTM) and gated recurrent unit (GRU) networks, but, to our knowledge, their placements have not been crossed with gate initialisation in a factorial design, and it is not known whether early gradient propagation tells which runs will learn long dependencies. We trained one‑layer LSTMs and GRUs on the adding problem and on delayed recall in a factorial grid of 720 runs, crossing conventional and chrono gate initialisation with up to six recurrent regulariser placements at one rate each, with pre‑specified follow‑ups, and recorded how sensitive the final hidden state was to the input at every lag. Because both tasks supply fresh or exhaustive training data, the study measures the regularisers' effects on optimisation and does not assess generalisation from a finite training set. At lengths chosen to make conventional initialisation difficult, all 360 chrono‑initialised runs reached the validation criterion within 5000 updates, as did 119 of 120 unregularised runs with a horizon about four times too short or too long, whereas conventionally initialised networks learned slowly or did not reach the criterion within the budget at the longer lengths. Under conventional initialisation, dropout generally delayed learning and zoneout helped only in some conditions. The early input sensitivity added little to predicting which conditions learn once the early validation loss was modelled per task, architecture and initialisation. Within conditions, a flatter early decay went with the seeds that learned, but in a pre‑specified test the association was modest (concordance 0.61), condition‑dependent and not robust to resampling seeds. The results support chrono initialisation, even with a rough horizon, on such tasks.
Authors:Yongsheng Lian
Abstract:
Estimating food mass from a photograph is central to image‑based dietary assessment, but whether trained models transfer to new cameras and foods is unclear. We fine‑tuned DINOv2‑base on 2,253 weighed Nutrition5k dishes to predict mass from one RGB photograph and compared it with a zero‑shot vision‑language model, Qwen3‑VL‑32B, on Nutrition5k and three external datasets: ASA24 studio portions, ECUSTFD whole foods and Food Portion Benchmark meal trays. In domain, the fine‑tuned model's mean absolute error was 24.8 g, less than half of Qwen's. On external datasets it did not beat Qwen. For a food it had never seen, it ranked portions by size, but no better than a count of food pixels. Its gram scale was wrong in two ways: it changed with how much of the scene the photograph showed, and it was off by a different factor for each food. Training on other foods or trays from the tested dataset cut the external error by 38 to 65%. DINOv2‑large, a larger backbone trained on the same dishes, did not reduce the external error. One labeled photograph of the same food cut the model's ASA24 error to 28.3 g, better than that photograph achieved alone. Prediction intervals that held the true mass for 90% of Nutrition5k dishes held it for at most half of external photographs. Nutrition5k often photographed the same plate several times, so a random test split shared near‑identical photographs with training and overstated accuracy. Single‑image mass estimators therefore need labeled photographs from the setting where they are used.
Authors:William Huang, David Li, Xiang 'Anthony' Chen, Xun Qian, Nels Numan, Zhongyi Zhou, Hyunsung Cho, Salman Muin Kayser Chishti, Ding Xia, Xincheng Huang, Jiahao Ren, Faraz Faruqi, Tianyu Xu, Yang Zhang, David Kim, Ruofei Du
Abstract:
Coding agents are rapidly advancing from single‑file generation to long‑horizon repository‑scale engineering, yet the evaluation of their capabilities to author interactive extended reality (XR) applications remains an open challenge. Unlike traditional software, XR applications operate within dynamic 3D physical environments where runtime robustness depends jointly on continuous user motion, spatial input, and environmental sensing. We introduce GenXR‑Bench, the first benchmark for the end‑to‑end evaluation of coding agents in XR software development. GenXR‑Bench contains 112 expert‑authored tasks spanning 24 feature implementations, 21 bug repairs, and 67 application generation tasks. Held‑out tests combine scripted and adaptive interactions with programmatic and visual assertions to evaluate core functionality and runtime behavior. GenXR‑Bench's testing suite achieves 96.5% accuracy against blinded expert judgments of 124 candidate applications executed on consumer XR hardware. Evaluating 10 models on GenXR‑Bench, we find that the strongest configuration achieves only 50.9% overall Pass@1. Failure analysis identifies recurring misunderstandings in asynchronous sensor lifecycles, physical grounding, and embodied interaction semantics. GenXR‑Bench establishes a reproducible testbed for evaluating coding agents in the interactive and physically situated conditions of spatial computing.
Authors:Jungseob Lee, Hogun Seong, Euijun Choi, Sugyeong Eo
Abstract:
Speculative decoding accelerates large language models by verifying drafted tokens in parallel. New draft‑training objectives are usually credited with the acceptance they add over cross‑entropy on externally written responses, yet many pipelines let the target rewrite them. We ask how much of that gain survives. With a vision‑language target and a block drafter, we train three objectives on four versions of one corpus: cross‑entropy on corpus tokens, token‑level distillation from the target, and distillation gated to positions that would survive verification. Distillation and the gate add large gains on external responses, but after greedy regeneration, distillation adds a smaller gain and the gate little or none. The shrinkage persists across data orders, position weights, verifiers, decoding temperatures and, for distillation, initializations and learning rates. Most of it remains at matched supervised length and at each objective's best measured training length. A second vision corpus shows the shrinkage, as does a text‑only replication, for distillation also with an autoregressive drafter. Objective gains should therefore be measured against a target‑regenerated baseline and across training lengths. Our code is available at https://github.com/js‑lee‑AI/TargetRegeneration.
Authors:Md Khalid Syfullah, Alvi Ataur Khalil
Abstract:
Clinical registries and wearable sensors support early health‑risk assessment, yet models trained on these records may leak sensitive information when released. Differential privacy (DP) bounds this leakage by injecting noise whose total magnitude grows with the number of trainable parameters, a cost especially severe for small health cohorts. This motivates parameter efficient models including quantum machine learning (QML), whose variational circuits can represent complex decision boundaries with few trainable parameters. It raises a question: at matched input dimensionality and privacy budget, does a quantum model retain more utility than a classical one? We investigate this question on three clinical cohorts and one wearable cohort, treating each clinical record or participant as a privacy unit. We compare capacity‑matched quantum, multilayer perceptron, and logistic regression models across five privacy budgets and five seeds. The quantum model leads on three clinical cohorts by up to 0.104 ROC AUC averaged across budgets; however, trails on the wearable cohort, where the small number of participants requires substantial noise. No difference survives Holm correction across 24 tests, so these results indicate a trend rather than a quantum advantage. An analytic gradient bound shows that lower clipping thresholds can improve utility by reducing noise despite clipping bias. Finally, an experiment on IBM quantum computer (ibm_fez, a 156‑qubit superconducting processor) using eight fixed test cases yields mean absolute probability errors of 0.046 (private) and 0.063 (non‑private) relative to simulation, supporting the feasibility of inference on noisy intermediate‑scale quantum (NISQ) hardware. Code is publicly available at https://github.com/TITANS‑Lab‑Artifacts/DP‑QML‑Health.
Authors:Md Khalid Syfullah, Alvi Ataur Khalil
Abstract:
As quantum computing moves from the noisy intermediate‑scale quantum (NISQ) era toward early fault‑tolerant quantum computing (EFTQC), quantum‑enhanced AI is shifting from a conceptual idea to experimental research. Quantum components are expected to extend the capabilities of existing AI systems, yet it remains unclear how such components will help within large multimodal models. In this study, we try to answer that question for vision‑language models (VLMs) using classically simulated quantum layers. With LLaVA‑OneVision as the backbone, we evaluate 4‑ and 8‑qubit basic‑ring and strongly entangling circuits at four insertion and four replacement locations. Insertion adds quantum layers to the classical model, while replacement substitutes them for classical components. Using the same setup, we compare performance and computational cost between classical and quantum‑classical hybrid models. On the Visual Spatial Reasoning dataset, inserting a quantum layer after the visual projector reaches 80.7% development accuracy, 2.28 percentage‑points above classical full fine‑tuning. Replacing the final decoder feed‑forward sublayer retains 79.22% accuracy with 13.07 million fewer trainable parameters. Quantum insertion can thus improve accuracy, and selective replacement can reduce parameters. These benefits depend strongly on where the quantum layer is inserted or which classical component it replaces: earlier replacements lower accuracy, and fewer parameters do not reliably shorten training or reduce computation. To assess potential security benefits, we evaluated the robustness of quantum‑classical hybrid models under six attack settings. Overall, our study maps where quantum layers help and hurt within VLMs, offering a starting point for hybrid VLM design; whether these gains generalize remains open. The code is available at https://github.com/TITANS‑Lab‑Artifacts/QuantumVLM.
Authors:Sergey Salishev, Zeev Volkovich, Oleg Granichin
Abstract:
Transformers are deployed as a special model class. Neural processing units (NPU) add dedicated operators for attention and normalization, and quantization relies on Transformer‑specific algorithms. We show the computation does not force this. Every Transformer block, with attention, LayerNorm and a GELU MLP, compiles exactly into a generic deep neural network built from fixed affine maps, routing and the square, sigmoid and inverse square root, plus GELU or its sigmoid approximation. Without activation‑activation products, every transform‑wise compilation adds at least N^2 scalar nonlinear evaluations per head for dense scores. Keeping that one operation type removes this cost; its elementwise form, present on every surveyed NPU, suffices with the sigmoid and inverse square root. For top‑k attention with exchangeable tail scores, we prove that exact or conditional‑mean tail mass gives an unbiased, minimum‑mean‑square‑error estimate of the attention distribution, while tail drop is conditionally biased. When queries or agents cluster, centroids choose the keys first, reducing the number of computed scores from N^2 to Nk per head. On DeiT‑Tiny, MambaIRv2‑Light and Pythia‑70M the compiled models match their references with all keys, standard post‑training quantization applies, and the exact tail‑mass estimate beats tail dropping with half as many keys on DeiT‑Tiny and, for k\le8, on Pythia‑70M.
Authors:Gonzalo Gómez-Nogales, Daohan Lu, Peiye Zhuang, Tinghui Zhou, Yi Zhou
Abstract:
Coarse‑to‑real (C2R) video generation converts coarse animation, simulation, or game‑rendered videos into photorealistic videos while preserving scene layout, camera motion, and dynamics. Training such models requires aligned (coarse, realistic) video pairs, which are expensive to create and confine prior systems to narrow synthetic domains. We exploit a simple asymmetry: paired videos are hard to obtain, but paired real‑to‑coarse images are easy to construct with existing editing and 3D reconstruction tools. Using these image pairs, we train the DINO‑R2C model to transfer realistic images to coarse images in DINO feature space, and apply it frame‑wise to ordinary real videos to synthesize pseudo‑coarse videos. The C2R# model then learns to generate each real video from its pseudo‑coarse counterpart, free from needing (coarse, realistic) video pairs. At inference, the DINO‑R2C model is discarded and actual coarse videos drive the C2R# model directly. Thus, our pair‑free training scales with any real‑video corpora rather than with paired synthetic videos. We will release the model and project webpage at https://gonzalognogales.github.io/coarse2real‑sharp/.
Authors:Audrey Lee, Nathaniel Dennler, Ilia Sucholutsky, Andreea Bobu
Abstract:
Trajectory demonstrations show robots how to behave but not why, leaving them to guess which features of the task actually matter. Incorrect guesses produce misaligned internal representations and lead to incorrect robot behaviors. Because task features are often semantically defined, we argue that natural language is better suited for specifying features rather than relying on demonstrations. Although existing representation editing methods have proposed using language to specify new features, only allowing additive edits produces feature sets that are often over‑specified or too narrowly scoped, especially in sequential editing tasks. We propose the Flexible Feature Editing Pipeline (FLEX), which uses language models to iteratively refine a robot's internal representation of task features from natural language corrections via three complementary operations: Add missing features, Generalize existing specific features, and Mask irrelevant features. We evaluate FLEX on 2D navigation scenarios, 7DoF Franka tabletop manipulation tasks, and a real‑world study using naturalistic human language corrections and real‑robot demonstrations. Comparisons against single‑operation and unscaffolded baselines show that all three operations are necessary in interactive scenarios, and that FLEX achieves consistently higher normalized reward compared to other feature editing approaches while maintaining compact, transferable representations. Project available at https://mit‑clear‑lab.github.io/FLEX/
Authors:Anrui Gu, Zhichao Wang, Nicolai Ouporov
Abstract:
Many social evaluations for multiagent systems study behaviors as they emerge from environments instead of systematically intervening in the environment to elicit such behaviors. We introduce Fig (Finding Interactions in Games), an agentic system that performs targeted elicitation interventions on game environments. It takes a natural language behavior description, decomposes it into sub‑behaviors, generates new social games or selects existing ones, and executes single or combined interventions to the games' engine, opponents, instructions, and context. To evaluate model behavior, it combines built‑in mechanical checks and synthetic rubric judgments. We validate the interventions, generated games, and evaluation mechanisms via human review. Models attempt the target behavior in 45.2% of episodes on average for valid scenarios generated by Fig for six test behaviors ‑‑ cooperation, exploitation, deception, covert coordination, betrayal, and sabotage. For game‑behavior pairs with available controls, Fig's interventions increase attempt rates by 10.9% on average compared to matched control games. Fig shows that multiagent environments can serve as programmable test beds for intentional, controllable, and reproducible behavior evaluation. Code is available at https://github.com/fleet‑ai/fig.
Authors:Matthew Strong, Ankush K. Dhawan, Julien Poffet, Giuse Pham, Jinho So, Monroe Kennedy
Abstract:
Human video scales easily and does not need robot hardware, but existing methods require depth sensing, overlay the robot onto 2D video, or generate data only for the demonstrated objects. We present Real2Sem2Real (R2Sem2R), a real‑to‑semantic‑to‑real pipeline that turns one monocular RGB video of a human demonstration into thousands of robot demonstrations across the semantic object instances of a category. R2Sem2R tracks the SE(3) pose of each object with DINOv3‑distilled Gaussian splats, outperforming existing methods and FoundationPose with estimated metric depth. R2Sem2R then aligns digital cousins, each generated from one or a few photographs, to the demonstrated objects with a semantic alignment. This alignment matches whole objects and the functional regions of target objects, such as a coffee machine's tray. R2Sem2R maps the human hand to a feasible robot grasp that extends to the cousins. Robot demonstrations are generated either in IsaacSim, for photorealistic rendering, or MuJoCo, for contact physics, and R2Sem2R can generate over 2,000 episodes in about two hours. On two real robots and four tasks, we train a vision‑language‑action policy on data from one human video per task, and policies trained on our data match or exceed policies trained on teleoperation across diverse object instances. The project page is available at https://real2sem2real.github.io/.
Authors:Aur Shalev Merin
Abstract:
Looped transformers reuse one block of layers many times and are often trained by backpropagating through only the last k loops. We measure how much of the weight gradient this throws away, exactly and for every k from one backward pass, on nine looped language models, and report three findings. First, on a trained model this lost gradient is set near the output: once the loop count is well above k, it barely changes with the loop count (within about 0.03 from 32 to 64 loops on the three models we tested, as a pre‑registered test predicted for Parcae). Where the backward signal shrinks, ρ^k, with ρ the signal's fitted shrink per loop, approximates it at moderate loop counts but overstates it at long ones, because the fitted ρ drifts up as the fit takes in more loops. On Ouro the signal grows, and keeping two of four loops still loses 81% of the final output's gradient. Second, the lost gradient falls during fine‑tuning, by 0.07‑0.22 within 300 steps on three of four models, on held‑out problems too. The fall is not just the read‑out's share of the gradient; training only the layers outside the loop reproduced up to half of it, and it also falls under truncated training. So a trained model's value overstates what truncation removes later. Third, we could not show that it predicts the training cost of truncation better than rules that need no measurement: on two new models it tied Parcae's cost at the same k, in a test too small to separate them.
Authors:Eric Yocam, Varghese Vaidyan, Paris Kalathas
Abstract:
Adoption of artificial intelligence in K‑12 schools has outpaced guidance for students, teachers, and administrators. Existing AI‑literacy frameworks emphasize understanding and use; cybersecurity frameworks predate AI‑specific threats, and neither defines AI security literacy as a competency domain in its own right. We propose an AI Security Literacy (AISL) framework of six dimensions spanning the school ecosystem, ground its technical dimensions in public AI‑safety benchmarks and a reproducible multi‑seed adversarial evaluation, and ground its attitudinal dimensions in nationally representative survey micro‑data from U.S. teachers and adults. Only 34.5% of teachers report a school AI policy, and teachers see their own AI use as easing their work but students' use as complicating it; on the technical side, a reported robustness figure depends as much on the attack and the particular trained model as on the system evaluated. NeuralFortify, a browser‑based K‑12 platform built on real AI models, is presented as an illustrative operationalization.
Authors:Gabriel Wedson Mendonça de Souza Veloso, Luan Borges Teodoro Reis Sena, Denis Oliveira Correa, Francisco Galuppo Azevedo
Abstract:
Progress on foundation model pretraining is gated by who can afford a trial, and relational pretraining is no exception. Recipes are developed inside individual labs, reported under private protocols, and do not accumulate. Speedrun leaderboards remove that barrier by decentralizing the search rather than the training run. The problem is fixed, the stack is open, entries are ranked by wall‑clock time to a quality target, and one GPU is enough to hold the record. We introduce modded‑nanoRT, the first such leaderboard for Relational Transformers. Which database to hold out decides whether the benchmark can resolve improvements at all, so we fix it by calibration, ranking six candidates by how sharply a 45‑minute learning curve separates improvement from seed noise. We open the leaderboard with typed trilinear attention, a relational operator that conditions attention scores and values on the foreign key traversed, together with Muon and SIGReg. The recipe reaches the target in 305.63 seconds on one NVIDIA H200, 8.11x faster than the reference implementation, and wins 13 of 15 pairs outside the development database. We will release the benchmark, its frozen artifacts, and the record runs. Our code is available at https://github.com/kunumi/nanoRT.
Authors:Zheng Qin, Junhua Xi, Yan Huang, Jiahong Lai, Anwen Huang, Qiong Li, Guangda Zhang, Kai Xu
Abstract:
We present Register Anything Model (RAM), a unified model for generalizable 3D point cloud registration that robustly estimates 6‑DoF transformations across diverse domains spanning sensors and environments. Existing registration methods are typically tailored to a single domain, with carefully tuned architectures and hyperparameters, and thus degrade markedly when applied to novel domains. We bridge this gap by training a single model on diverse data from indoor, outdoor, and object domains. A central challenge is the severe geometric discrepancy across domains, i.e., differences in scene scale, sensor noise, point density, and structural patterns, which hinders processing with a unified point‑based model. To address this, we introduce a geometry‑aware normalization strategy that jointly voxelizes and normalizes point clouds into a standardized geometric space, adaptively based on their geometric properties to preserve salient local structures while standardizing resolution. Built on this normalization, RAM follows a hierarchical coarse‑to‑fine paradigm, and employs an efficient geometric transformer that models local geometric consistency instead of globally over all points. This design mitigates discretization noise introduced by voxelization, and improves computational efficiency. Extensive experiments on 12 benchmarks demonstrate state‑of‑the‑art performance in terms of inlier ratio and registration recall. Notably, our method outperforms the counterparts trained on each separate domain, showing superior generalization capability.
Authors:Kaichen Zhou, Yuzhen Chen, Hannah Yao, Chonghe Jiang, Rao Fu, Xihang Yu, Wei Dai, Bashar Kabbarah, Haoyu Chen, Ishita Goluguri, Zhiyang Dou, Haonan Chen, Fangneng Zhan, Mengyu Wang, Wojciech Matusik, Antonio Torralba, Paul Pu Liang
Abstract:
In contact‑rich manipulation, contact conditions can change before a visual policy updates its action plan. Recomputing the plan through the visual policy adds latency, while tactile feedback alone does not specify how the ongoing motion should be revised for the task. We introduce T‑WLA, a world‑language‑action architecture that uses shared visual memory to guide both action generation and tactile correction. At each visual update, the visual branch integrates the current image, task instruction, and retrieved history into a visual memory that conditions base‑action generation. The same memory is cached for a lightweight tactile editor, which combines it with the active action chunk and incoming touch to predict residual corrections to unexecuted actions without rerunning the visual branch. Future‑image and future‑pressure prediction provide training‑only supervision, encouraging the visual memory and tactile features to capture interaction dynamics relevant to action generation and correction. We evaluate T‑WLA on tactile‑conditioned hand‑motion prediction using EgoTouch and Tachin, and on four real‑world dexterous manipulation tasks. T‑WLA reduces finger‑joint MPJPE by 30% relative to the strongest reported baseline and achieves an average success rate of approximately 50.0% across real‑robot manipulation tasks. Videos and code are available at https://anonymous‑submission‑20.github.io/T‑WLA.github.io/.
Authors:Wendy Zheng, Liang Wu, Yinhan He, Mayank Darbari, Jundong Li, Liangjie Hong
Abstract:
Agentic systems have demonstrated strong capabilities on complex tasks by enabling multiple interacting large language model (LLM)‑based agents to collaborate and execute subtasks in parallel. However, this collaborative structure introduces new challenges for reliably evaluating and diagnosing system failures, particularly failure attribution: tracing a system's execution to identify the agent step responsible for an incorrect final outcome. Because manually performing this task is time‑ and labor‑intensive, recent work has turned to LLMs as judges to automate the process, referred to as automated failure attribution (AFA). These approaches assume that a sufficiently capable model can infer the failing step directly from an interaction transcript. In practice, however, direct prompting often performs poorly, which motivates follow‑up work that explores various ways to improve AFA performance, including alternative prompting strategies and inference‑time interventions. In this paper, we examine a more fundamental question: whether this weak behavioral performance reflects a genuine capability limitation. Using linear probes applied to internal activations, we uncover a significant \emphknows‑but‑won't‑say gap: models encode sufficient information to attribute a failure correctly but fail to explicitly output the responsible step. Building on this finding, we propose \emphgrafting, a simple manual intervention that exploits knowledge already encoded in the model to improve AFA performance without requiring ground‑truth labels at inference time. Extensive evaluations across multiple LLMs and benchmark settings demonstrate that grafting consistently outperforms direct‑prompting approaches, underscoring the potential of leveraging latent model knowledge for more reliable failure attribution. Our code is available at https://github.com/wendyzheng22/mi4afa.
Authors:Pedro H. Barcha Correia, Matheus Azevedo de Sá, Paulo H. Gomes de Senna, Giovanni Mandel Martignago, Victor Takashi Hayashi, Charles Christian Miers, Marcos A. Simplicio
Abstract:
The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks. By embedding malicious instructions, these exploits manipulate language models to bypass intended behavior and operational boundaries. In traditional chatbots, such attacks are confined to a single session because state memory is not preserved across conversations. However, persistent memory features in modern conversational interfaces enable cross‑session attack propagation, establishing a threat vector known as memory injection. This work explores this emerging vulnerability in leading LLM‑as‑a‑service platforms, highlighting its real‑world severity and the need for countermeasures. While existing literature focuses on violations tailored to specific platform features, we present a general threat model. We demonstrate that all NIST attack goals are collectively achievable across ChatGPT Instant 5.3, Claude Sonnet 4.6, and Gemini 3 Flash under this model. We also show that memory injections can exploit multiple attack vectors, including personal memories, projects, skills, organization instructions, and Model Context Protocol (MCP)‑connected resources. Our findings reveal that all evaluated services can be manipulated to persistently spread disinformation, generate harmful content, or simulate offline unresponsiveness, even in fresh sessions. In the particular case of Claude, shared‑memory features allow an attacker to compromise multiple users within an organization and also exfiltrate user prompts. To address these vulnerabilities, we derive defense mechanisms from our proposed threat model and foundational cybersecurity principles. Our analysis indicates that nearly half of these mitigations remain unimplemented across the target services. Datasets and attack artifacts are available at https://github.com/PedroBarcha/Memory‑injection.
Authors:Piotr Keller, Fayyaz Minhas
Abstract:
In vision and scientific domains involving spatial data, a key question is whether node feature values vary independently across space or form coherent spatial structures. Classical spatial autocorrelation statistics such as Moran's I and Geary's C quantify this phenomenon but operate on a single scalar variable with fixed neighbourhood weights, limiting their use for high‑dimensional graph‑like data. We introduce SAIL (Spatial Autocorrelation Index Learner), a differentiable framework that generalises spatial autocorrelation into a learnable operator by jointly learning feature projections defining candidate scalar fields and inter‑node neighbourhood affinities. SAIL produces interpretable local and global spatial coherence descriptors, enabling supervised prediction and unsupervised discovery of spatial autocorrelation modes and hotspots. We show that SAIL exactly recovers classical spatial statistics under appropriate parametrisations and, for tasks driven by latent spatial autocorrelation, can approximate the Bayes‑optimal predictor with an inductive bias matched to such tasks. Experiments on synthetic data and large‑scale biomedical graphs demonstrate statistical correctness and discovery of spatial coherence with applications across vision, geospatial analysis, and biomedicine. The code repository is available at: https://github.com/pkeller00/SAIL.
Authors:Jiahan Li, Huangyuan Su, Shengbang Tong, Shusheng Yang, Ruihan Guo, Suozhi Huang, Hexu Zhao, Pratyusha Sharma, Sherry Yang, Jiatao Gu, Rob Fergus, Kyunghyun Cho
Abstract:
Most unified multimodal models treat images and text asymmetrically: a pretrained language model generates text, an attached diffusion head generates images, and outputs that contain both are produced in a fixed order, so the first output cannot adapt to the second. We ask whether both modalities can instead be generated by one process that is trained on them jointly from scratch. Pretrained encoder spaces hold both modalities in one continuous form and carry knowledge learned at scale, and recent work finds them effective as generative targets for vision and language. We therefore model the joint distribution directly in these spaces. We introduce JUICE (Joint Unified multimodal modeling In Continuous Encoder spaces), a single flow matching Transformer over frozen image and text representations, in which each modality has its own noise level. Which modalities are clean and which are generated then decides the task: image generation, captioning, visual question answering, or joint generation. Controlled pretraining runs test each design choice. At equal guidance, generating text as continuous states leaves image generation intact (48.41 GenEval, against 47.86 for a model trained only for text‑to‑image generation), while captions lose some exact wording. Training on joint and conditional denoising together beats either alone, and semantic encoder spaces beat VAE latents and raw pixels as targets. At 2.1B parameters, with no language model initialization and at 224x224, JUICE reaches up to 0.88 on GenEval and 73.1 on VQAv2. By sampling both modalities from the joint distribution (coupled sampling), it can also draw a route on a map while writing out the matching moves. On mazes with several valid solutions, joint success rises from 54.6% with independent sampling to 93.7%, and no modality has to go first. On visual spatial planning, JUICE solves 99.8% of maps at its training sizes.
Authors:Vahid Hosseinzadeh, T. Anderson Keller, Michael Puthawala
Abstract:
Objects move, cameras pan, and weather drifts with the wind: much of what changes in sequential data is motion. Motion changes where things are, not what they are, and is therefore a symmetry of the data. Yet standard recurrent networks are not equivariant to it: their memory stays put while the input moves on. Flow equivariant recurrent networks first brought this symmetry into sequence models, for motions of constant velocity, or flows, by enumerating a bank of velocities. We extend this idea in three directions: from simple recurrent networks to convolutional LSTMs, from flows to motions whose velocity changes in time, and from enumerating velocities to estimating them. Treating a sequence as a field over time, we show that motion becomes a gauge symmetry: a global symmetry made local in time. Built on this view, our motion equivariant convolutional LSTM (MELSTM) uses its velocity model as the gauge field and is provably equivariant to every motion. Across future‑frame prediction, classification of digits visible only through their motion, and action recognition under camera shake, MELSTM outperforms a convolutional LSTM and its flow equivariant counterpart with the same number of parameters: it is more accurate, generalizes to longer horizons and to unseen motions, and carries a fraction of the flow equivariant model's state.
Authors:Berke Arda, Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mubashara Akhtar, Mrinmaya Sachan
Abstract:
Current evaluation of LLMs on benchmarks typically aggregates performance over individual examples into a single scalar score. This approach is overly coarse and ignores the diverse scope of skills covered by benchmark datasets. To address this limitation and better exploit benchmark data, we propose SkillEval, a new evaluation framework that performs cross‑model, skill‑level analysis. The framework integrates advances from two areas: (1) it leverages the meta‑cognitive capabilities of LLMs to automatically discover interpretable skills from questions, and (2) it applies cognitive diagnosis models (CDMs) from psychometrics to estimate per‑skill abilities of LLMs. We conduct experiments on 3,811 LLMs evaluated on 9,523 items across five benchmarks spanning diverse domains (MATH, BBH, GPQA, MuSR, IFEval). Compared to the traditional average‑based evaluation paradigm, SkillEval produces reliable, more calibrated and informative yet concise model profiles that more accurately predict model behavior on unseen items. We also demonstrate the utility of SkillEval in two downstream tasks: (1) LLM routing, which dynamically assigns each query to a capable but efficient model, reducing inference cost by over 60% while achieving improved performance, and (2) Computerized Adaptive Testing (CAT), where an unseen model can be efficiently evaluated using a trained CDM. Furthermore, we show that these skill profiles provide useful insights, for example, how benchmarks share skills and how instruction‑tuned models differ from their base models.
Authors:Ming Zhong, Stacy Li, Nicholas Carlini, Matthew Jagielski
Abstract:
Frontier coding agents are increasingly framed as end‑to‑end automated researchers, and ML for science offers a demanding test of that promise: improving a scientific model often requires domain insight beyond routine ML engineering. We introduce Agent4Sci, a long‑horizon testbed for measuring how far today's agents can improve the published best models on scientific ML benchmarks, spanning 20 benchmarks across 11 domains. Rather than ranking agents on a leaderboard, Agent4Sci asks what kinds of progress agents can make on these tasks under in‑the‑wild conditions, and where they fall short. Empirically, today's frontier agents advance the published best on every benchmark, with Claude Opus reaching 10.4% mean relative improvement and no run flagged for academic misconduct. These gains, however, come almost entirely from a bag of ML‑engineering tricks rather than research‑level insight. Trajectory analysis surfaces three recurring limitations in this setting: novelty miscalibration, resource under‑utilization, and rapid collapse into local engineering sweeps. Lightweight plug‑in skills for research ideation, experiment management, and scheduled checklists raise Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, while yielding more diverse, domain‑relevant interventions. Agent4Sci gives the community a way to track and push frontier agents from ML engineering toward domain insight on real‑world ML‑for‑science problems. Code and data are available on GitHub (https://github.com/maszhongming/agent4sci) and Hugging Face (https://huggingface.co/datasets/MingZhong/agent4sci).
Authors:Chuanmeng Bian, Daren Chen, Peixin Chen, Zhigao Chen, Chong Deng, Zhiyun Fan, Zhifu Gao, Bo Gong, Qing Gu, Jiajun He, Yawei Hu, Yunjie Ji, Wei Ju, Jingbei Li, Xiangang Li, Xu Li, Zengxi Li, Zheng Li, Chengdong Liang, Baiji Liu, Yaming Liu, Ying Liu, Bin Ma, Yiping Peng, Yuezhang Peng, Zhendong Peng, Yu Pu, Yang Shi, Xin Shu, Jian Tang, Biao Tian, Peiyao Wang, Tianzi Wang, Wen Wang, Wupeng Wang, Cheng Wen, Yuzhong Wu, Zijian Xia, Yunchong Xiao, Nan Yang, Jianwei Yu, Jixing Yu, Binbin Zhang, Lei Zhang, Sitong Zhao, Guangdong Zhou, Yuan Zhou, Jianheng Zhuo
Abstract:
We present Qwen‑Audio‑3.1‑ASR, a family of audio‑language models that builds upon its predecesor Qwen‑Audio‑3.0‑ASR with improved speech recognition and expanded capabilities in speaker‑diarized transcription and general audio understanding. The family retains support for 30 languages and 16 Chinese dialects, together with instruction‑controlled capabilities including historical‑context conditioning, hierarchical hotword customization, and native transcription polishing. Speaker‑diarized ASR produces structured outputs containing speaker labels, timestamps, and recognized text, maintaining consistent speaker labels across turns and the representation of overlapping speech. The general audio understanding capabilities cover audio‑, speech‑, and music‑captioning, temporal grounding and sound event detection, as well as audio question answering and multiple‑choice reasoning. These tasks broaden the application scope from recognizing spoken words to interpreting speakers, acoustic events, and their temporal relations. To accommodate different deployment requirements, the family includes variants for asynchronous file transcription, low‑latency streaming, interactive voice input, and non‑streaming recognition, with speaker‑diarized transcription and general audio understanding available in certain variants. Evaluations span conventional ASR, multilingual and dialectal speech, speaker‑diarized transcription, and general audio understanding, demonstrating the capabilities of the Qwen‑Audio‑3.1‑ASR family across transcription and broader audio analysis tasks.
Authors:Amr Abdelkhalik
Abstract:
We study the 305M‑parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 (bit‑identical to that of Gemma 4 E2B except for the final audio‑embedding projection) for Saudi and Gulf Arabic speech recognition, without changing any of its pretrained weights; the adapters of our adapted system are additive and kept separate. A frozen system trains only a 22.6M‑parameter CTC reader over a learned mix of four intermediate encoder layers and the encoder output; an adapted system adds 3.1M low‑rank adapter parameters, a 41.5M‑parameter hybrid CTC/attention reader and joint decoding with a sub‑word language model. Against Cohere Transcribe Arabic (2.07B parameters), re‑run on identical audio, the frozen system trails in word error rate (WER) but is not significantly different in character error rate (CER) on five of six human‑labelled conditions, while running 3.4 times faster on CPU with greedy decoding. The adapted system is not significantly different in WER on a public Saudi customer‑service call (19.20% against 19.27%) and on clean Emirati speech, is significantly better on telephone‑coded Emirati speech, has a significantly lower CER on all six conditions of these sets, and beats a 0.6B recogniser fine‑tuned on SADA in WER on all six; it still trails Cohere in WER on long‑form Saudi speech. On a blind, human‑corrected public call‑centre sample it ranks 4 of 15 systems, behind three commercial APIs (significantly only one) and ahead of the other self‑hostable systems by point estimate. An optimised joint decoder runs it at 139 times real time on one GPU. We also find, on our data, that references made by correcting machine drafts favour the drafting system. Code and public‑set results are released.
Authors:Sion Israel Sion, Alain April
Abstract:
Coding agents can now implement a specification, write and run tests, repair failures, and declare the result ready for release. When the same agentic system produces both the software and the evidence of its correctness, a passing test suite is no longer independent evidence: visible tests become an optimization target, code and tests may share one misinterpretation of the specification, and the builder's own tests may simply be incomplete. This paper argues that agent‑produced software requires an assurance plane that is structurally independent of the build plane. We characterize four gaps between stakeholder intent and production (requirement, conformance, evaluation, and model gaps) and identify which of them verification can close. We propose a framework that separates building from verifying, introduces a Verification Contract stating what must be demonstrated before release, and organizes verification in a specification‑first phase without access to the implementation and an implementation‑aware phase based on mutation analysis, property‑based testing, and fuzzing, with an assurance report that leaves the release decision to a human. We evaluate the framework in a pre‑registered study on three stateful services, two of them written by third‑party models, and an EvalPlus pilot. The builder's programs were correct, but its own tests missed subtle interactions: a verifier in a fresh context detected more of the 46 seeded faults than the builder's tests (Holm‑corrected p = 0.006) and surfaced the ambiguities of the specification, at the price of rare false oracles. A verifier from another provider did not add detection.
Authors:Haichao Zhang, Zhixuan Liang, Chong Zhang, Zixi Chen, Wen Wang, Jia Wang
Abstract:
Generative recommendation over discrete Semantic IDs has become a powerful paradigm for item retrieval. Real‑world systems, however, frequently require real‑time controllability to balance accuracy against objectives such as diversity. Existing autoregressive methods generate tokens strictly from left to right. This fixed direction locks in early decisions, preventing holistic planning and making control difficult. Consequently, direct interventions often break the semantic structure, producing non‑existent items. To address this, we propose ReGen, a framework that shifts from sequential decoding to parallel iterative refinement. ReGen reformulates generation as a mask‑filling task, predicting all tokens simultaneously rather than one by one. This global view allows us to apply control signals directly to tokens with high‑level semantics without retraining the model. To ensure that decoded candidates map to real items, we introduce a topology‑constrained tree that iteratively corrects low‑confidence predictions while strictly enforcing valid codebook paths during beam‑based inference. Experiments show that ReGen provides a favorable accuracy‑diversity trade‑off under moderate guidance while maintaining high validity of generated Semantic IDs. The source code is available at: https://github.com/zhang‑haichao/ReGen.
Authors:Dong Liu, Yanxuan Yu, Eric Jiang, Shu Wang, Ying Nian Wu
Abstract:
Disaggregated \textscLLM serving separates prefill and decode across node pools, placing a network fabric between \textscKV‑cache computation and consumption. This breaks a key assumption of classical cache policies: a miss no longer incurs only local recomputation, but also network transfer to the decode node. Likewise, admitting a block to the global cache requires an additional transfer, so poor retention decisions waste both memory and bandwidth. We present KVLearn, a learning‑based framework for cost‑aware \textscKV‑cache retention in disaggregated serving. KVLearn includes: (i) a lightweight Prefix Reuse Predictor (\textscPRP) that estimates reuse probability from structural and temporal prefix features without modifying model weights; (ii) a Cost‑Aware Retention Score (\textscCARS) that converts reuse probability into keep/admit decisions using recomputation, transfer, and storage costs; and (iii) an Adaptive Threshold Controller (\textscATC) that adjusts admission thresholds online from observed hit rates and memory pressure. We integrate KVLearn into a globally disaggregated serving topology and evaluate it on text and multimodal workloads, including large image/video‑derived \textscKV blocks with heterogeneous reuse patterns. KVLearn reduces end‑to‑end time‑to‑first‑token by up to 56% versus No‑Cache, 38% versus LRU‑Pool, and 33% versus Mooncake‑style baselines, while cutting inter‑node \textscKV transfer volume by up to 53% versus LRU‑Pool. On MM‑Session, throughput remains within ~5% of oracle. Code is available at https://github.com/FastLM/KVLearn.
Authors:Aur Shalev Merin
Abstract:
Probing studies of latent reasoning often read a model's state where the answer begins and take a failed probe as evidence that a value is absent. We test this with a digit copied from the question, in the looped model Ouro‑1.4B and the standard transformer Qwen 2.5‑1.5B. Comparing prompts that differ only in that digit shows it is present at the last prompt token, in a code shared across questions (readout about 0.9). A probe that sees one prompt at a time, the usual setting, barely reads it: 0.36 in Ouro and 0.19 in Qwen even after 5,249 training questions. The digit's change is small rather than crowded: random noise lands on equally busy directions, while a random word in its place moves the state far more and is readable. Since every varied token leaves a trace, a successful probe is ambiguous too. We give a protocol, including a synthetic null for generic mixing, for testing computed values against copies and mixtures of the inputs. In pre‑registered studies of two‑step problems it finds a small signal for the unwritten intermediate a+b just before Ouro writes the answer's last digit, and its sign follows the operation, though mixing alone produces much of that sign‑following part. Holding that value at another problem's from the first loop on does not move the written digit toward the other problem's, while the same patch of the answer does: the value is readable but not used.
Authors:Changrui Zhu, Simon Julier
Abstract:
Online scene change detection requires integrating incoming observations to determine which regions have changed relative to a reference scene. We update change beliefs as new images arrive using generalised Bayesian evidence fusion. However, the prior probability of change and evidence models are unknown beforehand, and numerical Bayesian inference can be costly. We address these challenges by learning offline from automatic edits to a reference 3D Gaussian Splatting (3DGS) model. These edits generate labelled training views without manual annotations. Together with unchanged reference observations, they provide supervision for learning scene‑specific evidence models and estimating the effective pixel prior and response statistics. Online prediction combines current evidence with historical support, while closed‑form local Bayes factors, parameterized by the learned response statistics, update an accumulated 3D Gaussian field. Object‑guided refinement produces the final change masks. Our method achieves state‑of‑the‑art accuracy on PASLCD and CCS3D, with 67.64% and 75.82% mIoU, respectively. On PASLCD, the complete pipeline runs at 17.2\,FPS over complete query sequences, the fastest among the evaluated online methods. Project page: https://changruizhu96.github.io/work/lego‑scd/
Authors:Zhizhuo Kou, Heming Li, Jian Yang, Junyu Luo, Jingshu Peng, Sirui Han, Yike Guo
Abstract:
People adapt how they talk to the person in front of them. However, chat models tend to respond in Standard English regardless of the dialect used by the user. A uniform answer can hide whether training treats these users alike, and output audits cannot tell as well. We trace how each post‑training stage, from supervised fine‑tuning (SFT) to preference optimization (DPO) and reinforcement learning, changes a model's tendency to match the user, and read what each update prefers from two content‑matched replies, one matching the user and one not. Across 10 open model lineages, post‑training standardizes replies to every informal cue, yet after adjusting for how likely each reply already was, the DPO update still favours matching British and Australian spelling over AAE and Singlish on 10 lineages, and models still produce AAE features when asked. We then introduce calibrated accommodation pairs (CAP), preference data that reward matching users' dialect density while preserving Standard English for Standard‑English users. On a fully open model, CAP raises convergence on features it was not trained on without a detectable increase in dialect for Standard‑English users or a cost in instruction following. In contrast, one‑sided pairs reach a similar gain only by overshooting the user and imposing dialect. Our findings show that uniform replies can hide selective updates and audits should read the update as well as the output. Code: https://github.com/kouzhizhuo/cap‑accommodation.
Authors:Daniel Marew, Sangjun Park, Nisal Perera, Donghyun Kim
Abstract:
While large‑scale human motion capture datasets provide a rich source of demonstrations for humanoid robot learning, retargeting the diverse motions to different humanoid robots remains a nontrivial challenge. Existing kinodynamic trajectory optimization pipelines are computationally expensive and often require manual adjustments per motion. We present FRDV (Fast Retargeting with Dynamics Validation), a scalable framework consisting of rapid constrained kinematic retargeting and an additional dynamics validation pipeline. Its kinematic stage integrates a custom Sequential Quadratic Programming solver, Real‑Time Iteration, and parallel chunk‑based processing, achieving up to 9,156 frames per second, approximately up to 136 times faster than GMR and ProtoMotions, while robot‑specific constraints are incorporated into the optimization to reduce common retargeting artifacts, such as ground penetration and self‑collision. For motions requiring additional verification, a contact implicit iLQR controller is utilized to test the retargeted reference under full rigid body dynamics. This dynamics‑based validation identifies dynamically infeasible motions and refines contact‑related artifacts such as foot floating and foot skating. We evaluate retargeting quality by training motion tracking reinforcement learning policies using several state‑of‑the‑art pipelines and comparing their tracking fidelity and success rates. Furthermore, sim‑to‑real transfer is demonstrated on a Unitree G1 humanoid. Videos and code are available at: https://daroslab.github.io/frdv/.
Authors:Raúl Castilla-Arquillo, Miguel Olivares-Mendez
Abstract:
Lunar surface exploration increasingly relies on autonomous rovers that must perceive and traverse unstructured terrain under extreme illumination conditions. Near the lunar poles, low sun elevations produce long, high‑contrast shadows that challenge stereo vision, terrain segmentation, and localization. Developing perception and navigation algorithms for these conditions requires large amounts of sensor data with accurate ground truth, which field campaigns and analog facilities can only provide at high cost and with limited variety. This paper presents MARE (Modular Autonomy and Rendering Environment), an open‑source simulation suite for lunar rover perception and autonomy built on Unreal Engine 5 and ROS 2. MARE consists of three integrated components. Terrain Studio generates cratered and rocky lunar Digital Elevation Maps (DEMs) to build simulation scenes. A dedicated Unreal Engine project hosts two physics‑based rovers with plugins for stereo, lidar, event, and Single‑Photon Avalanche Diode (SPAD) cameras, streaming time‑stamped sensor data and pixel‑accurate ground truth. Lastly, a Guidance, Navigation, and Control (GNC) stack provides web‑based mission control. Validation shows a mean Intersection over Union (IoU) of 0.79 between predicted and rendered shadows at low sun elevations, and stereo depth analysis verifies decreased pixel coverage in shadowed regions relative to sunlit terrain.
Authors:Kartik Tandon, Alessio Borgi, Pavel Snopov, Inés García-Redondo, Davide D'Ascenzo, Sebastiano Cultrera di Montesano, Fabrizio Silvestri, Pietro Liò, Francesca Dominici, Randall Balestriero, Claudio Battiloro
Abstract:
Joint‑Embedding Predictive Architectures (JEPA) offer an appealing approach to self‑supervised learning, but are typically prone to representation collapse. LeJEPA resolves this by introducing a theoretically grounded distribution‑matching objective that drives the embedding towards an isotropic Gaussian. However, this approach is fundamentally tied to Euclidean assumptions, and its global target does not reflect domain‑specific geometric priors, such as the use of non‑Euclidean embedding geometries, e.g., equivariant CNNs or hyperbolic ViTs. We thus introduce LocaLeJEPA, a framework motivated by a manifold‑theoretic reformulation of LeJEPA's entropy‑maximization property. The accompanying LocaLeReg, a novel locally Euclidean distribution‑matching regularizer, enables a stable and scalable implementation of the LocaLeJEPA framework. We then perform extensive experiments in the vision setting on models trained on ImageNet‑1K. LocaLeJEPA significantly outperforms LeJEPA when paired with hyperbolic ViT‑S/8 and equivariant ResNet‑50 backbones, and also with ordinary Euclidean ViT‑S/8 and ViT‑L/14 backbones. This holds both in terms of linear and k‑NN classification performance on ImageNet and on a series of downstream tasks tested. We view the LocaLeJEPA formalism as a principled approach to the JEPA paradigm that better respects intrinsic data geometry while also accommodating generalizations to exotic embedding geometries. Code available at https://github.com/alessioborgi/LocaLeJEPA.
Authors:Zhenwei Tang, Di Bai, Jintao Liu, Xidong Wu, Qianfeng Wen, Xinwu Cheng
Abstract:
Agents are starting to sell to other agents. Seller agents write the listings of tools, services and products, and buyer agents read these listings when they choose what to use. We show that seller agents told only to be selected more often practice generative engine optimization on their own, and that this distorts the market for sellers and for buyers. We build tool markets from real data tools on Amazon and Instagram and run every tool on every question, so we know which tools can answer each question. Under all five production models we test, sellers rewrite their descriptions to claim more and more fields, including fields that their tools return for fewer than half of the targets. We call this process toolflation. Nothing in the market limits it, and how far it goes depends on the model, as the number of claims that contradict the sellers' own evidence differs by an order of magnitude between models. Toolflation shifts the buyers' choices between sellers, and the most capable tool loses share in seven of the ten combinations of seller model and platform. Under four models, the choices stay with capable tools and buyer agents keep their task success. Under the fifth, sellers copy one another until the descriptions no longer tell the tools apart. Buyer agents then pick tools by their position in the list, tools that answer almost nothing win a quarter to two fifths of the choices, and task success falls from 89% to about 30% on Amazon, close to random choice, and from 77% to 40% on Instagram. The sellers did only what their vendors asked, and the cost fell on buyers they never dealt with. We therefore argue that honesty toward third parties under competition is part of safety alignment, and that models should be tested as sellers before they are deployed as sellers.
Authors:Nikita Ganziuk, Innokentiy S. Humonen, Ilya Makarov, Pavel Lebedev
Abstract:
We present a tool for the comparative study of mythological and religious texts. Traditional manual analysis is highly labor‑intensive and difficult to scale. To address this, we use neural language models to embed texts into a shared semantic space, so that users can find semantic parallels across cultural traditions in an interactive interface. Embedded fragments form distinct semantic structures whose coarse geometry aligns with well‑documented historical relations between traditions. We quantify how much of that geometry is already explained by lexical overlap and by proper names. The current system is available at https://mythoscope.io, and the installable package is available at https://github.com/haafus/MythoSemantic/blob/main/RUN.md.
Authors:Xiaoxing Hu, Wenxi Yao, Chaoyuan Shi, Ziqi Ye, Xiaorong Guo, Peixuan Li, Zhuoran Sun, Ziyang Gong, Peiyuan Zhang, Mingxin Liu, Shuran Ma, Hongbo Liu, Xiaomin Yu, Yu Tian, Wentao Wang, Zhenjie Yang, Jian Hu, Yunhang Shen, Haoyu Cao, Xue Yang
Abstract:
Omni LLMs can naturally locate key audio and visual evidence and integrate information across modalities for audiovisual reasoning. Existing audio‑visual reasoning benchmarks that mainly rely on multiple‑choice questions usually score models solely based on answer correctness, making it difficult to assess whether a model truly forms a reasoning chain from evidence to the correct answer. In addition, textual or unimodal shortcuts may allow models to answer correctly without analyzing the necessary audio‑visual evidence, further weakening the reliability of evaluation benchmarks. To address these issues, we introduce EvaOmni, an evidence‑grounded benchmark containing 392 videos and 1,027 purely human‑annotated question‑answer pairs, covering 15 capabilities and three levels: retrieval and localization, basic understanding, and complex reasoning. We design a series of rigorous annotation procedures in which experts annotate audio and visual evidence and construct video‑based distractors; humans review whether questions meet strict inspection criteria, and finally models filter textual and unimodal shortcuts. Beyond high‑quality question‑answer pairs, we require models to simultaneously return the answer and supporting evidence intervals in order to evaluate the complete reasoning chain. We propose the Coverage Grounded Score (C.G.) to measure how much annotated evidence a model identifies, and the Bottleneck Grounded Score (B.G.) to penalize poor temporal localization in either modality; both are incorporated into answer correctness. Across 7 open‑weight and 17 proprietary Omni LLMs, Gemini3.7‑Flash achieved the highest overall accuracy (79.4%), C.G. (70.8%), and B.G. (35.1%). At Level 3, it reached 79.1% accuracy, but its B.G. was only 12.3%. These results show that even the most advanced models still have room for development in grounded audio‑visual reasoning.
Authors:Sahil Joshi, Prithvi Dixit, Aryan Katiyar, Agniva Chowdhury, Joseph E. Gonzalez, Anshumali Shrivastava, Aditya Desai, Kumar Krishna Agrawal
Abstract:
Large Language Model inference over long contexts is increasingly bottlenecked by the cost of attending to a growing Key‑Value (KV) cache. Sparse attention reduces this cost by accessing only a fraction of the cache, but existing methods face a trade‑off between deterministic retrieval, which captures high‑scoring tokens but discards residual attention mass, and sampling, which improves coverage at the cost of higher variance. Hybrid methods combine top‑k retrieval with uniform sampling, yet still exhibit a substantial gap from dense attention at aggressive sparsity. We introduce \emphStratified Sparse Attention (SSA), a unified framework combining deterministic retrieval with score‑guided sampling. SSA deterministically attends to high‑scoring tokens while sampling the residual using proxy attention scores and correcting for their inclusion probabilities, thereby recovering attention mass beyond the retrieval set. SSA encompasses top‑k, top‑p, and hybrid top‑k+sampling as special cases. Across RULER, LongBench, LOFT, and AIME, and five models ranging from 3B to 27B parameters, SSA consistently improves over the baselines at matched sparsity. On RULER‑HARD with Ministral‑3‑8B, SSA improves accuracy over the strongest practical baseline from 81.8% to 84.2% at 32K and from 63.8% to 69.3% at 128K under 50× attention sparsity, recovering 94% and 85% of dense‑attention performance, respectively. With optimized CUDA kernels, SSA achieves up to 2.96× decode speedup over FlashInfer at 1M context, demonstrating that its accuracy gains translate into practical long‑context inference acceleration. We open‑source our code at https://github.com/skylight‑org/SSA‑Paper‑Code.
Authors:Zhuchenyang Liu, Yao Zhang, Yu Xiao
Abstract:
Multimodal rerankers are usually dedicated models: a vision‑language model (VLM) is trained on retrieval data to score the candidates of a query. Scoring options is not specific to reranking. System One decision models such as Jev map a state and a typed question to one probability per option without generating text, and general models can make such decisions with little or no training. Reranking is one such decision, with the query and its candidates as the state and the candidates as the options. We therefore argue that a general VLM can serve as a multimodal reranker through System One decisions, and that decision tuning can improve its reranking. We ask a VLM one choice question over the top 16 candidates of a first‑stage retriever and rank the candidates by the probabilities of their labels, read from one forward pass. On MMEB‑V2 image retrieval, this choice scores 5.4 to 7.3 hit@1 above a yes/no question per candidate, and the yes/no question falls below the first stage. With the choice, zero‑shot Qwen3.8‑27B beats the supervised Qwen3‑VL‑Reranker‑2B on all three benchmarks and comes within 1.6 points of Reranker‑8B on each. It matches Reranker‑8B on MMEB‑V2 visual documents (84.79 vs. 84.78 nDCG@5), at 1.1 to 1.5 times its latency. JEV‑27B‑VL, the same backbone tuned on text decisions only, improves page reranking by 0.6 to 1.1 points, with gains on most tasks (sign test, p <= 0.017), and leaves natural images unchanged. The zero‑shot model is larger than the supervised rerankers and of a newer generation, so these comparisons are confounded; we discuss this and other alternative views, and the experiments that would test the position.
Authors:Junyuan Deng, Xinyi Wu, Zhenyao Wu, Song Wang, Yu Wang, Congchao Zhu
Abstract:
Generative‑prior‑based super‑resolution (SR) methods leverage implicit knowledge learned from large‑scale models to enhance low‑resolution inputs. However, such priors are inherently stochastic and optimized for diverse generation, conflicting with the deterministic, high‑fidelity reconstruction required for SR. It is also tied to the base model, costly to update, and prone to hallucinations as real‑world data evolves. To address these limitations, we introduce WorldSR, a novel framework that augments SR with explicit, entity‑aligned visual world knowledge. WorldSR automatically decomposes the input into semantic entities and performs entity‑wise retrieval of structurally and semantically aligned images from a large‑scale external corpus. For efficiency and reliability, we incorporate a memory mechanism to reuse previously retrieved results and a multi‑step filtering process to refine candidate references. The selected references as world knowledge are then integrated through a novel multi‑reference super‑resolution model, which provides fine‑grained constraints for detail synthesis and structural recovery. Experiments on real‑world SR benchmarks show that WorldSR consistently improves structural fidelity and visual quality, outperforming existing generative super‑resolution methods. Our results suggest that entity‑level grounding in external visual world knowledge provides a simple and effective complement to model‑bound generative priors.
Authors:Spiros Maggioros, Guray Erus, Gareth Harman, George Aidinis, Alexander Getka, Aristeidis Sotiras, Christos Davatzikos
Abstract:
Deep learning models have become essential tools in medical imaging, yet training them end‑to‑end for diverse tasks remains challenging due to limited labeled data and computational constraints. To address this, we introduce Brain2Vec, a foundation model pre‑trained on 74,425 T1‑weighted Magnetic Resonance Imaging (MRI) scans pooled from multiple studies. Brain2Vec can serve both as a weight initialization backbone for fine‑tuning downstream tasks and as a standalone feature extractor. We evaluated Brain2Vec in three tasks: downstream predictive performance, convergence speed, and embedding quality. Using Brain2Vec's pretrained weights as initialization and fine‑tuning on labeled data for a brain age prediction task, results demonstrate improved final accuracy and faster convergence compared to random initialization. Additionally, its frozen embeddings outperform anatomically defined volumetric features on four of five downstream classification tasks and perform competitively with a state‑of‑the‑art foundation model pre‑trained on substantially more data. Lastly, we show that Brain2Vec encodes information primarily from a spatially localized and neuroanatomically meaningful set of brain regions, and that manipulating a subset of latent features produces monotonic, region‑specific effects on the reconstructed anatomy. Brain2Vec code and trained models are available as an open‑source package on https://github.com/CBICA/Brain2Vec. Users can apply Brain2Vec locally by installing the package, or directly through the NiChart cloud platform https://cloud.neuroimagingchart.com, where Brain2Vec features can be derived without requiring local installation or specialized infrastructure.
Authors:Jinghui Zhang, Lang Gao, Zongfang Liu, Ruihong Zeng, Zirui Song, Rui Yan, Kentaro Inui, Xiuying Chen
Abstract:
Precisely reproducing a target span verbatim within a long context is a fundamental capability test for LLMs: even when the target string is present in context, models often still drift within the span or fail to stop correctly, revealing limitations in source grounding and boundary control. This capability has broad practical relevance. For instance, code generation often requires embedding sensitive strings such as API keys exactly as they appear in context. In this work, we construct a benchmark to test exact span reproduction ability across contracts, scientific literature, code, and random sequences, and find that current models still make surprisingly frequent errors. To address this, we propose CopyGen, a lightweight copy‑constrained plugin for frozen LLMs that turns copying from a byproduct of next‑token prediction into a controlled process. Our key idea is to decompose copying into three distinct decisions: when to copy, how far to copy, and when to stop. Concretely, CopyGen first determines whether to enter copy mode, then predicts a tentative copy length, and finally verifies the generated span token by token. During verification, we adaptively adjust the logits of copy continuation tokens to incentivize or suppress continuation, and emit only the longest valid prefix. Across three LLM backbones and four benchmark tasks, CopyGen improves average exact match by 35% relative without model finetuning, and integrates seamlessly with SFT to further improve it by 4%, while substantially accelerating generation. These results suggest that reliable exact copying is not something LLMs obtain for free, but a decoding‑time control problem that requires explicit modeling. Code is available at https://github.com/mbzuai‑nlp/CopyGen.
Authors:Jiawei Wang, Yucun Li, Xueqi Li, Shenghao Wang, Xinquan Gong, Dan Zhang, Zheqi Lv, Yuquan Le, Shaozu Yuan, An Zhang, Da Cao, Tat-Seng Chua
Abstract:
Visual tool use enables vision‑language models (VLMs) to acquire task‑relevant evidence, but its effectiveness can be limited by predefined visual operations and the tool‑use strategies learned during training. Representing these operations and strategies as reusable external skills allows them to be refined through self‑evolution without updating model parameters. However, skill refinement can introduce performance regressions. Meanwhile, inspecting accumulated task execution trajectories together with skill evolution history may lead to evolution evidence overload. To address these challenges, we propose VisualEvolve, a self‑evolving framework that evolves executable visual operations and tool‑use strategies for VLMs through a Select‑‑Execute‑‑Evolve‑‑Evaluate cycle. Skill‑Library Snapshot Selection uses performance on a fixed validation set disjoint from execution tasks to select an effective historical snapshot for subsequent evolution, limiting the propagation of performance regressions. Progressive Evidence Access (PEA) guides capability‑gap diagnosis through a coarse‑to‑fine hierarchy comprising the Skill‑Performance Report, Evolution Log, Trajectory Summaries, and relevant raw trajectory segments. Across four VLM backbones and six evaluation sets from three benchmarks, VisualEvolve improves over the Initial Skill Library in 23 of 24 settings, with a maximum gain of 15.1 percentage points on VisualProbe Hard using Qwen3.5‑Plus. Ablation studies support the contributions of Snapshot Selection and PEA, while further experiments show that skills can be improved using experience from different data settings and directly reused across backbones. Moreover, VisualEvolve enables effective skill self‑evolution with over an order of magnitude fewer task‑solving trajectories than estimated for representative RL‑based methods.
Authors:Zijun Dou, Kun Luo, Chaofan Li, Shuqi Lu, Zheng Liu
Abstract:
Long‑horizon deep search requires LLM agents to iteratively decompose complex information needs, gather evidence from external sources, and maintain a reliable research state under multiple constraints. A central challenge is that early mistakes can be amplified over the search trajectory: agents may anchor on an incorrect candidate, collect partially supporting evidence, overlook contradictions, and accumulate errors that misguide subsequent reasoning. We propose FALCA, a falsification‑aware framework that formulates deep search as a propose‑falsify process. FALCA adopts a collaborative Searcher‑Falsifier architecture, where the Searcher proposes candidate answers or sub‑answers, while independent Falsifiers actively seek counterevidence, check unmet constraints, identify conflicts, and produce concise falsification reports for research‑state revision. To strengthen this loop, we introduce falsification‑aware training, combining SFT warmup on falsification trajectories with agentic RL based on parallel rollouts and hierarchical credit assignment over research stages, state transitions, and actions. Experiments on five search benchmarks show that FALCA improves long‑horizon deep search performance over backbone and harness ablations, while reducing confirmation bias and error accumulation through more effective falsification and research‑state revision.
Authors:Tianyu Wu, Zhuohan Wang, Jonathan Geuter, Yu Yao, Weifan Jiang, Chengxi Zhang, Ju Li, Minlan Yu
Abstract:
In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution. D‑PACE (Wu et al., 2026) trains such drafters with a loss that weights each position by its contribution to the expected accepted length. Both its weights and its cross‑entropy (CE) loss use only the draft's probability of the recorded target token, treating the target as a point mass. Under rejection sampling, however, a sampled draft token is accepted with probability equal to the overlap of the draft and target distributions, one minus their total variation (TV). We derive D‑PAL (Dynamic Position‑wise Acceptance Loss) from the exact expected accepted length, using this overlap as the acceptance signal. This yields overlap‑based position weights with log acceptance (LA) as the per‑position loss, and the objective reduces to D‑PACE for a point‑mass target. Since the weights are detached, any per‑position loss can replace LA in practice. We show that forward KL divergence and Rényi divergence of order 1/2 match or exceed LA, and that Rényi‑1/2 provably bounds the true acceptance rate from above and below. Replacing CE with Rényi‑1/2 improves acceptance length under both D‑PACE's weights and ours. On six benchmarks with a Qwen3‑4B target, D‑PAL raises mean acceptance length over D‑PACE from 4.50 to 4.57 at T=0 and from 4.20 to 4.28 at T=1, and consistently achieves higher throughput at both temperatures.
Authors:Sumaiya Fazal, Rizwan Qureshi, Laleh Behjat, Samira Ebrahimi Kahou
Abstract:
Realistic few‑shot adaptation of medical vision‑language models (VLMs) can leave low‑prevalence classes without any labeled support. The training‑free SS‑ Text+ solver remains operational in this setting, but its zero‑support prototypes receive no visual adaptation and fall back to a purely textual class prototype. We in‑ troduce Gated Knowledge‑Guided Prototype Transfer (GProtoT), a pre‑repulsion module that transfers supported‑class visual residuals to missing classes using VLM similarity and an ontology‑based clinical distance. A validation‑free evidence gate scales transfer based on semantic relevance and residual agreement, thereby limiting unsupported extrapolation. In 50 paired realistic one‑shot trials on MESSIDOR with FLAIR features, severe and proliferative diabetic retinopathy were absent from every support set. Relative to SS‑Text+, gated ProtoT improved missing‑class recall by 5.23 points, average class accuracy by 1.37 points, and the missing/supported recall harmonic mean by 1.83 points. It reduced negative‑ transfer frequency from 46‑48% for ungated transfer to 2%, while at the cost of reductions of 1.20 percentage points in supported‑class recall and 0.45 percentage points in top‑1 accuracy. These results indicate that evidence‑based gating provides a safety mechanism for the knowledge source. GProtoT is based on SS‑ Text+ and requires no gradients, validation data, or additional model inference, of‑ fering a low‑compute path toward accountable adaptation under incomplete clini‑ cal supervision. Code is available: https://github.com/sumaiyaafridi/GProtoT
Authors:Atharva Thakur, Abulhair Saparov, Bruno Ribeiro
Abstract:
Reliable retrieval‑augmented generation (RAG) requires recognizing when neither retrieved documents nor parametric knowledge supports an answer. We introduce DeclineCraft, a dynamic abstention benchmark whose unanswerable questions target Minecraft feature additions after a corpus cut‑off, with audits of possible parametric exposure. Later cut‑offs support evaluation of future models. Across 27 LLMs, we measure Correct Abstention Rate (CAR; how often models acknowledge insufficient knowledge on unanswerable questions), Answerable Attempt Rate and Answer Accuracy. We find that (1) single‑shot retrieval significantly raises CAR in 38 of 40 configurations (model and thinking setting) versus no retrieval. (2) Iterative RAG, in which the model self‑assesses its context and can request up to two further retrieval rounds, significantly lowers CAR versus single‑shot RAG in 30 of 40 configurations (mean 46.4%), with little change in Answer Accuracy (68.5% to 69.3%). Adding query rewriting and reranking to hybrid (dense & sparse) retrieval lowers CAR for all 12 models tested. (3) Larger models answer more accurately (Spearman ρ= +0.60 with total parameters), but neither model size nor release date shows a significant association with CAR. (4) Without retrieval, enabling thinking on the same weights lowers CAR significantly for 8 of 13 models (by 14% points on average), and the responses shift from acknowledged uncertainty to premise denial.
Authors:Aymen Lassoued, Angelique Loesch, Romaric Audigier
Abstract:
Instruction‑based image editing models can now add, remove a specified number of objects or change their category while preserving the scene. Yet these models often fail to produce the correct number, and these failures grow sharply as scenes become more complex and the requested count increases. Current benchmarks largely miss this problem: they score edits with holistic similarity metrics or with vision‑language model (VLM) judges that are themselves poor at counting, so numerical errors go unnoticed or are graded incorrectly. As a result, the true numerical ability of today's editing models has never been accurately measured. We address this issue with ABACUS, a new benchmark, evaluation protocol and dedicated scoring metric for numerical image editing. Our protocol is judge‑free: no VLM or MLLM scores an edit or counts objects (a VLM only confirms the category of rare ambiguous crops). Instead, it verifies each edit at the instance level against panoptic ground truth confirmed by detector consensus, checking both that the exact requested count was achieved and that the rest of the scene was preserved, across the add, remove and change operations. By evaluating leading image editors, our method reveals substantial numerical editing failures that worsen with increasing scene complexity and object count; failures that prior benchmarks often overlooked or misjudged.
Authors:Tianyi Lorena Yan, John Hewitt
Abstract:
Monitoring the chain‑of‑thought (CoT) of language models (LMs) is a key strategy for detecting and preventing misbehaviors for AI safety. As such, it is critical to identify reinforcement learning (RL) practices that lead to unmonitorable chain‑of‑thought. One such practice may be unavoidable: reward misspecification. As RL environments become more complex, it is infeasible to have perfect verification, and models are rewarded for behaviors that are different from the true task intent. We show that reward misspecification alone already degrades monitorability. We create a testbed with pairs of true reward aligned with task intent and misspecified rewards that favor a different behavior, so that reward hacking can be induced by training and labeled verifiably. Across three models and three tasks, CoT monitorability remains low as models' hack rates increase: the CoTs keep reasoning about the task intent while the outputs violate the intent and get rewarded. To mitigate unmonitorability without prior knowledge of the true reward or the hack, we collect rollouts with diverse rewards across training steps and use a LM to infer a reward description. We then inject the description into the input, which teaches the model to verbalize the reward. This reward description training improves monitorability by an average of 24% or 42% absolute, depending on the metric, across all models and tasks, and yields CoTs that reason about the misspecified reward explicitly. Code is available at https://github.com/Lorenayannnnn/reward‑misspecification‑degrades‑cot‑monitorability.
Authors:Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Ayesha Mohsin, Sarmad Bilal, Dean F. Hougen
Abstract:
Test‑time scaling allocates additional inference compute to refining existing solutions or generating new ones. This raises a fundamental question: Should I keep improving this solution, or stop working on it and try a completely new solution? Existing methods make this decision using fixed refinement schedules, self‑verification judgments, or learned controllers, which can waste compute or rely on signals that may be incorrect or require additional training. We introduce Adaptive Replay Refinement (ARR), a training‑free method that detects when a stationary revision reaches a fixed point and reallocates the released compute to a new independent sample; replay decoding treats the current solution as a draft and verifies unchanged tokens in parallel, enabling efficient fixed‑point detection. Across seven models from five families and seven benchmarks, ARR matches majority voting on three of five mathematical reasoning benchmarks at equal compute, while reducing critical‑path serial passes by up to 2.4 times. On MATH500, ARR achieves 84.6% accuracy with only 41% of the serial passes required by majority voting. These results demonstrate that detecting refinement exhaustion enables adaptive reallocation from revision to independent sampling, improving inference efficiency while highlighting the limits of self‑revision. Code is available at https://github.com/AhsanBilal7/adaptive_refine.
Authors:Ivan Ilin, Peter Richtárik
Abstract:
Teams of large language model (LLM) agents are increasingly applied to complex tasks, but their collaboration protocols are mostly designed empirically, without convergence guarantees. We formalize collaborative problem solving as block optimization: each agent improves one part of a shared solution, guided by its local objective plus a correction for how its changes affect the other parts. A coordinator checks every proposal against the global objective and merges proposals that do not interfere with each other. We prove linear convergence up to an error floor that vanishes when the corrections are exact. Interfering proposals can also be merged once their combination has been checked; the rate then depends on how well the proposals actually combine, not on worst‑case interactions. With GPT‑OSS‑120B agents on robot routing, a three‑agent team finds collision‑free plans in every run, better than those of a single agent that controls all robots. On a larger routing task, a six‑agent team completes its search about twice as fast as the single agent (in median). Merging the best checked combination of proposals, rather than only proposals that do not interfere, cuts the team's search time by about 30%. With constants measured on the runs, our convergence bound holds in every round and follows the observed progress. We also test the theory on query planning in SQLite and on a synthetic task.
Authors:Huachen Gao, Yang Yang, Xiaoqian Liang, Xingyu Chen, Youjia Zhang, Qi Zhang, Baorui Ma, Anpei Chen, Wei Yang
Abstract:
Geometric Relative Positional Encoding has become the primary mechanism for injecting spatial inductive biases into multi‑view transformers. Existing designs condition attention on camera‑level frustums or ray‑marched depth, associating cross‑view tokens through the sensor geometry. Such sensor‑centric formulations, however, describe only where a token is observed from, leaving the rich surface geometry of the scene region depicted by the token. In this paper, we advocate a shift toward surface‑aware positional encoding that explicitly grounds patch tokens in the geometry of physical surfaces. We introduce HRoPE, which represents the local geometry of each patch as a slanted 3D plane, following the slanted support‑window assumption of classical multi‑view stereo. We inject token‑level, plane‑induced homographies into attention as relative transformations. This equips the transformer with a geometric prior for rectifying perspective distortions and aligning patch features across viewpoints. Our encoding requires no ground‑truth geometry: the plane parameters can be predicted end‑to‑end under a view‑synthesis objective alone, or derived from foundation models. Extensive experiments on RealEstate10K, CO3Dv2, and DL3DV show that our best variant of HRoPE consistently outperforms the evaluated geometric positional encodings for novel view synthesis. Extensive ablations and analysis show the effectiveness of the proposed encoding. Please refer to our project page at: https://gaohchen.github.io/HRoPE
Authors:Tapan Parikh
Abstract:
A model behind an API can be identified from its one‑word answers to everyday questions such as "Name a sound" or "Name a poet". A model's profile records how often it gives each answer to each question. We attribute new answers to the model most likely to have given them. On a panel of 94 models, 40 answers identify the exact model 92.5% of the time, against 1.1% by chance. Profiles built through OpenRouter identified all ten models served by DeepInfra under blinded labels. A system prompt lowers accuracy by 6 to 16 points. We applied the method to 96 endpoints of 13 open models on OpenRouter, and 92 identify as the model they claim. Of the other four, one matches the claimed model with reasoning turned off. One serves Gemma 4 under the name Llama 4 Maverick: it names itself Gemma 4, and it is billed Gemma 4's token counts on every message. Two remain unresolved. Billing records also show context added to first‑party models. Grok 4.7 is billed 1,247 input tokens for a one‑line question that costs GPT‑4o only 12 tokens, and five models state the current date without being billed for it. The questions come from a public census of model defaults, and the method costs cents per endpoint.
Authors:Chucheng Xiang, Runze Wang, Zhi Deng, Ruchao Bao, Yuanwei Zhang, Yuxuan Xie, Hanliu Wang, Liangzhen Fei, Wenzheng Wu, Cheng Wan, Peifeng Li, Zhongyuan Liu, Ligang Liu
Abstract:
Reasoning about fine‑grained placement constraints, such as support and containment, is essential for realistic 3D scene creation. It remains a significant challenge for existing methods that rely on simplified object representations like oriented bounding box (OBB), which are blind to the detailed geometry. To address this limitation, we introduce GALA, a geometry‑aware language model with explicit point cloud perception for controllable object placement. Our model performs autoregressively by predicting the 6D pose of one target object at a time conditioned on the scene context and placement instruction. We utilize a two‑stage training strategy: the first stage aligns geometric and textual modalities through pretraining tasks, and the second stage fine‑tunes the model for placement. To facilitate this task, we also curate a multi‑source scene dataset covering diverse room types and fine‑grained furniture arrangements, which will be open‑sourced for community benefit. Experiments on our benchmark show our model significantly outperforms baselines without explicit geometric reasoning. We further develop a multi‑stage plan‑place‑verify agent pipeline that uses the model for text‑driven scene generation. Project Page: https://xccelephant.github.io/paper/gala/
Authors:Zhiyuan Zhao, Hengrui Kang, Guang Liang, Bin Wang, Conghui He
Abstract:
Accurate and efficient document layout analysis is critical for modern document parsing systems. However, although the YOLO framework represents a leading paradigm for the real‑time detector, its adaptation to heterogeneous documents remains underexplored. To address this gap, we propose DocLayout‑YOLO, a document‑specialized real‑time detector that integrates diverse synthetic document pretraining and page‑axis context modeling. From the data perspective, we introduce MeshLayout‑1M, a synthetic pretraining corpus generated by the proposed Mesh‑candidate BestFit algorithm, which provides large‑scale and diverse document layouts for pretraining and improves adaptation to heterogeneous document scenarios. From the model perspective, we propose Page‑Axis Layout Attention (PALA), which captures long‑range dependencies along the horizontal and vertical page axes while remaining computationally efficient. Extensive experiments demonstrate that DocLayout‑YOLO achieves state‑of‑the‑art performance on four heterogeneous document benchmarks, obtaining 72.4%/75.5%/96.0%/73.1% mAP50:95 on D4LA, M6Doc, HJDataset, and PRIMA‑LAD, respectively. Compared with existing real‑time detectors, DocLayout‑YOLO establishes a superior accuracy‑efficiency trade‑off in document layout analysis. Code, model, and data are available at https://github.com/opendatalab/DocLayout‑YOLO/tree/release‑v2.
Authors:Xiaoxu Xu, Hao Li, Hongtao Zhang, Yili Tang, Jiafei Cao, Junhao Cai, Haoxiang Ma, Weixin Li, Weinan Zhang
Abstract:
Effective manipulation requires a policy to capture not only about the current scene geometry, but also about how that geometry will evolve under interaction. Existing approaches either augment policies with geometric information from current observations or introduce future prediction as auxiliary supervision, leaving open how predictive geometry should be represented for action generation. Therefore, we propose a two‑stage framework, which learns complementary predictive representations from multi‑view video clips. During pretraining, a frozen 3D foundation model provides dense geometric features that are compressed into predictive geometric representations and optimized to preserve scene structure and its temporal evolution. Action supervision is then applied to geometric transitions to extract a complementary motor representation that emphasizes action‑relevant transitions. During downstream post‑training, predictive geometry is transferred through geometry queries, while the motor representation guides intermediate features along the action‑generation pathway. Future observations are used only as privileged supervision during training and are not required at inference time. Across LIBERO, LIBERO‑Plus, SimplerEnv, and real‑world bimanual and single‑arm tasks, our approach consistently improves downstream policies, including an absolute gain of 10.4 percentage points on the WidowX evaluation. We further demonstrate transfer across multiple VLA architectures and WAMs, with improvements in both future‑world and action prediction. The code will be available at https://github.com/xuxiaoxxxx/CPR.
Authors:Sivan Talaei, Fatemeh Daneshfar, Abdulhady Abas Abdullah, Mourad Oussalah
Abstract:
Medical image segmentation remains challenging when targets exhibit limited visual distinction, irregular structure, or substantial variation across imaging modalities. Medical text can reduce this ambiguity, but indiscriminate fusion may introduce unreliable guidance. We present RELiF, a vision‑‑language framework that integrates text through reliability‑guided decoding. Its Reliability‑guided Fusion Gate (RFG) adaptively controls cross‑modal updates, while the Text‑guided Skip Refinement Adapter (TSRA) refines high‑resolution features using text guidance. RELiF is evaluated on radiographic, CT, and endoscopic segmentation tasks, where it shows competitive performance across diverse imaging characteristics and target appearances. Additional analyses examine efficiency and stability. The complementary uncertainty representation derived from the reliability response also shows an association with prediction difficulty, with higher uncertainty values occurring more frequently in pixels associated with segmentation errors. Code: https://github.com/SeivanTalaie/RELiF.
Authors:Zijian Yu, Hangwei Zhang, Jun Lan, Huijia Zhu, Weiqiang Wang, Jianfu Zhang
Abstract:
AI‑generated images linked to news events and online claims can mislead audiences, enable fraud, or cause public controversy. We introduce WildAIGC‑Bench, built through recurring collection across news websites, fact‑checking sources, and related social media content. The pipeline combines broad coverage, timely updates, and human‑verified labels through scheduled discovery, automatic screening and preliminary annotation, and human review of image‑specific evidence. The benchmark contains 407 verified AI‑generated or edited images, each accompanied by a source webpage link and an evidence summary. Images retain the conditions encountered online, including compression, screenshots, and recapture. Evaluation of dedicated visual detectors, multimodal large language models, and a simple Perception Encoder baseline reveals substantial recall gaps relative to existing benchmarks. WildAIGC‑Bench provides both a practical detection challenge and a basis for continued data expansion and source‑based verification. The dataset is released at https://github.com/YuZijian/WildAIGC‑Bench.
Authors:Taiheng Pan
Abstract:
Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users. We measure how often this happens. We trace seven open‑source frameworks for RL post‑training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026. Of 1,598 papers that propose a general RL training method, 37 reached at least one framework. Across five quarterly cohorts of papers, the number included within 180 days stayed between 5 and 7 while the cohorts grew from 72 to 386 papers. Compared with papers from the same quarters, included papers more often trained policies of 30B parameters or more, at 41 against 10 percent, and more often changed a coefficient, clip, mask or weight in the existing loss, at 29 against 9 percent. Both gaps hold without papers from the companies that maintain the frameworks, while code release was equally common in both groups. The frameworks' issues and pull requests show that few methods are turned away in public. More than nine in ten of the general methods no framework included have no thread there that cites or names them, and many proposals that do appear stay open without a reply. Two further records, the frameworks' issues and the technical reports of 13 large open‑weight models, show that users rarely report training failures in public and that the reports state 8 of the 13 widely included methods. We release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.
Authors:Cui Enhe, Jinyue Yang, Bojian Yin, Guoqi Li
Abstract:
Diffusion models generate samples through a multi‑step denoising trajectory, yet condition every step on the same instruction, so the trajectory cannot be addressed at the level of individual operations. We introduce Dancing Diffusion, which assigns each span of timesteps a dedicated continuous condition block and thereby turns conditioning into a latent program. A compiler maps text features into this program, and a fixed pretrained generator executes its blocks over successive sampling intervals. The trajectory can then dance, keeping its executed history while its continuation is redirected. A shared prefix is executed once and reused, and a later operation is rewritten while content fixed by earlier operations is preserved. The compiler is trained through the images it produces, with visual feedback on both the requested change and the retained content. On SANA, reusing an executed prefix preserves markedly more of the scene than regenerating from noise at a similar strength of edit, learned conditions preserve more background than native prompt switching from the same saved state, and the learned program improves generation quality over native SANA. Later branches weaken the edit, which leaves semantic control over individual operations limited. By converting a single‑instruction trajectory into a program of addressable operations, Dancing Diffusion lays a foundation for structured intermediate computation in diffusion models.
Authors:Jihai Zhang, Runjia Qian, Kaiqin Kong, Biao Jiang, Puyi Wang, Siru Chen, Tianle Li, Guanjie Chen, Junyi Wu, Chubin Chen, Yuanhang Li, Kaichen Huang, Haobo Zhang, Yangguang Li, Wei Yu, Guibin Chen, Yu Cheng, Yahui Zhou
Abstract:
Moving from controllable videos to genuinely playable worlds requires persistent world state, executable game rules, and low‑latency interaction. We outline a roadmap for the next generation of Matrix‑Game that separates game execution from visual generation through an executable game harness, and introduce Matrix‑Game 4.0 as the first system built on this design. A lightweight game engine maintains world state and executes interaction rules, a multimodal coding agent translates player ideas into game code and dynamic scene captions, and a renderer based on autoregressive video model transforms engine‑rendered low‑poly proxies into detailed visuals, conditioned on dynamic captions and memory references. This framework provides continuous guidance for visual generation and enables natural‑language creation and modification of scenes and gameplay. Our renderer is trained on aligned low‑poly and high‑fidelity video pairs and distilled into a causal generator that generates in four‑frame chunks with one or two denoising steps. Together with targeted inference optimizations, Matrix‑Game 4.0 enables 480p, 60 FPS gameplay on a single NVIDIA RTX5090 with a key‑press‑to‑display latency of 112ms. Single‑step generation achieves 83FPS steady‑state throughput with 18.2GiB of peak GPU memory usage. With fixed prompts, steady‑state throughput also reaches 67FPS on an RTX5080 and 58FPS on an RTX5070Ti. We demonstrate customizable shooter and driving games with engine‑enforced mechanics, and release code, model weights, and low‑poly game assets to support research and creation of playable game worlds.
Authors:Antonio Langella, Giovanni Minelli, Giovanni B. Dessy, Lorenzo Amatucci, Giulio Turrisi, Victor Barasuol, Claudio Semini
Abstract:
Learning loco‑manipulation policies requires demonstrations that capture coordinated base and arm motion on the target embodiment. Robot‑free interfaces allow demonstrations to be collected independently of the target robot, increasing flexibility but without directly capturing platform‑specific commands, proprioception, and body‑mounted observations. On‑robot teleoperation allows to capture these signals directly, but requires an interface capable of commanding the complete system concurrently. This paper presents LocoPilot, a single‑operator bilateral teleoperation and data‑collection framework implemented on a 12‑DoF Unitree Go2 quadruped equipped with a 6‑DoF AgileX Piper L manipulator, an Intel RealSense D435 wrist camera, and an Intel RealSense D405 head camera. A second 6‑DoF Piper L arm, used as the leader and two handheld controllers provide simultaneous control of the base, body posture, manipulator, and gripper. The follower arm interaction forces are estimated without a dedicated force/torque sensor and rendered to the operator through a passivity‑constrained haptic channel. A ROS~2 pipeline records synchronized multimodal observations together with a unified 11D action shared by teleoperation and policy deployment. Compared to prior work, LocoPilot uniquely unifies robot‑in‑the‑loop legged data collection, force feedback without motion‑capture instrumentation, and the same action interface at deployment. We evaluate the interface across multiple loco‑manipulation tasks and validate the complete collection‑to‑deployment pipeline by fine‑tuning MolmoAct2 \citeMOLMO on pick‑and‑place demonstrations and deploying on the physical robot without action retargeting or additional inverse kinematics. Code available at: https://github.com/iit‑DLSLab/LocoPilot
Authors:Yukuo Ma, Cong Liu, Junke Wang, Junqi Liu, Haibin Huang, Zuxuan Wu, Xuelong Li
Abstract:
Long‑horizon video generation requires retaining an ever‑expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high‑dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self‑attention, reducing token count while retaining block‑level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual‑pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench‑Long overall score of 80.1 and competitive static‑scene revisitation performance on R2M‑Bench. It also leads on all four consistency metrics for off‑screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.
Authors:JunHyeok Oh, Byung-Jun Lee
Abstract:
Reasoning‑enabled language model agents can incur substantial computation before acting, although the reasoning needed varies across decisions. Existing stopping signals, such as confidence and intermediate‑answer stability, can vary in effectiveness across reasoning models; signals obtained from hidden states or token probabilities also require internal access that may be unavailable or impractical. We introduce StreamStop, which learns model‑specific stopping policies offline from prefix‑level action agreement. A frozen prefix reader encodes available reasoning text for a lightweight stop head, allowing stopping without access to the reasoning model's internal signals. We construct and publicly release a dataset of agent reasoning trajectories with prefix‑level action annotations. These annotations compare each prefix action with the full‑reasoning reference action, enabling offline comparison of stopping rules and exact expected‑return optimization of a discounted action‑agreement objective. Conditioning the head on the discount factor allows the agreement‑token trade‑off to be adjusted at inference without retraining. In closed‑loop evaluations on WebShop and ScienceWorld, StreamStop achieves higher observed success than most validation‑selected fixed‑budget settings while using fewer reasoning tokens than Full thinking. On WebShop with Qwen3.8‑27B, it uses 42% fewer reasoning tokens per episode with similar observed task success.
Authors:Wenbin Liao, Hao Zhu, Yike Ma, Hao Jiang, Feng Dai
Abstract:
The Segment Anything Model 3 (SAM3) has achieved significant progress in Promptable Concept Segmentation (PCS) by processing short noun phrases to generate segmentation masks with unique instance identifiers. While SAM3 excels at single‑concept segmentation, it falls short in complex, real‑world environments where multiple concepts must be segmented simultaneously. In particular, SAM3 typically fails to simultaneously process concepts with semantic overlap, leading to semantic misclassifications and redundant mask predictions. Moreover, as the number of concepts increases, the independent inference paradigm introduces prohibitive inference latency. To address these limitations, we propose the Dense Competitive Representations SAM (DCR‑SAM) framework. By decoupling cross‑modal interactions, the proposed method alleviates the inference latency. To resolve semantic misclassifications, DCR‑SAM dynamically aligns textual concepts with visual representations and incorporates a dense competition head to enforce explicit inter‑class competition. Furthermore, a learnable background token is applied to absorb non‑target objects dynamically. These mechanisms effectively suppress overlapping predictions and generate high‑fidelity masks. Extensive experimental evaluations across multiple benchmarks demonstrate that the proposed framework: (1) achieves a 2.3% ~ 10.7% performance improvement over state‑of‑the‑art methods, (2) delivers a 33× inference speedup compared to the native SAM3, and (3) exhibits strong robustness across diverse scenarios.
Authors:Byeonghun Bang, Sangwoo Park, Jinheon Baek, Kangsan Kim, Youngwan Lee, Sung Ju Hwang
Abstract:
Recent advances in large (vision) language models have enabled computer‑use agents that automate digital workflows via fine‑grained keyboard and mouse operations. However, such agents remain fragile in long‑horizon settings, where errors accumulate and executed actions leave unintended side effects, making the ability to recover from failures critical. Yet, existing remedies remain inadequate: environment resets are infeasible in proprietary settings, action replay is prohibitively expensive over long horizons, and API‑based rollback is limited to web environments, leaving model‑based recovery largely unexplored. To address this gap, we present ReTrace, a scalable evaluation framework for systematically studying recovery behavior in computer‑use agents. We formalize Transition‑Consistent Recovery: identical downstream actions from the recovered and target states must yield identical outcomes. We operationalize this criterion through goal‑oriented action replay following controlled error injection. Using ReTrace, we present the first systematic study of autonomous, model‑based recovery, revealing that (i) ideal recovery can unlock model's latent task completion capabilities, (ii) current computer‑use agents fail to exploit their latent recovery capabilities, and (iii) accurate model‑based recovery yields measurable end‑to‑end gains. Together, these findings establish recovery as a fundamental capability for building reliable computer‑use agents. Our code and dataset is available at https://github.com/BangProx/ReTRACE.
Authors:Yue Ma, Pengjie Song, Xinyu Wang, Yi He, Zeqian Long, Fangneng Zhan, Kaichen Zhou, Hongyu Liu, Hongfa Wang, Peihao Li, Haoyang Huang, Nan Duan, Qifeng Chen
Abstract:
Egocentric action transfer aims to decouple the semantic action from egocentric observations and reproduce them in new visual contexts, enabling scalable world simulation, embodied policy learning, and robotic data generation. Existing approaches to transferring actions rely on condition‑guided video generation, which converts the observation video into explicit geometric controls (e.g., hand poses or meshes) to drive synthesis. However, these methods introduce geometric estimation errors, making it difficult to preserve physically plausible interactions when transferred to new scenes. Alternatively, motion transfer methods directly extract motion patterns from the observation video, yet motion‑level features alone cannot encode rich contact dynamics, frequently leading to severe hand structural collapse and implausible contact layout. To address both limitations, we present EgoACT, a test‑time framework for egocentric action transfer that operates directly on the denoising process of video diffusion models without requiring additional geometric estimators. EgoACT uses Velocity‑guided Structure Anchoring to stabilize reference‑consistent hand‑object structure in the early denoising stage, and Sparse Correspondence Calibration to refine reliable local correspondences in the mid‑to‑late denoising stages. Together, these two components preserve transferable action semantics while improving temporal coherence and interaction realism. We further establish EgoActionBench, a benchmark for evaluating action preservation, visual quality, and hand‑object plausibility across diverse egocentric manipulation scenarios. Experiments show that EgoACT generates more coherent and physically plausible interaction videos than strong baselines.
Authors:Benjamin Donnot, Sami Tazi, Eva Boguslawski, Nourredine Henka
Abstract:
Learned surrogates for power‑flow (PF) computation are usually motivated by speed, compared against a single CPU‑based baseline. We show this comparison is fragile: across 7 open‑source CPU‑based solvers evaluated on identical inputs, throughput varies by roughly 2.5‑3 orders of magnitude, so the choice of baseline alone can flip the conclusion. We build a benchmark spanning 9 physics‑based solvers: 7 CPU‑based and 2 GPU‑based, including a new batched GPU solver, gumsim2grid ‑ and 3 published ML surrogates, evaluated under a shared, independently implemented KCL‑mismatch check across 15 grids from 14 to 70,000 buses. On the PF task itself, a sufficiently parallelized CPU solver or gpusim2grid matches or exceeds the throughput of the ML surrogates we test while remaining orders of magnitude more accurate; we find no evidence that these GNN‑style surrogates are yet competitive for PF specifically, though prior work reports genuine gains on related but more combinatorial tasks such as optimal power flow or power grid control. Beyond this finding, we hope the benchmark suite, code and reference evaluator released at https://github.com/Grid2op/gpusim2grid helps close a broader reproducibility gap in this space, where batched GPU‑based PF solvers are rarely open‑sourced or evaluated against a common set of baselines.
Authors:Jiahui Ning, Liang Chen, Yuchen Liang, Yi Zhang, Fenglei Fan
Abstract:
Model compression makes large language models deployable, but today's evaluation tells an incomplete story. Standard practice measures retained task performance and stops there. This misses a critical fact: compression perturbs the distributed knowledge embedded across billions of weights, and task scores alone cannot reveal which underlying abilities survive and which quietly collapse. To solve this question, we introduce ACE‑Compression, an ability‑centric evaluation framework. Instead of asking only "does the compressed model still answer correctly?", we ask "which of its core abilities remain intact?" Our framework probes four deployment‑facing dimensions: procedural reasoning and constraint execution, knowledge grounding and truthfulness, social cognition and affective understanding, and persona fidelity and stylistic consistency. Across more than 35 compressed checkpoints from six open‑weight LLMs‑covering quantization and pruning‑we find that compression redistributes ability profiles. Coarse, local, and surface‑level signals often persist, while fine‑grained, long‑tail, and context‑dependent abilities become fragile. Quantization consistently outperforms pruning in retention, but the practical meaning of that gap depends entirely on which ability an application requires. Based on our findings, deployers can choose compression methods by the ability profile their application needs, not just by a leaderboard average. And algorithm designers gain diagnostic signals to build compression methods that preserve what matters. We believe that ACE‑Compression turns compression from a black‑box accuracy drop into an interpretable intervention‑one we can measure, understand, and ultimately improve.
Authors:Marcel Dreier, Dakota Pyles, Nora Gourmelon, Moritz Koch, Fei Wu, Thorsten Seehaus, Matthias Braun, Andreas Maier, Vincent Christlein
Abstract:
Mountain glaciers and ice sheets are undergoing rapid loss of ice mass. To assess their current volume and ice‑thickness distribution, and to accurately forecast their future, ground‑penetrating radar systems are employed to measure ice thickness. These instruments produce radio‑echo sounding data that reveal the internal structure of the ice and its boundaries at the surface and beneath the ice body. However, delineating these interfaces is time‑consuming as the surveyed area can span several hundred kilometers. To accelerate the labelling process, automated methods have emerged, but they often struggle to delineate the ice bottom, particularly in temperate glaciers. The high water content inherent in temperate ice increases both scattering and attenuation of the radar signal, resulting in weak and ambiguous reflections at the ice bottom and generally noisier data. As a result, the visual features corresponding to the ice bottom become ambiguous. To address this issue, we propose several extensions to deep learning models that incorporate additional contextual information into the prediction. Specifically, we integrate modifications that incorporate additional spatial and geo‑spatial contextual information into the network. With these enhancements, we achieve a new state‑of‑the‑art performance on the IceAnatomy benchmark dataset, with mean errors of 10.9 m for the FAU subset and 43.4 m for the CReSIS subset in a single combined system. We also demonstrate that our method generalises well to previously unseen data, outperforming the baseline by 49.6 % on new campaign data from Patagonia. The model code and the new dataset are available at https://github.com/ki7077/ICIBS/tree/main and https://zenodo.org/records/23392580.
Authors:Seohyun Lee, Jongha Kim, Dohwan Ko, Ji Soo Lee, Seoung Choi, Jayun Ku, Hyunwoo J. Kim
Abstract:
Enterprise agents answer questions over workplace records that evolve through team collaboration. As information is shared, corrected, and revised across documents, chats, and figures, multiple versions of the same information accumulate in the workspace. Reliable question answering therefore requires agents to reason over sufficient evidence, resolve cross‑source and temporal conflicts, and recognize when the available evidence is insufficient to establish an answer. To systematically evaluate these capabilities, we introduce MMEnterprise‑Bench, a benchmark comprising 1,613 questions grounded in evolving multimodal records from simulated collaboration projects, covering three categories: Evidence Reasoning, Conflict Resolution, and Unanswerability. We evaluate 10 proprietary and open‑source multimodal LLMs and find that even the strongest models struggle when evidence contains cross‑source or temporal conflicts or spans multiple images. Moreover, increasing the retrieval budget or employing a larger retriever improves evidence recall but does not consistently translate into higher answer accuracy. These findings highlight the persistent challenges of reasoning over evolving workplace records and establish MMEnterprise‑Bench as a testbed for developing more reliable enterprise agents. Code and data are available at https://mlvlab.github.io/MMEnterprise‑Bench/.
Authors:Diego Miguel Lozano, Erik Derner, Juan Antonio Pérez-Ortiz, Nuria Oliver
Abstract:
Negation remains a persistent challenge for language models across NLP tasks and modalities. We show that this weakness can undermine LLM safety guardrails through a structural vulnerability we term polarity blindness: the failure to recognize harmful intent after polarity‑altering linguistic transformations. We study two forms of polarity inversion: grammatical negation and antonym replacement, and use synonym replacement as a polarity‑preserving control. We further test whether an adversarial textual wrapper amplifies these failures. Using a 260‑prompt benchmark across 10 state‑of‑the‑art LLMs and 3 languages (English, Spanish, Czech), stratified by harmfulness potential and evaluated on a validated 0‑5 harm scale, we find that polarity blindness affects most evaluated models, with mean harm scores reaching 4.12 for the most vulnerable model. The effect persists across typologically differing negation systems, suggesting that polarity blindness reflects a broader limitation in semantic robustness rather than a language‑ or grammar‑specific failure.
Authors:Amane Yamaguchi, Shohei Kamiguchi, Masashi Hamaya, Shigeo Yoshida, Takuji Narumi, Mai Nishimura
Abstract:
Imitation learning increasingly relies on novice operators, whose demonstrations fall short of experts' in number and inter‑trial consistency. Operators act on what they are shown, yet how the rendering of the robot affects their demonstrations, and the policies trained on them, has not been examined. We study this question with PointHand, a teleoperation interface that renders a multi‑fingered robot hand as sparse keypoints at its fingertips and joints, the same keypoints a policy can observe. In a simulated mug‑in‑box task, 18 novices and 4 experts collected 2,200 trials under three renderings: the real shape of the hand, all keypoints emphasized, or the thumb and index finger alone emphasized. The novices shown the thumb and index finger emphasized collected 441 successful demonstrations from 600 trials, against 320 for those shown the real shape. We then trained diffusion policies on these demonstrations, replacing the scene point cloud and proprioceptive state they observe with the hand keypoints and two object keypoints. Within 400 steps these demonstrations went from the weakest of the three novice conditions under the point‑cloud policy (46.3%) to the strongest under the keypoint policy (57.3%), a gain of +11.0 percentage points in all five seeds. Those collected with all keypoints emphasized changed by at most +0.7 percentage points, and in no more than three seeds. The value of a demonstration therefore depends on the rendering shown during collection and on what the policy observes together. A shared representation can pay off on both sides, in the number of successful demonstrations and in the success rate of the policy trained on them. Project page: https://omron‑sinicx.github.io/point‑hand/
Authors:Genshin Kakimoto, Atsushi Yanagisawa
Abstract:
Staged post‑training of multimodal foundation models chains vision‑language alignment, reasoning distillation, and domain tuning, yet how each phase trades capabilities against the others remains under‑characterised. Tracking an 8.4B‑parameter model from an un‑fine‑tuned baseline through three training phases on Japanese financial documents (evaluated on five text‑input benchmarks), we find the phases interact non‑additively: math‑only distillation overwrites the financial knowledge acquired earlier even as procedural math skill survives, and the domain phase repairs more than it installs. A skip‑phase ablation is best for domain knowledge but forfeits numerical‑reasoning gains, so per‑phase contributions are not decomposable and the best checkpoint is task‑dependent, with implications for checkpoint retention and scheduling rather than added stages.
Authors:Younghyun Kim, Wooseok Seo, Minseo Kim, Seunghyuk Oh, Minjae Lee, Donghoon Kim, Hyung Il Koo, Jaehyung Kim, Wonjun Kang
Abstract:
Recent studies in harness optimization have shown substantial performance gains in language model agents without updating model weights. A well‑designed harness more effectively leverages the underlying model's capabilities by shaping context, tool use, and execution workflows. However, naïve success rates alone do not fully characterize whether optimized harness yields more reliable success on tasks already within reach (elicitation) or a broader range of solvable tasks (expansion). This distinction is crucial for recursive self‑improvement and evolutionary search, as expansion may provide a broader range of successful experiences to push the capability boundaries. Despite the growing significance, this distinction remains underexplored in harness optimization. We conduct a systematic analysis to study the effect of gains from harness optimization. Building on pass@k analyses in the reinforcement learning (RL) literature that compare models before and after RL training, we compare optimized harnesses with their pre‑optimization counterparts. Specifically, we assess changes in reachable task coverage, compare harness optimization with model scaling and increased inference‑time compute, and examine how gains vary with task difficulty. Across static prediction and dynamic tasks, we find that pass@1 advantages of harness over stronger interventions can reverse at larger attempt budgets and aggregate gains conceal shortfalls on hard tasks, underscoring the need for a fine‑grained diagnosis.
Authors:Peng Zhang, Shuaihao Han, Tingfa Xu, Jianan Li
Abstract:
Multispectral aerial object detection aims to exploit spectral‑spatial cues to address object detection challenges in complex aerial scenes. Existing methods mainly focus on spectral‑spatial decoupling and object‑aware spectral‑spatial fusion to improve spectral utilization. However, two critical issues remain underexplored: 1) severe foreground‑background imbalance caused by large scene coverage hinders reliable object recognition; 2) aerial objects exhibit substantial local variations in scale, orientation, and structure, which hinder accurate localization. To address these issues, we propose SGLDet, a spectral‑aware global‑local dynamic modeling framework for multispectral aerial object detection, which is motivated by the observation that object semantics are jointly determined by global spectral consistency and scene context, while precise localization requires dynamic feature modulation conditioned on the input spectral and spatial properties. To realize these properties, SGLDet consists of three coordinated components: a Spectral‑driven Global Dynamic Modeling module that aggregates spectrally related features across the scene via similarity‑guided sparse matching; a Multi‑scale Global Context Integration module that constructs a compact scene memory for scene‑guided contextual reasoning; and a Spectral‑aware Input‑dependent Calibration Aggregation module that performs expert‑kernel aggregation with scene‑guided spectral‑spatial calibration to enable adaptive local feature modulation. Extensive experiments on MODA, HOD3K, and DroneVehicle demonstrate that SGLDet achieves superior performance with competitive computational cost. Code is available at https://github.com/Riiluo/SGLDet.
Authors:Peng Zhang, Tingfa Xu, Jinrui Sui, Yuyang Tian, Jianan Li
Abstract:
Multispectral object detection requires detectors to exploit material‑dependent spectral responses while maintaining consistent localization under geometric variations. Existing MOD detectors learn discriminative spectral representations, yet lack a unified mechanism that preserves spectral order, adapts spectral‑context fusion across pyramid levels, and regulates geometric responses under rotation. To address these limitations, we propose S^3E‑Det, a strict spatial‑spectral equivariant detector that couples ordered spectral modeling with transformation‑consistent prediction. Spectral Order‑aware Sampling Calibration (SOSC) imposes local order constraints on input‑derived spectral features through first‑order variation, while Common‑Differential Equivariant Fusion (CDEF) adapts spectral‑context aggregation across pyramid levels. A compact Geometric Alignment Head (GAH) reconciles invariant and equivariant output properties within a unified prediction stage. Across MODA, HOD3K, and DroneVehicle, S^3E‑Det combines high detection accuracy with a compact parameter footprint and reduced rotation‑induced variation. Code is available at https://github.com/Riiluo/S3E‑Det.
Authors:Chuanyang Ma, Chenye Wang, Yixuan Wang, Huiling Zhou, Xingqun Qi, Muyi Sun, Man Zhang, Caifeng Shan, Zhenan Sun
Abstract:
Anticipating how users will interact with their surroundings is essential for proactive systems that provide timely assistance in AR, VR, and embodied applications. Prior work typically forecasts hand‑object interactions directly from historical observations, leaving intention and contact geometry implicit. However, similar observation histories can precede semantically or geometrically distinct futures. Inspired by behavioral evidence that gaze can provide early cues for underlying intention, we propose Gaze2HOI, a gaze‑aware framework that formulates hand‑object interaction forecasting as a hierarchical problem comprising intention inference, contact grounding, and coupled hand‑object motion generation. Specifically, Gaze2HOI first weights gaze cues by fixation strength and recency, and uses them to modulate the visual representations of a vision‑language model (VLM) through Temporal Gaze Saliency Modulation (TGSM), thereby reducing semantic ambiguity in intention inference. Subsequently, an Anticipatory Contact Grounding (ACG) module maps these structured intentions into explicit contact, support, and placement fields, effectively mitigating geometric ambiguity. Driven by these explicit conditions, a diffusion‑based model jointly generates the 6‑DoF object trajectories and future articulated motion of both hands. To evaluate Gaze2HOI, we further introduce EgoAIM, a phase‑aware benchmark that organizes egocentric hand‑object interactions into four phases defined by evolving hand‑object relations and provides fine‑grained intention annotations. Extensive experiments demonstrate that Gaze2HOI achieves state‑of‑the‑art performance in both intention inference and motion forecasting, even approaching oracle‑level end‑to‑end performance in specific interaction phases. Project page: https://gaze2hoi.github.io/.
Authors:Shuqi Liu, Yingkai Zhang, Lin Gu, Tao Zhang, Ying Fu
Abstract:
Hyperspectral Image (HSI) reconstruction has made substantial progress with the deep unfolding framework by decomposing the problem into a projection module and a denoiser module. Nevertheless, existing methods still exhibit limitations of insufficient matching with HSI data. The issues lie in two aspects: 1) the data‑consistency projection module applies a fixed gradient descent step while ignoring the spectral content adaptivity of HSI; 2) the denoiser module either suffers from high computational complexity or is limited to the local receptive field, failing to efficiently capture the global spatial‑spectral dependencies. In this paper, we propose a deep unfolding framework centered on a two‑stage manifold learning strategy to obtain a degradation‑free structural prior, which is then strategically injected into both the projection and denoiser modules. In the data‑consistency projection, a prior‑guided spectral weight mechanism is introduced to adaptively modulate the weight of spectral band. In the denoiser module, a manifold‑guided Mamba is proposed to achieve the structure‑aware extraction of spatial‑spectral features through prior modulation. Quantitative and visual comparisons on synthetic and real‑world datasets show that the proposed method significantly outperforms existing state‑of‑the‑art approaches, while maintaining the structural interpretability of the unfolding framework. The code is available at https://github.com/SqLiu67/MPG‑DU.
Authors:Imre Boros
Abstract:
We consider global Chebyshev collocation for the ignition problems y'=y^q(1‑y), y(0)=δ, q\ge2; for q=2 this is the flame problem, a standard stiff test equation. After rescaling to [‑1,1] the solution has an interior layer of width O(μ), μ=(q‑1)δ^q‑1, at a δ‑dependent position. Two obstacles arise: plain Chebyshev collocation needs O(1/μ) nodes, and imposing the initial condition amplifies errors by a factor of order 1/μ, which makes the collocation equations severely ill‑conditioned. Our main contribution addresses the second one. Since all solutions are translates of one profile, the initial condition can be replaced by an equivalent interior condition. For an unknown u=φ(y), imposing it where the speed |φ'f_q| of u along the orbit is maximal bounds the propagator of the linearized equation by one; for u=\ln y this is y(x^\ast)=(q‑1)/q. For the first obstacle we use the sinh map of Tee and Trefethen: the time function is explicit, the complex singularities of the profile lie exactly at \pm(2k+1)πi, and the map parameters follow in closed form, so that O(\ln(1/μ)) nodes suffice. In double precision the pinned scheme reaches the interpolation error of the exact solution in five or six Newton iterations for μ down to 10^‑12, whereas the initial‑condition form fails or converges to wrong discrete solutions, also in an integral (Birkhoff) reformulation. Computations in 256‑bit arithmetic confirm the predicted rate down to errors of 10^‑70, AAA approximation locates the singularities to within 2%, and the method carries over to an Arrhenius reaction rate, with all parameters computed numerically.
Authors:Haocheng Wang, Youquan Liu, Weiyu Guo, Aokai Zhang, Donghang Wu, Hongyu Liu, Sha Zhang, Weidong Yang, Ben Fei
Abstract:
Vision‑language‑action (VLA) robustness is commonly summarized by aggregate task success, which obscures how individual executions change under distribution shifts and where failures emerge in long‑horizon tasks. We introduce RoboCasa‑Shift, a diagnostic robustness benchmark built on RoboCasa365 that combines controlled single‑factor shifts, matched clean‑perturbed rollouts in which each pair shares the same underlying settings and differs only in the target shift, and simulator‑grounded stage‑aware diagnostics. We evaluate GR00T N1.5 on four long‑horizon tasks that pass a predefined clean‑performance gate under four perturbation factors and three calibrated severity levels, including conditions that drive the evaluated policy to complete task failure, totaling 1,040 formal rollouts. Robustness responses vary substantially across factors and tasks, and increasing severity does not always produce monotonic degradation. For example, one camera condition yields Δ\mathrmSR=0 despite 8 of 80 matched episodes changing outcome. Stage‑aware analysis further shows that the same end‑to‑end degradation can correspond to different execution‑depth profiles. Camera L_2 and Gaussian blur L_1 both yield Δ\mathrmSR=0.188, yet their first‑stage failure rates are 19.0% and 36.4%, respectively. Our findings suggest that VLA robustness should be viewed as a multi‑dimensional property of execution rather than a single end‑to‑end performance measure, motivating evaluation protocols that jointly capture outcome degradation, paired outcome sensitivity, and failure progression. The code is publicly available at https://github.com/HaochengWang23/RoboCasa‑Shift.
Authors:Junha Song, Geonmo Gu, Jeongeun Park, Heejeong Nam, Lucas Maes, Byeongho Heo, Dongyoon Han, Sangdoo Yun
Abstract:
This paper shows that JEPA‑style latent world models (LWMs) can learn across diverse domains and data sources by adopting the language‑conditioned interface of video generation models. We establish this approach as an effective and efficient basis for planning with imagination and downstream policy fine‑tuning. To this end, we introduce LaLa‑World, a Language‑conditioned Latent World model that learns from simulation and robot trajectories together with broader image and video data. Two designs make this work. First, we describe both numerical actions and visual changes in language, enabling joint training of a single future predictor across heterogeneous datasets. Second, we build the predictor on a language model, allowing it to interpret language conditions while predicting future latent states. The resulting predictor achieves higher average MPC planning performance than existing LWMs trained separately for each domain or task. Adding image and video data further improves average planning performance and robustness to action phrasing. In a matched comparison with an RAEv2‑based video generation model baseline, LaLa‑World reaches comparable planning performance with 6× less training cost and 13.9× faster planning. Beyond planning, the pretrained LaLa‑World serves as an initialization for LaLa‑Policy, which jointly predicts actions and future visual representations. LaLa‑Policy performs competitively with Cosmos Policy on LIBERO and RoboCasa, with up to 5.5× faster inference.
Authors:Theresia Veronika Rampisela, Maria Maistro, Tuukka Ruotsalo, Christina Lioma
Abstract:
Recommender Systems (RSs) can be evaluated for many aspects, two of them being recommendation relevance and fairness. These two aspects can be evaluated either in isolation, using a measure for each aspect; or jointly with a single`relevance‑aware' fairness measure that quantifies fairness with respect to relevance. Yet, neither approach provides a simultaneous evaluation of both relevance and fairness: the first approach may result in two different rankings of models, each corresponding to an evaluation aspect; the second approach often provides an imbalanced evaluation of the two aspects. To address this gap, we propose Distance to Pareto Frontier (DPFR), a new approach to jointly evaluate RS fairness and relevance. DPFR is based on a Pareto frontier of relevance and fairness scores from existing measures. The frontier is generated from user‑item interaction data and represents various Pareto‑optimal trade‑offs between the two aspects. Thus, the distance from a point in the frontier provides a joint score of fairness and relevance. As DPFR can be used with existing measures, it is modular and intuitive. Our experiments with 4 recommenders and 3 fairness‑aware re‑rankers on 6 real‑world datasets reveal inconsistent associations between existing measures and DPFR, showing that they are not reliable proxies for evaluations based on Pareto optimality. We further examine the robustness of DPFR as a joint evaluation approach for fairness and relevance by conducting experiments with artificial recommendations and with various fairness‑relevance trade‑offs, as well as by analysing DPFR's agreement to simple measures of fairness and relevance. These analyses show that DPFR is more robust than existing joint evaluation approaches. Finally, we provide guidelines for evaluating fairness and relevance jointly with DPFR. Our code: https://github.com/theresiavr/robustness‑DPFR‑recsys‑evaluation.
Authors:Guanchen Li, Yixing Xu, Xuanwu Yin, Dong Li, Emad Barsoum
Abstract:
As large language models (LLMs) grow more powerful, their scale makes them costly to serve. Depth pruning offers a practical route to efficiency by directly reducing memory and latency without hardware‑specific support. However, we observe that transformer layers exhibit markedly different task‑dependent layer contributions, whereas existing depth‑pruning methods ignore this heterogeneity and thus tend to optimize for a narrow domain or benchmark rather than preserve broad capability. This suggests that LLM depth pruning should be formulated as a multi‑objective optimization problem rather than a single‑score ranking problem. We propose SliMOO, an interpretable multi‑objective evolutionary search framework for LLM depth pruning. SliMOO first uses single‑layer removal probing to construct a task‑layer importance map, which provides an interpretable view of task‑shared and task‑specific layers and serves as a search prior. It then performs a Pareto‑aware evolutionary search for pruning masks, where candidates are generated under the guidance of the task‑layer prior, scored by their deviation from the dense model on each task, and retained through NSGA‑II environmental selection. Across the Llama‑3.1 and Qwen3 model families, SliMOO consistently achieves stronger multi‑task trade‑offs than rule‑based, greedy, and scalarized baselines. For example, on Llama‑3.1‑70B at 25% sparsity, it improves the math‑domain average from 24.33% to 34.40% and the code‑domain average from 35.03% to 39.14%. Code will be made available at https://github.com/AMD‑AGI/SliMOO.
Authors:Oliver Hausdörfer, Felix Minzenmay, Mani Majd, Maximilian P. Christof, Shiyuan Zhang, Angela P. Schoellig
Abstract:
Action‑free videos are an appealing substitute for costly teleoperated data. Latent action models (LAMs) turn videos into a supervision signal by inferring abstract actions between frames. Existing LAMs observe a single camera view, which resolves 6D robot actions only ambiguously missing out‑of‑plane movements and potential occlusions. We propose a LAM that ingests synchronized multi‑camera views using cross‑attention to resolve depth and occlusions before compressing them into latent actions. The latent actions are fed into a policy in which a latent expert and an action expert are coupled by sequential flow matching around a frozen VLM backbone. We further evaluate in strict cross‑embodiment settings where task‑specific demonstrations come from other embodiments. We show that dual‑view LAM pretraining improves the decodability of ground‑truth 6D actions by 17% compared to single‑view, while scaling single‑view data by two orders of magnitude is not sufficient to fully recover out‑of‑plane motions. The downstream policy improves success rate by +13.5% compared to +6.2% for single‑view LAM. A single cross‑embodiment is sufficient for skill transfer, and adding a second viewpoint outperforms adding up to four further embodiments. The results transfer to hardware where demonstrations come from a human hand improving success rate by +20% over action‑only policies in 850 rollouts. We show that video and robot data are complementary, and that benefits of learning from video are task‑dependent. Website: https://learnsyslab.github.io/multiview_latent_action_pretraining/
Authors:Yang Li, Yike Wang, Di Wan, Chujun He, Zongsi Xu, Yitao Yu, Quanfeng Lu, Xiyu Chen, Sheng Zhong, Deng Binjie, Yichen Guo, Zhenwei Guo, Hou Jun, Kunhan Guo, Weihao Lin, Tuo Zhou, Hai Liu, Chenwei Wu, Jianyuan Zhong, Xilin Dai, Ye Luo
Abstract:
Simulating economic responses at society scale requires agents whose decisions reflect household circumstances. We introduce S^2‑MAS, a society‑scale multi‑agent simulation framework built on 500 million synthetic household personas grounded in real‑survey data. Its Genese engine separates decisions from state updates, while reusable financial scenarios and application ledgers track consumption, cash, debt and payments. Trained with SFT + RLCD on 2.93 million examples, sharp‑421m‑mlx achieves 74.55% accuracy and 38.37% question‑macro F1 on one‑shot consumer financial‑choice tasks derived from held‑out survey responses. Its accuracy exceeds GLM‑5.3‑Flash, Jev 1.13, DeepSeek‑V4.1‑Flash, GPT‑6‑Luna and GPT‑6‑Astra in the common benchmark (full test for the first four; matched 500‑group sample for Astra). We evaluate eight financial task families and simulate 12 million distinct household agents in paired purchase‑funding conditions. Jointly raising purchase goals and approved credit limits increases consumption, with responses differing across age and income groups. Education, employment and household‑size differences persist after age‑income standardization and mainly reflect transitions from deferral to purchase. Asset and debt groups differ in cash drawdown, repayment and closing debt. S^2‑MAS links heterogeneous household decisions to evolving consumption, cash and repayment paths, supporting economic experiments across household groups and population scales.
Authors:An-Sheng Jhang
Abstract:
Large language models (LLMs) are increasingly used as judgment assistants and as stand‑in participants in psychology experiments, yet little is known about how much their probability estimates move in response to a single new piece of information. We pair premise‑hypothesis items from δ‑SNLI with one strengthener and one weakener each, elicit the prior and the posteriors in separate calls, and measure the directional movement s: the mean posterior change signed by the annotated direction, so that the prior cancels out. We vary whether the update is stated ("Here is something else you learned") or reported ("Someone said that"), and whether the model gives one probability or five weighted estimates. We test seven configurations: two local models (Qwen3‑8B, OLMo‑2‑7B; 240 pairs) and five configurations run through agent command‑line harnesses (120 pairs), with hypotheses and analysis rules fixed before the formal runs (not publicly registered). Under stated, direct elicitation all seven move in the annotated direction (s = 18‑28 percentage points). Reporting lowers s in all seven point estimates (significantly in six, by 2‑5 points) and raises the share of reverse movements in all seven: the effect is consistent but small. Five weighted estimates lower the share of responses at \leq5% or \geq95% in all seven, but in the two local models they also reduce s by 43% and 68%, whereas in the harnessed configurations the point estimates of the change lie between ‑3.7 and +0.5 points. An elicitation format that reduces extreme values thus need not preserve sensitivity to evidence; the two should be tested separately. Because δ‑NLI annotates only direction, we make no claim about how large an update should be. Code, data and the frozen plan: https://github.com/asj252/llm‑judgment‑studies
Authors:Jiyoung Lim, Jee-Hyong Lee
Abstract:
Real‑world datasets are inherently class‑imbalanced, leading models to favor majority classes and struggle with minority ones. Existing logit adjustment methods address this by modifying logits based on class frequencies, but their class‑wise adjustments ignore instance‑wise confusion and often suppress informative signals from confusing non‑target classes. This results in insufficient learning of such classes, increased prediction uncertainty, and limited generalization. To address this, we propose two complementary methods. First, Confusion‑based Instance‑wise Logit Adjustment (CILA) adaptively adjusts logits based on instance‑wise confusion between the target and non‑target classes, restoring informative gradients that class‑wise adjustment suppresses. Second, Batch‑adaptation Noise‑Injection (BNI) perturbs the target logit during training, extending the confusion‑aware update to correctly classified but near‑boundary samples that the penalty alone cannot reach. CILA is a loss‑agnostic plug‑in module that can be seamlessly integrated into existing long‑tailed learning methods, and consistently improves performance across multiple benchmarks. Our code will be available at https://github.com/maya0395/CILA.
Authors:Zhipeng Cao, Taoran Wu, Weiru Cui, Luke Ong, Bai Xue
Abstract:
Learned barrier certificates provide a principled mechanism for establishing safety of stochastic systems, but their validity may deteriorate during deployment when the underlying dynamics differ from those used for training. We propose an anytime‑valid runtime monitor that audits a learned barrier certificate using observed closed‑loop trajectories. The monitor is intended to detect loss of statistical support for the deployed barrier condition; it does not by itself guarantee detection before unsafe entry or safety of a subsequent intervention. Specifically, the conditional barrier‑drift condition is formulated as an episode‑wise sequential hypothesis test, and observed barrier increments are transformed into a nonnegative test supermartingale, yielding finite‑sample false‑alarm control under arbitrary stopping times. Inspired by finite‑horizon safety verification, we further introduce a deterministic temporal shaping of the monitoring boundary, making the monitor more conservative early in a prescribed deployment horizon while preserving anytime‑validity. The monitor is designed to detect statistical evidence that the safety certificate is no longer consistent with the observed dynamics, rather than to predict unsafe entry. To assess the monitor empirically, we evaluate empirical alarm behavior under valid and invalid certificates. Code and data are available at https://github.com/FormalCert‑AI/runtime‑barrier‑monitor.
Authors:Jiushu Li, Haoran Wang, Yumeng Liu, Zhiyang Dou, Yuan Liu, Yuexin Ma, Cheng Lin
Abstract:
Long‑horizon human‑object‑scene interaction (HOSI) is central to embodied AI, animation, and game development. Existing human‑object interaction (HOI) and human‑scene interaction (HSI) models excel in their respective domains, but none alone covers tasks requiring multiple interaction skills. Training unified HOSI models is hindered by scarce and costly data. We introduce World2HOSI, a multi‑expert orchestration harness that generates continuous HOSI data from 3D Gaussian Splatting (3DGS) scenes and natural language instructions. Our key insight is that long‑horizon HOSI hinges not on a stronger single model but on orchestration. Given a shared, structured scene state and an orchestration harness grounded in it, existing experts can jointly accomplish tasks beyond the reach of any of them alone. World2HOSI realizes this insight as a closed loop around a persistent scene state built from decoupled background and object assets. A harness plans action stages over this state, invokes experts via their native conditioning interfaces, and writes each action's effects back, so that every invocation is grounded in the evolving world. To reduce physical artifacts, bounded residual optimization and contact‑driven physics simulation further ensure the physical plausibility of human motion and object manipulation, respectively, while preserving interaction intent. Experiments and a perceptual user study show that World2HOSI achieves the best task completion and interaction quality on the evaluated HOSI tasks, while generating more diverse data.
Authors:Hayden Moore, Abhinav Verma, Anshuman Dutt
Abstract:
Large language models can achieve strong Text‑to‑SQL benchmark performance while still producing schema and query‑logic errors. Most evaluations characterize such failures only in the final generated query, making it difficult to determine whether a related error was also observed in an earlier, simpler step or appears only at the final stage. We present a drill‑down evaluation framework that decomposes Text‑to‑SQL queries into progressively more complex executable sub‑queries and corresponding natural‑language questions, allowing rule‑based hallucination categories to be examined across a generation path. We distinguish recurrent hallucinations, in which the same rule‑based category is observed in an earlier step and again in the final query, from emergent hallucinations, in which the category is observed at the final step without an earlier detection in the same path. Across five LLMs evaluated on an expanded BIRD‑mini benchmark, several categories show appreciable recurrence probabilities, while clause abuse shows comparatively higher conditional emergence. Under the conversational‑history condition, recurrence probabilities are generally higher and emergence probabilities are generally lower. These observations indicate that the two conditional measures do not move uniformly in the same direction under additional context and motivate trajectory‑level analysis in addition to final‑output evaluation. Code and evaluation artifacts are available in the project repository. https://github.com/HaydenMM/bird‑mini‑drilldown
Authors:Injae Kim, Minseok Joo, Jihwan Park, Sehyung Kim, Minseong Bae, Hyunwoo J. Kim
Abstract:
Feed‑forward 3D Gaussian Splatting (FF3DGS) reconstructs 3D scenes from images in a single forward pass, making it well‑suited to applications that must operate within limited resource budgets. To further improve its practicality, recent budget‑controllable FF3DGS methods allow the number of Gaussians to be adjusted at inference, so that the output meets a given storage limit or rendering‑latency constraint. However, these methods overlook the cost of inference itself: to produce a representation with the target number of Gaussians, they first predict a dense set of Gaussians and then prune or select from it, requiring nearly constant latency and peak memory regardless of the budget. We therefore propose RLSplat, a resource‑linear budget‑controllable FF3DGS framework whose representation size, inference latency, and peak memory all decrease linearly as the Gaussian budget is reduced. Instead of pruning or selection, RLSplat learns a Gaussian allocation policy that distributes the budget across local scene regions, so that exactly the requested number of Gaussians is generated. These Gaussians are then refined through resource‑linear Gaussian decoding, in which each Gaussian attends only to nearby Gaussians found via an efficient hierarchical neighbor search, so that the decoding cost decreases linearly with fewer Gaussians. In zero‑shot evaluation, RLSplat outperforms prior budget‑controllable methods at every budget, surpassing the strongest baseline even with 4× fewer Gaussians, while running up to 4.1× faster with 9.6× less peak memory at low budgets.
Authors:Ruoyu Wang, Jiahan Li, Yong Liu, Kexin Zhang
Abstract:
Semantic occupancy mapping supports embodied perception during exploration and detailed scene understanding afterward, giving rise to two complementary requirements: online incremental mapping for maintaining an up‑to‑date global map under causal constraints, and offline global map refinement for improving the completed scene representation with additional computation. We present VEOcc++, a unified voxel‑centric framework that connects both stages through a persistent sparse voxel feature map. During exploration, the map expands with newly observed regions and integrates aligned local features through reliability‑guided updates, while context‑aware decoding produces the current semantic occupancy map. After exploration, cached local features are consolidated as complementary evidence and combined with the completed online representation for multi‑scale spatial refinement. On EmbodiedOcc‑ScanNet, VEOcc++ achieves 67.12% IoU and 58.40% mIoU, establishing new state‑of‑the‑art performance for online incremental mapping. With the local predictor fixed, our online mapping pipeline outperforms VEOcc fusion by 3.70 IoU and 3.74 mIoU points, demonstrating substantial gains from the proposed global mapping design. Offline global map refinement further improves the completed map to 67.79% IoU and 59.37% mIoU using only the observations acquired during exploration. Zero‑shot deployment on a mobile robot further demonstrates transferability to unseen environments, onboard feasibility, and scalability to large‑scale indoor mapping. Project page: https://wryzju.github.io/VEOccpp/.
Authors:Jinghan Gao, Siyu Wang, Dongyu Feng, Yiyuan Pan, Rong Xiong
Abstract:
3D scene graphs support language‑guided navigation by organizing physical entities and their containment and spatial relations into hierarchical maps. Building such structured maps requires a scene representation that consistently distinguishes semantic regions and object instances across viewpoints. To this end, we present PanoGraphGS, a unified framework that integrates panoptic 3D Gaussian representation learning with hierarchical 3D scene graph construction. The framework learns semantic and instance features on shared Gaussian geometry. Prototype‑based semantic learning and association‑based instance learning produce consistent panoptic predictions across viewpoints. The learned representation supports novel‑view panoptic segmentation and direct Gaussian‑level panoptic decoding. A semantic‑aware scene graph module further recovers a floor‑room‑stuff‑object hierarchy with containment and spatial relations. We further release NaviGS, a real‑world dataset for panoramic scene understanding. The dataset provides 3DGS, mesh, RGB, semantic, and depth data. For 3D scene graph identification, PanoGraphGS outperforms the strongest evaluated baseline in object‑node accuracy by more than 50% points on average across Replica and NaviGS. PanoGraphGS also achieves state‑of‑the‑art performance among the evaluated methods in 2D panoptic segmentation. These results demonstrate the effectiveness of PanoGraphGS in both hierarchical 3D scene understanding and 2D panoptic segmentation. The project website is available at: https://github.com/gaojh0333/PanoGraphGS.
Authors:Ziwei Zhao, Yuhang Liu
Abstract:
Long‑video understanding with multimodal LLMs is fundamentally constrained by the mismatch between video length and the model's limited visual context budget, making frame selection essential for practical reasoning. Existing query‑aware methods typically rely on heuristic pipelines: they assign one‑dimensional query‑conditioned scores to frames and then apply sampling, ranking, or extraction rules to form the final subset. While effective in practice, such methods are generally not derived from an explicit optimization objective and struggle to capture the heterogeneous evidence required for complex reasoning. In this paper, we revisit long‑video frame selection from a principled optimization perspective. We formulate it as a structured evidence‑allocation problem and propose OTFS, an optimal transport framework that maps multiple evidence sources onto video frames. To reflect the asymmetry of this problem‑‑‑important evidence should be preserved, whereas redundant frames need not absorb mass‑‑‑we instantiate this view with a semi‑unbalanced entropic optimal transport objective and an efficient solver, OTFS‑Sinkhorn. Combined with an exact dynamic program for coverage‑aware subset extraction, OTFS yields a practical two‑stage training‑free pipeline. We further show that, in the single‑source case, our formulation induces a Gibbs‑type distribution over frames, making Q‑Frame's temperature‑scaled distribution a special case of our framework while also providing a broader perspective for understanding related prior methods. Experiments on three long‑video understanding benchmarks show that OTFS consistently outperforms strong training‑free baselines. Codes are available at https://github.com/barry664/OTFS.
Authors:Thai Nguyen Van, Ibrahim Khan, Mury Fajar Dewantoro, Pratch Suntichaikul, Hai Van Pham, Ruck Thawonmas
Abstract:
This paper introduces a novel framework for efficiently evaluating in‑game sound designs using multimodal learning and reinforcement learning. Sound design significantly influences gameplay and accessibility; however, objective evaluation methods, defined here as automated assessments based on agent performance rather than human ratings, often require retraining agents from scratch for each sound design. Our approach first pre‑trains a multimodal encoder that encodes visual and auditory information into a shared latent space. A reinforcement learning agent is then trained using this latent representation. We fine‑tune the multimodal encoder for each sound design while keeping the trained policy fixed, allowing us to isolate the effects of sound changes on agent behavior. To evaluate the proposed method, we conducted experiments on games from multiple genres, comparing the performance of blind agents (agents using only sound input) under various sound designs. Experimental results suggest that this approach can detect differences in agent performance caused by degraded or altered sound design. Moreover, we significantly reduce the computational cost compared to retraining the agent from scratch. Experimental results suggest that this approach provides a practical and scalable solution for evaluating in‑game sound designs. The source code, data and supplementary document are available: https://github.com/thainv0212/tog‑reusing‑agents.
Authors:Deyuan Liu, Zecheng Wang, Xuanyi Liu, Jungang Li, Yihao Hu, Mingjie Tang, Bryan Dai, Jie Fu
Abstract:
Sampling complete solutions gives large language models more chances to solve a problem, but each attempt starts again. For tasks with several subproblems, all required answers must be correct in the same attempt. Retaining intermediate results can avoid repeating correct work, but errors in those results may affect later attempts. We study when this reuse is reliable and how much computation it requires. For local retries that stop when no candidate passes within the retry budget, we give conditions for correct completion and bound the average number of attempts. We also show how checking cost and the chance of passing a check affect total computation. Experiments across eight reasoning models examine composite mathematics and arithmetic. Across four DeepSeek models, average success on the second subproblem is 36.2 percent without first‑subproblem context, 57.2 percent with its answer, and 58.3 percent with its reasoning and answer. These scores measure second‑subproblem coverage and include continuations made when selection fails. Arithmetic experiments show that similar first‑attempt accuracies can lead to very different coverage after repeated attempts. Code is available at https://github.com/SempraETY/Intermediate‑State‑Reuse.
Authors:Wonsup Shin, Jingu Kim
Abstract:
Practitioners building training corpora with a pipeline of language models keep a preprocessing or refinement (post‑process) stage when switching it on raises the corpus quality. We ablate a production pipeline that turns conference speech into question‑answer pairs. The preprocessing stage rewrites the transcript, and the refinement stage filters the generated pairs. Switching each on and off gives four corpora from the same 40 Korean medical and finance sessions. On the corpus itself, quality rises as the stages switch on. Four LLM judges and six human domain experts rank the four corpora the same way, and the fully refined corpus leads by +0.18 and +0.22 on a 1‑5 scale. We then ask whether that gain carries downstream. We supply each corpus to six models at inference and measure retrieval accuracy. The ranking does not hold. Enabling the transcript rewrite lowers retrieval accuracy by 0.061, the only stage effect that survives correction for multiple comparisons. That stage also reads the whole session, so it almost doubles the cost of one. With both stages on, accuracy does not change. An equivalence test we declared in advance confirms this. The corpus itself is clearly valuable. It lifts retrieval accuracy from 0.231 to 0.748 in every model and domain we test. More than a third of its pairs carry knowledge that neither of two frontier models produces on its own. On our data, the null does not come from weak corpora. A rising score on a rubric of this kind did not predict what the corpus holds for a model.
Authors:Xu Hu, Haoming Li, Yuxiang Wang, Jinxin Yang, Kangshuo Li, Licheng Wang, Dingyi Kang, Jiayu Song, Tianhao Wang, Zhanhua Hu, Xu Yang, Qiannan Li, Bingzhe Li, Feng Chen
Abstract:
As computer‑use agents (CUAs) move to long‑horizon workflows spanning multiple applications, undetected errors can silently corrupt persistent state and cause cascading failures in subsequent actions. Reliable error detection and recovery have become critical bottlenecks for building robust CUAs. However, existing benchmarks focus primarily on end‑to‑end task performance, providing limited insight into whether agents can detect and recover from errors during execution. The few benchmarks that evaluate recovery rely largely on manually injected errors, which may not reflect the types or horizons of failures induced by real agent policies. We introduce CUA‑Recovery, a benchmark comprising test and training suites across multiple desktop applications. We construct it using ReRail, a framework for synthesizing long‑horizon tasks and trajectories in a persistent desktop environment. ReRail composes MyPCBench tasks into new long‑horizon workflows with executable ground truth, collects verifier‑confirmed failures from multiple policies, and turns them into verified recovery trajectories. CUA‑Recovery comprises a human‑verified test suite of 2,250 erroneous states over 100 tasks and a training suite of 8.6K trajectories over 377 tasks. Evaluation on the test suite shows that agents struggle both to recognize inherited errors and to complete the task. Across six agents and six takeover depths, Error Awareness Rate (EAR) reaches at most 44.0% and Pass@3 at most 25.5%. After fine‑tuning Qwen3.5‑35B‑A3B on the training suite, its average EAR rises from 19.9% to 35.2% on held‑out recovery states, and its Rubric Score goes from 5.0% to 16.1%. It also gains 15.3 points on MyPCBench and 10.8 points on Odysseys for Pass@3, showing that the training trajectories help both with recovery and with completing tasks from a clean start. Our code is available at https://github.com/xu‑hu‑2002/CUA‑Recovery.
Authors:Daisuke Oba, Hiroki Furuta, Naoaki Okazaki
Abstract:
Temperature scaling is typically applied as a post‑hoc transformation of model probabilities. In posterior‑induced flow language models, this creates two mismatches. First, posterior tempering implicitly changes both the endpoint distribution and the distribution of intermediate noisy states, while the denoiser remains trained on the original states. Second, the tempered posterior weights are combined with transport dynamics derived for the untempered flow. These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes‑Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow. Its learned realization trains the denoiser on tempered intermediate states and tempers the endpoint distribution using the corresponding tempered dynamics. We also derive training‑free variants for conventionally trained checkpoints. Across hyperspherical flow LMs with two denoiser architectures and a von Mises‑‑Fisher flow LM, BCT improves GSM8K generation across broad temperature ranges and achieves the strongest tempering performance. The gains also extend to Sudoku, demonstrating benefits on tightly constrained generation. At very low temperatures, the learned realization is also markedly more robust than the training‑free variants, supporting temperature as a property of the generative flow across training and inference rather than a post‑hoc probability adjustment. Code and model weights are available at https://daioba.github.io/bct .
Authors:Tingxu Han, Wei Song, Ziming Li, Yuchen Chen, Weisong Sun, Zhenpeng Chen, Chunrong Fang, Zhenyu Chen, Yang Liu
Abstract:
Automatically generating high‑quality documentation for software repositories remains challenging: existing approaches largely treat documentation as unconstrained text generation, lacking explicit quality specifications, depending heavily on long‑context prompts over raw code, and offering limited support for reproducible evaluation. We argue that code‑level documentation generation should instead be a standards‑guided software engineering task, in which documentation is generated and assessed against explicit, reusable quality requirements derived from authoritative standards. We define code documentation as natural‑language descriptions of repository functions and classes, organized by source‑code path. We present CODEDOC, a standards‑guided and dependency‑aware framework that identifies dependencies between code entities via static call‑graph and AST analysis and constructs dependency‑aware structured prompts combining local code context, repository paths, and documentation criteria drawn from authoritative standards and style guides. We further introduce an interactive LLM‑as‑a‑judge protocol that supplies criterion‑aligned feedback to refine generated documentation along fine‑grained dimensions from formal and practice‑oriented standards. Experiments on ten open‑source Python repositories show that CODEDOC produces higher‑quality documentation than RepoAgent and a long‑context baseline under standards‑aligned evaluation, with relative improvements of 11.6%‑17.0% and 21.4%‑29.6%, respectively, while consuming roughly an order of magnitude fewer tokens per documentation entity than the long‑context baseline. A human study confirms substantial alignment between LLM‑based and human judgments (Spearman's rho = 0.72), and the relative findings hold across different judge LLMs. Code is available at https://github.com/ControllableModel/CodeDoc.
Authors:Anmol Pandey, Aditya Jain, Liang Chen, Carsten Maple
Abstract:
Autonomous LLM agents act through tools under per‑action authorization: a local guard screens each tool call in isolation. We show that local authorization does not compose into global authorization. A compositional authorization failure (CAF) is a locally authorized execution, or a pair of locally authorized executions, that violates a global policy spanning intent, purpose, provenance, consent, and delegation. We characterize CAFs along two axes, expressiveness (single‑trace safety versus relational k‑safety) and observability (a ladder of monitor profiles). Our central result is an impossibility: relational purpose violations form a 2‑safety hyperproperty with no per‑trace locus of blame, so a per‑action monitor that never raises a false alarm detects none of them, independent of the underlying model or decision procedure; they become visible only under cross‑execution monitoring, which none of the six guardrails we survey implements. We make this constructive with AL‑SYNTH, a SAT/SMT synthesizer (using no language model) that emits minimal CAF witnesses, each checked against an exhaustive oracle. The empirical signature is sharp: across nine LLM judges the per‑action view approves the synthesized witness almost without exception, blocking it at a rate comparable to a benign control, yet the same models flag it when shown the two executions together. Per‑action authorization is therefore structurally insufficient for relational policy conjuncts not reducible to single‑trace safety: to bound them, a monitor must compare executions or accept false alarms.
Authors:Tingxu Han, Zijie Shen, An Guo, Lyuke Wang, Chunrong Fang, Zhenyu Chen, Wei Song
Abstract:
LiDAR‑based 3D object detection is a core perception service for autonomous driving, but deployment requires robustness to point‑cloud corruption while meeting latency and memory requirements. Existing approaches address robustness and resource efficiency separately, resulting in compact detectors that are fragile under corruption or robust detectors that remain costly to deploy. We therefore formulate LiDAR perception provisioning as constructing a detector that remains accurate under point‑cloud corruptions while meeting constraints on inference latency and peak GPU memory. To achieve robustness, we first craft corrupted samples that mimic extreme weather conditions. Then we train candidate detectors offline by ordering clean and corrupted samples by perturbation sensitivity, through which detectors learn more from hard corrupted samples. We estimate perturbation sensitivity using an ensemble of seven heterogeneous 3D detectors by tracking changes in confidence, BEV overlap, and unmatched detections. To improve efficiency, we narrow the network and use offline neighborhood‑density measurements to determine how many points and neighbors each sampling stage processes. We evaluate our method using AP (Average Precision) under clean and corrupted conditions, inference latency, and peak allocated GPU memory. On KITTI for Car detection with PointRCNN, our method improves Moderate AP under corruption by 3.32 points over the original detector while reducing mean latency by 19.8% and peak allocated GPU memory by 53.9%. Across two datasets, two detector architectures, and three object classes, our method improves AP under corruption by 4.03 points on average over the corresponding original detectors, while reducing mean inference latency and peak allocated GPU memory by 14.4% and 38.0%, respectively. Our code is available at https://github.com/drunklibai/robust‑efficient‑lidar‑perception.
Authors:Yubang Wang, Zezheng Huai, Yifei Zhang, Jinhao Wu, Yitong Chen, Haozhe Zhang, Sixian Li, Jingjing Gong, Zhenyang Lin, Ziyang Chen, Xipeng Qiu
Abstract:
Can embodied task agents use tactile experience to perform fine manipulation without task‑specific model training? We investigate this capability through Tactile‑ETA, a system that incorporates tactile experience into an embodied task agent through in‑context learning. Successful expert demonstrations show how contact changes during manipulation. We pair each recorded robot movement with its camera views and bilateral tactile observations. We keep tactile observations at the start and end of each movement and add two neighboring time points when the tactile images change markedly in between. The agent combines these demonstrations with current vision, tactile observations, and proprioception to select actions through general‑purpose robot tools, without updating model parameters. Across four simulated manipulation tasks, adding tactile observations to the same two expert demonstrations increases average success from 15.75% to 24.00%, with current sensing and execution interfaces held fixed. Further experiments show that performance depends on the sensory content and representation of demonstrations, while comparing agent models on Lift Bottle reveals substantial differences under the same system configuration. On the real‑robot tasks, demonstrations with aligned visual and tactile observations increase the task success rate from 0% to 60%. These findings demonstrate the feasibility of using tactile experience in context for fine manipulation and suggest a path toward extending embodied task agents to additional sensory inputs.
Authors:Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, Zhineng Chen
Abstract:
Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications. STE typically requires paired images before and after editing for training. However, recent approaches rely heavily on synthetic paired data and complex disentanglement strategies. They struggle to faithfully capture the diversity and complexity of natural scenes, often producing artifacts or hallucinations. In this work, we challenge the common practice and investigate whether STE can be driven by turning each real sample into its own paired supervision. Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self‑paired training data, and build a strong baseline (TextRIRO) upon a conditional diffusion architecture. Our experiments show that combining character‑column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image. With this concise yet real‑oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real‑world benchmarks, substantially mitigating the synthetic‑to‑real domain gap. Furthermore, we leverage TextRIRO to build TextRIRO‑3M, a large‑scale realistic dataset for long‑text recognition by concatenating edited and original images. Experimental results show that training recognition models with TextRIRO‑3M significantly improves recognition accuracy, demonstrating that TextRIRO provides highly useful training resources for downstream tasks. Overall, TextRIRO lifts STE to realistic, reliable, and widely applicable new levels. Code is available at https://github.com/YesianRohn/TextRIRO.
Authors:Wonsup Shin, Jingu Kim
Abstract:
When unique pretraining data is scarce, models see the same tokens many times and are thought to memorize. Random‑token replacement, an input‑corruption augmentation, is one remedy reported to lower validation loss, but a lower loss alone does not reveal which source of error it reduces. On a controlled source whose entropy rate is known in closed form, we show that replacement shrinks predictions toward the corpus average. We also introduce a decomposition that separates validation loss into approximation, estimation, and memorization terms. This shrinkage view predicts that replacement's bias cost needs no memorization but its benefit does. Memorization raises validation loss after its minimum, so we report the effect at two points, the minimum validation loss and the end of a fixed budget. Where replacement helps, it reduces this rise by about 60%. At the end of the budget, it turns from harmful to helpful as the controlled source becomes less predictable; at the minimum, it helps at no predictability level under the fixed recipe. On Korean and Japanese web text, replacement removes about a third of the loss rise after the minimum by the end of the budget, but its effect at the minimum is much smaller. Across ten real corpora, replacement lowers the final loss on nine. It harms only the census corpus, program‑generated JSON that is the most redundant of the ten and much more redundant than any of 64 further prose and code corpora we measure.
Authors:Koshiro Saito, Daisuke Oba, Danushka Bollegala, Naoaki Okazaki
Abstract:
Diffusion language models (DLMs) are emerging as a practical alternative to autoregressive language models (ARLMs), yet their fairness remains largely uncharacterized. DLMs introduce sampling controls with no direct autoregressive counterpart, such as block length and remasking strategy, that directly set their speed‑‑quality operating point. This raises a basic question: does a DLM's measured social bias remain stable across these settings? Across LLaDA, LLaDA‑1.5, and Dream on Open‑BBQ, we find that all three models exhibit social bias, but both the magnitude of measured bias and the relative ordering of models by bias are highly sensitive to the sampling configuration. This sampling dependence also changes what successful debiasing means. While mitigation strategies established for ARLMs transfer effectively to DLMs and reduce bias without degrading downstream performance, sampling dependence persists even after mitigation. Our results show that fairness in DLMs cannot be characterized independently of the sampling configuration, even after debiasing. Project page is available at https://pakapaka333.github.io/Beyond‑Model‑Weights/.
Authors:Yuxuan Li, Xiangchen Guan, Kunrong Li, Jin Wang, Kaiwen Luo, Yang Xiao, Yufei Guo, Zhisheng Chen, Liang Lin, Kun Wang
Abstract:
Full‑duplex speech agents must update persistent memory while listening and speaking, yet it remains unclear which events should be stored, revised, or excluded. Existing benchmarks isolate interaction timing from memory outcomes, obscuring the state transitions behind a correct answer. We introduce DuplexLedger, a bilingual benchmark of 1,094 persistent sessions aligning four full‑duplex scenarios with event‑level Write/Update/Ignore/Retain supervision, explicit pre/post memory states, and 6,564 same‑session questions and checks. Evaluating five full‑duplex models reveals an interaction‑memory gap: no system combines strong boundary control with coherent final‑state memory, and the best final‑state fact coverage is 2.01%. We propose ACT‑Mem, an 8.92M‑parameter sidecar comprising an event controller, a versioned transactional ledger, and a typed retriever coupled with a frozen MiniCPM‑o 4.5 backbone. On 219 held‑out sessions, ACT‑Mem increases latest‑value accuracy from 4.11% to 8.68% while reducing stale assertions and distractor adoption. It improves interaction boundaries but slows interruption stopping. These results motivate selective, auditable full‑duplex memory rather than indiscriminate transcript retention. Code is available at https://github.com/lyx‑researcher/DuplexLedger.
Authors:Yuxing Peng, Zhiqing Tang, Weijia Jia
Abstract:
Low curvature trace, an effective rank condition, is a common explanation for why randomized zeroth‑order methods scale to high‑dimensional tasks such as language model fine‑tuning. Can deterministic methods achieve the same savings? We show that they cannot. We study finding \varepsilon‑stationary points of L‑smooth nonconvex functions on \mathbbR^d from exact function values, assuming that below the starting value the Hessian is dominated by a fixed matrix with trace at most LE; this is a fixed‑matrix form of the effective rank condition used to explain zeroth‑order fine‑tuning. When ET\le d, the Gaussian two‑point method that such fine‑tuning uses needs only O(ET) calls if it rejects steps that raise f, where T=LΔ/\varepsilon^2 is the iteration count of gradient descent from initial gap Δ, and we prove that every randomized method needs Ω(ET) calls, so this cost is optimal up to constants. In contrast, every deterministic method needs \widetildeΩ(d\min\T,\sqrtET\) calls, and adaptive subspace search attains O(d\min\T,\sqrtET\): a deterministic method pays about d calls per discovered direction, and the trace budget allows about \sqrtET directions. Determinism thus multiplies the cost by a factor of order d/(E+\sqrtET) up to logarithmic factors, which grows linearly with d for fixed E and T; once E exceeds T by a logarithmic factor, deterministic methods need Ω(dT) calls, as without any trace budget. Our lower bounds hold for C^\infty functions with the matrix bound on all of \mathbbR^d, even when each call also returns gradient projections priced by rank.
Authors:Zhiping Zhang, Franklin Mingzhe Li, Zihao Huang, Tianshi Li
Abstract:
When an LLM agent acts on behalf of a user, it is expected to act within the authorization scope the user grants. Existing agent evaluations largely measure task completion or harmful consequences and can therefore overlook actions that violate this delegation without producing conventional safety or privacy harms. We introduce AuthorizationBench, a benchmark of authorization alignment grounded in documented real‑world cases. A scalable construction pipeline produces 286 executable tasks spanning situational triggers, the presence or absence of direct harmful consequences, and ten application domains. We execute every task with four models at two reasoning‑effort settings across three agent harnesses in an authorization‑aware sandbox that keeps authorized and out‑of‑authorization paths executable and observable, yielding 6,864 trajectories. Agents act out of authorization in 19.2% of trajectories, with rates ranging from 10.5% to 26.6% across the 24 configurations. Task completion does not imply authorization. Increasing reasoning effort leaves the aggregate rate nearly unchanged, with effects varying across models. On controlled pairs, agents act out of authorization at similar rates with (17.9%) and without (20.8%) direct harmful consequences, and more often under benign pressures than malicious injections (23.5% versus 17.6%, p=0.002). Our results show that task capability, additional reasoning, and consequence‑oriented safety evaluation do not by themselves ensure authorization alignment. Moreover, agents that consult the principal before completing a task exhibit lower rates of out‑of‑authorization behavior (14.0% versus 21.6%, p<0.001), pointing to a tension between agent autonomy and preserving users' decision authority that calls for further human‑centered study. Dataset and code are available at https://github.com/PEACH‑Research‑Lab/AuthorizationBench .
Authors:Yuqian Zheng, Hyoju Ahn, Hyeongjin Ahn, Eunil Park
Abstract:
Zero‑shot composed image retrieval (ZS‑CIR) aims to retrieve a target image based on a reference image and modification text. Recent approaches commonly rely on large vision‑language models to generate target image captions for retrieval. However, these captions may include redundant semantic details, introducing noise when retrieving the target image and leading to suboptimal retrieval results. To address this issue, we propose Less‑is‑More CIR, a two‑stage training‑free framework that combines Concise Retrieval and Critic Re‑ranking. In the first stage, concise target‑oriented captions are generated that preserve only the most discriminative modification semantics, thereby reducing semantic noise in the retrieval. In the second stage, a critic re‑rank module further evaluates the retrieved candidates from the perspectives of modification consistency and reference preservation. Extensive experiments demonstrate the effectiveness and robustness of the proposed framework for the ZS‑CIR. Our code is available at: https://github.com/dxlabskku/LM‑CIR.
Authors:Shiza Charania
Abstract:
Capability claims in robot manipulation are unverifiable, and existing leaderboards are each closed, simulated, or gameable. We propose HATCH, a robot escape room: an openly specified, physically replicable, sensor‑scored, and unrehearsable benchmark. Tasks span five skill categories at three difficulty levels and are chained so that the sensor confirming one task's completion unlocks the next, turning a room into a single long‑horizon evaluation. Success is confirmed by instrumentation rather than human judgment, scoring is embodiment‑agnostic, and object placements are randomized by a run‑day seed. Entrants submit attested self‑runs (uncut video, sensor logs, and the seed), with occasional organized events for calibration. Task specifications, build guide, bill of materials, and scoring rubrics are released under an open license.
Authors:Ningzhi Li, Xiaojun Ma, Xiaofei Wang
Abstract:
Although large language models support a wide range of tasks, hallucinations remain a major obstacle to user trust. When a model generates an answer, both its internal processing and final output vary with the question. Internal states are often expected to offer richer signals for hallucination detection. However, studies using these states typically report their standalone discrimination performance, while output‑side comparisons often use simple baselines with weaker modeling capacity. We integrate output‑side information available at inference time into a combined baseline and compare detection performance before and after adding hidden states under identical data splits and evaluation rules. Experiments cover open‑ended generation and constrained multiple‑choice tasks across different models. Adding hidden states improves AUROC by approximately 0.004 and 0.039 on the two tasks, respectively. Hidden states thus provide additional predictive information, but strong standalone detection performance can still yield only limited additional gains.
Authors:Greg Sidebottom
Abstract:
We give a scheme that multiplies two 3×3 matrices with 23 multiplications and 55 additions over any ring, one addition below the previous record of 56 (Sun, 2026) and the first below 56 for exact non‑commutative 3×3 multiplication without a change of basis. The scheme forms 13 signed combinations of the entries of A and 14 of B, takes 23 products, and recombines them with 28 further additions; its correctness is machine‑checked in Lean 4. The previous records, from 60 down to 56, were obtained by greedy common‑subexpression elimination, which only bounds each side of a scheme from above. We minimize both input sides exactly by chain covering, and the output side exactly through the transposition principle: the 9‑by‑23 output map costs its transpose plus 14, and the transpose is a 9‑input map that the input‑side minimizer solves exactly. Applied to Perminov's cn122 scheme (class i19w225c4efh, published at 58 additions) this gives 55 = 13 + 14 + 28, where greedy elimination reaches only 29 on the output side. The same method shows that Sun's representative is output‑optimal at 56, and an exact orbit‑wide computation over the Heule‑Kauers‑Seidl database shows that the record class is the only database class attaining 55.
Authors:Jiashuo Sun, Hao Zhang, Xueqiang Xu, Heng Wang, Yixuan Xie, Pengcheng Jiang, Zhenrui Yue, Jiajun Fan, Peiran Li, Ge Liu, Shaowen Wang, Jiawei Han
Abstract:
On‑policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowledge transfer. How can we assess whether a newly generated question is valuable for distillation, and learn to generate better questions‑‑‑without access to its correct answer? We introduce Ask‑to‑Distill (\method), a framework that treats synthetic questions as distribution elicitors and optimizes the teacher supervision they expose along student‑generated trajectories. A student‑initialized synthesizer is optimized using a distributional surrogate for question value, computed from a frozen student‑‑teacher pair without reference answers, correctness rewards, or per‑candidate student updates. The reward combines teacher‑‑student disagreement with teacher probability mass on the student's candidate support, together with length and diversity controls. The generated questions are filtered and deduplicated into a prompt‑only bank for standard OPD. Across mathematical reasoning, code generation, and agentic tasks, our method improves average performance and teacher‑gap recovery over unoptimized student‑generated prompts and KL‑filtered real questions, while matching or exceeding real‑bank OPD in the evaluated settings. Signal analyses further show that optimized questions can elicit denser supervision and more total signal from shorter student responses, supporting elicited distributional supervision as an objective for synthetic distillation data. Our code, data and models are publicly available at https://github.com/GasolSun36/Ask_to_Distill.
Authors:Thamirawaran Sathiyalogeswaran, Navindu De Silva, Chamikara Siriwardane, W. K. Alwis, Jason Mars, Udaya Sampath K. Perera Miriya Thanthrige, Ranga Rodrigo
Abstract:
Low‑rank adaptation (LoRA) attaches small low‑rank matrices to a frozen backbone, and public repositories host many LoRAs per backbone, each for a different concept or task. Combining several LoRAs in one generation forces a trade‑off. Merging their weights into one update needs a single forward pass, but LoRAs trained for different concepts interfere and quality drops. Running each LoRA in its own forward pass (composite) preserves quality, but inference time grows linearly with the number of active LoRAs. We study a property of LoRA pools overlooked by prior multi‑LoRA work. In real LoRA pools, the stacked low‑rank factors need far fewer directions than their summed ranks. Using singular value decomposition, we find the minimal subspace that keeps 99% of the factor energy. On three diffusion pools, only 46% to 57% of the summed LoRA ranks is needed, and the subspace grows sub‑linearly with pool size. A 48‑LoRA sub‑pool of LoRA‑Hub on FLAN‑T5 tasks shows comparable overlap (62%), while the diversity‑curated LoRA‑Retriever benchmark sits near the no‑overlap baseline (96%), so pool composition rather than modality predicts where compression is large. We build on this overlap with LoRASpace, a training‑free method that computes the subspace once per pool and reuses it in two inference modes, a static mode at merge cost and a dynamic mode for input‑dependent routing. On the diffusion pools, static LoRASpace matches composite quality at merge cost, and dynamic LoRASpace is more than 6x faster than composite with six active LoRAs on a 17‑LoRA pool, mainly because it runs one backbone pass instead of six. In an ablation, static variants at the same rank match LoRASpace, so the quality gain does not depend on the pool‑wide basis. On the near‑orthogonal LoRA‑Retriever benchmark, compression is small and gains come from coupling with external retrieval.
Authors:Erik Skalnes, Michael Oberst
Abstract:
Variance reduction techniques improve the efficiency of drawing conclusions from data. Recent proposals for variance‑reduction use two complementary techniques to reduce variance: incorporating machine learning predictions into estimators and weighted sampling. A popular weighted sampling algorithm is Poisson Sampling (PS), which independently flips a biased coin at each unit and includes all heads in the sample. A fundamental flaw with PS is that it has a random sample size, which presents practical challenges and introduces variance into the estimator. We show that a fixed‑size alternative, known in survey sampling as Conditional Poisson Sampling (CPS), is a drop‑in replacement for PS on general multivariate Z‑estimation problems, which include convex M‑estimation considered in prior work. Notably, we show that the asymptotic variance of CPS is no worse than that of PS, and can be substantially better in realistic scenarios where predictions are imperfect. We illustrate these advantages on both synthetic and real datasets, and release a Python package implementing CPS. In light of our contributions, there is no need to suffer a variable sample size.
Authors:Tianhao Peng, Ho Man Kwan, Fan Zhang, Shan Liu, Dave Bull
Abstract:
Neural image codecs face a fundamental trade‑off between pixel fidelity and perceptual quality. Distortion‑optimized codecs preserve pixel‑wise fidelity but produce blurred reconstructions at low bitrates, whereas generative codecs leverage learned priors to improve perceptual quality but tend to exhibit saturating PSNR at higher bitrates and require separate models for different bitrate targets. We propose PerQ, which reframes perceptual reconstruction as inverse generative modeling with bounded support. Specifically, we define the quantization cell as the set of latents that yield the same transmitted symbols and derive a transport bound showing that reconstructions from this cell remain within a rate‑determined radius of the codec output. This motivates performing generative reconstruction within the codec‑induced quantization cell, rather than sampling from an unconstrained image prior. PerQ implements this principle by attaching a rate‑conditioned flow‑matching compensator to a frozen, multi‑rate distortion‑optimized backbone, enabling perceptual enhancement while preserving the backbone's coding performance. Experiments on Kodak and CLIC2020 show that PerQ achieves perceptual performance comparable to generative codecs such as MS‑ILLM in LPIPS and FID, while maintaining competitive PSNR relative to distortion‑optimized codecs such as ELIC. A single model checkpoint supports a wide bitrate range with fast decoding, and the compensator provides consistent gains across three different backbones. The implementation is available at https://jasminepp.github.io/PerQ/.
Authors:Arnav Grover
Abstract:
Frontier models can control robots through agent harnesses that loop on one question: what motion comes next toward the user's goal? We show that this default loop carries out harmful instructions. With GPT‑6 Astra controlling a low‑cost SO‑101 arm through the Inspect Robots harness on three household tasks whose hazard is common knowledge, such as "put the knife in the toy basket", the policy refused 0% of 15 trials, attempted the harm every time, and completed it in 27%. Its one exit, give_up, is meant for goals that cannot be achieved, and it never cited the hazard. We add an optional runtime safety check to the loop: before the first motion, the model is asked whether it would carry out the instruction, given the same camera frames. The check would have stopped all 15 hazardous instructions, including the 4 that completed the harm, and flagged none of the 15 harmless controls. Code and data: https://github.com/Arnav‑Gr0ver/Refuses‑When‑Asked‑Complies‑When‑Acting.
Authors:Zerui Chen, Changrui Chen, Fei Ni, Jiankang Deng
Abstract:
Vision‑language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine‑grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as zooming, cropping, or code execution. While effective, their multi‑turn interactions with VLMs introduce redundant token computation and slow inference, and separate region inspection from the reasoning step that depends on it. To address these issues, we propose Look Before You Reason (LoRe), an implicit visual thinking framework for efficient visual reasoning within a single‑turn VLM interaction. Given an image and a question, LoRe predicts the most relevant image region, aligns it with the corresponding image patch features, and strengthens the associated key‑value (KV) cache entries, enabling subsequent decoding to directly exploit the selected visual cues without additional visual inputs or explicit tool calls. We further employ reinforcement learning to guide the VLM toward identifying informative image regions and leveraging them during reasoning. To support fine‑grained evaluation of multimodal reasoning, we introduce COCO‑ObjQA, a large‑scale benchmark for object‑centric multimodal reasoning. Experiments on three challenging multimodal benchmarks demonstrate that LoRe consistently improves VLM reasoning performance with low additional inference overhead.
Authors:Yihua Xu, Yan Lyu, Jize Li, Jiani He, You Lin, Yidan Hu, Rubing Qiang
Abstract:
Industrial artificial intelligence (AI) planning systems combine predictive models, optimization logic, controls, and governance rules, but improvement in the complete policy does not show whether one addition made an independent contribution or whether that contribution justifies its burden. We address this attribution problem with an observable evidence standard that separates opportunity to act, executed‑plan change, operational consequence, equivalence, uncertainty, and operating scope. Using forecast‑to‑plan replay on Favorita, M5, and Walmart data, we apply the standard to heterogeneous planning additions. Three limits of single‑indicator evaluation emerge. First, a composite policy can conceal negligible component contribution: one control changed only 15 of 681 executed plans. Second, changing execution does not guarantee stable benefit: the evaluated multi‑period capability helped in several long‑lead settings but was 7.6% more costly under tight capacity. Third, indirect evidence can prioritize review but cannot complete attribution. Independent runs produced different paths in 19 of 599 cases that a rule‑based check predicted would be unchanged. Although a lower‑cost diagnostic matched the full‑evaluation winner in only 55.0% of cases, it selected an option within the 10^‑6 cost‑equivalence tolerance in 79.5%; the remaining cases prevented the proxy from authorizing acceptance. These experiments establish capability‑level contribution evidence; the organization completes the value decision by weighing it against implementation and sustaining burden. The resulting capability evidence record links the observations to validation, deployment scope, fallback, monitoring, redesign, and retirement decisions.
Authors:Avi Gupta, Nilotpal Sinha, Vishnu Raj, Sambuddha Saha, Pratik Joshi, Koteswar Rao Jerripothula, Tammam Tillo
Abstract:
Class‑Incremental Learning (CIL) aims to learn new classes continuously without forgetting the previously learned ones. Recent works in CIL have significantly advanced across various modalities; however, at the intersection of audio‑visual modality, this problem remains underexplored. While representations from multimodal foundational models (MFMs) encapsulate rich static priors, our preliminary analysis shows that these representations struggle in incremental settings. Motivated by this, we propose a framework that generalizes frozen MFMs to audio‑visual CIL. Specifically, we leverage their dense audio and visual representations and employ an attention‑based contextual guidance module, in which the audio representations serve as task‑stable contextual guidance for the visual representations, enabling effective class recognition in the incremental setting. To further mitigate catastrophic forgetting and preserve stability across tasks, we employ feature‑ and logit‑level distillation, which are primarily responsible for the reduced forgetting. Extensive evaluations across various audio‑visual CIL benchmarks demonstrate that our approach consistently outperforms state‑of‑the‑art methods. Code is available at https://github.com/avigupta2798/MFM‑CIL.
Authors:Abolfazl Babanazari, Kaveh Fathian
Abstract:
Maximum clique and maximum consensus are widely used formulations for robust point cloud registration in robotics. Maximum consensus seeks one transformation that aligns as many matches as possible, while maximum clique selects matches through pairwise distance tests. Maximum clique is often used as a substitute for or an approximation to maximum consensus, with empirical evidence suggesting a close relationship. However, how closely the two formulations agree is not fully understood. We study this relationship theoretically. We show that a maximum clique can have arbitrarily large alignment error and be arbitrarily larger than the maximum consensus set at any fixed positive tolerance, and derive geometric conditions that bound this error. When rotation is fixed, we derive sharp error bounds and tests that exactly characterize maximum consensus for the translation subproblem. Finally, we study group‑k consistency, which tests groups of k matches, and determine the smallest group sizes that guarantee alignment error bounds independent of point cloud scale and geometry. These are d+1 for rotations and translations and d+2 when reflections are allowed, in dimensions d\ge 2. Interactive examples and proof verification using the Lean theorem prover are available at https://ariarobotics.github.io/mc‑vs‑mc‑registration/.
Authors:Hao Li, Tongyan Hu, Weidi Luo, Shanghao Shi, Xiaogeng Liu, Aiden Suh, Yulong Cao, Ning Zhang, William Yeoh, Yevgeniy Vorobeychik, Chaowei Xiao
Abstract:
Agent systems are shifting from task‑scoped tool‑use agents to persistent, personalized assistants such as OpenClaw. These claw‑style agents maintain configuration files (e.g., AGENTS.md), long‑term memory, and reusable skills. Once poisoned, this persistent state is repeatedly reloaded into the agent's context, creating persistent threats across tasks and sessions. Existing security evaluation systems struggle to evaluate these complex claw‑style systems since they rely heavily on manually constructed test cases and environments, which limits their scale and coverage; in addition, these manually designed cases quickly become outdated as agents evolve, and their evaluation settings leave a gap with how claw‑style agents are deployed in practice. To address this, we introduce AutoRedClaw, an automated red‑teaming framework that generates end‑to‑end attacks together with the environments needed to execute them. AutoRedClaw combines a strict quality‑checking protocol with execution‑based validation to ensure the correctness, realism, and effectiveness of generated samples, and adopts task‑driven environment construction to keep costs low. To achieve broad coverage, it organizes generation along risk categories, attack strategies, attack stages, and attack surfaces, and uses experience‑guided exploration to steer generation toward under‑explored regions. AutoRedClaw achieves an effective attack rate above 30% at less than 0.10 per successful sample. From its outputs, we construct RedClaw‑Bench, a 330‑case benchmark that yields substantially higher attack success rates (ASRs) than AgentDojo, AgentDyn, and DTap. Across seven harnesses and four frontier models, most agent systems exceed 50% ASR, showing that our red‑teaming pipeline provides a strong evaluation signal even for the most advanced agent systems. Our code and benchmark are released at https://github.com/SaFo‑Lab/AutoRedClaw.
Authors:Jonas Rohweder, Adrian Mülthaler, Faridoun Mehri, Sebastian Gerstner, Hinrich Schütze
Abstract:
Linear relational embeddings (LREs) approximate a language model's subject‑to‑object mapping with an affine function. Their faithfulness varies enormously across relations, from near zero for person_father to the ceiling for country_capital_city. We investigate the factors that contribute to this variation and demonstrate that it can be significantly predicted from a readout that is not aware of the underlying relation. To do so, we propose a novel form of the Jacobian lens, the entity lens. Whereas the Jacobian lens is estimated across all tokens and concepts, the entity lens EL_t is trained to predict what the model will output for an entity of type t under the neutral prompt " is known for"; e.g., we train ELs for countries and persons. After estimating an entity lens, we measure how highly it ranks the model's answer for a relation without using any relation‑specific information in estimation and prediction. Across 28 relations of the LRE dataset on Olmo‑3‑7B this relation‑agnostic score correlates with LRE faithfulness at ρ= 0.7355 (p < .01). This suggests that linear decodability is largely a property of what a subject's representation already carries, rather than of the relation being decoded. Moving the unit of analysis from the relation to the individual fact, we conclude that new relation datasets and analysis methods are needed to address several problems in existing datasets, most prominently semantically weak first tokens (e.g., most predicted universities' first token is "University") and reference competition (e.g., the LLM confuses obscure actor names with competing non‑actor entities that have the same last name). Our full pipeline is available at https://github.com/jonasrohw/entity‑lens
Authors:Kristina Mustakova, Alexey Sorokin
Abstract:
We present a study on controlled automatic simplification of Russian texts using preference‑based optimization algorithms with a focus on adapting texts to the A1, A2 and B1 proficiency levels. We fine‑tune YandexGPT using Rejection Sampling and Direct Preference Optimization (DPO) methods to improve control over the complexity of the generated text. We compare several training configurations that differ in dataset composition, text type and the strategy used to construct preference pairs for DPO. For automatic evaluation we train a regression model based on ruRoBERTa‑large to predict continuous complexity scores corresponding to CEFR levels. We also consider semantic similarity, lexical coverage and quantitative readability metrics. Human evaluation assesses simplicity, fluency and adequacy. Our best model achieves 87.15% target level accuracy, an MAE of 0.35 CEFR levels and a target interval MAE of 0.11. Manual assessment shows that 75% of the evaluated simplifications match the target morphological and syntactic level, and fluency and adequacy scores are especially high for A2 and B1. Although A1 remains the most challenging level, the model significantly reduces the proportion of vocabulary and grammatical constructions beyond the target level while generally preserving meaning. Our model is freely available on \hrefhttps://huggingface.co/kmustakova/RuSimplifier_DPOHuggingFace.
Authors:Zhiwei Liu, Zhujun Lan, Jielin Qiu, Zixiang Chen, Liangwei Yang, Ming Zhu, Akshara Prabhakar, Wenting Zhao, John Emmons, Jason Wu, Shafiq Rayhan Joty, Junnan Li, Shelby Heinecke, Silvio Savarese, Huan Wang
Abstract:
Enterprise adoption of LLMs in agentic workflows is gated by evaluations that span a wide arc of abilities: CRM operations, heterogeneous‑document retrieval, personal data understanding, multi‑turn planning, tool use, function calling, long‑horizon research, and long‑context software engineering. These evaluations live today in separate repositories with divergent runners, dependencies, and scoring conventions, which makes cross‑benchmark comparison difficult and obscures what a single aggregate number actually represents. We release EGI‑BENCH, a unified LLM agent benchmark suite that consolidates eight enterprise facing agent benchmarks (CRMARENA, HERB, PERSONABENCH, USERBENCH, MCP‑UNIVERSE, MCPEVAL, BENCHFORCE, LOCOBENCH) behind a single command‑line interface with two runtime‑budgeted execution modes. On top of the suite we define the Enterprise General Intelligence (EGI) Score, a six‑dimensional framework (M1‑M6) whose buckets are populated by fine‑grained sub‑metrics routed from each benchmark rather than by benchmark‑level averages. We describe the suite, the sub‑metric routing, and the quick/deep budgets, and report results for fourteen frontier and cost‑efficient LLMs spanning the GPT, Claude, and Gemini families under quick mode. Top‑line EGI ranges from 39.6 (GPT‑4o mini) to 65.6 (gpt‑5.5), and no single model dominates every capability axis: gpt‑5.5 leads task outcome, planning, and retrieval grounding; Claude Sonnet 4.5 leads tool actioning and safety/abstention; and Claude Haiku 4.5 leads engineering quality. Code is available at https://github.com/SalesforceAIResearch/egi‑bench.
Authors:Tao Jiang, Benjamin Navet, Yousouf Taghzouti, Fabien Gandon, Louis-Félix Nothias, Élise Dumont
Abstract:
Chemistry has recently entered a high‑data regime as molecular databases from high‑throughput platforms or computational studies have become available. Increasing automation within the chemical sciences leads to a more digital and precision chemistry. Dedicated scientific agents are becoming central to guiding the chemist through an ever‑increasing amount of data and findings, including negative ones. Here, we introduce Magnolia, a persistent‑memory artificial intelligence agent framework that integrates computational protocols, from docking to quantum chemistry, lab notebooks and chat logs relying on large language models (LLMs). It relies on a three‑tier memory system: session‑level observations are automatically captured and distilled into structured learnings, confirmed and elevated to project‑tier entries and, when human‑validated, promoted to cross‑project rules that actively guide subsequent work and provide direct chemical feedback. We illustrate Magnolia on four case studies: quantum‑chemical evaluation of aromatic systems (COMPAS database), molecular dynamics of butane and of modified DNA, and a protein‑peptide docking campaign. Across these projects, Magnolia tracked 317 runs and accumulated 116 project‑tier entries, from which four cross‑project rules were promoted; in controlled replays of four tasks, runs with memory passed the checks that runs without it failed (for example, ten of ten docking‑configuration checks, against one to four of ten). By combining automation, reproducibility, and human‑in‑the‑loop control, our Magnolia framework streamlines docking, molecular dynamics and quantum chemistry calculations with direct analysis and LLM‑augmented feedback, offering accelerated access to computational (bio)chemistry.
Authors:Leaf Jiang
Abstract:
The Jetson AGX Orin is a common computer for robots and vehicles, yet measured accuracy, speed and energy of current depth estimators on it are hard to find. We benchmark nine model families as of October 2026: the monocular Depth Anything V2 and 3, Depth Pro, MoGe‑2, Metric3D v2 and UniDepth V2, and the stereo methods libSGM, NVIDIA ESS and Fast‑FoundationStereo. Every method runs on the Orin as a TensorRT FP16 engine or native CUDA code at up to three input sizes, up to the full 5.7 MP, and is scored for metric accuracy against Middlebury 2014 ground truth (non‑occluded pixels of the 10 scenes whose disparity stays below 256 px), GPU time, energy and licence. Stereo is far more accurate. At 30 fps, NVIDIA's ESS reaches 0.70% AbsRel in 8.6 ms and libSGM 0.73% in 25 ms; with more compute, Fast‑FoundationStereo reaches 0.33%. The best monocular models reach 12% with the true focal length (Depth Anything 3 Metric‑L in 33 ms, Depth Pro in 803 ms), and 3.8‑4.0% only with an oracle per‑image scale (MoGe‑2‑L, UniDepth V2‑B, Metric3D v2‑L). Stereo error falls as the input resolution grows; no monocular model improves, and the metric scale of most drifts away from the truth outside their training resolution. At depth discontinuities, which AbsRel barely registers, the monocular models scatter 17‑46% of the edge points between foreground and background, learned stereo 2‑9% and libSGM at full resolution 1%. We give recommendations per frame‑rate budget and release all code, per‑scene results and engine build scripts.
Authors:Abdelrahman Abdallah, Mohammed Ali, Adam Jatowt
Abstract:
Dense bi‑encoders and late‑interaction retrievers are usually treated as alternatives, yet they fail on different queries: one vector captures what a text is about, token‑level matching finds its exact entities and phrases. Using both today means two models, two indexes and a score‑fusion step. We present UIBK‑Embedding‑v1, a 179M‑parameter English encoder that produces both representations in one pass and scores them with one unmodified MaxSim. The dense embedding is written into the list of token vectors as three extra vectors, and a one‑dimensional tag keeps the two kinds of vectors from matching each other, so that plain MaxSim equals the token‑level MaxSim plus a weighted dense cosine. The model therefore runs in a standard late‑interaction index without changes; we test it with PLAID. A shared 18‑layer trunk with a separate 4‑layer top per head, and a cross‑encoder distilled into the fused score, turn the two heads into one retriever. On BEIR, UIBK‑Embedding‑v1 reaches 59.79 nDCG@10. This is 2.57 points above LateOn, the strongest model of its size, and 2.10 points above it when both models read documents of the same length; UIBK‑Embedding‑v1 is ahead on all 15 datasets. On a decontaminated version of BEIR it stays ahead on average (61.55 against 60.36) and on 8 of 12 datasets. Among the 156 models with complete BEIR results in the MTEB results repository, only models with at least 1.5B parameters score higher. Code, model weights, and a technical blog post are available at https://github.com/DataScienceUIBK/uibk‑embedding , https://huggingface.co/DataScience‑UIBK/uibk‑embedding‑v1 , and https://huggingface.co/blog/abdoelsayed/uibk‑embedding‑v1 .
Authors:Alec Laprevotte, Haonan Chen, Xiaoshen Han, Yilun Du
Abstract:
Diffusion policies with linearly composed scores have emerged as a promising approach to multitask robotic learning by combining separate factors to enable modular, task‑wise adaptation, compositional generalization, and interpretable functional decomposition. However, current training objectives do not encourage factor specialization, often resulting in uninterpretable denoisers whose composed prediction is an average over identical copies, undermining their architectural premise. To address this, we propose a drop‑in score‑decorrelation auxiliary loss, motivated by projective composition theory. Our approach encourages specialization by penalizing overlap directly between component noise predictions, with little architectural change or computational overhead. We demonstrate across multiple benchmarks that our decorrelation loss consistently reduces measurable double‑counting between components, improves task decomposition, and produces behaviorally meaningful specialization without compromising multitask performance. Website: https://al3cl1.github.io/robotics‑score‑decorrelation.
Authors:Yubo Ye, Sweekar Piya, Xiajun Jiang, Linwei Wang
Abstract:
Learning latent dynamics is central to assessing the current states and forecasting the future trajectories of high‑dimensional time series. For locally‑stationary latent dynamics \mathcalF(\mathbfz_<t; \mathbfc) with latent dynamics state \mathbfz_t and environment variable \mathbfc, prior identifiability results have largely focused on \mathbfz_t when conditioned on predefined label u of dynamics environments. This leaves two limitations: reliance on predefined labels that hinder generalization to unseen environments, and a limited understanding of the identifiability of \mathcalF and \mathbfc which ‑‑ although offering important structural properties for the identifiability of \mathbfz_t ‑‑ are learned jointly with \mathbfz_t. We address these limitations with Meta‑iLaD, a novel identifiable latent dynamics framework attained by meta‑learning across dynamics environments. Meta‑iLaD introduces a novel conditional prior of \mathbfc, modeled as a feedforward meta‑learner to rapidly extract \mathbfc from few‑shot examples. Meta‑iLaD further establishes identifiability for \mathbfz_t, \mathbfc, and \mathcalF simultaneously, for a general formulation of \mathcalF(\mathbfz_< t;\mathbfc) without restricting the dimension of \mathbfc or how it modulates \mathcalF. On synthetic and real data, we provide strong empirical evidence that 1) conditioning on few‑shot examples enables generalization to out‑of‑distribution environments, and 2) identifiability for \mathbfc and \mathcalF is critical for accurate forecasting beyond reconstructing observed trajectories. Code is available at https://github.com/GoForIt‑YYB/Meta‑iLaD.
Authors:Ping Zhang, Quang-Huy Nguyen, Yingtie Lei, Sameera Shridhar, Zachary Leveroni, Ellie Quattrocchi, Jai Y Yu, Wei-Lun Chao
Abstract:
How learning progresses over time and varies across individuals is a longstanding question in cognitive and behavioral science. However, a major challenge is identifying observable behavioral measures that faithfully capture learning progression. Existing approaches commonly rely on predefined behavioral features or task‑level performance, while fine‑scale posture and movement in freely behaving animals remain less explored. In this paper, we ask: can short‑term movement trajectories reveal learning progress and individual identity? We introduce a self‑supervised representation learning framework built on a simple hypothesis: behavioral characteristics associated with identity and learning remain relatively stable over nearby time intervals. Using temporal proximity as supervision, our framework learns compact representations directly from movement trajectories without manually defining behavioral states or kinematic features. Using behavior data from freely behaving rats, we show that the learned representations not only improve prediction of identity and learning‑related readouts over existing methods, but also provide a tool for investigating broader scientific questions about when and how these signals appear in behavior. Specifically, trajectories as short as one second contain informative learning and identity signals. Egocentric pose dynamics are more predictive of identity, whereas allocentric motion is more predictive of learning‑related variation, and both signals are strongest at decision junctions in the alternation cohort. We further use a representation‑conditioned diffusion model to inspect movement patterns associated with identity and learning progress. Our findings establish short‑term movement as an informative behavioral "fingerprint" for studying learning and individual differences. Our code is available at https://github.com/pizhn/TRAIL.
Authors:Oussema Dhaouadi, Zuria Bauer, Johannes Michael Meier, Marc Pollefeys, Daniel Cremers, Olaf Wysocki
Abstract:
Autonomous Unmanned Aerial Vehicle (UAV) navigation in urban environments requires precise pose estimation. Because Global Navigation Satellite System (GNSS) signals are often unreliable in cities, visual localization serves as a crucial complement. Within this domain, textureless Level‑of‑Detail (LoD) city models provide a lightweight and privacy‑preserving substitute for heavy 3D textured meshes. However, localizing against abstract geometry is challenging. Existing pipelines typically score 2D silhouette overlaps, which can suffer from severe geometric ambiguities. We present MatchCityLoc, a framework that shifts this paradigm to direct feature matching from a noisy 6‑DoF pose prior. By extracting surface normals from the query image using a zero‑shot foundation model and matching them against rendered LoD normals, we establish explicit 3D‑to‑2D correspondences. This enables a Perspective‑n‑Point (PnP) pose solve, which we iteratively refine through a render‑and‑match loop. Experiments demonstrate strong pose recall and accurate translation and rotation among the evaluated off‑the‑shelf LoD‑based methods with very few renderings. By bridging the modality gap between aerial imagery and texture‑free geometry, MatchCityLoc enables accurate localization using lightweight, privacy‑preserving city models.
Authors:Yuxuan Chen, Yang Xiao, Jack Sun, Chak Sun, Brick Li, Sam
Abstract:
Spoken interaction is moving from alternating turns to full‑duplex conversation, where an agent listens while speaking. Learning this coordination requires natural dialogue with persistent speaker‑resolved tracks, a data requirement that remains difficult to satisfy at scale. Generative recovery offers a route from ordinary recordings, but recovering two voices is not enough: their track assignments can drift as the conversation unfolds. We address this missing link with speaker‑anchored routing before diffusion: a 1.84M‑parameter module matches the separator's coarse streams to fixed enrollment identities and orders them before a frozen generator reconstructs the speech, without retraining the generator or altering the reconstructed audio. On 25‑minute remote conversations, routing cuts wrong‑track time from 43.9% to 4.1%, below the 6.2% of matched‑enrollment waveform re‑identification, with its largest advantage when one participant holds the floor (17.8% versus 28.6%), and raises fixed‑track activity F1 from 0.46 to 0.84 while per‑chunk reconstruction quality is unchanged. The same router transfers to unseen face‑to‑face and telephone recordings, and a margin‑gated verification step brings their wrong‑track time to 0.9‑1.7% while embedding 20‑36% of the output, where full re‑identification embeds all of it. Recovered voices thus become the participant‑consistent tracks that full‑duplex dialogue modeling needs. Audio examples: https://elpsykongloo.github.io/beyond‑two‑tracks/
Authors:Marcel Rogge, René Schuster, Didier Stricker
Abstract:
3D Gaussian Splatting grows a dense scene from a sparse initialization through adaptive density control (ADC), driven by per‑Gaussian statistics gathered during rendering. These statistics pass a frustum‑based visibility filter blind to occlusion: fully occluded Gaussians still count as observed, diluting the averaged gradient signal and suppressing densification precisely where geometry is occluded. We correct this with an occlusion‑aware filter that restricts densification statistics to views in which a Gaussian actually contributes, and replace the criterion's raw view‑count normalizer with an average‑contribution one. Integrated into a StopThePop‑based framework, the two together improve every image‑quality metric over the occlusion‑blind baseline on occlusion‑heavy scenes (up to 6% lower LPIPS), at less than half the Gaussians a view‑count normalizer requires. Under the longer schedules these large scenes require, the count stays bounded where most criteria diverge, and on smaller benchmarks our method stays competitive at a fraction of the competitors' count.
Authors:Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, Ran He
Abstract:
Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio‑visual reasoning, yet cross‑modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations. To address this issue, we propose Omni‑Woodpecker (OWP), a training‑free framework that shifts hallucination mitigation from output‑level manipulation to conflict‑conditioned representation rebalancing. Specifically, OWP first identifies cross‑modal conflicts from complementary modality‑specific views. It then employs orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. Finally, OWP adaptively rebalances fused representations by suppressing misleading priors and restoring ignored evidence before decoding. Extensive experiments demonstrate the effectiveness and strong cross‑architecture generalization of OWP. Using Qwen2.5‑Omni‑7B as the backbone, OWP improves performance by 12.9 and 8.6 points on CMM and AVHBench, respectively, while consistently achieving gains of 0.67‑2.25 points across four general OLLM reasoning benchmarks. Furthermore, OWP generalizes effectively to VideoLLaMA2‑AV, improving performance by 4.6 and 3.9 points on the same benchmarks. Code is publicly available at https://github.com/ZiiJianLiu/Omni‑Woodpecker
Authors:George Nigmatulin, Valentin Mohl, Sascha Frey, Kang Li, Aramis Marti-Shahandeh, Mihai Cucuringu, Anisoara Calinescu, Jakob Foerster, Stefan Zohren
Abstract:
Realistic market impact is essential for generative limit order book (LOB) models to serve as world models for trading agents. If a simulator misprices this mechanism, agents learn policies that are optimal in the simulator but wrong in the real market. Existing metrics assess generated streams purely through distributional comparison with historical data, leaving counterfactual response to an agent's own orders largely unexplored. Here, we show that market impact emerges naturally in neural LOB generators trained solely for next‑token prediction. We inject execution‑style metaorders into the generated message flow of five autoregressive models across five NASDAQ stocks and measure the response using estimators from the empirical impact literature, validated on baselines whose physics is known in advance. Our benchmark evaluates any message‑level generator in three phases: free generation (reading order‑flow persistence against price diffusivity); metaorder injection (yielding the impact law's exponent and amplitude); and a post‑execution window (measuring relaxation). In addition, comparing each visible injection to a blind, simulator‑only control isolates the model's learned response from standard book mechanics. For every model on every stock, we measure how the impact responds as a function of cumulative metaorder participation. Interestingly, we observe emergent impact in generative models of market data, with exponents in the range of 0.63 to 1.21, slightly above the traditional 0.5 of the square‑root law of impact. In the family of models tested, we are so far unable to observe impact decay during relaxation. We tentatively attribute this to the still‑short context windows of the models. We release the benchmark as open source to validate future generative models for market data.
Authors:Lijia Wu, Zheng Li, Yuxuan Liu, Jingwen Zhang, Penghai Zhao, Xiang Li
Abstract:
Scientific method figures often provide researchers with a clear visual entry point into a paper's methodology and core contributions, yet their dense layouts and complex information flows are difficult for Vision Language Models (VLMs) to interpret. Existing benchmarks predominantly focus on coarse‑grained tasks, such as context‑based retrieval or holistic comprehension, overlooking the fine‑grained content and inter‑module relations within complex methodological diagrams. To address these overlooked aspects, we introduce Scientific Figure Benchmark (SciFigBench), a benchmark for evaluating core capabilities required for scientific method figure understanding. Specifically, it targets three capabilities: dense content perception, visual relation comprehension, and information‑flow tracing. For this purpose, we curate 4,352 scientific method figures from real‑world research papers and construct 40,670 QA pairs to operationalize these capabilities as measurable question‑answering tasks. All annotations are generated by dedicated pipelines and then manually verified. We further propose SciFigCue, a training‑free visual prompting module that externalizes arrow‑induced dependencies as visual relational evidence for method‑figure reasoning. Experiments across open‑source and proprietary VLMs show that even strong models remain substantially limited in visual relation comprehension, while SciFigCue improves performance across several tasks without parameter updates. The code and related resources are available at https://github.com/LeNcyer/SciFigBench.
Authors:Wonjung Woo, Hyeonwoo Kim, Hanbyul Joo
Abstract:
Generating human‑object interactions (HOIs) with multiple objects is challenging because the resulting motions arise from different underlying causes. Human motion reflects semantic intent, intentionally manipulated objects must remain coordinated with the human, while surrounding objects move primarily as physical consequences of contact and collision. We present BUMP, a text‑driven framework for multi‑object HOI generation based on a simple principle: learn intentional motion and simulate physical consequences. Our method first generates full‑body human motion from text and scene context, then predicts trajectories for a small number of intentionally manipulated driver objects conditioned on the human motion. We then obtain responder object motions through physics simulation, allowing the framework to handle varying numbers and configurations of surrounding objects without expanding the learned trajectory output space. To address the scarcity of paired full‑body multi‑object HOI data, we combine heterogeneous supervision from human‑only motion, full‑body HOI, and hand‑centric manipulation data augmented with plausible body motion. We further introduce outcome‑conditioned interaction editing, which infers a local modification of the causal driver object trajectory from a desired responder object behavior without requiring an explicit target driver object trajectory. Experiments demonstrate strong performance on both single‑object and multi‑object HOI benchmarks. Qualitative results showcase diverse scenarios involving inter‑object relationships and edits, highlighting the advantage of our unified model formulation.
Authors:Jing Wang, Yanlin Li, Mong-Li Lee, Wynne Hsu, Shengqiong Wu, Hao Fei
Abstract:
Recent advances in unified visual editing have begun to consolidate image and video editing within a single model, while 3D scene editing remains largely separate and constrained by scarce paired training data. Given the rich image and video editing supervision now available and the common editing semantics shared across modalities, a natural question is whether this knowledge can be transferred to data‑scarce 3D editing. We present VitronEdit, a universal framework for image, video, and 3D scene editing. VitronEdit adopts a shared sequence representation and progressive multimodal training to transfer editing knowledge across modalities. We further introduce geometry‑aware post‑training, which leverages a pretrained geometry reconstruction model to provide depth and camera‑pose supervision. Surprisingly, applying geometric supervision only to 3D samples yields the best overall performance, while image and video editing also improve despite receiving no direct geometric supervision, suggesting that geometric regularization transfers through the shared backbone. Our analysis further shows that image and video training mainly transfers editing semantics to 3D, whereas 3D supervision primarily improves multi‑view consistency. The project page: https://vitron‑llm.github.io/VitronEdit/.
Authors:Zeyu Zheng, Yiyang Liu, Yuyang Zhao, Congmin Zheng, Jiachen Zhu, Jianian Chen, Kangning Zhang, Kayue Daniel Wong, Lingyu Yang, Rong Shan, Shihao Chen, Thomas Zhu, Wentao Zhang, Xiaopeng Li, Yan Hu, Yongxi Lin, Yuxiang Chen, Zhuoying Ou, Jianghao Lin, Jeremy Avigad, Prasad Tetali, Jingbang Chen, Sean Welleck
Abstract:
As AI systems begin to resolve open research problems, reliable verification becomes increasingly important. Formal mathematics provides a trusted framework for mathematical reasoning, enabling agents to produce machine‑checkable proofs while conducting mathematical exploration. Existing formal mathematics benchmarks often focus on known results or a small set of prominent open problems, which can become difficult to evaluate reliably once solutions are discovered and widely disseminated. Existing evaluation protocols are also not always aligned with how modern coding agents operate. We therefore introduce ReuF2F, a Lean 4 benchmark of 244 formalized open problems drawn from Research Experiences for Undergraduates (REU), spanning entry‑level open research problems across a broad range of mathematical subfields. We expect these less prominent open problems, which tend to attract less attention and may remain unsolved for longer, to be better suited for held‑out evaluation of end‑to‑end open problem resolution in Lean. We adopt a standard coding‑agent evaluation setting, placing each problem and its logical negation into a Lean project in a Docker container. We also build Leanimum‑agent on mini‑SWE‑agent to provide a unified harness for fair comparisons across models. Our evaluation finds that even the best‑performing model resolves only ~40% of the problems, leaving substantial room for improvement on formalized open research problems.
Authors:Rong Wang, Mulong Luo
Abstract:
Anonymous peer review protects reviewers from retaliation, yet reviewer anonymity may also be weakened by artifacts released through the review process itself. This risk is growing as many venues require submitting authors to review, creating substantial overlap between authors and reviewers within the same review cycle. We identify review‑to‑submission linkage, where an anonymous review can be associated with the reviewer's own submission, substantially shrinking the reviewer anonymity set. To assess whether this linkage is practically exploitable, we develop ECHO, which combines technical‑expertise and writing‑style signals across reviews, papers, and rebuttals, and evaluate it on R2S‑Bench, a ground‑truth benchmark we construct. ECHO correctly identifies the reviewer's own submission in Top‑1 for 34.6% of reviews, in the Top‑5 for 54.0%, and in the Top‑10 for 63.4%, compared with 0.5%, 2.6%, and 5.3% under random ranking, from the candidate submissions. With a stronger Stage‑2 reranker, ECHO further reaches 56.4% Top‑1 and 66.4% Top‑10. These results demonstrate that review‑to‑submission linkage is feasible using only routinely released peer‑review artifacts. Code and benchmark are available at https://github.com/kaimu11111/ECHO‑Revealing‑Review‑to‑Submission‑Linkage‑in‑Anonymous‑Peer‑Review.
Authors:Feng Chen, Pei Zhou, Lixuan He, Yunchao Yao, Li Sun, Shenghua Gao, Mingyu Ding, Kaifeng Zhang, Kaichen Zhou, Yi Ma
Abstract:
Foundation models offer a promising basis for robotic agents, but long‑horizon tasks require them to sustain coherent reasoning and reliable physical execution across interdependent stages. We introduce Embodied Playground, a benchmark for evaluating how foundation‑model agents coordinate physical, strategic, and spatial reasoning throughout such tasks. Agents must track task progress, satisfy intermediate constraints, and revise their plans using execution feedback as earlier actions shape subsequent decisions. The playground comprises 410 task environments spanning basic manipulation, sequential puzzles, and multi‑stage household activities, with variations in planning depth, geometric constraints, and physical execution demands. We curate EP‑100, a representative selection of 100 tasks across five suites and 16 task types, together with the EP‑Easy and EP‑Hard difficulty splits and the compact EP‑20 selection. A shared protocol specifies observation and tool access, resource budgets, repeated trials, and independent checks of goal completion, task‑rule compliance, and physical validity. Embodied Playground provides a unified framework for studying whether foundation‑model agents can sustain effective reasoning and action over extended task horizons. https://winniechen2002.github.io/Agent‑bench‑website/
Authors:Chen Chen, Chenggang Wu, Yongbiao Chen, Weidong Qiu, Zhengwei Qi, Haibing Guan
Abstract:
Learned image compression (LIC) has achieved promising rate‑distortion performance, but most methods lack scalability for diverse platforms with varying computational and storage constraints. Existing slimmable LIC methods achieve flexible complexity via consecutive‑channel activation but suffer from two limitations: forced consecutive‑channel selection degrades narrow‑model performance at low bitrates, and the lack of stable variable‑rate control limits adaptability to varying bandwidth. We propose a channel‑level mask‑guided slimmable image compression method. A learnable channel‑wise mask with sparsity‑promoting regularization enables non‑contiguous top‑w channel selection, letting narrow models use the most informative channels. After training, mask merging and channel reordering convert the model into a standard slimmable network via Mask‑to‑Slimmable (M2S) conversion, enabling efficient inference without architectural changes. For variable‑rate control, we combine lambda‑ and width‑conditioned modulation with quantization‑step scaling and a progressive training pipeline stabilized by a freeze‑then‑fine‑tune strategy. Experiments on Kodak across E2E, Hyperprior, and Checkerboard backbones show superior rate‑distortion performance across widths and bitrates, with large gains for narrow models at low bitrates. At w=48, it achieves up to 19.22%, 24.11%, and 22.76% BD‑rate savings over the same‑width Slimmable baseline on E2E, Hyperprior, and CKBD, respectively. A single VBR model replaces 40 independent models, reducing storage by 14.8x to 15.9x. Code: https://github.com/sjtu‑tcloud/MGSIC.
Authors:Lanfeng Zhong, Xiangyu Sun, Guixuan Xu, Rongzhao Zhang, Jie Xu, Shaoting Zhang, Guotai Wang
Abstract:
Histopathology is central to cancer diagnosis, but morphology in Hematoxylin and Eosin (HE)‑stained images does not directly reveal the molecular states that characterize disease heterogeneity and inform tumor characterization, prognostic assessment, and biomarker discovery. Spatial Transcriptomics (ST) measures spatially resolved gene expression, providing molecular information within tissue architecture, yet differences in assay coverage and spot informativeness make paired HE‑‑ST observations unequally reliable for representation learning. We propose Reliability‑Guided Spatial Transcriptomics (REST) supervision with Biological Program Bank (BPB), a program‑structured and reliability‑guided ST‑supervised training framework for predicting molecular information from H\&E images. REST constructs BPB from curated signature, pathway, and cell‑type gene sets to provide a shared structure for ST encoding, reliability estimation, and molecular reconstruction. Guided by BPB, Spot‑Reliability Contrastive Alignment (SRCA) measures the reliability of each spot from assay‑level program coverage and spot‑level activity dispersion, and reweights HE‑‑ST alignment accordingly. Biology‑Informed Program Reconstruction (BIPR) further uses learnable program queries to reconstruct member‑gene expression from histological morphology. REST's training corpus comprised 209 samples (407,475 spots across 16 organs); evaluation used 70 disjoint samples (230,056 spots across eight organs) for gene‑expression, pathway‑activity, and cell‑signature prediction. REST achieved the highest Macro correlation among state‑of‑the‑art ST‑supervised training methods and outperformed existing pathology foundation models. Code is available at https://github.com/lanfz2000/REST.
Authors:Yoo Sung Jang, Michael Ryoo
Abstract:
Latent world models let an agent plan in latent space. The planner searches for an action sequence that minimizes a cost, and that cost is often the \ell_2 distance between the predicted terminal latent and the goal latent. The latent, however, is partly noise: the regularizer that prevents collapse during world model training promotes variance in the directions carrying no signal. We show that the same regularizer also supplies the structure to prune that noise in the latent. SIGReg drives the latent marginal to a standard Gaussian at every step, and assuming further that the latent sequence is a Gauss‑Markov process, the transition is linear with its covariance pinned by the map itself, z' \mid z ~ \mathcalN(Az,\, I ‑ AA^\top). Fitting the state dynamics A therefore yields the covariance at no additional cost, and in its eigenbasis each latent direction carries its own uncertainty. Furthermore, constraining A to be a normal matrix fixes that basis across timesteps, so one eigenbasis measures the uncertainty over an arbitrary number of steps and serves as a stable basis to compute the planning cost on. We then restrict the \ell_2 cost to the k‑dimensional subspace the state dynamics carry forward with the least uncertainty. The correction is post hoc: the encoder, the predictor, the rollout, and the planner are all left untouched, and only a single constrained matrix, A, is fit. We evaluate across six environments and show large gains on the 2D navigation tasks, TwoRoom and PointMaze, raising success rate over the plain \ell_2 cost by up to +54.7 points under a harder planning setting. Code is available at https://github.com/yjang43/lp2.
Authors:Fuping Wu, Xi Jia, Le Zhang, Jiahua Li, Rima Mustafa, Thomas E. Nichols, Bartlomiej W. Papiez
Abstract:
Auxiliary learning (AL), which can improve the performance of a primary task by leveraging related auxiliary tasks, is commonly regarded as a special case of multi‑task learning. Existing AL approaches typically require manually annotated auxiliary labels, increasing annotation costs, particularly in medical imaging where expert labeling is expensive. In this work, we investigate whether segmentation performance can be improved using auxiliary tasks whose labels are automatically derived from existing segmentation annotations, requiring no additional manual labeling. To address this, we propose a novel auxiliary learning framework, formulated as a bilevel optimization problem. Our method enables effective knowledge transfer between tasks via a feature communication module, enhancing the primary segmentation task. We evaluate the method on three medical image segmentation benchmarks under limited‑label settings. The results demonstrate consistent improvements over state‑of‑the‑art methods, achieving superior label efficiency. The code is publicly available at https://github.com/FupingWu90/ALBO.
Authors:Xiaocheng Luo, Yanping Chen, Zaifu Zhan
Abstract:
Relational triple extraction identifies entity relations from raw sentences without depending on entity type information. The performance of traditional methods is seriously degenerated by ``unseen'' entities, because these methods can't build contextual dependencies relevant ``unseen'' entities. This paper addresses this challenge by proposing a multi‑granularity interaction network that leverages word‑word, entity‑relation, and head‑tail interactions to capture deep semantic dependencies. By incorporating multiple iterations, the multi‑granularity interaction network is effective to refine the representations corresponding to head and tail entities. It enhances the global semantic features associated with relational triples, aiding in the remediation of triples that may have been overlooked or omitted in earlier steps. Strong and consistent performance is achieved by our method on two rearranged datasets, with a significant improvement in identifying relational triples, including those involving ``unseen'' named entities. For example, our model improves F1‑score by 3.1%/20.5% over previous approaches on the NYT‑R/WebNLG‑R datasets in the Unseen metric. Code: https://github.com/xiaocen‑luo/Interaction‑Network.
Authors:Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov
Abstract:
Retrieval‑augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity‑driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document‑level context that resolves cross‑passage references. We introduce evidence‑based generation (EBG), requiring models to return an answer with line‑level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence‑grounded multi‑document QA benchmarks‑ MultiHop‑RAG and QASPER‑MP, a multi‑paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two‑hop haystack and an evidence‑augmented NoLiMa spanning 10K‑1M characters. Across ten proprietary frontier and three open‑weight LLMs, EBG isolates two RAG limitations that long‑context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi‑fact queries, and chunking destroys document meta‑information needed to resolve identity‑bearing queries. EBG also exposes grounding failures invisible to answer‑only evaluation, including incomplete citations in two‑needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC‑EBG.
Authors:Rostislavs Rostovskis, Aleksandrs Platonenko
Abstract:
Double‑precision matrix multiplication, real (DGEMM) and complex (ZGEMM), dominates the run time of dense linear algebra and electronic‑structure codes. On affordable GPUs, however, FP64 throughput is a small and shrinking fraction of the throughput of the low‑precision matrix engines. Ozaki scheme II closes this gap by reducing a floating‑point matrix product to a handful of exact INT8 products followed by a Chinese‑remainder reconstruction; the number of products sets the accuracy. We present what is, to our knowledge, the first implementation of Ozaki scheme II for Intel GPUs. Written in SYCL on top of oneMKL, it runs entirely on the device and exposes the full BLAS dgemm/zgemm interface. Making the scheme fast on the Xe2 (Battlemage) architecture required keeping FP64 arithmetic out of all memory‑bound kernels, a branch‑free residue computation, an FP32 evaluation of the CRT quotient with a proven safety margin, 16‑bit limb arithmetic in the reconstruction, and work‑arounds for performance cliffs of the vendor INT8 GEMM. On an Intel Arc Pro B60 (24 GB) the emulated DGEMM reaches 17.3, 12.1 and 9.3 TFLOP/s with s=8, 12 and 16 moduli at N=8192, and up to 21.1 TFLOP/s overall, i.e. 12‑28x the native FP64 DGEMM of the same board. The emulated ZGEMM, which uses the 3M form exactly in integer arithmetic, reaches up to 27.4 TFLOP/s and is 13.7‑17.8x faster than native ZGEMM for N\ge4096 at s=16. With s\ge16 the results are correctly rounded for well‑scaled inputs and, using an extended‑width residue path (s=20‑24), also for inputs with wide exponent ranges, whereas native DGEMM rounds only 2‑13% of the entries correctly. We also report worst‑case accuracy, phase breakdowns, energy, an ablation study, a comparison with Ozaki scheme I, and a validated performance model.
Authors:Xinping Wang, Jie Ma, Rui Xing, Wensong Guo, Xiayi Zhang, Jianhao Deng, Zihao Zhou, Boqian Yang, Feijuan He
Abstract:
Mathematical reasoning is a fundamental capability for educational applications of Large Language Models (LLMs), where reliable problem solving requires not only correct answers but also coherent and verifiable reasoning processes. Multi‑agent systems (MASs) have shown growing potential in this domain by leveraging collaboration and verification among multiple LLM agents. However, existing evaluations primarily focus on the correctness of final answers in text‑only settings, offering limited insight into multimodal reasoning capabilities, the reliability of intermediate reasoning steps, and robustness against reasoning errors. To address these limitations, we introduce MAMath‑Bench, a process‑aware benchmark designed to evaluate the multimodal mathematical reasoning capabilities of MASs. MAMath‑Bench comprises 1,003 problems spanning 11 mathematical categories, multiple difficulty levels, and several modalities, including text, images, and audio. It features fine‑grained milestone annotations to evaluate systems across three dimensions: problem‑solving correctness, reasoning‑process reliability, and robustness to adversarial reasoning‑chain perturbations. Our evaluation yields three key findings. First, multi‑agent collaboration does not consistently improve final‑answer accuracy over strong single LLMs. Second, collaboration can substantially enhance the coverage of intermediate reasoning, but such gains may come at the cost of increased reasoning redundancy. Third, current systems remain vulnerable to adversarial reasoning errors, particularly those involving subtle structural perturbations. The dataset is available at https://github.com/mira‑ai‑lab/AgentMath.
Authors:Ziyun Chen, Yijun Wang, Chuanyi Zhang, Liang Yao, Yuexuan An, Xiang Gu, Wei Zhou, Fan Liu
Abstract:
Referring expression segmentation (RES) aims to localize targets at the pixel level guided by human instructions. However, current models predominantly follow a text‑centric paradigm that treats discrete text as the primary interaction medium for these human instructions. This assumption is especially limiting in unmanned aerial vehicle (UAV) applications, where rapid response, hands‑free operation, and natural communication are often critical. In this paper, we reconstruct UAV referring expression segmentation through an audio‑centric perspective. Grounded in information theory, we demonstrate that raw audio provides a higher upper bound for extracting rich semantic features than transcribed text. To achieve this, we construct an audio‑driven UAV referring expression segmentation benchmark named Aerial‑AudioRES by augmenting VisDrone with synthesized speech, forming "audio‑text‑image‑mask" quadruples. Specifically, we employ an ambiguity filtering mechanism to ensure referential uniqueness, and incorporate eight distinct timbres alongside 4 realistic noises to simulate authentic environments. To drive the paradigm shift from text‑centric to audio‑centric, we propose the end‑to‑end Audio‑centric Referring Expression Segmentation Network (ARES‑Net). Distinct from discrete text‑visual matching, our approach learns to establish a direct soft alignment between continuous acoustic manifolds and multi‑scale visual representations. Comprehensive experiments show that our method obtains a peak accuracy of 64.23% in oIoU and demonstrates robust generalization on a collected real‑human test set. Furthermore, it significantly reduces total interaction latency to one‑third of the text‑based method. Our code is available at https://github.com/LemonGauss/ARES‑Net.
Authors:TingTing Gu, JinCheng Luo, WenQiu Guo
Abstract:
Ambivalence and hesitancy (A/H) are subtle behavioural states that can be expressed through both what a person says and how it is spoken. This paper presents an audio‑‑text framework for video‑level A/H recognition. Full‑video transcripts and speech waveforms are encoded using pre‑trained multilingual BERT and wav2vec~2.0, respectively, and projected into a shared representation space. Two stacked bidirectional cross‑attention blocks allow linguistic tokens to attend to acoustic frames and acoustic frames to attend to linguistic context without requiring explicit temporal alignment. The resulting sequences are aggregated through masked mean pooling and combined using concatenation, element‑wise products, and absolute differences for binary classification. We evaluate the model on the subject‑independent split of the Behavioural Ambivalence/Hesitancy (BAH) dataset. The proposed system achieves 72.19% accuracy, 75.00% precision, 81.13% recall, 77.95% A/H F1‑score, and 70.16% Macro‑F1 on the test set. These results demonstrate the potential of jointly modelling linguistic and acoustic information for automatic A/H recognition.code at:https://github.com/DorriG/ECCV.
Authors:Jiaxuan Luo, Siqi Ouyang, Lei Li
Abstract:
Retrieval‑augmented speech translation improves rare terminology, but systems such as RASST still assume that users provide a glossary before each session. That assumption is inconvenient for live talks, while merging every domain into one growing glossary eventually places too many irrelevant terms in the same retrieval and prompt competition. We present AutoTerm‑SST, an interactive system that removes session‑time glossary upload and domain selection. It organizes a large offline terminology catalog into topic slices, matches recent streaming translation context to bilingual slice prototypes, and activates a compact multi‑slice working set under fixed retrieval and prompt budgets. On five ACL and five ESO oncology talks, AutoTerm matches a known‑domain reference in terminology accuracy (88.3% vs. 88.1%) and mWER‑resegmented BLEU (45.09 vs. 45.19), while requiring no domain input. Compared with a one‑million‑entry merged glossary, it improves terminology accuracy by 3.9 points and BLEU by 2.30. Routing plus four‑slice retrieval remains lightweight: the complete system runs at 0.406 RTF, and the vLLM host sustains 32 real‑time sessions. The live demo exposes active slices and retrieved terminology as the topic changes, providing zero‑setup terminology‑aware streaming translation.
Authors:Jiaming Liu, Xinghao Zhu, Yiming Zhang, Shanghang Zhang, Yinxi Wang, Yichu Yang, Zhuoyang Liu, Dantong Niu, Soha Pouya, Juekun Li, Yan Chang
Abstract:
Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language‑based reasoning captures task semantics but struggles to express fine‑grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning‑before‑acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state‑of‑the‑art success rates of 90.3% on visuotactile UniVTAC and 99.6% on vision‑based LIBERO‑Long, and outperforms competing methods on real‑world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long‑horizon manipulation. Project page: https://nvidia‑isaac.github.io/video_to_data/physicalword/
Authors:Yi-Chuan Huang, Hao-Jen Chien, Zheng-Hui Huang, Yu-Lun Liu, Kaipeng Zhang, Zhixiang Wang
Abstract:
Procedural generation plays a key role in synthetic data creation and game development, offering greater controllability, editability, and diversity than static mesh representations. However, VLM‑based code generation often produces relatively simple 3D assets, while sophisticated photorealistic procedural systems require extensive manual development. Existing VLM‑guided procedural methods remain largely constrained by predefined factories and APIs, limiting their ability to create novel asset types. We introduce InfiniDream, a VLM‑driven framework for open‑ended procedural 3D world generation. First, we propose Symbolically Constrained Asset Generation, which leverages existing procedural algorithms to create novel, photorealistic assets while preserving editability and seed‑based diversity. Second, we develop an end‑to‑end text‑to‑world pipeline that combines semantic scene planning with Deterministic Frustum‑Guided Grid Placement to construct large‑scale 3D worlds with precise object placement, procedural population, and animation. Extensive evaluations on 100 object and 20 scene generation tasks demonstrate improvements across four perceptual metrics. Diversity analysis, ablation studies, and human preference evaluations further validate the effectiveness of our framework. Project page: https://yichuanh.github.io/infinidream‑page/
Authors:Zhuoshi Pan, ZhiYuan Feng, Yukuo Cen, Honglin Lin, Junru Lu, Zhongxia Chen, Shuo Yang, H. Vicky Zhao, Jie Tang
Abstract:
Harness development plays a crucial role in building capable AI agents. Recent approaches improve harnesses through recursive self‑improvement (RSI), but their performance gains often come with higher execution costs. We introduce HarnessPro, a label‑free method for evolving harnesses to improve task performance while reducing execution cost. In this setting, candidate harnesses are evaluated using LLM‑generated tests without access to ground‑truth solutions or official tests. Specialized agents diagnose execution trajectories, plan updates with different performance‑cost priorities, and implement them as candidate harnesses. Rather than returning a single harness, HarnessPro uses normalized weighted squared‑distance selection to provide cost‑focused (lite), balanced (medium), and performance‑focused (heavy) harnesses for different deployment budgets. Selected harnesses enter the next iteration, where execution feedback guides further updates. Our experiments cover Terminal‑Bench 2, SWE‑Bench Pro, and ProgramBench, using GPT‑5.5 with Codex and GLM‑5.2 with Claude Code. Across these benchmarks using GPT‑5.5 with Codex, the heavy mode of HarnessPro achieves an average pass‑rate gain of 2.0 percentage points and an average per‑task cost reduction of 26.1% relative to RHO. This advantage also holds when using GLM‑5.2 with Claude Code. Code is available at https://github.com/pzs19/HarnessPro
Authors:Akihiro Miki, Shun Hasegawa, Yoshimoto Ribayashi, Yuta Sahara, Kento Kawaharazuka, Kei Okada
Abstract:
Human skin contains diverse mechanoreceptors, including Merkel cells and Pacinian corpuscles, each localized in specific regions. Although their spatial organization is known to be shaped by molecular signaling and neural induction during development, experimentally examining its functional significance under controlled conditions in living systems remains challenging. In particular, physically comparing alternative receptor arrangements that do not occur biologically is difficult. In this study, we developed a layered biomimetic robotic skin designed to mimic key structural features of human skin tissue by embedding receptor‑analog sensors within compliant soft media. Using this platform, we implemented both biologically motivated and counterfactual sensory arrangements and quantitatively evaluated their performance across three passive tactile tasks: detection of weak contacts, a two‑point discrimination‑like assessment, and texture identification. Arrangements reflecting key features of the biologically motivated sensory organization exhibited higher performance than the alternative configurations examined. This study addresses a biological question concerning how spatial organization influences tactile function by presenting a biomimetic robotic approach that enables physical and empirical comparison of biologically motivated and counterfactual sensory arrangements that are difficult to investigate in living systems.
Authors:Zhaoxi Zhang, Hao Liang, Linxuan Wu, Yiming Xu, Yifu Yuan, Kun Liang, Chenming Tang, Meiyi Qiang, Wentao Zhang, Weikang Li
Abstract:
Spatial intelligence requires agents to actively acquire visual evidence and integrate observations into a coherent representation of the three‑dimensional world. Yet spatial question answering offers limited evidence of whether models can construct accurate metric maps through autonomous exploration. We introduce SpatialScout, a benchmark for evaluating vision‑language model (VLM) agents on active object‑level 3D mapping. SpatialScout comprises 80 diverse indoor scenes containing 3,140 target objects. Given an object inventory, RGB observations, camera calibration, and camera poses, agents control a camera to explore each scene and estimate object centers in a shared coordinate frame. The task requires coordinating viewpoint selection, cross‑view association, geometric estimation, and map revision under bounded interaction budgets. Evaluation jointly measures localization accuracy and target coverage through object‑scale overlap scores that penalize missing predictions. Experiments with frontier VLMs reveal substantial limitations: the strongest evaluated model, Claude Opus 5.5, achieves a mean overlap score of 41.71%, with 47.23% of targets exceeding an intersection‑over‑union (IoU) threshold of 0.5 and only 3.25% exceeding 0.9. These results reveal a substantial gap between recovering coarse object layouts and constructing complete, precise metric maps. SpatialScout provides a quantitative testbed for developing agents that build reliable 3D spatial representations through active perception. Code is available at https://github.com/JesseZZZZZ/SpatialScout.
Authors:Rohan Menon, Chih-Hong Cheng
Abstract:
Attack success rate (ASR) evaluates jailbreaks against large language models without accounting for repeated calls to the victim model. Yet each additional call gives the deployed system another opportunity to detect the attack and intervene. We introduce \mathrmASR_\mathrmSTEP, which penalizes additional victim‑model calls while retaining binary success judgments. Our evaluation considers GCG, PAIR with a step budget, and a single‑step PAP baseline on JailBreakBench across three model families. GCG's scores decrease by 7.0‑8.7 percentage points after discounting, while PAP's remain unchanged. While for PAIR attack, we see a wider discounting range between 3.4‑12 percentage points, showing that the "step" quantity differs per iterative strategy. Reporting both metrics makes the discount due to repeated victim‑model interaction explicit. Code is available here: https://github.com/SuELS‑Group/beyond_asr.
Authors:Arjun Ashok, Mihir Parmar, Palash Goyal, Bhavana Dalvi Mishra, Chun-Liang Li, Irina Rish, Alexandre Drouin, Jinsung Yoon, Tomas Pfister
Abstract:
Recent time series foundation models (TSFMs) can consume covariates in‑context, extending zero‑shot forecasting to problems driven by external factors. This enables scenario analysis: forecasting how a target responds to events that set covariates to chosen future values. Whether they can do this reliably is an open question. We introduce WhatIf‑Bench, a benchmark measuring how well forecasters use covariates to forecast a target's response to future events. WhatIf‑Bench is semi‑synthetic: time series come from mechanistic simulators of data centers, power grids, road traffic, and water networks, with controlled covariate‑target relationships grounded in each system. Each task presents a history of events intervening on covariates, then asks to forecast a future event that either replays seen values (Replay) or sets them to unseen ones (Novel). Varying the number of covariates, historical events, and event length yields 37,665 tasks. Evaluating TSFMs, frontier LLMs, and simple regression baselines, we find the LLMs reach the highest skill (improvement over a naive baseline), ahead of the best TSFMs. Every model forecasts well on Replay futures (48 to 85% skill) but drops to ‑7 to 50% on Novel futures, with the regression baselines falling the most. Skill on Novel futures also falls as events set more covariates jointly. More historical events raise skill on Novel futures across the board, but large differences remain. WhatIf‑Bench reveals generalization to Novel futures as the core challenge, and offers a testbed toward arbitrary scenario analysis.
Authors:Xiaomeng Li
Abstract:
Photo‑based estimation of food energy could ease dietary assessment, and general‑purpose multimodal large language models (LLMs) make such estimates easy to obtain. Most evaluations use one photo taken before eating and compare with energy served, while dietary assessment needs energy eaten. We evaluated eight models from three vendors (Claude Haiku, Sonnet and Opus 5.5; GPT‑6 luna, 6.1 sol and 6 astra; Gemini 3.8 Flash and 4 Argon) on 802 meals from ACETADA, a controlled feeding trial with weighed intake, with the before photo only, before and after photos in separate calls (subtraction), or both photos in one request. Three results ran against intuition. Subtraction under‑estimated intake for six of eight models, because the models still report hundreds of kilocalories for a nearly empty tray. Scaling the before‑only estimate by the average share eaten (0.87) matched much of the gain from the after photo; beyond that constant, the second photo lowered meal‑level error by only 2.2 to 4.4 points, mainly in meals with food left. With one photo, two models had a near‑zero intake bias only because they under‑estimated the served energy and ignored uneaten food. In ablations, adding the after photo improved accuracy under both prompt formats for seven models, while prompt effects varied across models. Over 148 feeding days, five models kept the 90% confidence interval of their mean error within a pre‑set 10% margin; one model also met the pre‑set 30% limits of agreement, narrowly (‑28% to +27%; limits outside 30% cannot be excluded), and the other seven did not. Protein was under‑estimated by every model, most in protein‑dense meals; repeated calls changed error by at most 1.4 points; a high reasoning effort helped only the smallest model. A low average bias is not evidence of accurate individual intake estimation; such estimates should support, not replace, assessment by a dietitian.
Authors:Guannan Lai, Zi-Yang Bai, Jun-Peng Jiang, Han-Jia Ye
Abstract:
Self‑improving agents aim to continuously improve their behavior by learning from experience after deployment. However, accumulating and reusing more experience does not necessarily lead to better decisions, particularly when generation, interaction, and memory resources are jointly constrained. Under such stringent constraints, we find that several existing lifelong‑memory methods provide limited benefits over simple retrieval or even no‑memory agents. This reveals a fundamental challenge for resource‑efficient self‑improvement: agents must learn not only which past experiences are relevant, but which are actually worth reusing to improve future decisions. When memory capacity and opportunities for reasoning and recovery are limited, retrieving an ineffective experience can consume scarce resources and hinder downstream execution. To address this challenge, we propose TokenWorth, a lightweight feedback‑driven memory adaptation framework that learns query‑dependent memory worth from sparse execution feedback. Without updating the backbone model, TokenWorth uses the learned worth to guide both experience retrieval and bounded memory management, enabling the agent to continually adapt how it reuses accumulated experience. Across three LifelongAgentBench environments and both local and API‑based deployments, TokenWorth achieves the highest task success rate while using the fewest output tokens among the evaluated methods under the specified resource budgets. These results suggest that effective self‑improvement under resource constraints depends not on accumulating more experience, but on learning what is worth remembering. Our code is publicly available at https://github.com/AIGNLAI/TokenWorth.
Authors:Ruotian Peng, Yuanyi Ren, Shuaiqi Liu, Xin Guo, Fang Zhu, Yangyi Huang, Yue Wu, Zheng Wang, Yandong Wen, Zhihang Fu
Abstract:
Recursive self‑improvement (RSI) requires agents to diagnose system bottlenecks and coordinate improvements across training and inference. We refer to the agents' capability to jointly optimize training data, the trainer, and the inference harness as cross‑component co‑design. We introduce VibeTraining‑Bench, a benchmark for evaluating this capability across tasks spanning terminal operation, software engineering, multi‑turn tool use, and mathematical reasoning. Within each task, research agents start from an identical initial agent system under a fixed resource budget. Guided by validation feedback, they iteratively refine data, trainers, and inference harnesses, then select and submit a final agent system for evaluation on the test set. To prevent reward hacking, we enforce three safeguards: test data isolation, separate research and grading containers, and network restrictions for research agents. Our evaluation results show that frontier research agents substantially improve target agent systems. For example, on SWE‑bench Verified, GPT‑6‑Astra raises the target agent system's performance from 32.9% to 53.7%, comparable to the human‑developed instruct model's 55.4%. However, there is still room to improve their ability to co‑design and iteratively refine data, trainer, and harness to achieve cumulative performance gains. We hope that VibeTraining‑Bench will serve as a unified platform for advancing RSI, deepening our understanding of cross‑component optimization for agent systems. Code and data are available at https://github.com/D2I‑ai/VibeTraining‑Bench.
Authors:Michael Glöckler, Timo Ropinski
Abstract:
Vision‑language models (VLMs) achieve remarkable performance on 2D medical images, such as X‑rays or pathology images, while 3D VLMs that directly process CT/MRI volumes lag behind due to limited 3D training data and the predominance of 2D foundation architectures. To leverage powerful 2D VLMs for analyzing medical volumes, which contain hundreds of slices, it is therefore necessary to focus only on those few slices, which are diagnostically relevant. To enable such a prioritization, we propose XGLASS, a method that transfers medical expertise from 2D X‑rays to 3D CT for task‑aware slice prioritization. XGLASS uses disease‑conditioned importance maps derived from language‑aligned radiologist gaze on X‑rays and spatially aligns them with coronal CT projections, bringing gaze priors and CT content into a shared 2D representation. The fused gaze‑prior/CT projection is then used to predict a disease‑specific ordering of axial slices, prioritizing diagnostically relevant slices. XGLASS significantly improves budgeted retrieval over 2D and 3D‑native baselines, achieving a top‑10 hit rate up to 75.9% for the best‑performing disease. In practice, this means that after inspecting only the top 10 slices (4.17% of a 240‑slice scan), XGLASS identifies a diagnostically relevant slice in more than 75% of cases. For downstream slice‑wise finding classification, an XGLASS‑selected 30‑slice candidate set reduces model calls by 87.5% while improving F1 by 11.3 percentage points over all‑slice inference, 4.2 points over the strongest 2D selector, and remains competitive with 3D‑native selectors while requiring up to 130.8 times less time and 10.9 times less GPU memory per finding, supporting its use as a lightweight preselector for volumetric CT analysis. The code and pretrained checkpoint are available on https://github.com/michaelgloeckler/XGLASS.
Authors:Emirkan Burak Yilmaz, Yakup Genc
Abstract:
We present PnP‑Transformer, a neural pose solver designed as an iterative‑free, differentiable alternative to classical PnP‑RANSAC algorithms. PnP‑Transformer regresses the 6D object pose by processing predicted semi‑dense 2D‑3D correspondence fields alongside confidence scores, while mapping geometric inputs into a continuous latent space. To evaluate its practical utility, we integrate PnP‑Transformer into a zero‑shot 6D pose estimation pipeline, where the correspondence module and pose solver are trained separately. PnP‑Transformer achieves a 41.2% mean AR score with low latency (0.46 s/obj) on seven core BOP benchmark datasets, surpassing all prior baselines (such as RayPose with 39.1% AR) but one (Co‑op with 58.4% AR). When the pose solver is replaced with EPnP‑RANSAC (similar to Co‑op) while keeping the correspondence module unchanged, the RANSAC alternative outperforms PnP‑Transformer on six of the seven datasets by reaching 52.2% mean AR at a cost of an additional 60 ms per object, revealing an accuracy‑latency trade‑off. Code and pretrained models are available at https://github.com/ebylmz/pnp‑transformer.
Authors:Yinghao Li, Tianyu Zhang, Zhuohao Li, Yi-Lin Wei, Fa-Ting Hong, Qing Zhang, Wei-Shi Zheng
Abstract:
Physical interaction is a key capability for humanoid collaboration, requiring coordinated whole‑body motion and precise contact. However, learning diverse dual‑humanoid interactions faces two major challenges: (1) human‑robot morphological differences disrupt contact interfaces and spatial relationships during retargeting, and (2) jointly learning motor skills and partner coordination makes it difficult to retain control fidelity as interaction diversity grows. In this paper, we propose OmniInteract to address these challenges through surface‑aware modeling and scalable residual policy learning. First, we introduce Surface Contact‑Aware Retargeting, which leverages dense human‑robot surface correspondences to jointly retarget both humanoids, preserving explicit interaction contact interfaces and their near‑field geometric context. Second, we propose a Scalable Flow‑based Interaction Policy for partner‑conditioned residual generation. It factorizes dual‑humanoid control into a reusable motor foundation and interaction‑specific adaptations: partner‑aware experts learn corrections over a frozen whole‑body motion prior, while conditional flow matching unifies them in a shared residual action space. This enables scalable skill consolidation while retaining pretrained motor capabilities. Extensive experiments demonstrate that OmniInteract improves retargeting fidelity and dual‑humanoid coordination while better retaining interaction performance as skill diversity grows. Real‑world deployment further demonstrates successful contact establishment, stable whole‑body coordination, and recovery from external perturbations.
Authors:Kesav Kumar Jayakumar, Karthigeyan Thilak
Abstract:
We set out to reproduce a recent hallucination‑mitigation result on the POPE benchmark. The LLaVA‑1.5‑7B baseline we measured, 82.21 F1 on the adversarial split, sat 1.21 to 8.67 points above the adversarial baselines we planned to compare against (81.00, 79.30, and 73.54). We looked at the evaluation code and found a readout choice that moves F1 on its own. A two‑token logit readout decides yes versus no by comparing two vocabulary IDs, 3582 ("yes") and 1217 ("no"). Across 9,000 greedy‑decode questions spanning all three splits, neither ID is generated even once. The model emits " Yes" (token 3869) and " No" (token 1939), because SentencePiece prepends a space byte after the prompt suffix "ASSISTANT:". Read with an eight‑token lookup that covers the space‑prefixed surface forms, LLaVA‑1.5‑7B scores F1 0.8221, 0.8498, and 0.8713 on the adversarial, popular, and random splits. That 6.13‑point adversarial shift exceeds the adversarial gain reported by five of the six papers we surveyed. None of them states its readout, and our reference ran under 4‑bit NF4 quantization, so we cannot say how much of any reported gain a matched evaluation would keep. That comparison stays unresolved until readout and precision are matched. On the fixed baseline we run nine inference‑time corrections of our own (none beat it) and dissect why the adversarial split resists logit‑level intervention. In a 3,000‑question probe, wrong predictions draw more image attention than correct ones (Cohen's d=‑0.46, p=1.4×10^‑24), the opposite of what grounding‑based corrections assume. A four‑readout audit across four further VLMs from three tokenizer families confirms the confound is protocol‑level, not model‑specific: porting the LLaVA‑1.5 IDs verbatim drops F1 to 0.00 on the two disjoint‑vocabulary models, whereas a tokenizer‑derived readout holds between 0.80 and 0.92.
Authors:Feiyang Hong, Yujie Wei, Bo Zhao, Xiu Su, Hongxun Yao, Shuo Yang
Abstract:
Robot manipulation often depends on short‑term cues ‑ such as brief contacts, temporary occlusions, and intermediate object states ‑ that are ambiguous in a single frame and can only be resolved by integrating observations over time. However, injecting historical information solely by enlarging the history window is computationally costly and frequently yields limited gains due to substantial redundancy and task‑agnostic, inflexible sampling of observations. We propose CondenseVLA, a multi‑frame VLA policy with a learnable decoder‑side history condensation module for using short‑term visual context in action decoding. Given a sequence of cached per‑frame features from the VLM backbone, CondenseVLA distills observation history into a fixed set of slot tokens using learnable queries and multi‑head cross‑attention. Simple auxiliary regularization penalizes overlap among slot attention maps and decorrelates slot embeddings, encouraging complementary attention and reducing slot redundancy. Across the evaluated tasks, CondenseVLA improves Simpler‑WidowX success from 58.7% to 69.8% (+11.1 percentage points), achieves 97.2% on LIBERO, and reaches 52.8% on real‑world tasks. It achieves 21.6% success on memory‑dependent RMBench tasks. These results support selective condensation of bounded recent visual evidence for action decoding, while harder multi‑observation tasks expose the limits of a finite history window. Project page: https://github.com/FeiyangHong/CondenseVLA
Authors:Jianhao Zhang, Ou Wu, Yi Du
Abstract:
Catastrophic forgetting is a central obstacle in single‑stage fine‑tuning of large language models (LLMs) on a new task, especially when the original pretraining data are unavailable. Synthetic data offers a practical mitigation path, but candidate pools are often noisy and redundant, making effective selection essential. Existing methods often rely on indirect proxy signals whose alignment with actual training benefit can be limited, while utility‑‑diversity selection over large candidate pools remains computationally demanding. To address these challenges, we propose NAvigator‑guided Data Selection (NADS), a fine‑tuning framework for LLMs. NADS first follows the target fine‑tuning recipe without preservation regularization to obtain a navigator model, whose deviation from the pretrained model exposes recipe‑induced capability drift. It then uses the predictive divergence between the navigator and the pretrained model to score forgetting‑aware utility for each synthetic candidate, and constructs a constraint set through an efficient utility‑‑diversity selection strategy. Finally, NADS fine‑tunes on the new‑task data while distilling pretrained behavior on the selected constraints to preserve general capabilities. Experiments show that NADS delivers a stronger balance between new‑task performance and general capability preservation, while reducing utility‑‑diversity selection overhead. Code at: https://github.com/zjh‑vinky/NADS.
Authors:Xiao Ma, Zhonghao Zhan, Yi Tang, Qi Chen
Abstract:
Modern generative models can produce visually plausible images while violating exact identity constraints. Evaluating such failures requires distinguishing violations of the requested identity from permissible variation in appearance. We introduce Low‑Entropy Visual Identity (LEVI), which characterizes visual identities determined by compact facts and explicit rules. The low‑entropy requirement concerns identity uncertainty, while appearance may remain diverse. We formulate identity fidelity in terms of the probability that a conditional image distribution assigns to the identities permitted by the prompt. Visual Identity Spaces (VIS) formalize this distinction by specifying admissible identity values and which different realizations express the same identity. Building on this formulation, we develop Verifiable Visual Grammar (VVG), a declarative language whose typed primitives and compositional constraints describe identity‑bearing structures. Using this grammar, we build a benchmark for image generation. It assesses each output against the task's identity requirements, drawing evidence from the final pixels. The curated benchmark contains 1,375 tasks spanning 120 base identity spaces across six domains. Among ten evaluated model configurations, the highest whole‑task pass rate is 58.69%. These results reveal a persistent gap between visually plausible images and faithful realization of their specified identities. Project page: https://maxiao0234.github.io/LEVI/.
Authors:Senhe Zhang, Tao Zhang, Yujia Zhang, Zeyu Zhang, Dong Gong
Abstract:
Long‑horizon human‑scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text‑driven motion models are largely scene‑agnostic and lack closed‑loop correction, while geometry‑conditioned approaches depend on paired scene‑motion training or explicit 3D preprocessing that limits generalization to novel layouts and open‑vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long‑horizon human‑scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step‑wise recovery decisions (continue/redo/replan) and Memory‑Augmented Reflection for multi‑round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training‑free baselines. Code: https://github.com/AIGeeksGroup/InteractAgent.
Authors:Antonio Scardace, Silvia Ingala, Sebastiano Battiato, Daniele Ravì
Abstract:
Spatiotemporal forecasting of 3D anatomical changes is a critical generative vision challenge for modeling Alzheimer's Disease (AD) progression. While brain MRI forecasting from single‑modality data has advanced, other key complementary multimodal biomarkers remain underutilized. In this work, we propose the Multimodal‑Conditioned Latent Diffusion Model (MC‑LDM), a novel framework for patient‑specific brain MRI synthesis. MC‑LDM leverages multimodal biomarkers‑‑‑MRI, tau‑PET, Clinical Dementia Rating (CDR), and APOE4‑‑‑during training, but relies only on baseline MRI, APOE4, and patient metadata at inference, reflecting realistic clinical availability. Our contributions are threefold: (1) a Latent Diffusion Model (LDM) fine‑tuned via Low‑Rank Adaptation (LoRA) for data‑efficient learning; (2) a multimodal conditioning that integrates molecular and clinical signals to model heterogeneous disease trajectories; and (3) auxiliary predictors estimating unobserved multimodal biomarkers from MRI features and patient covariates to condition the diffusion model, removing the need for these assessments at inference. Trained on 1,355 visits from 852 subjects in the Alzheimer's Disease Neuroimaging Initiative (ADNI), MC‑LDM reduces average regional volumetric MAE by 18% over the second‑best baseline on an independent test set of 664 visits. To our knowledge, MC‑LDM represents the first diffusion‑based framework to leverage multimodal priors for MRI forecasting. This conceptual advancement enables patient‑specific trajectory generation that captures aspects of AD heterogeneity not observable from structural anatomy alone, bridging the gap between rich multimodal supervision and the strict constraints of routine clinical deployment. Code is available at: https://github.com/antonioscardace/MC‑LDM.
Authors:Wenyi Lian, Zheng Zhao, Thomas B. Schön
Abstract:
Test‑time scaling of diffusion models with sequential Monte Carlo (SMC) has become a promising strategy for sampling under user‑specified rewards, such as human preference scores. However, the standard SMC framework typically relies on stochastic proposals and therefore does not directly transfer to flow matching models, in which inference is governed by deterministic ODEs. In this work, we introduce noise‑rejuvenated SMC, an inference‑time scaling method tailored for text‑to‑image flow matching models. Our key idea is to perturb each resampled particle by changing its source noise and constructing a new intermediate state using its current endpoint estimate. This moves the particle to another state that remains representable under the flow‑matching interpolation form, enabling particle exploration without converting the sampler to an SDE. We further introduce a look‑ahead mechanism to refine resampled particles before weighting. Experiments on Stable Diffusion 3, FLUX.1‑dev, and Z‑Image demonstrate consistent improvements in reward‑guided generation, while ablation studies validate the contributions of noise rejuvenation and look‑ahead refinement. Code is available at https://github.com/shermanlian/NR‑SMC.
Authors:Houyuan Chen, Fanzhou Wang, Xiangyu Fan, Zesong Qiu, Yukun Wei, Songlin Yang, Zhitao Yang, Lvmin Zhang, Pei Yang, Hao Zhao, Dahua Lin, Lei Yang, Anyi Rao
Abstract:
Multi‑turn image editing aims to modify an image according to a sequence of instructions, enabling more effective interactive visual content creation for users. However, existing methods often exhibit noticeable drift after only a few editing turns, manifesting in two aspects: 1) unstable visual quality, with accumulating artifacts and degraded details; and 2) inconsistent image content, with unintended changes to preserved regions and prior edits. To address these challenges, we propose MT Forcing, a novel training paradigm for multi‑turn image editing with unified multimodal models (UMMs) based on autoregressive diffusion. Specifically, we propose Single‑Turn Distribution Matching Distillation (Single‑Turn DMD), which supervises student self‑rollouts with a teacher that applies cumulative edits to the source image in a single turn, providing visual‑quality guidance without recursive error accumulation. Additionally, we introduce Teacher‑Forcing Guidance, which supervises history‑dependent transitions on reference trajectories during warmup and as a regularizer during distillation, encouraging cross‑turn content consistency beyond what per‑turn distribution matching alone can ensure. Extensive experiments demonstrate that our approach significantly improves the multi‑turn image editing capabilities of UMMs, yielding more stable visual quality and stronger cross‑turn consistency.
Authors:Markus Pobitzer, Chang Liu, Niccolò Biondi, Nicu Sebe
Abstract:
Digital art generation has undergone a massive paradigm shift, moving from static image synthesis to dynamic video generation that can mimic the human painting process. While existing models can synthesize visually stunning time‑lapses of paintings being created or "unpainted," they largely treat the creative process as a black box. Current conditioning mechanisms are either too rigid, relying on static geometric cues like Canny edges or depth maps, or too holistic, utilizing simple scalar progress variables that offer no localized control. Consequently, users lack the agency to steer how a painting evolves over time. In this work, we argue that true control over the video generation of artwork requires a joint understanding of two fundamental dimensions: where an artist acts, and when those actions take place. To achieve this, we introduce the Painting Change Map, a novel spatio‑temporal control mechanism that explicitly tracks normalized structural changes across a 1,000‑timestep continuum, a resolution sufficient to capture fine‑grained stroke‑level changes while remaining computationally tractable. By mapping localized visual differences into an encoded RGB space, our representation enables fine‑grained guidance for both the forward painting and reverse unpainting processes. Extensive benchmarking demonstrates the efficacy of our approach. We outperform existing control baselines and achieve competitive or superior results compared to state‑of‑the‑art painting methods. Furthermore, under oracle conditions, our method surpasses all baselines by a significant margin. Code is available at https://github.com/Markus‑Pobitzer/painting‑change‑map.
Authors:Yushi Lan, Zeren Jiang, Koichi Namekata, Xingang Pan, Chuanxia Zheng, Andrea Vedaldi
Abstract:
Synthesizing a monocular video from a new camera trajectory requires preserving the observed scene geometry and motion while completing regions that become newly visible. Feed‑forward 4D reconstructors recover the structure of the observed scene, whereas video diffusion models provide a generative prior for completing unseen content. We introduce Holo4D, a video reshooting framework that uses reconstructed 3D trajectories as a holistic source‑view 4D scaffold. We project these trajectories into the requested views and encode their geometry and source appearance to guide a pretrained video diffusion model. Together with the source video and target cameras, the scaffold grounds generation in reconstructed observations while leaving unsupported regions to the generative prior. Experiments demonstrate the value of trajectory‑based conditioning for camera control and novel‑view synthesis. The framework accommodates different 4D reconstructors without retraining the video generator, allowing it to benefit from future advances in source reconstruction. More details can be found in our project page.
Authors:Boxin Bu, Simon Boche, Haoming Zhang, Jaehyung Jung, Jiaxin Wei, Hannah Schieber, Daniel Roth, Angela P. Schoellig, Stefan Leutenegger
Abstract:
In this letter, we present GOFusion, a real‑time visual mapping system for autonomous robot navigation that anchors Gaussian primitives to local occupancy maps, providing both spatial awareness for path planning and high‑fidelity appearance that enables information‑driven robot navigation. GOFusion builds 3D Gaussian maps from pose estimates and local occupancy maps obtained via visual‑inertial SLAM and enables pose corrections to maintain both geometric and visual consistency. Furthermore, it restricts the 3D Gaussian Splatting optimization to active submaps to bound GPU memory usage. Building on the appearance‑rich Gaussian map, we introduce a rasterizer‑integrated Fisher approximation that quantifies per‑primitive sensitivity online, serving as an information‑driven heuristic for Gaussian pruning and next‑best‑view planning. We evaluate GOFusion on public datasets and in real‑world drone flights. It not only outperforms several baselines in mapping quality, but also achieves real‑time, high‑fidelity and uncertainty‑aware mapping at scale. Finally, we demonstrate fully autonomous drone exploration for Gaussian map refinement using the proposed Fisher‑informed heuristic. Code is available at https://github.com/Bennnoo/GOFusion. We provide a supplementary video demonstration at https://youtu.be/pLQKbxQvb6I.
Authors:Kaidi Wang, Wenhao Guan, Peijie Chen, Weijie Wu, Xuetao Lin, Lin Li, Qingyang Hong
Abstract:
Automatic video dubbing (AVD) requires temporally aligned speech, transcripts, and video, yet existing corpora vary in audio quality, transcription reliability, and speaking‑face visibility. We introduce VoxDub, a bilingual lip‑synchronized audio‑visual dataset containing 766K clips and approximately 1.58K hours of in‑the‑wild Chinese and English recordings, covering varied speakers, scenes, speaking styles, and linguistic content. Its curation pipeline combines speech‑centered segmentation with dual‑ASR transcription checking, speech quality filtering, face visibility checks, and lip‑speech synchronization verification. Text‑to‑speech (TTS) experiments show that training on VoxDub yields lower word error rates and higher speaker similarity than training on the compared audio‑visual corpora, LRS3 and CN‑CVS. AVD experiments show broadly comparable speech quality and subjective synchronization. These results support the utility of VoxDub for TTS and AVD training. More details are available on the project website (https://voxdub‑dataset.github.io/VoxDub/).
Authors:Yuchen Xi, Yuhan Guo, Chenwei Hou, Zixu Liu, Jin Fang, Jason Liu, Ruigang Yang
Abstract:
Novel view synthesis under trajectory changes is essential for autonomous driving simulation, yet existing methods struggle to generate consistent videos when extrapolating to unseen viewpoints such as lane shifts. Reconstruction‑based approaches maintain geometric consistency but degrade in visual quality under large viewpoint changes, while reconstruct‑then‑restore methods often produce visually plausible frames yet suffer from geometric and temporal inconsistencies. We present GeoV2V, a geometry‑grounded video‑to‑video diffusion framework that synthesizes temporally coherent driving videos along novel trajectories by integrating LiDAR and depth‑derived geometric priors with full‑video conditioning in the Wan 2.1 backbone. A key challenge in learning cross‑lane transformations is the lack of synchronized multi‑trajectory supervision in real‑world datasets. To address this, we also introduce Para4D, a synthetic dataset capturing synchronized multi‑lane driving videos within dynamic scenes, which provides explicit supervision for lane‑shift generation and serves as a strong training prior for geometry‑consistent view synthesis. Experiments on Waymo, nuScenes, and Para4D demonstrate state‑of‑the‑art performance in visual quality, geometric consistency, and robustness to complex lighting dynamics under large trajectory shifts. Project page: https://xiyuche.github.io/GeoV2V/
Authors:Haotian Gan, Weidong Tang, Hanbin Sun, Yi Luo, Xintian Li, Jiayong Wang, WanYue Li, Yudong Li
Abstract:
Evaluating LLMs for crisis communication requires more than judging isolated responses, because public feedback and competitor actions can reshape trust over time. Existing benchmarks remain largely static and single‑turn, offering limited evidence about long‑horizon risk‑sensitive communication. We introduce BrandEval, a two‑track benchmark for product‑crisis communication, focusing on public brand responses to product‑related safety, quality, service, and trust incidents. BrandEval‑Static evaluates stakeholder‑aware strategic analysis, while BrandEval‑Dynamic stress‑tests response policies in BrandPolis, a controlled multi‑agent sandbox with consumer feedback, competitor responses, and trust‑risk trajectories. We also construct BrandSRD, a Chinese crisis‑response dataset, and instantiate a reference Strategic Agent using Strategic Rationale (SR). Across proprietary and open‑source models, we find that static strategic quality is informative but incomplete: it aligns with some end‑state outcomes but does not reliably predict volatility and delayed recovery; dynamic stress tests further expose tail‑risk failures under manipulative or weakly grounded policies. The code and dataset will be available at https://github.com/htgan‑ai/BrandEval.
Authors:Haotian Gan, Yudong Li, Xintian Li, Jiarui Han, Weidong Tang
Abstract:
Medical vision‑language models often exhibit hidden stratification, where high overall performance masks failures on subtle visual findings and overlapping clinical conditions. We trace this failure mode to modality imbalance during fine‑tuning, where strong linguistic priors can suppress visually grounded learning and induce plausible but weakly grounded rationales. Existing data selection methods usually score image‑text pairs as holistic units, without separating visually necessary samples from shortcut‑solvable ones. To address this problem, we propose ViCoR, a counterfactual‑residual‑based data selection method for Med‑VLM fine‑tuning. ViCoR estimates visual necessity through image‑dependent optimization residuals, calibrates the resulting scores with medical supervision compatibility, and constructs a diversity‑aware fine‑tuning subset. Across two backbones, four external generalization benchmarks, and one multi‑disease robustness benchmark, ViCoR‑selected 20% subsets achieve the strongest average performance compared with full‑data fine‑tuning and selection baselines. Gradient‑level analysis and perturbation audits further show that ViCoR enriches image‑dependent training signals and strengthens image‑level and evidence‑region dependence. The code will be available at https://github.com/htgan‑ai/ViCoR.
Authors:Simon Josef Durstewitz
Abstract:
An increasing number of disciplines recognize the explanatory potential of meaning‑bearing mental structures through which individuals interpret the world and guide their behavior. Concept representations constitute a core class of these structures, emerging from individuals' experiences with concept referents and encoding and organizing their understanding of them. Recognizing a private level of meaning, where concepts are not objectively defined by rules in the world but constitute subjective structures in individuals' minds, raises a fundamental question: do different individuals actually mean the same thing when they invoke the same concept? As latent structures, concept representations are difficult to access and model empirically. The present study introduces a measurement framework that makes such representations empirically accessible through textual traces elicited under standardized conditions. Using transformer‑based sentence embeddings, the framework maps standardized responses elicited for concepts into a high‑dimensional shared vector space, opening concept representations to the systematic analysis of interindividual variation. It develops three analytical avenues: (1) comparing representational sharedness across concepts, (2) identifying variables that correlate with representational variation, and (3) examining how focal actors or subpopulations deviate from a reference population. Each avenue is illustrated through an empirical proof‑of‑concept drawing on a large‑scale free‑association dataset. Together, the demonstrations show how researchers from across disciplines can move from a research question and data source to analysis and interpretation. An accompanying GitHub repository provides a ready‑to‑use implementation of the framework for existing and custom datasets.
Authors:Karthik Iyer, Madan Dabbeeru
Abstract:
A retailer makes different decisions at different grains of the product, store and time hierarchy: store orders for an item in a store, supplier funding for an item across stores, category performance by store, and the financial plan for the chain. Which forecasting model to use depends on the grain. This paper compares models at each grain on four public data sets (M5, six Dominick's categories, Breakfast at the Frat, and Corporación Favorita, the last kept out of all development), over rolling forecast origins with paired tests and model confidence sets. It covers price and promotion through the plan inputs these data record, not product, assortment or place. The models are a world model that carries each item's state forward under the price and promotion plan, the same network without that state, LightGBM, the Temporal Fusion Transformer, TimesFM, Chronos‑Bolt, Timestral, seasonal naive, and an equal‑weight average of the world model, LightGBM and TimesFM given the plan. The answer changed with the grain. For an item by store and an item across stores, the average had the lowest error on seven and eight of nine data sets, and the 90% confidence set held a median of 1 and 2 models. For a category by store, a category and the chain total, the lowest‑error model differed between data sets and a median of 4 of nine models were not separated by the four to six windows available. For category‑by‑store planning cost, the average or LightGBM was lowest on seven of eight data sets, and the world model was lowest only on M5; it was behind LightGBM item by store on six of eight. Every trained model used the plan, and on the weekly promotion‑driven data pretrained models were less accurate without it. These results hold for these data sets; a forecast solved across all four Ps, or a retailer's own data, may give different results. Every model, test and change that lost is reported.
Authors:Amirreza Mohseni, Gerasimos Spanakis
Abstract:
People ask conversational AI models about their legal rights, obligations, procedures, and personal situations that may have legal consequences. To study this behavior, we first need to distinguish requests for legal guidance from other legally themed discussions and unrelated conversations. We contribute a multi‑axis taxonomy of legal queries capturing whether legal guidance is sought, how a request is formulated, and which legal domains it concerns; apply it to 2,122 naturally occurring WildChat‑1M conversations; and create a human‑reviewed gold evaluation set of 200 conversations. All annotations, model checkpoints, and evaluation code are publicly available. In this enriched sample, 71.1% of conversations containing legal content seek personalized or practical guidance. Rights‑and‑options advice is the most common primary request type among legal‑content conversations, and 25.3% of guidance‑positive conversations span multiple legal topics. The gold evaluation set supports a two‑stage comparison of ModernBERT‑base and ModernBERT‑large against a TF‑IDF cascade. Our framework supports large‑scale studies of legal guidance in conversational AI, and the public release of the annotated data facilitates reproducible research and future model development.
Authors:Yakiv Shavidze
Abstract:
Repeatedly sampling short sequence windows from a human pangenome requires an access representation as well as compression. panvram stores each assembly as a reference‑relative block archive and pools these archives on a GPU. For 558 haplotype assemblies, the default q4k representation occupies 16.642 GB of persistent pools including the decoded reference; a packed‑reference, compact‑directory representation occupies 11.634 GB. These quantities are not peak application memory. In the C1 experiment, default and fully compact forms delivered 743,271 and 736,762 windows/s; a separate public Colab run delivered 741,931 and 680,760 windows/s. All are batch API measurements and must not be pooled as repeated trials of an identical environment. Full‑decoding checks compared reconstructed FASTA checksums with the manifest. On the CPU, the M6 shuffled‑window workload at the same cohort cardinality reported 26 requests/s for AGC and 145,222 requests/s for refrel3 q4k. This comparison concerns one‑thread requests of 4 KiB on the recorded warm‑cache workload, not all genome‑access patterns. AGC provides the smaller incremental archive: 5.40 MB/assembly versus 13.04 MB/assembly for refrel3 q16k, with different exclusions; the size pair is not the same refrel3 configuration as the q4k timing pair. We derive an expected‑work expression for fixed independent granules and distinguish it from an empirical median‑latency predictor. Cohort‑scale counter and DRAM measurements motivate a locality hypothesis, but do not establish a causal per‑miss latency or independently reproduce a model‑failure verdict. The window‑cost verdict at N=558 is published with its harness and raw timings.
Authors:Amirhossein Ghaffari, Susanna Pirttikangas, Ekaterina Gilman
Abstract:
Traffic forecasting models based on spectral graph convolution offer an interpretable account of how information is mixed between sensors, but they rely on an eigendecomposition of the graph Laplacian. It costs cubic time, is fixed by the road network, and provides a symmetric operator that cannot represent the directed propagation of traffic congestion. Attention‑based spatio‑temporal transformers remove these limitations but generate a dense sensor‑by‑sensor attention map at each layer and time step, so memory, rather than accuracy, becomes the binding constraint on large networks. We introduce the spectral adaptive graph transformer, whose spatial operator is learned rather than derived from the Laplacian, directed rather than symmetric, and conditioned on the current traffic state rather than a static state, at a cost linear in the number of sensors. A sparse road graph diffusion branch provides the strictly local propagation that a low‑rank operator cannot express, and a zero‑initialized gate learns the balance between the two. Before spatial mixing, a unit‑mean‑gain frequency‑bin attention module reweights each sensor's short‑term periodicities according to its identity and current context. Across six public benchmarks spanning 170 to 1843 sensors, evaluated over three random seeds, the proposed model attains the lowest error of any single end‑to‑end model on one benchmark and remains competitive with the strongest current attention models on the others, while using about a quarter of their memory and training about twice as fast.
Authors:Shuo Xu, Hanrui Ying, Debin Liu, Jianjun Li, Zhiyuan Ma
Abstract:
Fine‑grained rectification of existing generations under a limited additional budget remains a key problem in test‑time optimization (TTO) for image generation. As for prior approaches, they could identify residual errors, but could not directly ensure that the model acts on the feedback during reasoning. Such errors stem from multiple sources, such as the initial noise or text prompt, and ultimately induce an execution gap of failing to handle potential deviations along the trajectory generating. However, classic models generally cannot remedy this gap. To narrow it, we propose Reflection‑Guided Dynamics Rectification (RGDR), which turns reflections on completed outputs into corrective interventions on the process of sampling dynamics at the trajectory level. Specifically, RGDR could impact every sampling step by adapting to instantaneous intervention. Moreover, RGDR generates flexible candidates while adjudicating between them and the reference. As experiments show, rectification from RGDR starts from the existing base trajectory and reflection, and RGDR is substantially more effective than former trajectory exploration. Under matched budgets for additional image generation, RGDR's gain in the cross‑model average final checklist score over the reference is 1.51× that of TIR prompt refinement and 1.81× that of Dynamic CFG.
Authors:Faisal Mohammad, Mohamed Younis
Abstract:
Recognizing a vessel's type from an image supports maritime surveillance and sanctions monitoring, but a classifier can reach high accuracy by leaning on the generic hull instead of the more distinctive superstructure, and is then fooled when a vessel is re‑hulled or occluded. De‑biasing methods claim to remove such reliance, yet the field certifies them with worst‑group accuracy, a proxy that needs group labels and only infers reliance from accuracy. We measure the reliance directly. HullProbe swaps a registered anatomical part of a vessel for another instance's and reads how much that part drives the decision, using a same‑class swap to control for compositing artifacts. Because a surface vessel is a hull and a superstructure arranged along one axis, its parts form a 2x3 grid whose 2^6 = 64 swaps we evaluate exhaustively, and a single Walsh transform returns each part's interventional influence and its interaction spectrum. On planted shortcuts, where the true reliance is known, the measure tracks worst‑group accuracy (r = ‑0.94 over 45 configurations), and replacing its learned segmenter with human annotations changes the hull‑share by less than the audit's resolution. On the 26 vessel types of MARVEL, the classifier places about 46% of its corrected sensitivity on the generic hull yet follows the superstructure when the two conflict, and the group‑label‑free de‑biasers available there do not detectably reduce the hull's share. The findings hold across three architectures and a second maritime dataset, and the method extends to fine‑grained vehicle recognition on Stanford Cars.
Authors:Xi Zhu, Yuanlin Duan, Changyi Yang, Kai Mei, Riyang Bao, Shichao Zhu, Dimitris N. Metaxas
Abstract:
Skill‑augmented agentic RL has emerged as an effective approach to improving LLM agents on long‑horizon tasks. Despite this progress, individual skills may follow distinct trajectories as the policy evolves, with some becoming redundant after the model internalizes their capabilities, others remaining valuable as external scaffolding, and still others turning harmful. However, existing skill library systems fail to identify each skill's true contribution due to free‑riding among co‑retrieved skills, difficulty confounding across tasks, and the absence of skill‑specific counterfactual evidence, which prevents reliable skill lifecycle management. We propose TrackRL, an agentic RL framework where skill tracking drives skill library full lifecycle management. Specifically, within each GRPO group, we turn grouped sampling into online randomized controlled experiments. Independent random injection and within‑group contrasts on the same task isolate each skill's true contribution, and reliability‑weighted aggregation accumulates evidence across groups. As evidence accumulates, we continuously track skill utility, which measures the skill's causal contribution as external scaffolding, and model mastery, which reflects how far the corresponding capability has been internalized. Along this line, we formulate skill lifecycle management as an evidence‑gated state‑transition process that governs each skill from generation and admission through protected incubation to retention, retirement, or removal, while a soft capacity bound controls library growth. Experiments show that TrackRL outperforms recent skill‑augmented RL baselines in both library‑assisted and skill‑free evaluation, with further analyses revealing how skill value shifts during policy learning and how evidence‑driven state transitions shape skill library evolution. Our code is available at https://github.com/xizhu1022/TrackRL.
Authors:Yanyan Wu, Guanyu Zhu, Changbin Zhao, Jin Zhang, Dexu Yu, Tianmin Sun, Jining Luan, Chunxiao Li, Hanwen Du, Youhua Li
Abstract:
Offline sepsis treatment aims to learn treatment policies from retrospective ICU trajectories. Decision Transformer‑based methods provide a natural sequence modeling framework for this task, but they mainly rely on raw physiological states and historical clinician actions. Such raw trajectory tokens may not explicitly capture clinical semantics that clinicians use in practice, such as patient severity, recent treatment response, and current risk priorities. To address this limitation, we propose ClinSDT, an LLM‑assisted Decision Transformer framework for offline sepsis treatment. ClinSDT uses a frozen LLM as a semantic bridge rather than a treatment decision maker. It constructs three clinical semantic views from ICU trajectories, namely Profile, Context, and Advisory, and inserts their encoded representations as independent tokens into a Decision Transformer policy. To support policy refinement and rollout, ClinSDT further trains a Treatment‑Response Forecaster to estimate the next physiological state, SAPS‑II change, and predictive uncertainty from state‑action inputs. In the second training stage, the forecaster screens local counterfactual treatment candidates, and only low‑uncertainty candidates with lower predicted SAPS‑II increase are distilled into the policy after semantic regeneration. During inference, ClinSDT performs closed‑loop treatment rollout by refreshing semantic inputs from the currently available patient state. Experiments on MIMIC‑III and MIMIC‑IV show that ClinSDT achieves consistent relative improvements over representative imitation learning, offline reinforcement learning, and Decision Transformer baselines under the same rollout protocol. Code and data are available at https://github.com/KAG778/ClinSDT.
Authors:Jingwen Bai, Guochao Bai, Canbo Zhang, Shuangqi Li, Minchao Hu, Weichun Huang, Shuang Liu
Abstract:
Embodied AI lacks the data‑rich pretraining that gave language models their agents and skills, so new manipulation capability costs demonstrations per task or a frontier model in every episode. We study a third route: a strong general model acts only as an offline author that writes, runs, diagnoses and repairs a manipulation skill in simulation until it passes an exam on layouts drawn after freezing; the frozen skill then runs as code alone or under a cheap caller that reads only its description. We call this skill‑level recursive self‑improvement Skill‑RSI and specify a compile protocol (isolated, audited author; hidden validation layouts; no‑feedback and inherit‑only controls; tamper‑evident ledger) and a reuse protocol that runs the identical files with executors from another model family. On 7 RoboTwin 2.0 tasks with cluttered tables, Claude Code compiled 4 tasks into 7 frozen skills, 5 from a first draft needing only qualification, at USD 4.29‑8.22 of author cost each (USD 55 per skill counting the 13 failed campaigns). The other three tasks did not compile in 18 campaigns, including five with GPT‑6 and a pre‑registered 150‑layout run: on the hardest task every campaign scored 78‑95 of 100 but had 3‑16 safety aborts, failing the gate of 95% success with zero aborts; a 100‑episode exam bounds an abort rate only to 3.6%. On hidden layouts, repair that inherits the previous version beat from‑scratch rewriting in 8 of 8 controlled campaigns (sign test p = 0.004); an inherit‑only lineage attributes most of the gap to inheritance, with development feedback adding a little (3 of 3, p = 0.125). Write‑once campaigns found no detectable first‑draft effect of practice (minimum detectable effect 14 points). The frozen skills kept 148/150 successes on new layouts with no model and 147/150 with a DeepSeek‑flash caller at USD 0.00028‑0.00054 per episode; no direct‑control baseline was run.
Authors:Galen J. Wilkerson
Abstract:
The explore/exploit trade‑off is the core problem of reinforcement learning (RL). We show that a minimal foraging swarm approaches its optimum ‑ the ideal‑free / Nash allocation of a multi‑agent resource‑sharing game (an equilibrium, not a return maximum) ‑ with no learner and no reward signal. Agents explore by heavy‑tailed (Levy) flights, recruit neighbours to a discovery through a threshold cascade on their proximity network, and disperse as a patch crowds; this cascade cycle balances exploration and exploitation and drives foragers toward the ideal free distribution. The swarm attains the shape of this optimum (committed foragers track richness with slope 0.94) but not its exact equilibrium ‑ it stays measurably exploitable ‑ reaching it dynamically (~150 steps) through the coupling of recruitment to movement: remove the coupling and matching collapses (0.94 ‑> 0.08). The RL sits at the level of the problem and the allocation reached, not the micro‑rule: no step evaluates payoff (reward‑free by construction, though the crowd signal is informationally tied to payoff), and the swarm reaches the allocation without learning it (quality plateaus and re‑settles after resources move, no cumulative gain). Against a trained baseline the comparison is two‑sided: in an easier, disembodied game a learner is more precise, while in the same embodied environment the reward‑free swarm reaches ~65% of a validated oracle ceiling at zero training and leads until trained PPO and centralised‑critic (CTDE) learners overtake it (~74‑75%, 16 seeds, all above the swarm mean) after ~4,000 of 15,000 training steps ‑ its advantage is in budget, not asymptote. In this one setting we read it not as beating RL but as a strong zero‑shot, reward‑free operating point: a demonstration that the self‑organizing physics of a minimal swarm can carry a collective close to the optimum RL seeks, with none of RL's machinery.
Authors:Abdul Karim, Matt Keon
Abstract:
As model reasoning gets cheaper, the physical experiment becomes the slow and costly step in discovery. Each experiment returns a noisy reading: a high value can mean a good candidate or a lucky draw. With a fixed budget, an AI agent must keep choosing between exploring a new candidate and confirming a promising one by re‑testing it. We measure that choice in Wellocate, a simulated assay campaign with known true values. It is built on a generated landscape and on two real protein datasets, GB1 and PhoQ, scrambled so no model can answer from memory. We test five language models as minimal agents, with no tools and no memory beyond the history in each prompt. They misallocate in both directions. Three over‑confirm, spending most of the budget re‑testing. On the generated landscape one under‑confirms, trusting single readings, and there every model ends with a worse final pick than a basic scripted strategy. Each failure has its own cause. The over‑confirmers run short of new candidates worth trying: a list of untried candidates cuts their re‑testing and, on real protein data, makes two of them pick much better. A running summary of their readings or a description of the landscape changes it by at most five percentage points. For the under‑confirmer, re‑testing its top candidates in a final round, replayed on its own campaigns, recovers only about a quarter of what it loses by picking the wrong candidate. This suggests confirmation has to run through the whole campaign. We release Wellocate as a benchmark and reinforcement‑learning environment with frozen splits and scripted baselines. With it, anyone can measure how an agent splits its budget before trusting it with real experiments, and train agents to split it better. Code, data and the environment are available at https://github.com/Xstendyl/wellocate.
Authors:Jeonghwan Choi, Jibin Hwang, Hayoon Park, Hyeonjae Cheon, Hwanjun Song
Abstract:
Enterprise retail workflows require agents to chain evidence across heterogeneous sources, from structured product and order records to unstructured inquiries and reviews, yet existing agentic RAG benchmarks rarely capture this challenge. To enable controlled evaluation of such workflows, we introduce RetailBridge, a compositional retail benchmark built from persona‑driven simulations. Its questions are constructed by linking entities and intermediate answers across heterogeneous sources, requiring agents to identify the relevant sources, carry evidence between them, and produce a grounded answer. To systematically evaluate LLMs on these retail workflows, we develop RetailRAG, a retail‑aware framework equipped with auto‑schema briefing, source‑routed plan chaining, and diagnostic plan revision. RetailRAG consistently outperforms existing agentic RAG baselines across all evaluation axes, serving as a strong reference system for assessing LLM agents on retail workflows. On RetailBridge, evaluating ten LLM backbones with RetailRAG reveals multi‑source chaining as the dominant bottleneck across providers and model scales, suggesting that reliable retail agents require explicit cross‑source orchestration rather than scaling alone. Code and data are publicly available at \urlhttps://github.com/DISL‑Lab/RetailBridge.
Authors:Kaiwen Wei, Kejun He, Jie Zhang, Xiaomian Kang, Yuming Yang, Aijia Zhou, Xie Zhou, Yining Wang, Nayu Liu, Li Jin, Zhenyang Li, Jiang Zhong, He Bai, Jingxuan Wei, Junnan Zhu
Abstract:
Generative recommendation constructs semantic IDs for catalog items and predicts the next ID from user interaction histories. However, existing methods face limitations in both stages. First, conventional quantizers may produce imbalanced code assignments and lack coherent intermediate semantic units between individual tokens and complete items, limiting history reconstruction. Second, recommendation models remain dominated by next‑item prediction, treating historical interactions mainly as conditioning context and providing limited supervision for recovering persistent preferences across the historical path, particularly when relevant preference signals are distributed over distant interactions. To address these limitations, Structure‑Aligned History Learning (SAHL) reconnects the two stages by using the structure created during quantization directly for history reconstruction. Its Semantic Residual Balanced Quantization (SRBQ) forms a balanced token‑subspace‑item hierarchy, and SAHL reconstructs history at these same granularities. Predictive entropy prioritizes uncertain transitions, while curriculum optimization progressively shifts training to inference‑aligned next‑item prediction. On four Amazon Review categories, SAHL improves NDCG@10 by 18.6%‑30.0% over RPG and 2.7%‑6.0% over strong method MHL. Controlled analyses verify balanced code use, the benefit of following quantizer‑defined groups during reconstruction, and dependence on user history. The relevant code is available at https://github.com/CQU‑MM‑Intelligent‑Lab/SAHL.
Authors:Junwei Wen, Deshui Miao, Guangming Lu, Ming-Hsuan Yang, Xin Li, Wenjie Pei
Abstract:
Video Reasoning Segmentation (VRS) requires a model to identify an implicitly described target and predict its pixel‑level mask trajectory, while resolving the spatial relations and temporal events that determine the answer. Text‑only Chain‑of‑Thought (CoT) can explain a target choice without making the underlying visual references explicit, leaving a \emphvideo reference gap between linguistic inference and the evidence distributed across frames. We introduce Temporal Chain of Video Primitives (TCoVP), a spatiotemporally grounded form of Video CoT that organizes query‑relevant observations into a chronological reasoning chain. Each primitive associates an entity reference with temporal support and timestamped spatial evidence, and the chain interleaves these references with language to guide reasoning about which entity is relevant, when the queried event holds, and where its evidence appears. The selected target primitives then provide box prompts and temporal support for video mask prediction. We construct TCoVP‑SFT‑15k and TCoVP‑RL‑13k to learn this representation through supervised fine‑tuning and chain‑aligned reinforcement learning. Extensive experiments across five VRS benchmarks and three model backbones demonstrate the strong performance of TCoVP, highlighting the effectiveness of organizing spatial and temporal evidence within the reasoning process. All models and code will be publicly released. Project page: \urlhttps://victor‑wjw.github.io/TCoVP/
Authors:Dengyuxing Wang, Yonggang Zhang, Qianshu Cai, Huajiang Zheng, Jun Song, Yangqiu Song, Yike Guo
Abstract:
Automated AI research agents aim to improve machine learning methods by iteratively proposing, implementing, and evaluating candidate methods. Existing work uses the outcome of each evaluation to determine how the next candidate should be revised for performance improvement. However, a candidate often combines multiple design decisions, and its overall outcome alone does not reveal which decisions produced their expected effects, whether their required conditions were satisfied, or which decisions should be retained or revised. Inspired by ablation studies in the machine learning community, we propose DAGLoop to test the expected effect of a design decision by comparing implementations with and without that decision while preserving the other decisions it requires. Specifically, DAGLoop translates each design decision's rationale into a testable expectation and pairs it with a targeted experiment, forming an experimental specification for assessing the expected effect under its required conditions. Using these specifications, we formulate ablation planning as the construction of a directed acyclic graph (DAG) from the candidate method's design: nodes associate decisions with tests, and edges encode dependencies. The DAG specifies which decisions each experiment must retain and supports joint tests when an expected effect requires several decisions to act together. Outcomes are recorded with the tested decisions, expectations, and conditions to form structured research memory. This memory guides the agent in retaining experimentally supported decisions, revising unsupported ones, and planning further experiments when the evidence is unresolved. Experiments on dozens of research tasks demonstrate substantial performance improvements achieved by DAGLoop across diverse executable research settings. Code and evaluation scripts are available at https://github.com/deng34567/DAGLoop.
Authors:Sunghoon Kim, Sangyeop Kim
Abstract:
Large Language Models often generate factually correct responses that fail to maintain user‑requested perspectives, blending multiple perspectives and forcing users to filter through unwanted information‑‑‑a perspective control failure that factual accuracy metrics cannot detect. We introduce PerspectiveBench, a benchmark quantifying perspective alignment by constructing scenarios where multiple valid reasoning paths coexist, and propose the Perspective Alignment Score (PAS) to measure adherence to specified perspectives. Experiments reveal that state‑of‑the‑art models successfully include target content but fail to exclude non‑requested perspectives. Providing documents through RAG improves factual grounding but does not resolve perspective drift, as models prioritize comprehensive coverage over strict user constraints. Code and data are available at https://github.com/sunghoon014/PerspectiveBench.
Authors:Pavel Sokolov, Alexander Kuklev
Abstract:
We present a dependently typed pure type system that contains a circular axiom s : s for a certain sort s and is nonetheless strongly normalizing. Girard's paradox is avoided because our sorts are not closed under a Π‑former: function types are placed in sorts according to their rank. Turns out that this system has a unique set of characteristics: it has similar normalization behaviour as STLC (so complexity of normalization is TOWER‑complete) while at the same time allowing non‑uniform representation of arbitrary elementary recursive functions; furthermore, it has a nontrivial set‑theoretical model in ZF+GCH and allows to carry out nontrivial proof developments. Base normalization proof is surprisingly simple (modulo general metatheoretical properties of pure type systems) and has been fully formalized in vanilla Rocq (formerly Coq) without additional axioms.
Authors:Shenqi Zhou, Qiran Zou, Wenhao Zhao, Zhenyu Liao, Dianbo Liu
Abstract:
Autonomous agents are increasingly capable of developing machine learning (ML) models end to end. In medical practice, however, strong performance on the target metric does not fully characterize how good the models are in real‑world use cases. For example, medical ML models should also provide well‑calibrated risk estimates and remain robust across patient populations. Yet autonomous model development is typically guided by task objectives that emphasize simple predictive discrimination (e.g. AUROC). This raises a key question: are these objectives sufficient to guide agents toward models that meet the broader requirements of real‑world medical use? In this study, we introduce Auto‑MD‑bench, a benchmark of 15 real‑world medical tasks from 10 clinical datasets, spanning survival analysis, classification and sequential prediction, with external validation across cohorts, hospitals and time periods. We evaluate four agents: AIDE, AI Scientist v2, Codex, and Autoresearch. The strongest agents achieve competitive primary scores, yet their predicted risks are less calibrated than those of the simple baseline models, which could lead to severe medical accidents. Moreover, models with nearly identical primary scores can exhibit markedly different calibration, while performance varies substantially across population shifts. These findings show that primary task performance alone is insufficient to assess autonomous clinical modeling and multiple different metrics widely used in healthcare practice need to be taken into consideration.
Authors:Tao Zhang, Bin Liao
Abstract:
In heterogeneous GPU clusters, GPUs are costly and scarce, yet placing a job on a machine does not mean that the job becomes ready to run. A large amount of allocated capacity can remain occupied for long periods without producing useful computation, further aggravating the waste of compute resources. On the Alibaba GPU cluster trace we find that, even when GPU capacity is sufficient and placement is complete, 52.70% of execution records never became ready. We term this phenomenon ineffective occupancy. Building on this diagnosis, and keeping the placement policy fixed, we propose Readiness‑Aware Scheduling (RAS): it uses only nine attributes observable at admission time to predict readiness probability and reorders admission accordingly. Under a five‑fold split grouped by machine, the gradient‑boosted model reaches an ROC‑AUC of 0.8187. A feature‑group contribution analysis shows that the model's score is driven mainly by job profile, duration, and GPU model, with the request‑size group contributing least, followed by the scheduling and concurrency group. In GPU terms, RAS reduces the ineffective‑occupancy ratio (the share of cluster GPUs held under ineffective occupancy) from 43.64% to 27.15% and raises the effective coverage of large requests (eight GPUs or more) from 46.11% to 63.18%, without sacrificing capacity utilization. Relative to the upper bound obtained by ordering with ground‑truth labels, RAS captures 65.1% of the attainable improvement, well above the interpretable rules built only from the contribution‑analysis conclusions; the gain therefore comes mainly from predicting with all nine attributes jointly rather than from combining rules built on any single attribute group. These results indicate that readiness status is a measurable, predictable, and optimizable dimension of admission decisions in heterogeneous GPU clusters.
Authors:Parsa Razmara, Seyedarmin Azizi, Ardalan Aryashad
Abstract:
Agent Skills are instruction files that tell a coding agent how to do one job. Users install skills from many authors, so two skills for the same job can be active at the same time. We ask what a model does when both skill files are in its context: does it combine their compatible rules, or follow one skill and drop the other? We build SkillClash, 16 pairs of same‑job skills in which code checks every rule and a reference output shows the best result that can be reached. On seven models from five families, from 4B open models to Qwen3‑235B and GPT‑4.1, a skill's rules are followed 93 to 99% of the time when it is loaded alone and 61 to 62% when a same‑job skill is loaded next to it. Most outputs show takeover: every rule that only one skill asks for is followed, and every rule that only the other skill asks for is dropped. The worked examples inside the skills drive this. When each skill file instead ends with one example that follows both skills, takeover falls to 1 to 15% on the six models we tested. Telling the model which skill has priority does not bring the dropped skill back, and telling it to apply both skills brings the dropped skill back only on the two strong models and leaves the interacting rules broken. One merged skill file written by the strongest model we used, Claude Opus 5.5, lets all seven models follow 96 to 99% of the achievable rules. Most of its gain over a plain list of both skills' rules comes from its one worked example, and such an example helps only when it is correct. Code and data: https://github.com/prazmara/skillclash
Authors:Shyam Sundar Saravanan, Tie Luo, Sanjay Madria
Abstract:
Flow Matching (FM) is an efficient simulation‑free approach for training flow‑based generative models, making it a promising foundation for scalable time‑series anomaly detection (TSAD). However, standard FM learns a monolithic vector field in which nominal transport and local density‑changing behavior are entangled. This creates a central challenge for TSAD; local expansion and contraction carry crucial anomaly‑sensitive information, yet isolating this signal requires expensive Jacobian‑trace estimation over the entire field. We propose \emphHelmholtz Component Flow Matching (HCFM), an unsupervised TSAD framework that parameterizes the vector field prediction network with separate transport, compression, and residual heads. The transport head is constrained to be divergence‑free, so divergence estimation is restricted to the compression head, while a lightweight residual head absorbs the remaining mismatch. This structured parameterization improves scalability and detection performance by separating volume‑preserving nominal transport from density‑changing compression, yielding cleaner anomaly signals from distinct deviations of nominal behavior. Experiments on large‑scale univariate and multivariate TSAD benchmarks show that HCFM consistently improves detection accuracy while maintaining efficient inference compared with 16 baselines. The code is available at https://github.com/TrustAIoT/HCFM__time‑series‑anomaly/
Authors:Sinan Ekmekçibaşı, James D. Trotter, Johannes Langguth, Xing Cai, Didem Unat
Abstract:
We introduce a novel triple‑overlap implementation of the pipelined preconditioned conjugate gradient (Pipe‑PCG) and pipelined stabilized biconjugate gradient (Pipe‑BiCGStab) solvers, methods widely used in scientific applications. By leveraging a collective communication library, our approach extends conventional computation‑communication overlap to concurrently execute collective operations, irregular P2P communication, and compute kernels, enabling computation‑communication‑communication overlap across multiple GPUs. We evaluate the design on the MareNostrum 5 (H100) and Leonardo (A100) supercomputers, comparing against PETSc and the state‑of‑the‑art two‑stream NCCL implementation using SuiteSparse matrices and three massive instances scaled up to 512 GPUs. The three‑stream design outperforms the two‑stream baseline on every instance, with the margin widening from 2‑6% at 4 GPUs to 18‑47% at 64 GPUs. At 64 GPUs, the geometric‑mean speedup over the two‑stream baseline is 1.33x on MareNostrum 5 and 1.23x on Leonardo for Pipe‑PCG, and reaches 1.30x for Pipe‑BiCGStab. In large‑scale experiments, we observed a speedup up to 35% in Pipe‑PCG and up to 33% in Pipe‑BiCGStab compared to the two‑stream design. For PETSc, the margin is even wider, up to 3.27x speedup in both Pipe‑PCG and Pipe‑BiCGStab. Related code for the proposed method is available online: https://github.com/ParCoreLab/aCG
Authors:Hamin Chang, Fei Teng, Gyu-Sub Lee, Jae-Won Chang
Abstract:
Modern power grids increasingly rely on power electronic interfaces, requiring steady‑state power flow analysis to accommodate diverse operating modes and network structures. Conventional approaches often rely on system‑specific formulations and sequential approaches to handle such complexity. However, these approaches may limit applicability to diverse grid configurations and lead to convergence and computational issues. To address these limitations, this paper proposes a generalized and unified Newton‑Raphson (NR) power flow framework for AC/DC hybrid grids. For generalized operational modeling, conventional and droop operating modes of generators and interlinking converters are represented through a common set of coefficients. For generalized structural modeling, models of AC transformers and DC/DC converters accommodate different transformer winding configurations and converter operating modes. For a unified solution, augmented variables are introduced to incorporate droop deadbands, output limits, and converter station losses into a single NR formulation, eliminating the need for sequential outer loops. To validate the effectiveness of the proposed framework, its results are compared with those obtained from existing power flow methods and electromagnetic transient simulations across transmission networks, distribution networks, and microgrids. The implementation is made publicly available to facilitate its use in analyzing AC/DC hybrid grids.
Authors:Zhenyue Peng, Lin Yang, Jie Wang, Xiaoqi Ni, Hanzhu Chen, Xize Liang, Yanjun Wu, Rujing Wang, Kaiwen Zhou, Jianye Hao
Abstract:
Flexible switching between reasoning trajectories (i.e., thought switching) has significantly enhanced the reasoning capabilities of Large Reasoning Models (LRMs). However, existing models often switch excessively yet fail to sustain promising reasoning thoughts, a phenomenon termed "under‑thinking". While recent efforts suppress switching to mitigate this, such over‑correction may discard valuable trajectories. To address this challenge, we propose Steady Thought (ST), a novel thought‑level preference optimization framework. ST formalizes under‑thinking as a preference issue at switching points, forcing single‑path continuations from identified points to yield high‑quality reasoning. Then, ST performs thought‑level preference optimization by treating the newly generated response as preferred and the original one as dis‑preferred. Experiments across multiple models and datasets show that ST effectively reduces token consumption, achieving reductions of up to 63.8%, while maintaining strong reasoning accuracy. Further analysis suggests that ST helps models acquire a generalizable ability to reduce redundant switching across domains and languages. Code is available at https://github.com/MIRALab‑USTC/SteadyThought.
Authors:Sicheng He, Yi Pan, Qiuting Yu, Jiaming Huang, Qi Lu, Shuyan Wu, Hongyan Feng, Xuanyu Liu, Jun Zhang, Jie Zhang, Wenqi Zhang, Xuhong Zhang, Jianwei Yin
Abstract:
Flow‑matching vision‑language‑action (VLA) models generate an action chunk by iteratively denoising Gaussian noise, but typically execute only a short prefix before replanning with updated observations. Under this receding‑horizon paradigm, the unexecuted remainder‑‑‑the action tail‑‑‑is discarded, and the next chunk is regenerated from fresh noise. We show that this discarded tail retains transferable denoising progress: action‑tail initialization can match or outperform a native 10‑step Gaussian baseline using fewer denoising steps. However, the optimal denoising budget varies substantially across samples, indicating that tail reuse should be adapted to the updated environment. Based on this observation, we introduce VLA‑Rolling, a lightweight inference framework that reuses action tails through three stages: action‑tail inheritance, per‑action confidence estimation, and confidence‑guided refinement through renoising and denoising budget allocation, while the underlying VLA remains frozen. Across π_0.5, SmolVLA, X‑VLA, and MolmoAct2 on LIBERO, MetaWorld, LIBERO‑Plus, and RoboTwin 2.0, VLA‑Rolling reduces action‑chunk inference cost in all 16 model‑‑benchmark pairs, achieving up to a 1.48× speedup, while improving success rate in 14 pairs by up to 15.08 percentage points. These results show that action tails are not merely discarded predictions, but reusable intermediate states that enable more efficient and accurate VLA inference. These results demonstrate that the action tail can support the generation of better actions with fewer denoising steps. Our project is available at https://chenbao123ww.github.io/VLA‑Rolling/.
Authors:Jike Zhong, Ming Li, Yuxiang Lai, Xi Xiao, Sheng Zhang, Ritwick Chaudhry, Tianle Chen, Jingyu Xie, Zheda Mai, Shao-Yuan Lo, Konstantinos Psounis, Yuanyuan Lei
Abstract:
On‑policy distillation (OPD) trains agents on their own trajectories using supervision from a stronger teacher. We uncover a counterintuitive phenomenon in multi‑turn agentic tasks: a smaller, initially weaker student can substantially outperform its teacher during training, only to regress as distillation continues. We term this phenomenon distillation overshoot. Through decision‑ and task‑level analyses, we trace overshoot to two competing effects of OPD: repairing weak student decisions and eroding successful student choices that the teacher disfavors. Early in training, repair outpaces erosion, allowing the student to combine the teacher's corrections with its own strengths and surpass the teacher. As training continues, erosion overtakes repair, causing these gains to vanish. Motivated by this finding, we propose Success‑Preserving On‑Policy Distillation (SPOD), which selectively masks conflicting teacher supervision on successful student choices and introduces a soft probability floor to counteract their erosion, while retaining teacher guidance elsewhere. SPOD requires only binary episode success and model probabilities already available during OPD. Across two model families and three agentic benchmarks, SPOD consistently outperforms ordinary OPD and evaluated variants, improving final success by up to 15.2 percentage points and surpassing the teacher by up to 20.0 points. Moreover, SPOD's final checkpoints outperform even ordinary OPD's best intermediate checkpoints in all six settings, demonstrating that beyond‑teacher gains can be not only preserved but further extended. Code: https://github.com/Jike338/SPOD
Authors:Jiwon Jeong, Anjelo Jann Laroza, Chaewoon Yoon, Arjun Pillai, Pranav Kokati, Christian Hoang, Seunghyeok Hong
Abstract:
Large language models sometimes reply in an unexpected language, a failure known as language confusion. We ablate individual attention heads in 12 model checkpoints spanning seven model series, using FLORES‑200 continuations in five languages and Language Confusion Benchmark (LCB) chat replies. In ten models, zeroing one head causes substantially more departures from the expected language than zeroing any other head in its layer; its LCB pass‑rate drop persists, though reduced, under mean ablation. Seven heads move 8.0 to 51.5 percent of all FLORES continuations out of the prompt's language, versus at most 1.4 percent for other same‑layer heads; the other three mainly affect explicitly requested languages. These seven heads matter less in base models given plain text. In two models, replacing the values a head reads at the requested‑language name with another language's mean makes replies fail and shifts some into that language, unlike a random change of equal size. Replacing a head's output with another language's mean redirects 84 to 99 percent of scored non‑English monolingual LCB replies into that language in five models. Yet the requested language's mean improves crosslingual replies beyond every control head only in the two Gemma models, whose baselines are low. Code and results are available in the public repository at https://github.com/christian‑hoang‑04/anonymous‑release.
Authors:Stavros Damianakis, Konstantinos Psounis
Abstract:
Prompt injection is the dominant security risk for large‑language‑model (LLM) applications and agents, yet the field still lacks a coherent map of how these attacks are constructed. More than twenty public datasets describe prompt injection using incompatible vocabularies, often organized around delivery vector, attacker goal, or generation method. In this Systematization of Knowledge, we present PIAtlas, a taxonomy organized instead around how the injected instruction is constructed to subvert the underlying model. Its six classes, one per subversion mechanism, form the roots of a hierarchy of techniques and sub‑techniques that researchers can use to describe attacks consistently and design more systematic evaluations. We derive the taxonomy from how LLMs are trained, prompted, and influenced by human‑like framing, and compare it with historical attack data. To ground this comparison, we release PICorpus, to our knowledge the largest license‑clean collection of public prompt‑injection datasets, and map thousands of its records to PIAtlas under a human‑verified protocol. The mapping shows that attacks usually combine several techniques drawn from only a few mechanisms. It also reveals gaps that existing labels obscure: widely used agent benchmarks test only part of the attack space, and adaptive attackers favor mechanisms that curated benchmarks underrepresent. PIAtlas thus provides a common foundation for understanding this attack space, improving benchmark coverage, and guiding red teaming and defenses.
Authors:Josep Marques, Hyunyoung A. Cho, Youngeun Kim, Youjia Zhang, Aecheon Jung, Kyuhong Shim, Sungeun Hong
Abstract:
Recursive Transformers depend not only on how many parameters are shared, but also on how shared parameters are organized across depth. Existing static schemes adopt two extreme reuse topologies: layer‑wise repetition, where each layer is reused independently, and global cyclic reuse, where the entire stack is repeatedly applied as a single shared function. We argue that sharing granularity is an important factor in recursive parameter sharing. This study proposes Block Recursive Transformers (BRT), which partition layers into contiguous blocks and recursively reuse each block before moving to the next. This intermediate topology enables local refinement within blocks while preserving depth‑wise transitions, without adding routing modules, auxiliary states, or extra training objectives. Across decoder‑only models up to 1.7B parameters, with additional encoder‑decoder and encoder‑only studies, BRT improves validation negative log‑likelihood and downstream accuracy over static recursive sharing baselines. Ablations and representation analyses further show that intermediate block granularity better balances local refinement and depth‑wise differentiation. Project website: https://aim‑skku.github.io/BRT/
Authors:Yongchao Long, Chunrui Zheng, Yuxuan Zhou, Guangkun Nie, Tian-Chi Ma, Chao Pang, Shuohuan Wang, Yuchen Ding, Xianbin Wen, Dianhai Yu, Junyuan Shang, Yuxi Zhou, Shenda Hong
Abstract:
The throughput gains of speculative decoding depend on the trade‑off between acceptance length and execution latency. We ask when additional acceptance is worth the cost of obtaining it. We establish a comparison protocol spanning training, inference, and evaluation, remeasuring released checkpoints and retraining six drafter recipes on shared target‑generated data across ten benchmarks and two target models. Alignment changes the picture: gaps that appear large in published comparisons shrink substantially under shared training and inference conditions. This narrowing motivates two deeper questions: how is acceptance improved at different costs, and when do these gains improve throughput? We find that acceptance can be improved through nearly free objective and context changes or more expensive token‑to‑token conditioning. DCore illustrates the former route as a strong baseline with little additional measured drafting overhead. The payoff is conditional: task uncertainty and serving conditions affect acceptance returns, execution costs, and the preferred drafter and budget. These findings inform future drafter design and deployment choices.
Authors:Chongkai Gao, Wenlong Huang, Haonan Chen, Lin Shao, Yilun Du, Jiajun Wu, Ruohan Zhang, Li Fei-Fei
Abstract:
Video models provide rich priors for generalizable robotic manipulation, but plausible visual futures are not always executable. We introduce DreamCan, a hierarchical policy that grounds video models through sparse visual planning: a high‑level planner adapted from a pre‑trained video model and a low‑level visuomotor policy. Rather than predicting dense trajectories, the high‑level planner generates sparse visual subgoals that balance reachability and task utility. From offline, action‑free robot videos, we estimate these quantities and use dynamic programming to select subgoal sequences without manual annotations. The resulting subgoals are more reliably attained than uniformly spaced alternatives, and the full system outperforms language, dense‑video, and uniform‑subgoal planners, reaching high planning success rates on Language‑Table tasks and 91.2% success on LIBERO‑Long with a 0.1B‑parameter low‑level policy. Additional suboptimal videos and test‑time compute further improve success. The system also performs challenging real‑world bimanual tasks, including box folding and packing with unseen object combinations. More results are available at https://dreamcan.github.io/.
Authors:Yerin Jung, Jongheon Jeong
Abstract:
Recent advances in video generative models have enabled flexible conditional generation and editing pipelines, such as text‑to‑video, image‑to‑video, and reference‑guided synthesis. However, these workflows are inherently one‑directional, mapping conditions to videos without providing a mechanism to invert this process. In complex media production settings, the ability to recover controllable textual conditions from a given video is highly desirable, yet remains underexplored. While inverse prompting has been studied for image generation, directly extending these approaches to video is challenging due to the need to capture temporal dynamics beyond static appearance, as well as the substantial computational cost of per‑video optimization. In this work, we propose the first practical framework for efficient video inverse prompting. Specifically, we formulate inverse prompting as a meta‑learning problem and introduce Meta Video Inverse Prompting (MVIP), which learns a meta‑prompt optimized over temporal dimensions to enable fast and effective prompt inference. Our approach amortizes the inversion process, eliminating the need for costly iterative optimization at test time. Experimental results demonstrate that MVIP recovers higher‑quality inverse prompts with significantly improved reconstruction fidelity compared to image‑based inversion and iterative captioning baselines, while achieving substantial gains in efficiency. Code and video results are available at https://ueriniuno.github.io/mvip‑page/.
Authors:Changho Lee, Janghoon Han, Rajdeep Mukherjee, Yeda Song, Lening Nick Cui, Xinya Du, Honglak Lee
Abstract:
Reasoning post‑training on long trajectories has substantially improved the reasoning capabilities of large language models, but scaling such training to large datasets is increasingly expensive. We study target‑aware reasoning data selection, where a small reference set specifies a target capability and guides the selection of training trajectories for specialization. We find that existing target‑aware methods can exhibit limited target discrimination on long reasoning trajectories, while dominant population‑level structure in candidate gradients can obscure finer target‑dependent variation during gradient matching. To address these limitations, we introduce Deconfounded Gradient Matching (DeGMatch), which first constructs a target‑relevance gate and then reranks candidates using model‑specific gradients. DeGMatch centers gradients using the candidate‑population mean and removes leading principal directions before target matching, reducing the influence of dominant population‑level structure and improving target discrimination. Across mathematics, science, and code, DeGMatch outperforms strong heuristic, target‑aware, and reasoning‑specific data selection baselines, while remaining effective across selection budgets, model families, and scales. Our code is publicly available at https://github.com/CHLee0801/degmatch.
Authors:Kazem Faghih, Yize Cheng, Shoumik Saha, Mobina Pournemat, Armin Gerami, Soheil Feizi
Abstract:
Language model benchmarks assign each question a single outcome from one fixed wording, even though the same underlying request can be expressed in many equivalent ways. We argue that the quantity of interest should therefore not be correctness on one wording, but the accessibility of an inquiry, which we define as the probability that a model succeeds across meaning‑preserving formulations. The distribution of accessibility across inquiries reveals a structure that accuracy and existing best‑, worst‑, or average‑case prompt scores collapse: how broadly a model's capabilities extend across inquiries and how dependably those capabilities can be elicited. Two models can have the same average performance while one succeeds across more inquiries unreliably and the other succeeds across fewer inquiries but more consistently, leading to opposite preferences depending on whether the task allows multiple retries until one success or when repeated consistent success is required. Across six benchmarks and more than twenty models, this tradeoff is common: for about half of the model pairs with nearly equal mean accessibility, which model looks better depends on whether one success across five formulations is enough or all five must succeed, quantities we call Access@5 and Reliable@5, respectively. For some pairs the two measures diverge significantly, by up to 10 points in opposite directions. Accessibility also reveals that scaling, reasoning, and prompt optimization can redistribute capability across inquiries in ways aggregate accuracy obscures, sometimes broadening the set of inquiries a model can succeed on while making success on them less repeatable. Accessibility turns formulation sensitivity from a nuisance into a measurable property of capability, distinguishing what a model can sometimes succeed on from what users can actually depend on.
Authors:Thai-Khanh Nguyen, Nhi Y. Pham, Uyen N. B. Vo, Thieu N. Vo, Cuong Pham, Tan M. Nguyen
Abstract:
Next‑token prediction leaves hidden‑state trajectories largely unconstrained, while existing geometric regularizers primarily control the direction rather than the magnitude of trajectory deviations. We introduce \emphControlJEPA, a trajectory regularization framework based on a discrete‑time Lyapunov contraction condition from practical Input‑to‑State Stability theory. Its Lyapunov Tube Loss penalizes violations of this condition on a normalized, scale‑invariant measure of transversal deviation. When the contraction condition holds at every step, the hidden‑state trajectory remains within an explicit tube around the reference secant. In the partial‑ISS regime, we bound mean transversal energy, temporal variance, and average tube width using the Lyapunov loss and contraction parameters, with horizon‑independent variance bounds under uniformly bounded violations. ControlJEPA requires no architectural modifications. Across six benchmarks and six model families, it consistently improves accuracy and data efficiency over standard fine‑tuning and prior trajectory regularizers. The resulting smoother transversal profile also persists empirically during inference, when the regularizer is inactive. The code is publicly available at: \urlhttps://github.com/t‑khanusa/ControlJEPA
Authors:Hanjun Luo, Xuecheng Liu, Xiucheng Zhang, Zhuoning Xu, Yingbin Jin, Yankai Chen, Xue Liu, Hanan Salam
Abstract:
Language models behind coding agents are post‑trained with outcome rewards, and recent methods concentrate credit on the turns that decide the outcome. We find that defensive actions beyond what a task requires escape both. They begin at silent forks, the turns where every next action leads to the same outcome. Such actions leave the success rate unchanged, and frontier models take the disproportionate branch there in up to 91% of decisive actions under implicit budgets. The most direct fix, a checker in the trajectory reward, reaches these forks only through fully proportionate trajectories, leaving 74% of GRPO groups with identical rewards and behavior unchanged. This motivates a rethink of where learning signals belong, namely where behavior diverges, which a behavioral checker can locate and the outcome cannot. Our method, ForkDPO, contrasts a proportionate and a disproportionate action at the same prefix, at silent forks on both sides of the required behavior. On a test set of unseen repositories, it lowers a 4B model's disproportionate behavior from 70.3% to 58.9%, against 67.7% for DPO at randomly chosen turns, establishing behavioral divergence as a learning signal for how agents act beyond whether they finish. The code is publicly available at https://github.com/Astarojth/silent‑forks.
Authors:Quan Wang, Yiran Mao
Abstract:
CSS3 @font‑face rules with unicode‑range descriptors partition web fonts into downloadable slices. While Latin fonts need ~20 KB in WOFF2, Pan‑European (LGC) and East Asian (CJK) fonts contain 2,297‑30,759 codepoints (158 KB‑5.9 MB in WOFF2), where heuristic partitioning causes unused‑glyph downloads, high HTTP RTT, CSS bloat, or missing‑glyph fallbacks. We present MLFont, a machine learning and optimization framework that learns font slicing strategies from corpora via three stages: (1) a Zipfian High‑Frequency Tier (most) isolating top‑frequency codepoints; (2) a Latent Co‑Occurrence Topic Tier using Online Variational Bayes LDA (Online LDA) or a Deep Neural Network Topic Model (DNNTM) with Shannon Entropy Difference Loss and Sinkhorn‑Knopp normalization; and (3) a Memetic Range‑Compact Genetic Algorithm (GA) with Potts‑Viterbi dynamic programming and island bridging that minimizes topic discrepancy and CSS fragmentation. Across five Google Noto fonts (LGC, KR, JP, TC, SC), MLFont cuts 3G per‑glyph latency by 56.7%‑75.3% over single‑file delivery and 46.1%‑66.3% over equal‑bin baselines on CJK with 100% coverage, reduces CSS intervals by 17.9%‑41.7%, and achieves 1.56x‑6.09x cold‑cache speedups (65.4%‑84.9% CJK and 44.7% LGC bandwidth savings) across 20 websites in simulator and Headless Chrome benchmarks (45/45 exact slice‑set parity). Code: https://github.com/wq2012/mlfont Models: https://huggingface.co/collections/wq2012/mlfont
Authors:Sen Fang, Liwei Che, Dimitris N. Metaxas
Abstract:
We present Stable Signer V3, an interactive high‑resolution sign language avatar system built on 4D Deformable Primitive Splatting. It addresses three limitations of existing avatars: restricted facial expression, inaccurate identity‑specific body deformation, and loss of fine appearance details. Shared Face Primitive Splatting adds transferable superquadric primitives for richer expression control; Test‑Time Body Primitive Splatting learns an identity‑specific virtual muscle layer for more faithful motion; and Test‑Time Tunneling Super‑Resolution uses TunnelingNet to recover identity‑specific high‑resolution details. Both test‑time modules are optimized once per signer identity; inference then runs entirely from the stored identity parameters. Together, they reduce facial geometry error by 71.4%, body‑boundary error by 35.1% across ten identities, and high‑resolution RGB reconstruction error by 24.2% on held‑out evaluations, while retaining interactive streaming.
Authors:Yulin Sun, Shenghua Li, Yong Dou, Xinwang Liu, Cheng Yang, Kele Xu
Abstract:
Few‑shot audio classification requires audio‑‑language models to distinguish variable acoustic events from limited labeled data. PALM's main formulation adapts encoded class representations without changing pretrained encoders, but its class representations remain input‑independent and classification uses audio features without class‑conditioned refinement. We introduce DF‑PALM, a dynamic feature‑space prompting framework that combines audio‑conditioned class representations with asymmetric cross‑attention for audio refinement. Class representations are conditioned after text encoding, held fixed across fusion layers for each input, and reused for final scoring. Both pretrained encoders remain frozen, avoiding audio‑conditioned text‑encoder passes. With PENGI as the frozen backbone, the main 16‑shot evaluation covered 11 benchmarks and six baselines. DF‑PALM achieved 80.69% mean accuracy and 80.32% macro‑averaged F1, exceeding PALM by 4.11 and 4.30 percentage points, respectively. Paired benchmark‑level tests supported both gains after Holm correction. Across four shot settings, DF‑PALM achieved the highest mean accuracy and macro‑averaged F1 among the four evaluated prompt‑learning methods. The gains extended to LAION‑CLAP and OpenFLAM, with significant improvements over PALM in both metrics. At 50 candidate classes, operator‑counted inference floating‑point operations increased by 0.39% relative to PALM. These results support instance‑conditioned feature‑space adaptation as an effective approach to few‑shot audio classification. Our code is publicly available at GitHub https://github.com/sunyulin0421/DF‑PALM.
Authors:Cantao Su, Anna Wegmann, Esther Ploeger, Dong Nguyen
Abstract:
Diversity is a critical property of text corpora, with direct implications for training data curation, test set design, and model generalization. However, its measurement remains fragmented and poorly understood. Existing embedding‑based diversity measures vary widely in how they are computed and can yield substantially different diversity scores on the same data, yet no systematic comparison of their empirical behaviour exists. We make two contributions toward addressing this gap. First, we present a taxonomy of 21 embedding‑based diversity measures, organizing them into four families based on their underlying computations. Second, we introduce EmbDivBench, a benchmark that evaluates these measures along three conceptually distinct components (variety, balance and disparity) on both simulated and natural text data. Although Mean Pairwise Distance is the most commonly used measure, our experiments show that it is outperformed by several alternatives on the three diversity components, with Graph Entropy being the best choice overall. Lastly, we distil these findings into practical recommendations for diversity measure selection.
Authors:Riddhi Chatterjee, Ritish Shrirao, Kushagra Mehrotra, Viswanath Gopalakrishnan
Abstract:
Goal‑conditioned manipulation policies are steered by a goal image, a picture of the desired post‑manipulation scene synthesized from the current observation and a language instruction. The prevailing assumption is that better image‑editing quality yields better goals, so goal generators are built and compared using general image‑editing metrics. We show this assumption is false. Across two independent goal‑conditioned policies on CALVIN, the image editors that top aggregate editing‑quality leaderboards (Gemini 3 Pro Image and GPT Image 1.5) produce worse control targets than our proposed far smaller finetuned open models, and the gap widens sharply for the multi‑step (long‑horizon) tasks robots actually face. A per‑dimension analysis explains why. Downstream policy success is governed almost entirely by manipulation adherence (selecting the right object and the right final state; Spearman ρ\approx 0.9 with success), while photorealism and aesthetic fidelity, precisely where large proprietary editors excel, are uncorrelated with success (ρ\approx 0). This dissociation holds on both CALVIN and the real‑world‑aligned SIMPLER‑Bridge benchmark. We then present SGE‑Goal, a spatially‑grounded decomposed editing framework that predicts initial and final 2D edit‑region masks before goal‑image synthesis. SGE‑Goal turns open base models (FLUX.1 Kontext, FLUX.2 klein) into the strongest goal generators we evaluate on CALVIN, and second only to Gemini 3 Pro Image on SIMPLER‑Bridge. Base models fine‑tuned with SGE‑Goal also hold up under chaining far better than the best proprietary editor, sustaining roughly seven times the success rate at the longest (five‑step) chains. We release code, datasets, checkpoints, and a human‑aligned diagnostic evaluator. Our results argue that goal generators must be selected by downstream utility, not by editing‑quality leaderboards.
Authors:Adarsh Vatsa, William Eiers
Abstract:
A general observation in agentic workflows, especially bounded ones, is that the decision space of the LLM, when we separate a decision from generation, starts approaching saturation: the agent meets the same states again and again, and after enough conversations it stops meeting new ones. LLMs generally don't treat these decisions as an optimization problem and instead re‑derive them from their context/prompt on every turn. We show that treating decision making within reasonable bounds as an optimization problem, solving it once, and reusing the solved decisions can often make an agent faster, cheaper and more reliable. Our approach, which we call Discretize, Then Decide, starts from data such as the workflow's own logs or, when none exists, runs of the LLM agent itself. It reads each conversation into discrete states and estimates the probability of each outcome and the cost of each move, until no new states appear and the probabilities stop changing. It then solves the resulting Markov decision process. The solution runs as a controller around the model, sending moves whose content is fixed in the state, such as a tool call or a templated reply, and any state it hasn't seen is sent back to the LLM. We call an LLM run this way a Bellman language model (BLM), after Richard Bellman's equation for the best move in each state. On τ^2‑bench telecom, the controller with no model solves 79% of tasks. As a BLM, Qwen3.8‑27B, an open model, goes from 29% to 96% with 93% fewer model calls. For Claude Opus 5.5, which solves 96% of tasks on its own, running it as a BLM cuts model calls and cost per task by about 98% and lets it solve 2 more tasks. On InventoryBench, the solved policy with no model beats every published method. Where the hard part is writing content rather than choosing moves, such as resolving SymPy issues, few decisions can be solved in advance, and a BLM helps little.
Authors:Galen J. Wilkerson
Abstract:
Foraging recruitment is a contagion: an agent that finds a resource triggers a linear‑threshold cascade, the model built for rumour and adoption spread on social networks, now embedded in space. Because activation is degree‑normalized, the same rule that spreads the cascade also stifles it as neighbourhoods crowd, so the swarm aggregates on a resource and then releases to explore again. The cascade runs on the communication network, but acting on it moves agents in physical space, and that motion rewires the network: network and space are one coupled system, and this feedback, not any tuning, makes the coordination self‑limiting. Because recruitment chains along the graph rather than reaching near neighbours alone, whether a discovery stays local or spans the swarm is a question of percolation. The swarm holds itself in a cascade window that is locally near‑critical though globally subcritical, a self‑organized, self‑limiting drive‑release cycle, sandpile‑like but non‑conservative. Nothing is tuned, yet behaviours that foraging biology documents separately emerge together: recruitment to discoveries, marginal‑value patch‑leaving, and Levy search with a switch to local exploitation. These follow from one least‑committal principle, the resource field and the search both as maximum entropy, with Levy search matched to the field's sparsity and recruitment to its clustering, so active‑matter order and optimal foraging appear as one system seen in two domains. The gains are real but conditional: up to 7.6x the harvest of solitary search when resources are sparse and clustered, shrinking to nothing when search is already easy. That sparse, clustered field is also the generic one, because resources almost inevitably aggregate, so one untuned rule should recur widely.
Authors:Kien T. Pham, Yuhao Liu, Chong Hou Choi, Zhenqi He, Yanghao Wang, Chenyang Lei, Rui Liu, Qifeng Chen, Long Chen
Abstract:
Unified audio‑video creation requires a single model capable of text‑conditioned joint synthesis, bidirectional cross‑modal generation, and instruction‑guided editing across both modalities. Existing methods rely heavily on paired corpora, which are scarce for joint generation and even rarer for editing. Consequently, they are spent learning single‑branch generative and editing capabilities that abundant unimodal data could readily teach, rather than on the cross‑modal coupling. Furthermore, prevailing systems apply a single shared noise schedule to both modalities, neglecting two critical limitations. First, due to the significant discrepancy in information density, video requires prolonged high noise to organize spatial and motion structure, whereas the sparser audio signal can settle earlier during the synthesis process, the dynamics a uniform schedule might compromise. Second, lockstep denoising leaves asymmetric workflows, such as conditioning on one modality to synthesize or edit another, untrained. We present UniAVC, which addresses both obstacles. It first trains each branch on unimodal dataset to full generative and editing competence, and subsequently dedicates paired data to exploit cross‑modal correspondence. Modality‑decoupled schedulers meanwhile warp the noise schedule independently from a shared timestep, while subtasks with one clean modality directly supervise asymmetric configurations. These settings respectively enable intent‑aware editing, which preserves one modality intact while enlisting it to constrain the other, and post‑inference self‑refinement, which regenerates only an unsatisfactory branch against an accepted one. UniAVC thus spans multiple creation tasks from a single set of weights without task‑specific adapters.
Authors:Hamidreza Dastmalchi, Ali Cheraghian, Aijun An, Abdelwahed Khamis, Lars Petersson
Abstract:
Reasoning Video Object Segmentation (RVOS) aims to localize and segment objects referred to by natural‑language expressions throughout a video. Although large vision‑language models (LVLMs) implicitly encode localization cues in their attention maps, these attentions are often noisy, diffuse, and scattered across layers, heads, and modalities, limiting their direct use for segmentation. Existing methods typically rely on handcrafted attention‑selection heuristics or computationally expensive attention refinement. We propose ATLAS, an attention adaptation framework that learns to transform raw LVLM attentions into object‑centric localization cues. ATLAS treats attentions from all layers, heads, and modalities as structured features, combining high‑resolution frame‑level attentions that capture spatial information with low‑resolution video‑level attentions that encode temporal context. A lightweight U‑Net equipped with temporal attention and FiLM‑based modulation fuses these complementary representations to generate coarse segmentation heatmaps, which are propagated across the video using SAM2. Extensive experiments on challenging referring and reasoning video segmentation benchmarks show that ATLAS outperforms existing frozen‑backbone methods. Code and models are available at \urlhttps://hamidreza‑136.github.io/atlas/.
Authors:Sjoerd van Straten, Alessandro Padella, Marwan Hassani
Abstract:
Predictive Process Monitoring (PPM) forecasts future events of ongoing business processes based on event logs. However, deep learning PPM approaches are often limited by labeled data scarcity and the low variability of real‑world logs. To address this, we introduce SiamSA‑PPM, a self‑supervised framework combining Siamese learning with statistically grounded data augmentation. It employs novel transformation methods that leverage control‑flow semantics and frequent behavioral patterns to generate realistic, semantically valid trace variants, learning generalizable prefix representations without labeled supervision. Extensive experiments on real‑world event logs with a wide range of variability, incorporating exhaustive hyperparameter tuning, demonstrate the value of this process‑aware augmentation. Compared to state‑of‑the‑art baselines, SiamSA‑PPM achieves competitive or superior predictive performance in both next activity and final outcome prediction tasks. An open source implementation of our method is publicly available under https://github.com/SvStraten/SiamSA‑PPM‑Extension..
Authors:Sumit Anthony Gomes, Yasin Rahman, Taeeba Tasnia, Faiza Nooren Suhita
Abstract:
This paper introduces BanglaBBQ, a question‑answering bias benchmark designed to capture stereotypes grounded in the Bangladeshi context, in both Bangla and English. Following KoBBQ and PakBBQ, BanglaBBQ adapts the original Bias Benchmark for Question Answering (BBQ) by classifying its templates into three groups: directly translated (DT), modified to align target groups or contexts with Bangladesh (TM), and removed when deemed unsuitable for the Bangladeshi context (SR). This process retains 140 templates across six categories, comprising 76 DT and 64 TM templates. Five newly authored (NA) categories, based on stereotypes common in Bangladeshi society, are then added: marital status, age and political perception, regional origin, educational background, and mental‑health status. Native Bangladeshi annotators judged all 466 newly authored templates, and 372 were retained because a strict majority of English annotators identified the intended target and rated the attitude as common enough. Each newly authored template is linked to a documented source and a verbatim quote from it. Each evaluated template also has a counter‑stereotypical version of its disambiguated context. The dataset was evaluated using two open‑weight models (Qwen3‑4B and Qwen3‑8B) and two closed models (GPT‑4.1‑mini and Gemini 2.5 Flash). All four lean towards the stereotyped person when the context is ambiguous, in both languages, and the effect of language differs by model, so English results do not predict behaviour in Bangla. On the adapted BBQ templates, both Qwen3 models are also less accurate when the context contradicts a stereotype than when it confirms it. On the newly authored templates, the closed models follow counter‑stereotypical evidence almost perfectly, and all four models are more biased on the templates that annotators validated than on those they rejected.
Authors:Chris Dzoba
Abstract:
A single‑track Gray code of length n is a cyclic Gray code of n‑bit words in which every column is a cyclic shift of the first. For prime n the longest possible period is 2^n‑2. Whether it is attained for every prime has been open since Hiltgen, Paterson and Brandestini introduced these codes in 1996, and optimal codes were known for primes n\le 19. We show that a conjecture of Degni and Drisko (2007) on the reflected Gray code order of necklaces implies optimal single‑track Gray codes for every prime. Their list of necklaces, conjecturally a Gray path for prime n, closes into a Hamilton cycle of the necklace graph after (n‑3)/2 explicit Pósa rotations, and such a cycle lifts to an optimal code. This implication is formalized in Lean 4. Together with computation it gives optimal single‑track Gray codes for every odd prime n\le 47: for n\le 37 explicit codes checked word by word by a program independent of the generator, and for n=41, 43 and 47 codes obtained from the implication after the Degni‑Drisko property was verified at these lengths by two independently written programs.
Authors:Dengke Zhang, Fagui Liu, Quan Tang, C. L. Philip Chen
Abstract:
Open‑vocabulary semantic segmentation aims to classify pixels without being restricted to a fixed set of categories. Although Contrastive Language‑Image Pre‑training (CLIP) demonstrates remarkable potential for open‑vocabulary recognition, its segmentation capability is impaired by noisy inter‑class patch interactions. To this end, this work proposes a unified training‑free object‑centric framework, named CorrCLIPv2, from three complementary aspects of patch correlation, feature representation, and output regularization. Specifically, for patch correlation, we leverage class‑agnostic mask generators to explicitly disentangle inter‑class interactions. Moreover, we utilize spatially‑coherent features from self‑supervised models to compute attention scores to mitigate mask inaccuracies. Regarding feature representation, we devise a refinement mechanism to simultaneously strengthen spatial details and semantic representation. Within the output space, we unify predictions via voting in each mask to enhance spatial consistency. To mitigate contextual bias and enhance prediction accuracy of objects, we further extend the framework by incorporating a cross‑image object retrieval paradigm, which incurs minimal computational overhead. We build a memory bank of CorrCLIP‑enhanced features from unlabeled images and implement a regime of multi‑CLIP ensemble. By aggregating semantically consistent object features across diverse contexts retrieved from the memory bank, CorrCLIPv2 mitigates context‑induced interference and obtains more robust predictions. Extensive experiments demonstrate that our approach establishes a new state‑of‑the‑art on 10 standard benchmarks. Remarkably, it even surpasses fully supervised open‑vocabulary methods across 22 multi‑domain datasets. Code is available at: https://github.com/zdk258/CorrCLIP.
Authors:Hangwei Zhang, Nathan Rouillé, Chuanxia Zheng
Abstract:
Flow‑matching models generate samples through iterative velocity predictions, but typically do not revisit these predictions using the intermediate representations that produced them. We introduce CorrFlow, a self‑correcting flow‑matching framework that reuses this information through a lightweight refiner. Under rectified‑flow interpolation, the error of the one‑step clean prediction equals the realized velocity residual scaled by the noise level. Motivated by this identity, the refiner learns to predict residual corrections from the base model's clean prediction, conditioning, noise level and hidden states from the same forward pass. The correction is added to the base velocity before each sampling update, so subsequent predictions are made from the corrected state. This requires no additional base‑model evaluations and only one lightweight refiner evaluation per step. Our theoretical analyses establish conditions under which feature reuse outperforms raw‑input affine predictors and refiner training achieves greater residual‑risk reduction than continued base‑model training under a matched training compute budget. Experiments on class‑conditional image generation, text‑to‑video generation and image‑to‑video generation show consistent improvements over the corresponding base models, including gains in visual quality and physical consistency. Project Page: https://fogradio.github.io/corrflow‑project
Authors:Mich-Seth Owusu, Stephen E. Moore, Akwasi Asare, Lawrence Adu Gyamfi, Paul Azunre, Joel Budu, Jonathan Asiamah
Abstract:
Adapting a pretrained machine translation (MT) model to new terminology usually requires new parallel data and another round of fine‑tuning, which is costly for low‑resource languages. We ask whether controllability can be added to MT, and the fine‑tuning burden reduced, without degrading the pretrained model. GLOSS‑MT looks up the source's noun phrases in an offline English‑Twi dictionary, replaces every phrase found by its Twi translation in place, and fine‑tunes a translation model once to translate these mixed inputs. We apply the same data and pipeline to NLLB‑200 (600M, 1.3B, 3.3B), OPUS‑MT en‑tw and TranslateGemma 4B. On a document‑level English‑to‑Asante Twi benchmark (84 documents, four domains), every fine‑tune scores above its pretrained base in chrF2 (+2.85 to +5.48), and the three NLLB fine‑tunes are the best open systems (50.12‑50.80). A direct controllability test on human‑translated Bible passages shows that the three fine‑tunes tested write 79‑97% of the dictionary terms, and that replacing every dictionary entry by a different term changes the output accordingly (70‑97% of the new terms appear) with no retraining. We release the models, data and code.
Authors:Meng Shu, Wei He, Hang Zhou, Runke Liu, Kai Han, Yunhe Wang
Abstract:
Agent Skills express open‑ended task knowledge in natural language, but composing them requires persistent execution order and upstream‑output handoffs. We propose selective formalization at Skill boundaries: coordination relations with stable references and machine‑checkable constraints become executable, while substantive task judgments remain within Skills. We introduce MetaSkill, a hybrid representation that preserves complete Skill packages and local instructions while making invocation dependencies and predecessor‑text bindings executable through a directed acyclic graph (DAG) runtime. This interface supports explicit coordination without prescribing how each Skill performs its local task. Across 20 multi‑Skill artifact tasks and two LLMs, we compare natural‑language procedures, prompt‑visible graphs, and MetaSkill derived from a common frozen plan per task, using the same designated Skill packages. Additional comparisons evaluate AgentSkillOS with the same available Skill packages as MetaSkill, as well as agent systems using their configured tools. Blinded artifact judgments yield favorable preference point estimates across all reported comparisons, with 95% task‑bootstrap intervals excluding parity against Prompt Graph under DeepSeek‑V4‑Flash and OpenClaw under GLM‑5.2. In audited GLM‑5.2 execution traces, all planned nodes complete and all declared precedence constraints are satisfied. Together, these findings support MetaSkill as a practical interface for composing open‑ended Skills, combining local linguistic flexibility with executable dependencies, explicit predecessor‑text bindings, and auditable coordination. Code is available at: https://github.com/TokenRhythm/opensquilla/tree/paper/metaskill‑lp4fm‑2026
Authors:Qisheng Zhou, Zhen Xiong, Qiaoyu Tan
Abstract:
Diffusion language models (dLLMs) are emerging as an alternative to autoregressive (AR) language models, yet dLLM agents largely inherit an interaction protocol designed for AR generation: once intermediate state is written, later observations can only be incorporated by appending more text. This is a poor match to a defining capability of dLLMs ‑‑ reconstructing earlier text conditioned on context that appears both before and after it. We ask whether this capability can support a different agent‑state paradigm in which intermediate planning state is directly revisable as evidence arrives. We introduce DiffuPlan, a diffusion search agent that maintains a single editable planning state and reconstructs only the affected steps when later evidence resolves an entity or changes the search route. To support variable‑length revisions, we introduce Expansion‑Partition Marginalization (EPM), which marginalizes over equivalent expansion paths. We further adapt diffusion trajectory RL to multi‑turn search. Across four multi‑hop QA benchmarks, DiffuPlan achieves 50.24 average F1, outperforming strong AR search agents. More importantly, under identical data and training, editable planning state improves the diffusion agent by 2.44 F1 versus only 0.47 for its AR source model. These results show that dLLMs need not imitate append‑only AR agents: revising committed state is a distinct agentic advantage.
Authors:Ruodai Cui, Shuaizheng Liu, Rongyuan Wu, Lei Zhang
Abstract:
Despite the significant progress in image enhancement under adverse illuminations, such as low‑light image enhancement, exposure correction, and backlit image enhancement, existing models show limited generalizability to different scenarios. A major bottleneck lies in the lack of large‑scale and diverse training data, as these tasks typically rely on manually captured, precisely aligned image pairs that are expensive to scale up. Although synthetic data have been explored as an alternative, existing synthesis methods are limited in modeling complex adverse illumination conditions and camera imaging pipelines. To address this issue, we propose a novel framework to synthesize realistic training pairs for generalizable image enhancement in the wild. Specifically, we first derive multi‑exposure sequences (MES) from high‑bit RAW images by rendering them under different exposure settings through an emulated ISP pipeline. Using these RAW‑derived MES as supervision, we train a diffusion‑based RGB‑to‑MES generator to synthesize plausible exposure sequences from a single 8‑bit RGB image, extending the synthesis pipeline from limited RAW collections to large‑scale in‑the‑wild RGB data. For each source image, we randomly sample one synthesized exposure variant as the degraded input and use the original high‑quality RGB image as the target. With the proposed MES Synthesis (MESS) approach, we construct a dataset of 500K training pairs and train a lightweight network, which demonstrates significantly better generalization performance than existing models across various image enhancement tasks. Code and data are available at https://github.com/PolyU‑VCLab/MESS.
Authors:Qijian Tian, Jiayu Ying, Ke Fan, Lizhuang Ma, Xin Tan
Abstract:
Vision‑language models (VLMs) have achieved remarkable progress in 2D visual understanding tasks, yet the passive view‑centric perception paradigm limits their capacity for 3D spatial reasoning. Such reasoning often requires actively acquiring spatial evidence beyond the currently visible views, such as camera poses and cross‑view spatial relationships, to infer 3D relationships that are not explicitly represented in 2D images. However, existing methods either lack active exploration or merely provide additional views during inference, without explicitly acquiring such spatial evidence beyond 2D images. In this paper, we propose View‑to‑World (V2W), an active exploration framework that enables VLMs to actively explore 3D worlds and lifts VLMs from passive view‑centric perception to active world‑centric spatial reasoning. Given a spatial reasoning task with multi‑view images, V2W first reconstructs an explicit 3D world with geometry, camera poses, and language‑grounded semantics, and exposes it to VLMs through visual and linguistic interfaces. Through multi‑turn interaction with these interfaces, VLMs iteratively acquire spatial evidence from the constructed 3D world, enabling active world‑centric spatial reasoning. V2W can be integrated with existing VLMs in a training‑free manner and can be further enhanced by optimizing the exploration policy via reinforcement learning. Experiments on spatial mental modeling benchmarks demonstrate that V2W substantially improves VLM baselines in the training‑free setting and achieves state‑of‑the‑art performance with the learned exploration policy, highlighting the importance of active exploration for spatial intelligence. Project page: \urlhttps://fangzhou2000.github.io/projects/v2w.
Authors:Siyu Zhang, Yuquan Lu, Yoshua Y. Li, Yifan Ding, Kun Zeng, Yu Huo, Yifu Guo, Rongxiang Weng, Xunliang Cai, Xiaoying Tang
Abstract:
Reinforcement learning with verifiable rewards (RLVR) substantially improves mathematical reasoning, but what it adds to the base model remains unclear. We investigate this through a sequence of generation‑time interventions on Qwen3‑4B‑Base, measuring how much of its RL‑trained counterpart's gain each intervention recovers on 120 AIME problems. Power sampling with the same 8,192‑token output limit brings no gain beyond greedy decoding. Yet borrowing the RL model's token only where the two models' greedy choices differ, roughly 4% of generated positions, recovers 90.5% of the pass@1 gain, with similar recovery across three model pairs on MATH‑500. We find that even a single opening token has a substantial effect: forcing the base model to begin with "Alright" recovers 35% of the RL gain. The opening alone is insufficient, however. Later replacements after the same opening add another 5.2 percentage points to pass@1, and the fixed rules we test for inserting reflective words such as "Wait" do not reproduce this benefit. Sparse replacement also retains much of the base model's diversity while staying close to the RL model at pass@1. At k=256, it solves 67.5% of the problems, significantly more than the RL model (61.4%) and a mixture of whole responses matched in pass@1 (61.45%). The comparison is at equal attempt counts rather than equal compute, since rl‑replace consults both models at every decoding step. These results show that the opening can recover a meaningful part of the RL gain, but a larger contribution comes from later choices, which can be borrowed while retaining much of the base model's diversity. Code is available at https://github.com/CyrusZhang04/WhatRLBrings.
Authors:Omar El Bachyr, Saad Ezzini, Tegawendé F. Bissyandé, Jacques Klein
Abstract:
Document preprocessing choices can substantially affect retrieval in Retrieval‑Augmented Generation (RAG) pipelines, yet they are often selected heuristically and are difficult to compare under consistent conditions. We present RAGit, an open‑source experimentation workbench for the systematic and reproducible evaluation of document preprocessing choices through downstream retrieval. RAGit represents parsing, chunking, indexing, retrieval, and evaluation as configurable experimental stages, enabling controlled comparisons while the remaining downstream settings are kept fixed. It preserves provenance from retrieved chunks to source pages and maps heterogeneous retrieval units to a common page‑level ranking for evaluation. The current implementation supports multiple parsers and chunking strategies, dense and late‑interaction retrieval, reusable intermediate artifacts, and multi‑configuration benchmarking. We demonstrate RAGit through controlled experiments comparing alternative preprocessing configurations under a fixed retrieval setup, illustrating how preprocessing can be treated as an explicit experimental variable. RAGit and its documentation are publicly available on GitHub (https://github.com/OmarElbachyr/ragit). A demonstration video is available at https://youtu.be/tsst‑Bhfz4I.
Authors:Naman Vats
Abstract:
Agent scaffolds increasingly delegate small decisions (whether to stop, verify, or change approach) to cheaper models. We ask whether a typed decision model, Jev 1.13, consulted only at deterministic decision points, makes a Claude Sonnet 5.5 terminal agent cheaper without making it worse. In a pre‑specified study on Terminal‑Bench 2.1, we hold one agent loop fixed and vary only the controller: none, deterministic rules, a small LLM (Claude Haiku 5.5) answering the same questions, Jev, and a placebo that samples Jev's development‑phase choice frequencies at the same moments. On 21 held‑out tasks (315 trials), Jev raised cost per attempt by 19% (geometric‑mean ratio 1.19, 95% CI 1.02‑1.36) at an identical pass rate (73.0% vs. 73.0%). Jev was cheaper than the small‑LLM controller (ratio 0.81) but 22% more expensive than the placebo, and no pass‑rate difference between arms was detectable. Offline, Jev ranked finish requests better than the alternatives (AUROC 0.78 vs. 0.62 for rules and 0.60 for the small LLM) but was poorly calibrated: its mean probability that a run was complete was 0.50 when 87% of those runs passed, so it asked the agent to verify work that was already correct. No controller ever chose to abandon a run. Informative decision signals did not become useful interventions; we discuss what that would require.
Authors:Yuhua Zheng, Xueyi Chen, Xinze Zou, Xin Li, Haoji Hu, Huan Wang
Abstract:
Recent works show that LLMs can generate kernels with strong performance on server GPUs. Edge deployment presents distinct challenges because the hardware is heterogeneous, the software stacks are fragmented, and the resource budgets are tight. LLM kernel generation across diverse edge backends has yet to be studied systematically. We introduce EdgeKernelBench, which combines a unified IR deployment framework with a hierarchical benchmark for edge workloads. The unified IR captures reusable computation, scheduling, and memory decisions and connects to each target backend through a compiler interface. The benchmark spans progressively larger program scopes, from individual operators to compact model graphs, across representative edge application domains. We instantiate the design with TVM as a reference compiler and validate it across heterogeneous edge backends. With a single generation attempt, the best model achieves 64.5% Correct@1 but only 25.5% Fast@1.0. This gap shows that functional correctness does not consistently translate into performance gains. We therefore introduce the Layered Edge Kernel Agent (LEKA), which organizes optimization through a layered IR. On the agent evaluation subset, LEKA achieves 43.5% Fast@1.0, compared with 13.0% for direct generation and 26.1% for Claude Code. LEKA and Claude Code use the same base model and optimization budget. Together, EdgeKernelBench and LEKA support unified evaluation and layered optimization of kernels generated by LLMs across edge backends.
Authors:Megh Shukla, Edo Collins, Prashanth Chandran, Karel Petránek, Sergio Orts-Escolano, Mathieu Salzmann, Alexandre Alahi
Abstract:
We study the problem of mapping parameter‑space variance of a 3D Morphable Model (3DMM) to per‑vertex variance. Traditionally, per‑vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter‑space variance to per‑vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed‑form Local Uncertainty Estimation (CLUE), which transforms the parameter‑space distribution of 3DMMs to per‑vertex through a closed‑form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in‑the‑wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty‑to‑error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps. We make our code available at https://github.com/vita‑epfl/CLUE
Authors:Yunhui Liu, Xudong Jin, Qizhuo Xie, Chunhui Zhao, Xiao Luo, Tao Zheng, Bin Chong, Tieke He
Abstract:
While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text‑attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse‑to‑fine granularity inherent in real‑world data. To bridge this gap, we propose SHiFT, which synergizes the reasoning power of LLMs with the structural representation capability of GNNs within an Expectation‑Maximization (EM) paradigm. In the E‑step, we harness LLMs as expert taxonomists to induce an explicit semantic tree, introducing a lightweight Mapping & Diagnosis strategy that reduces inference costs by surgically updating the hierarchy only when anomalies are detected. In the M‑step, we align the GNN encoder with this induced hierarchy from complementary perspectives, including clustering partition, topological skeleton, and semantic concepts. Theoretical analysis interprets SHiFT as a generalized Variational EM algorithm. Extensive experiments demonstrate that SHiFT not only achieves state‑of‑the‑art performance but also uncovers high‑quality, human‑readable taxonomies. Our code can be found at https://github.com/Cloudy1225/SHiFT.
Authors:Matteo Spanio, Antonio Rodà
Abstract:
Native real‑time audio engines meet audio deadlines but are hard to connect to machine‑learning workflows, while tensor frameworks integrate with machine learning but process audio as offline batches without timing guarantees; runtime layers built on them inherit the host's scheduling and deployment footprint. This paper presents Fluxion, a Rust audio DSP runtime for cascaded second‑order sections, FIR filters, and delay‑line effects. Fluxion implements the parts that are specific to DSP: closed‑form filter design, forward kernels, analytic adjoints for the recursion, and stability checks. It relies on existing libraries for the autodiff graph, GPU code generation (tested here on CUDA), and array interchange. One graph algebra is lowered to a differentiable training engine and to an allocation‑free real‑time engine; between them, a trained graph is checked for stability and frozen into a versioned artifact that an edge device checks again before playback. We compare Fluxion with six framework‑hosted and native baselines under one float32 protocol. Among the systems that compute the exact recursion, Fluxion has the highest GPU throughput when data stays on the device and the highest throughput on most CPU workloads, while its CPU training step is slower than those of torchaudio and GRAFX. Its test suite checks the audio thread for allocations with an instrumented allocator, and we report measured latency tails. Fluxion is open source at \urlhttps://github.com/matteospanio/fluxion.
Authors:Qibin Gu, Yunfeng Yan
Abstract:
Visual accuracy, geometric validity, and agreement with applied forces are distinct requirements for visual dynamics. We introduce a diagnostic framework that connects reaction‑free SI measurements to controlled interventions on coordinate readout, training integration, and actuator response. Reference replay calibrates the measurement path, while continuous scoring exposes progress against fixed engineering targets. With three seeds per setting, we evaluate KeyCLD and CON across 343 planar assemblies at 64/256 pixels and V‑JEPA + AC and LingBot‑VA on 100 shared spatial test assemblies. CON reduces image MSE by 37.6%/37.2% relative to persistence, yet neither planar model attains valid native geometry. At 256 pixels, calibrated RGB replay places forecast errors far above the reference measurement floor. Exact‑state KeyCLD has identical geometric pass rates under Euler and RK4 training, yet its score rises from 52.65 to 64.30 as force and energy errors decrease. In both spatial pathways, CAD/IK reconstruction restores geometry while mean actuator residuals exceed five times the engineering targets. These diagnoses distinguish coordinate correction, numerical improvement, and persistent force‑response errors, guiding targeted improvements to visual dynamics.
Authors:Lukas Amiet, Gabriel Sánchez-Blanco Repgen, Mayank Mittal, Victor Klemm, Andrei Cramariuc, Marco Hutter, Cesar Cadena
Abstract:
Pedipulating a freestanding articulated object requires controlling its pose and internal configuration while maintaining balance. We demonstrate this capability by autonomously solving a giant Rubik's Cube with a quadruped using only its feet. The task demands repeated precise interactions, since unintended layer motion can invalidate subsequent operations. We develop an onboard RGB‑D perception pipeline that estimates the cube's metric pose and its full color state from partial views acquired through locomotion and flipping. A planner converts the computed solve into robot‑executable skill sequences. Learned whole‑body skills reorient the cube while preserving its layer configuration and turn its layers with one foot pinning the cube as the other pushes. We train these skills using reinforcement learning with privileged observations and distill them into policies that map onboard depth and proprioception to joint targets. The deployed policies tolerate variations in relative robot‑cube placement and recover from external disturbances. The system completes three autonomous hardware solves, each about 30 steps over 8‑9 minutes. These results show that combining onboard perception with coordinated pedipulation enables a quadruped to complete extended multi‑step physical tasks. Additional material can be found at: https://leggedrobotics.github.io/rubik‑skills/
Authors:Gengyu Liao, Yechun Yong, Zheng Chen, Jianze Li, Ziqing Zhang, Linghe Kong, Yulun Zhang
Abstract:
Open‑domain video super‑resolution (VSR) requires temporally coherent details across widely varying scenes and subjects. However, existing training datasets are often semantically imbalanced and insufficiently diverse. We address this data bottleneck with a structured curation pipeline for open‑domain VSR, SCOPE. In this pipeline, we combine task‑oriented video selection with hierarchical semantic rebalancing. Applying this pipeline to a large‑scale collection of data sources, we construct the SCOPE‑VSR dataset for training and SCOPE‑test for testing. SCOPE‑VSR has approximately 27k videos and provides balanced and diverse coverage of various real‑world scenarios and subjects. We further develop GraCo‑VSR, a novel framework for one‑step video restoration. GraCo‑VSR adopts a reliability‑aware design with explicit supervision to reduce degradation‑induced distortions in motion and texture guidance, allowing the model to benefit more effectively from the diverse training content in SCOPE‑VSR. Evaluations show that our model achieves strong reconstruction fidelity and perceptual quality. Our code is available at: https://github.com/gyliao2768/SCOPE‑VSR.
Authors:Yanlan Kang, Ruiqi Liu, Xinpei Gu, Tzu-Yi Lei, Shuai Xu, Xukun Zhang, William Cheng-Chung Chu
Abstract:
Traditional Chinese medicine (TCM) research connects classical texts, medicinal entities, molecular assays, computational predictions, and clinical studies. An autonomous research agent must preserve the inferential boundaries between these evidence types while coordinating analysis across heterogeneous resources. We introduce TCMScience, a scientific knowledge‑discovery agent that separates language‑model proposals from execution and release authority. Its architecture combines a trusted kernel, a scientific compiler, and claim‑support contracts: plans are compiled into typed operations, execution is bounded by explicit permissions, and candidate findings are checked against source provenance, study design, and claim scope before release. The full capability registry contains 117 public‑source connectors with 444 typed operations across 29 domains and 150 native tools across 12 domains; the TCM data hub catalogs 136 databases, 76 dataset specifications, and 84,294 formula‑source records. These resources support model‑independent workflows through browser and local execution interfaces. In a network‑pharmacology analysis of Gegen Qinlian Tang, replacing the whole‑proteome background with the assayed‑protein background changed the number of significant pathways from 66 to zero; incorporating inactive screening records identified nine pathways, including drug‑metabolism signals retained as mechanistic hypotheses. A governance ablation over 121 constructed erroneous outputs spanning 26 classes released 121 errors without governance, 57 with the domain layer, seven with the kernel, and zero with both layers. All 10 compliant controls were retained. These results establish complementary control of known failure modes and show how explicit evidence contracts change the conclusions an agent is permitted to report.
Authors:Abdullah Al Shafi, Kazi Saeed Alam, Sumaiya Rahim Suma, Sk. Imran Hossain
Abstract:
Motion‑deblurring models lose substantial accuracy when transferred from professional cameras to smartphone‑derived imagery. This paper studies whether a severity descriptor, partly derived from the paired sharp target, helps adaptation on the iPhoneBlur consumer‑device benchmark. Under a shared update budget, input concatenation of a severity map is compared with global feature modulation by capture metadata, spatial feature modulation by the severity map, and a tier‑contrastive output objective, together with an unconditioned spectral‑loss control, all fine‑tuned from the same domain‑adapted latent‑diffusion baseline. The primary experiment is an oracle diagnostic: target‑derived information is supplied at inference. For the evaluated model pair, input concatenation improves overall peak signal‑to‑noise ratio from 31.23 to 31.30 dB, whereas feature modulation, the contrastive objective, and the spectral‑loss control do not exceed the baseline. The gain is positive on 87.2% of test images and on 14 of 15 held‑out source videos; this video‑level analysis describes the evaluated checkpoint pair and does not establish robustness to retraining, and three training seeds are reported separately. The input‑level gain recurs on a transformer backbone. In one exploratory run, zeroing the channel at inference retains most of the gain in peak signal‑to‑noise ratio but lowers structural similarity below the baseline. A normalized‑step diagnostic does not support the expected larger‑gradient explanation. The results characterize the evaluated configurations and do not establish injection location as the sole cause, a spatial‑locality mechanism, or a reliable deployment advantage.
Authors:Xiaowei Ji, Lishan Xu, Jianing An, Jie Luo, Wenjun Wu, Lei Huang
Abstract:
Adam is widely used for training large language models (LLMs) owing to its strong optimization performance and training stability, but its element‑wise estimation for the first and second moments incur substantial memory overhead. Although numerous memory‑efficient optimizers have been proposed, many incur additional computational overhead or require additional tuning of hyperparameters. Consequently, Adam remains a dominant and reliable choice for LLM pre‑training. We propose AdamPM (Adam with Pooled Moments), a simple and memory‑efficient variant of Adam that groups adjacent moment entries along the row or column dimension and represents each group by a single shared mean. AdamPM can well inherit Adam's hyperparameter settings and is readily integrated into existing Adam optimizer implementations, with less memory overhead. We systematically investigate pooling of both first and second moment estimates and identify second moment pooling as the effective design. With a group size of s, AdamPM reduces second moment memory to 1/s of the original requirement. Extensive experiments on language model pretraining, fine‑tuning and multimodal learning demonstrate that AdamPM maintains model performance and training speed while substantially reducing optimizer‑state memory. Our code is available at \urlhttps://github.com/jixiaowei95/AdamPM.
Authors:Hangyu Li, Yiming Meng, Zhaohui Liang, Yunqing Xiao, Yuxuan Chen, Bofeng Cao, Shixiao Liang, Jiaxi Liu, Haotian Shi, Xiaopeng Li
Abstract:
Understanding how a particular automated vehicle reacts to nearby traffic can improve motion prediction, control development, traffic simulation, and safety evaluation. However, commercial automated driving systems are proprietary, and single encounters may provide only a few seconds of noisy data. Standard physical models are interpretable but inflexible, whereas neural models are often data‑hungry. We propose a physics‑informed Koopman framework that combines a calibrated car‑following law with a correction learned from the observations. A weak‑form estimator learns this correction from velocity measurements integrated over time, avoiding noise‑sensitive numerical differentiation. The resulting continuous‑time model predicts the follower's response and provides interpretable measures of driving behavior. Experiments on OpenACC and CATS‑CF evaluate 5‑, 7‑, and 11‑second histories. Averaged over these histories, the constant‑time‑headway branch reduces speed prediction error by 16.0% and 15.4% on OpenACC and CATS‑CF, respectively, relative to the best neural baselines. The PI‑Koopman branches achieve the lowest mean speed and gap errors among the physical and neural comparators. Weak‑form estimation reduces acceleration‑field reconstruction error by 74.5% in noisy known‑law simulations and noise‑induced field perturbation by 91.6% in paired real‑vehicle measurements. The same PI‑Koopman system identification supports following‑mode recognition with an approximately eleven‑fold difference in median residual variance between human drivers and automated vehicles, and a single residual threshold achieves 85.4% balanced accuracy. The branch informed by the intelligent driver model takes a median 17 to 26 ms for identification and forecasting on the tested central processing units, supporting frequent target adaptation.
Authors:Agus Sudjianto, Denis Burakov
Abstract:
Traditional weight of evidence assigns a credit characteristic a value based on its association with default across a population. Credit analysis also requires an assessment of that characteristic in the context of an individual applicant's record. We study conditional weight of evidence for this purpose. It measures the change in default log odds from observing a characteristic after the remaining credit information is known. Its sign identifies whether the characteristic adds adverse or favorable information for that applicant, and its magnitude measures the strength of that information. The same characteristic value can therefore receive different assessments for applicants with different payment histories or other credit attributes. We develop estimators using gradient‑boosted trees and evaluate their accuracy in a bureau‑style simulation. Regressions of the full model's predicted probabilities recover conditional evidence more accurately than separate models fitted to default outcomes, provided the full model is accurate. They also recover a deliberately constructed file‑age effect whose direction varies with utilization and is often missed by marginal weight of evidence. Local class‑frequency estimates improve on marginal weight of evidence but remain inaccurate for some weak effects. An application to FICO HELOC data illustrates estimation when the population evidence is unknown. Conditional weight of evidence provides an applicant‑specific interpretation of the information in a credit characteristic; its population summary, a conditional information value, splits the joint information in the record across blocks when averaged over orderings. Neither is an adverse action reason, which explains the fitted score difference between the applicant and chosen passing records.
Authors:Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash, Kirill Afendulev, Alexey Dontsov, Ivan Oseledets, Elena Tutubalina, Anton Korznikov
Abstract:
We study feature vectors: directions in a language model's activation space that correspond to human‑interpretable concepts, such as honesty or deception. Existing unsupervised methods like sparse autoencoders attempt to recover the whole feature space, often at the cost of feature quality and interpretability. We take a more pragmatic route: focusing on concepts relevant to AI safety and alignment, we generate synthetic concept‑antagonist story pairs and use the difference of their mean model activations as a feature vector. We scale this approach to 1,036 alignment‑relevant concepts using 2M synthetic stories across multiple languages and genres, which we publicly release. Across six models from three model families, our feature vectors reliably generalize to detecting their corresponding concepts in chat dialogues and web text. They form an interpretable geometry shared across models and can causally steer model behavior when injected into activations. We further use these features to analyze model representations during jailbreak attacks, mathematical and agentic reasoning. Our targeted feature construction thus offers a scalable route to studying concepts in language models, potentially extending to millions of features across arbitrary domains.
Authors:Ahad Bin Islam Shoeb, Jamal Uddin Tanvin, Kamrul Hasan, Alve Reduan, Imtiaz Ahmed
Abstract:
Instance‑dependent early stopping (IES) reduces training computation by skipping backpropagation for examples whose losses have stabilized, but the second‑order loss‑difference criterion used by IES lacks a theoretical justification. We develop Theory‑Guided Instance‑Dependent Early Stopping (TG‑IES), which analyzes this criterion under explicit assumptions and derives a noise‑calibrated mastery threshold and a conditional stability bound for instance exclusion. We show that second‑order differencing has a favorable bias‑variance trade‑off over first‑ and third‑order differences in a specific signal‑to‑noise regime, and that its signal contains both gradient‑drift and curvature terms. TG‑IES derives its stopping threshold and patience from training‑run statistics rather than using the manually selected threshold of IES. Across CIFAR‑10 and CIFAR‑100, TG‑IES saves 53‑58% of backpropagation instances in its main convolutional settings while keeping accuracy within 0.1‑0.9 percentage points of full training; in the primary ResNet‑18/CIFAR‑10 setting, this corresponds to 35.7% FLOP‑normalized savings. The method also transfers to ViT‑Tiny, where it achieves smaller savings and incurs a modest accuracy cost. These results provide a theoretical basis for second‑order instance‑dependent early stopping and show that theory‑guided calibration can substantially reduce training computation under the assumptions and experimental settings studied.
Authors:Jin-Shiang Hu, Bo-Yuan Cheng, Li Su
Abstract:
This paper presents an optical music recognition (OMR) system tailored for PDF scores of string quartet pieces. The approach begins with a SegNet‑based segmentation model to localize musical regions, followed by convolutional neural networks and image processing techniques to classify and extract musical symbols. To improve rhythmic precision, dynamic programming methods enforce time signature constraints and explicitly handle tuplets. Evaluation using a longest common subsequence metric across multiple feature matrices demonstrates robust accuracy in recognizing pitch, onset, duration, and accidentals on a benchmark dataset. Code and data are available at https://github.com/lattellie/25‑omr.
Authors:Jingyi Xiang, Yixuan Wang, Jeannette Bohg, Wenzhen Yuan
Abstract:
Contact‑rich manipulation underlies many everyday human activities, yet evaluating robots on such tasks remains difficult: the tasks involve diverse contact behaviors and are hard to reproduce across laboratories. Drawing on observations of everyday human manipulation, we identify eight essential skills that provide a practical framework for studying contact‑rich manipulation. We introduce ContactBench 1.0, an extensible benchmark that instantiates these skills through 36 reference tasks and more than 40 object assets. The assets capture task‑relevant geometry and contact behavior, can be fabricated reproducibly on consumer‑grade 3D printers, and are each paired with a matching simulation model. Controlled task variations in both the real world and simulation enable evaluation of how well robot systems generalize. We evaluate recent representative vision‑only and visual‑tactile policy learning methods on six benchmark tasks. The results reveal that while vision‑only policies struggle to perceive or react to contact events, visual‑tactile policies do not meaningfully improve their performance, highlighting that contact‑rich manipulation remains a significant challenge for current policy learning methods. By providing common task definitions alongside physical and simulated assets, ContactBench 1.0 offers a shared testbed for developing, evaluating, and comparing contact‑rich manipulation systems. The benchmark will be open‑sourced at https://jingyi‑xiang.github.io/contactbench.
Authors:Qingyue Chen, Huaiyuan Yao, Wanpeng Xu, Varun Shourie, Qianwen Wang, Hua Wei
Abstract:
Timely pedagogical feedback is difficult to scale in online classes, where long and noisy videos make student responses hard to locate and automated systems often produce generic advice weakly grounded in actual classroom behavior. The key challenge is to connect the instructional goal of a specific lesson segment with temporally aligned multimodal evidence of student responses, and then translate the observed gap into an actionable teaching move. We introduce MAVE (Multi‑Agents for Video‑based Educational Feedback), a multi‑agent framework for grounded pedagogical feedback from online‑class videos. MAVE decomposes feedback generation into four roles: the Curator structures the lesson and infers expected student activity; the Observer extracts aligned speech, visual, and interaction evidence; the Assessor estimates the activity supported by student behavior; and the Advisor generates gap‑conditioned teaching recommendations. To evaluate this capability, we introduce MAVEBench, a benchmark designed to measure whether feedback systems can ground diagnoses in multimodal student evidence, and generate actionable pedagogical feedback. MAVEBench contains 173 knowledge‑point segments from approximately 20 hours of authentic online‑class videos across STEM and Humanities, together with comprehensive simulation. Experiments show that MAVE outperforms direct prompting, Reflexion, and TextGrad in diagnostic correctness, instructional alignment, and actionability. Our code is available at https://github.com/DaRL‑GenAI/MAVE
Authors:Kahim Wong, Kemou Li, Yiming Chen, Haiwei Wu, Jiantao Zhou
Abstract:
As AI‑assisted image editing becomes increasingly prevalent, Text‑Centric Image Forgery Localization (TFL) is essential for protecting trust in financial and legal records. However, we observe that existing TFL detectors predominantly rely on Full‑Parameter Fine‑Tuning (FPFT) and fail to generalize to unseen forgery types because FPFT can induce low‑rank feature collapse and overfit training artifacts. Meanwhile, prior detectors capture diverse forgery traces with a single unified representation, making subtle traces entangled and difficult to learn. To address these challenges, we propose RiSE, a Residual Subspace Expert model that preserves pre‑trained priors by freezing principal SVD components and adapting only the residual subspaces with a ViT‑only segmentor to avoid feature collapse. RiSE introduces forgery‑aware residual experts, where each expert is trained independently on a training subset with a particular forgery type, and we show that cross‑domain localization performance improves consistently as more experts are added. At inference, we introduce Latent Perturbation Confidence (LPC), which selects the most confident expert by analytically computing the latent‑perturbed output. LPC enables efficient confidence estimation with a single forward pass, avoiding repeated forward passes on perturbed samples and improving expert selection. By training on synthetic forgery data, RiSE with 1 expert outperforms prior methods on real‑world forgery images by 20.4% while reducing training steps by 10 times. Scaling the number of experts to 9 yields a 35.8% gain. RiSE remains effective with DCT feature, even when the majority of the parameters is frozen with RGB‑only pretraining. Code: https://github.com/KahimWong/RiSE.
Authors:Tongshun Zhang, Zixuan Zhong, Pingping Liu, Jin Kuang, Yubing Lu, Dong Liu, Qiuzhan Zhou
Abstract:
Frequency‑domain methods have shown strong potential for low‑light image enhancement (LLIE) because they can effectively regulate global illumination. However, most existing methods assume that amplitude and phase can be restored independently, while overlooking their inherent spectral coupling. This artificial decoupling often leads to information inconsistency and structural distortions. To address this issue, we propose a frequency‑spatial framework that jointly models spectral interaction and reliable prior guidance. We first empirically analyze the coupling between amplitude and phase and develop an Amplitude‑Phase Adaptive Compensation Fusion (APACF) module. Instead of enforcing a hard separation between the two components, APACF enables their bidirectional interaction, resulting in spectrally coherent restoration. In addition, directly extracting priors from raw low‑light inputs is unreliable, since severe noise and underexposure destroy critical visual content. To alleviate this problem, we introduce an Endogenous Reliability Prior (ERP). Specifically, we treat an intermediate result, preliminarily restored in the frequency domain and already close to the ground‑truth illumination, as a high‑fidelity source for extracting complementary pseudo‑infrared and gradient cues. Based on ERP, we further design a Dual‑Modal Guided Refinement Network (DGRN) to inject these purified structural cues into the decoder, thereby recovering missing fine textures more accurately. Extensive experiments on benchmark datasets show that our method achieves state‑of‑the‑art performance and consistently surpasses existing methods in both quantitative metrics and visual quality. Our code is available at https://github.com/bywlzts/Amplitude‑Phase‑Coupling‑Net.
Authors:Chamuditha Jayanga Galappaththige, Jason Lai, Lloyd Windrim, Timothy Patten, Donald G. Dansereau, Niko Suenderhauf, Dimity Miller
Abstract:
Online scene change detection requires an agent to detect what has changed in a previously mapped environment while it is still on site, observing from unconstrained viewpoints and without access to future observations. This causal setting is strictly harder than its offline counterpart, and online methods have consistently trailed offline accuracy. O‑SCD closed that gap by treating change as a persistent property of a 3D Gaussian splatting map of the reference scene, surpassing the strongest offline methods in change detection accuracy while operating at over 10 frames per second. O‑SCD optimizes a change field over the Gaussians of the reference map, refitting it to all past cue maps at every frame, a step that consumes more than half of the per‑frame budget and forces accuracy to be traded against speed. We present a replay‑free formulation that eliminates this trade‑off. Because the reference map stays fixed, rendering the change field is a linear operation. Each new observation can therefore be carried onto the map's Gaussians directly, by a single backward pass of the renderer, which reduces multi‑view fusion to an exact two‑scalar recursion per Gaussian. Past observations are never stored or revisited. The self‑supervised objective that O‑SCD minimizes by descent also determines the decision threshold exactly, by sorting, yielding a label‑free per‑frame cutoff whose margin provides a selective‑prediction signal. On PASLCD, our method improves mean intersection over union from 0.489 to 0.575 and throughput from 11.8 to 25.6 frames per second, even exceeding O‑SCD's offline result while remaining strictly causal.
Authors:Ishir Garg, Vivek Myers, Qiyang Li, Catherine Ji, Benjamin Eysenbach, Sergey Levine
Abstract:
Large, unlabeled datasets of prior experience have become increasingly available for agentic tasks in domains such as robotics and coding. A common recipe for these tasks is to train a policy using the prior data, and then further improve it with online rollouts. But while much work has studied how to use both the offline and online data for policy extraction, far less work has been done to understand how the prior data can be used to efficiently explore during the online learning phase. Previous work has attempted to formalize this idea by defining the empowerment, a measure of how many future states an agent can realize; states with higher empowerment require more exploration as there are more futures to explore. However, existing methods for estimating empowerment require online policy rollouts in the environment. We propose the first method for estimating empowerment from unlabeled prior data, without any environment interaction. By jointly learning an empowerment‑maximizing policy together with its discounted state‑occupancy measure, our estimator can be trained on unlabeled trajectories generated from any unknown policy. We demonstrate that our empowerment estimator qualitatively gives intuitive empowerment measures and show that the pretrained estimator can be used to improve exploration in the online goal‑conditioned reinforcement learning setting. Code is available at \urlhttps://github.com/ishirgarg/offline‑empowerment.
Authors:Junsu Kim, Naeun Kim, Dongyoon Han, Seungryul Baek
Abstract:
We introduce avatar referring segmentation, which selects regions of animatable Gaussian avatars from language queries. A query can also specify a pose or motion to select matching intervals. This lets users interact with avatars for local editing and motion inspection. However, making these selections with existing language Gaussian splatting methods poses two challenges: adjacent parts and garments are hard to separate and may be referred to indirectly (region ambiguity), while fine‑grained motions can look alike in RGB (motion ambiguity). We address these challenges with HumanSkill‑4D, a training‑free framework that shares a multimodal large language model (MLLM) and geometric operators across avatars. For each avatar, a semantic memory links body‑part and garment categories to persistent Gaussian regions and appearance descriptions. The MLLM uses category names and descriptions to select categories by appearance, relation, or function. To find matching intervals, it composes operators into programs that check query conditions on driving joint trajectories stored in a motion memory. These category choices and operator outputs make selections traceable, while composing or adding operators supports new motion conditions. To evaluate this task, we introduce three human‑annotated benchmarks and show that HumanSkill‑4D outperforms the baselines in part segmentation and 3D and 4D referring segmentation, with compact memories and only minutes of setup per avatar.
Authors:Qiang Zou, Jiajun Shen, Zhijie Yang, Yuhao Wang, Shitianyi Leng, Dongshuo Yin
Abstract:
Recent advances in general‑purpose AI models and agentic systems have enabled them to perform some CAD tasks. A systematic assessment is now needed to establish what these systems can accomplish and how well their outputs meet engineering requirements. This paper presents an empirical study of GPT‑6 Astra, a SOTA general‑purpose AI model, across four basic CAD tasks: model understanding, generation, editing, and evaluation. We design test cases spanning different levels of complexity and engineering requirements. The results show that GPT‑6 Astra can support multiple stages of CAD workflows, with successful outcomes in cases involving nontrivial geometric complexity. This suggests that general‑purpose AI is beginning to offer practical value in almost all steps in CAD workflow. However, gaps remain between generating visually plausible geometry and producing valid, functional, and editable CAD models. Recurring errors in freeform geometry, parameter dependencies, and mating relationships between parts limit engineering usefulness. Agentic iterative refinement can improve results but does not consistently resolve critical errors; long completion times further constrain practical use.
Authors:Weiqiang Zhao, Xian Li, Yuzhe Gui, Tianzhu Liu, Yanfeng Gu
Abstract:
Multispectral filter array (MSFA) imaging enables snapshot spectral acquisition, but its per‑band sparse sampling causes severe spatial aliasing. Although missing spatial structures can be inferred across bands, wavelength‑dependent contrast variations make direct cross‑band transfer unreliable. Existing model‑based methods lack explicit magnitude‑invariant mechanisms, and learning‑based methods require layout‑specific training or per‑image tuning, limiting generalization. To address these issues, we propose S3D‑Mosaic, a model‑based spectral demosaicing method built upon a subgradient‑guided cross‑band structure transfer framework. We first establish a subgradient‑guided regularization framework with a generalized cosine‑similarity interpretation, providing a convex mechanism for magnitude‑invariant information transfer. Then we design a compact Directional Sparse Equivariant Difference Convolution (DSE‑DConv) for band‑level extraction of orientation‑selective, high‑frequency structure features at low computational cost. Instantiating the framework with DSE‑DConv on spectrally adjacent bands yields a cross‑band self‑structure similarity regularizer for spectral demosaicing, which we optimize via a dual‑scan ADMM algorithm that decouples the bidirectional dependency while propagating magnitude‑robust structural information across bands. With DSE‑DConv's two parameters trained once via self‑supervision, S3D‑Mosaic outperforms state‑of‑the‑art unsupervised methods on CAVE and NTIRE 2022, and generalizes robustly to varying MSFA layouts, pan‑mosaic fusion, and real‑world UAV data with fixed reconstruction hyperparameters. The source code is available at \urlhttps://github.com/bestwishes43/S3D‑Mosaic.
Authors:Jaeyeon Kim, Chao-Han Huck Yang, Luoyi Zhang, Chan-Jan Hsu, Fernando Ruiloba Portilla, Jinchuan Tian, Shinji Watanabe, Carlos Busso
Abstract:
We introduce Long Listening Thoughts (LLT), a framework for constructing post‑training datasets that improves audio reasoning by pairing deliberate perceptual grounding with explicit cognitive behaviors. On audio understanding and reasoning benchmarks, large audio‑language models (LALMs) are limited primarily by perceptual errors, yet existing post‑training datasets rely on audio descriptions of limited diversity. Through a four‑stage pipeline, LLT (1) builds multi‑aspect dense audio descriptions spanning semantic, acoustic, and temporal information, (2) generates more challenging questions, (3) synthesizes incorrect reasoning traces, and (4) extends short reasoning traces into long ones via a thought‑continuation strategy that elicits cognitive behaviors such as verification and backtracking. Using LLT, we synthesize 1.6M reasoning traces and train Qwen2.5‑Omni‑7B with a two‑stage pipeline of supervised fine‑tuning (SFT) followed by direct preference optimization (DPO). This improves accuracy on MMAU Test‑mini (71.5 to 78.2), MMAR (56.7 to 63.7), and MMSU (60.6 to 66.0).
Authors:Xinyu Chen, Liangjian Wen, Jiang Duan, Dongkai Wang, Yong Dai, Jiayu Bai, Jianzhuang Liu, Zhen Tian, Guoping Qiu, Zhao Kan
Abstract:
In multimodal learning, modality imbalance often lets one modality dominate optimization and keeps the model from fully exploiting cross‑modal interactions. Balance‑oriented methods such as the Multimodal Competition Regularizer (MCR) perform well by alleviating this imbalance during training. These dynamic balancing methods, however, mainly adjust modality contributions and do not consider how imbalance affects the formation of synergistic information during fusion. Synergy is a major source of multimodal advantage, because it is predictive gain that cannot be recovered from unimodal evidence alone. We propose the Multimodal Synergistic Competition Regularizer (MSCR), a fusion method that jointly promotes balanced modality utilization and the discovery of synergistic information. MSCR has two complementary objectives. An anti‑dominance objective suppresses excessive unimodal influence and leaves room for synergistic information, and a synergy objective encourages the fused predictor to outperform a non‑synergistic unimodal reference. An adaptive gate further concentrates the regularization on samples that need stronger collaborative modeling. Experiments on five benchmarks show that MSCR consistently improves performance over MCR and strengthens synergistic capability. It raises conditional recovery, the accuracy of the fused prediction on samples that the unimodal predictors misclassify, as well as the share of the model's output that InterSHAP attributes to cross‑modal interactions. Its gain in conditional recovery persists when the fusion module is Transformer‑based. Code is available at https://github.com/chenxinyu991‑collab/MSCR.
Authors:Wonjun Lee, Vaidehi Patil, Zaid Khan, Yue Zhang, Hyunji Lee, Suhyun Kim, Mohit Bansal
Abstract:
Recent work shows that attackers can implant dormant malicious behaviors that remain hidden at release but emerge after benign fine‑tuning. As foundation models are used as agents, whether dormant poisoning extends to harmful agent actions and multimodal settings remains unclear. We show two agentic risks: (1) harmful behaviors activated by benign fine‑tuning can extend to unseen actions; (2) multiple harmful behaviors can coexist and be jointly activated by benign downstream fine‑tuning. Existing defenses often rely on external safe responses, which are hard to obtain for agent actions, or a trusted clean reference model. We exploit a key property of dormant poisoning: the attacker must preserve safe behavior at release to conceal poisoning, and this behavior can serve as a safety reference. Leveraging this property, we automatically construct a Safety Buffer pairing harmful inputs with the released model's responses and benign versions generated by the same model to remove harmful intent. The Safety Buffer can replace external safe responses in existing defenses, but these defenses do not distinguish updates that weaken safety from those that preserve it, often increasing over‑refusal or reducing utility. To address this limitation, we introduce LookAhead Defense, which previews benign fine‑tuning updates using the Safety Buffer and penalizes only updates predicted to weaken safe behavior on harmful inputs or increase the likelihood of refusals on benign versions in the Safety Buffer. Across language models, language agents, and multimodal agents, LookAhead Defense reduces harmful behavior in agents from up to 93.9% to at most 14.2% and harmful responses in language models from up to 92.5% to at most 2.0% versus vanilla fine‑tuning, while maintaining competitive utility and low over‑refusal. It remains effective against other poisoning methods.
Authors:Shidu Ren, Rong-cheng Tu, Hang Zhou, Xiao Luo
Abstract:
As multimodal agents accumulate experience, their long‑term memories contain many similar observations of recurring scenes, actions, and entities. Answer quality depends on the evidence retrieved from these memories into the model's context. However, query‑relevant memories may contain overlapping information, and a fixed retrieval depth may return too little or too much evidence for a given question. To address these challenges, we propose PACE‑Mem, Prefix‑Aware Complementary Evidence, which constructs a complementary evidence sequence and determines its length from the same selection trajectory. For structured memories, query‑guided pooling combines records within each clip to obtain relevance scores and representations. Using these relevance scores and representations, maximal marginal relevance (MMR) selection balances query relevance against overlap with previously selected evidence. From the resulting score trajectory, a prefix‑length selection algorithm determines retrieval depth by comparing length‑normalized adjacent declines, accounting for both score changes and the amount of evidence accumulated. PACE‑Mem is a training‑free, plug‑in retrieval method for clip‑based agent memory systems, leaving memory construction and answer generation unchanged. Across three long‑video benchmarks and two agent frameworks, PACE‑Mem improves question‑answering accuracy by an average of 1.6% over Adaptive‑k, the strongest baseline at comparable mean retrieval depths. Further analysis reveals that jointly adding and removing evidence is associated with larger net accuracy gains than either change alone.
Authors:Jeff Shen, Zhejian Zhou, Ryan Lee, Jonathan May, Shirley Ho
Abstract:
Speculative decoding accelerates large language model (LLM) inference by using a lightweight drafter to propose a block of tokens that the target model verifies in one parallel pass. Autoregressive drafters generate this block one token at a time, whereas block diffusion drafters such as DFlash generate it in one pass, making drafting faster. However, these diffusion drafters predict each token independently, ignoring dependencies between tokens in the block. We introduce FlashMap, a continuous flow drafter that conditions a flow map language model on the target model's hidden states and generates the whole block in one forward pass while capturing inter‑token dependencies. On Qwen3‑8B, FlashMap is faster than a matched DFlash baseline on every benchmark, with 7% higher average speedup and up to 6.1x lossless acceleration.
Authors:Liang Su, David Chen, Jason Li, Kevin Guo, Teresa Lv
Abstract:
Vision‑language‑action (VLA), video‑action (VA), and world action models (WAMs) form the backbone of modern physical AI systems, yet their real‑time single‑batch inference remains a defining bottleneck. Latency is dominated not by individual kernel computation, but by system‑level overhead: thousands of short kernel launches, intermediate data movement across operator boundaries, and redundant recomputation across consecutive decisions. General‑purpose graph compilers cannot address these costs, as they optimize individual calls in isolation and lack visibility into cross‑call state semantics and global pipeline structure. We present FLASHRT‑PAI, a low‑latency inference engine built around model‑hardware pipeline co‑design rather than isolated kernel optimization. FLASHRT‑PAI combines three core mechanisms: deep kernel fusion with launch overlap eliminates inter‑operator boundaries and launch gaps; an execution contract formalizes the pipeline's stage DAG, update rates, and named buffers to enable cross‑decision state reuse and selective refresh; and execution‑state capsules enable fast snapshot, restore, and fork of persistent pipeline state. The system also supports drop‑in acceleration of unmodified PyTorch models via parity‑ and latency‑gated kernel replacement. Evaluation covers six policies on RTX 5090 and four on Jetson AGX Thor. At matched 16‑bit precision, FLASHRT‑PAI is 1.3‑3.7x faster than official compiled paths. Combining pipeline co‑design with reduced precision delivers 3.38‑12.66x speedup over official eager implementations. In LIBERO‑10, four‑bit pi0.5 achieves 94.04% success on Thor and 93.32% on RTX 5090, compared with 92.4% for the measured official FP16 reference. FLASHRT‑PAI is open source, distributed via Hugging Face Kernel Hub, and its opt‑in pi0.5 kernels have been adopted into the LeRobot framework.
Authors:Iris Ma, Zahra Shekarchi, Stuart Mcilroy, Mahtab Farrokh, Fattane Zarrinkalam, Joel Stremmel
Abstract:
Reciprocal Rank Fusion (RRF) is widely used to combine sparse and dense retrievers in hybrid information retrieval, but a critical limitation is that it assigns a fixed, uniform weight to each retriever regardless of the query. Query‑aware methods treat the per‑query optimal weight as a single point, an assumption whose empirical validity has not been examined. We present an empirical study of eight dynamic fusion weight prediction strategies across four datasets and four sparse‑‑dense retriever pairs. When we compare these approaches to static fusion and a per‑query oracle that bounds achievable performance, several findings emerge. First, the per‑query optimal fusion weight is typically an interval rather than a point. Queries can have multiple optimal intervals, and these can take on a variety of shapes. Second, 74 of 128 dynamic fusion methods tested were significantly better than standard RRF after false discovery rate correction. Third, dynamic fusion methods exhibit differing quality and latency characteristics that should be considered when choosing an appropriate fusion method. To address this last finding, we present a tiered decision framework that maps practitioner constraints, dataset properties, label availability, hardware, and latency budget to an appropriate method class. We release the code for our experiments at https://github.com/thomsonreuters/query‑based‑rrf.
Authors:Minbo Ma, Honglin Zhang, Bangtao Zhou, Chunguang Yu, Dasheng Ju, Yichen Shen, Fengli Xu, Yong Li
Abstract:
Advances in reasoning, coding, and tool use enable LLM agents to solve operations research (OR) problems through iterative problem modeling, program construction, and verification. Yet passing self‑authored verification does not ensure that the solutions satisfy task constraints and meet optimization quality. This gap can arise from correlated specification errors, where the generated program and its checks share omissions or misinterpretations of task requirements, alongside incomplete verification coverage and ineffective optimization strategies. We propose VERAS, a verification‑guided adaptive search harness for reliable OR that uses verification evidence to prioritize feasibility repair, constraint‑coverage review, and solution‑quality improvement. VERAS combines task‑specific solution checkers with a separate verification agent to examine candidate solutions against the task requirements. It uses verification feedback to guide feasibility repair, additional constraint checking, and objective improvement, while candidate acceptance criteria screen revisions for deterioration in output validity or constraint satisfaction. On ORAgentBench, VERAS achieves a 42.99% pass rate, exceeding the reported 35.51% of Codex using the same LLM. Across two industrial‑scale OR problems, VERAS reduces inter‑datacenter peak bandwidth by 10.34% relative to native Codex and lowers annualized distribution network planning costs by 6.28% compared with expert‑designed solutions. Code and evidence are available at https://github.com/holimario/VERAS.
Authors:Javid Baydamirli, Didem Unat
Abstract:
RDMA networking enables low‑latency, high‑bandwidth communication on large systems. Despite its adoption, however, observability is lacking. Existing tools target specific communication libraries or only report aggregated data, missing per‑operation behavior. The gap is widest for GPU‑initiated networking, whose in‑kernel RDMA operations are invisible to the host. We present SigmaTrace, a library‑agnostic RDMA profiling framework providing fine‑grained, per‑operation visibility for CPU and GPU‑initiated networking. It traces operations at the libibverbs/mlx5dv layers, the common denominator beneath communication libraries on InfiniBand and RoCE, and employs a low‑overhead memory interposition technique that, to our knowledge, gives the first observability into GPU‑initiated RDMA without source patching or binary instrumentation. Host‑side tracing adds 1.3% overhead on NCCL AllReduce at 128 GPUs and under 2% on RCCL at 1024 GPUs. On a persistent‑kernel BFS and Mixture‑of‑Experts inference, SigmaTrace exposes an in‑kernel barrier spread and 2.13x inter‑node traffic skew invisible to existing profilers.
Authors:Yuntao Du, Harsh Chaudhari, Yuetian Chen, Ron Heichman, Ashish Jayamohan, Kumar Pallav, Mahim Patel, Kaiyuan Zhang, Kyle Polley, Jerry Ma, Ninghui Li
Abstract:
AI agents increasingly interact with external environments to complete complex user tasks. These interactions can lead to unexpected behavior and expose agents to security threats such as indirect prompt injection. Current agent security systems largely protect agents by restricting their actions, which can result in a substantial security‑utility trade‑off for complex, open‑ended tasks. To address this, we propose a quantitative risk‑based approach to access control for AI agents that regulates measurable risks accumulated during task execution. As a concrete realization of this approach, we introduce RATION, a security framework that tracks measurable risks associated with agent actions and enforces quantitative constraints throughout execution. RATION first constructs a task‑specific policy by combining explicit constraints derived from the user request with implicit risks calibrated from historical executions of similar tasks. During task execution, RATION continuously tracks the accumulated risks and applies different enforcement strategies depending on the type of risk. Extensive experiments on four security benchmarks show that RATION provides strong security while maintaining high task utility across diverse models and environments.
Authors:Niko Covic, Baraq Lipshitz, Alexander Padula, Stefan Visnjic
Abstract:
We study the Muon optimizer using sharpness and provide the first empirical evidence that the Flat Minima Hypothesis also applies to Muon. Further, we derive Muon's implicit preconditioner to analyze its potential Edge of Stability behavior. Experiments on CIFAR‑10 show sharpness strongly correlates with generalization gap within each optimizer (Pearson r = 0.45‑0.69, p \leq 10^‑4), confirming the hypothesis holds for Muon similarly to Adam and SGD. However, Muon operates above the classical sharpness‑based Adaptive Edge of Stability threshold that bounds diagonally‑preconditioned optimizers like Adam and RMSprop, yet remains stable. Our findings suggest Muon operates in a fundamentally different regime than diagonally‑adaptive methods. Defining the exact stability boundary for Muon remains an open question. All code is available at https://github.com/NikoCovic/DL‑project.
Authors:Sho Sonoda, Luca Galimberti, Yuanchao Xu, Isao Ishikawa, Masahiro Ikeda
Abstract:
Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite‑dimensional networks or by Stone‑‑Weierstrass‑type density; these prove that approximants exist but say little about how the parameters depend on the target. We take a constructive route. We extend the ridgelet transform, an integral transform related to the Fourier, Radon, and wavelet transforms, to continuous maps on a separable infinite‑dimensional Hilbert space, replacing the Lebesgue measure that is unavailable in infinite dimensions by Gaussian weights with a homogeneous mixture on the direction space. The resulting transform has a Plancherel identity and reconstruction formulas incorporating the frame operator, the correction induced by the weights, and it assigns to each target an explicit coefficient measure, one for every admissible analysis filter. From these representations we obtain uniform approximation on compact sets for all continuous non‑polynomial activations of polynomial growth, including ReLU, and, by Monte Carlo sampling of the coefficient, finite‑width networks with a dimension‑free O(N^‑1/2) error in the width N for globally Lipschitz activations and coefficients with finite variation and second moments. Explicit examples include Gaussian targets and neural‑operator layers. A companion Lean~4 formalization provides machine‑checked proofs, with its correspondence to the manuscript documented at https://shosonoda.github.io/lean‑operator‑ridgelet/ .
Authors:Mohsinul Kabir, Md. Tariquzzaman, Mohammad Rashidujjaman Rifat, Tharindu Ranasinghe, Nishat Raihan
Abstract:
Safety classifiers for large language models (LLMs) are typically built and evaluated on standard‑script text, yet Bangla, like many lower‑resource languages, is routinely written in native script, in Latin‑script romanization, and code‑mixed with English, Hindi, or Urdu. We introduce BanglaShield, a resource and model for graded prompt‑safety classification across these written forms. We consolidate 10,245 Bangla prompts from six sources under a four‑level taxonomy (Safe, Offensive, Harmful, Severe) and render each prompt in three aligned forms that share labels and a group‑aware split, enabling strictly paired comparisons. Benchmarking six contemporary LLMs, including a Bangla‑specialized one, on 36,882 classification queries and 24,318 judged responses, we find that LLM classifiers label 47.0% of unsafe native‑script prompts as Safe, rising to 63.4% under romanization, while response‑level safety accuracy drops by 10.3 points. Three multilingual guard models also pass 30.3‑57.3% of unsafe native‑script prompts. Using a model's own classification as an input filter reduces harmful compliance by only about five points. In contrast, BanglaShield, a compact fine‑tuned encoder, reaches 87.8 macro‑F1 on native‑script and 84.4 on romanized prompts, outperforming the best LLM on each form by 42.8‑46.1 points with ~130x fewer parameters than Gemma 4 31B. We release the corpus, splits, judgments, and model.
Authors:Dariush Wahdany, Jesse Cresswell, Naiqing Guan, Atiyeh Ashari Ghomi, Franziska Boenisch, Adam Dziedzic
Abstract:
Tabular foundation models (TFMs) are increasingly deployed through black‑box APIs and trained on large collections of real‑world tabular datasets. As this training paradigm becomes more widely adopted, private or proprietary datasets may be incorporated into pre‑training corpora, yet there are currently no dedicated methods for determining whether a given tabular dataset was used to train a TFM. As a solution, we introduce the first dataset inference method for TFMs, aiming to infer whether a suspect dataset was part of a model's pre‑training data. We systematically analyze a broad collection of candidate signals that can be observed from a black‑box TFM via input manipulation and find that individual signals provide only weak evidence of training data membership. However, by combining multiple complementary signals, we can infer dataset membership for several state‑of‑the‑art TFMs trained on real tabular data, achieving up to 0.99 ROCAUC dependent on the TFM. We then study factors that influence dataset identification, including pre‑training data composition, model capacity, and the use of real vs. synthetic data. Our results show that our dataset inference method is a practical auditing tool for detecting privacy leakage and the use of proprietary datasets in TFMs.
Authors:Egor Zverev, Juhee Kim, Leon Trampert, Boris Köpf, Aashish Kolluri, Shruti Tople, Santiago Zanella-Béguelin
Abstract:
Browser agents automate complex workflows by reading and acting on arbitrary page content, but this flexibility exposes them to prompt injection attacks and unintended data exfiltration. Recent work argues that LLM‑driven browser agents erode traditional web security boundaries such as the Same‑Origin Policy (SOP) by acting across origins with the user's authority and no built‑in isolation. We argue that restoring SOP is neither sufficient (a same‑origin public sink can leak private data) nor necessary (legitimate tasks require authorized cross‑origin flows) for protecting the security of users. We propose SPIDES, which instead enforces information‑flow control by gating primitive browser actions such as typing, clicking and navigating on labels propagated from element‑level HTML confidentiality and integrity annotations. Untrusted and confidential content is masked from the agent's context, while policies block unauthorized flows, providing security guarantees against both prompt injection and data exfiltration that hold independently of the model driving the agent. To evaluate SPIDES and baseline defenses under a common threat model, we build MicroNet, a closed ecosystem of 28 interlinked websites with element‑level ground‑truth labels and paired benign/adversarial tasks. SPIDES achieves 0% attack success rate by design while retaining high task completion rate and autonomy. Our findings demonstrate that information‑flow control is an effective technique to extend web security to browser agents, gaining on autonomy and utility against state‑of‑the‑art baselines. The source code and data are openly accessible at https://github.com/microsoft/SPIDES.
Authors:Eric Zhang, Terry Jingchen Zhang, Chijioke Ugwuanyi, Jerick Shi, Bernhard Schölkopf, Zhijing Jin
Abstract:
As LLM‑based agents are deployed with increasing autonomy in real‑world environments, a critical failure mode emerges: agentic misalignment, where agents choose harmful actions over failure. However, existing safety benchmarks primarily evaluate whether agents comply with explicitly harmful instructions, leaving the conditions that cause agents to choose harmful actions poorly understood. To address this limitation, we propose AM‑Bench, a unified taxonomy and 35 scenario evaluation benchmark spanning seven industries for agentic misalignment grounded in the fraud triangle, a well‑established criminological framework explaining goal‑directed misconduct. We adapt this framework to agentic environments and construct our evaluation suite and detection systems along three dimensions: perceived threat, perceived opportunity, and reasoning. We operationalize each dimension into structured subcategories rooted in prior theory: five threat categories derived from the instrumental convergence thesis, seven opportunity categories drawn from the Emergent Strategic Reasoning Risks (ESRR) taxonomy, and a novel four‑question binary reasoning framework synthesized from prior literature. Our evaluation of 10 frontier models reveals a 62‑percentage‑point gap in misalignment across scenarios, with Gemini 3 Flash and Gemini 3.1 Pro exhibiting the highest levels of agentic misalignment while Claude Sonnet 4.6 displays near‑zero rates across all conditions. Our reasoning framework further reveals that models may exhibit harmful behaviours without explicit situational awareness or explanation, complicating both mitigation and detection. AM‑Bench scenarios and code are available at https://github.com/AI4Collaboration/Agentic‑Misalignment‑Bench
Authors:Wen-Zhi Li, Sainyam Galhotra, Claire Cardie
Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) improves the reasoning capabilities of large language models, but often reduces the diversity of their reasoning behaviors. We study this phenomenon from the perspective of finite‑sample learning dynamics and show that limited on‑policy rollouts can progressively suppress semantically distinct but equally rewarded reasoning modes. Once a mode becomes sufficiently under‑sampled, standard on‑policy RLVR rarely recovers it. Motivated by this observation, we propose TAR, a Targeted Adaptive Replay framework that preserves diverse correct reasoning behaviors already present in the base policy. TAR constructs a compact memory bank of diverse base‑model solutions and selectively replays them through an adaptive mechanism that activates only when stored behaviors are being suppressed during RLVR. Across six mathematical reasoning benchmarks and two Qwen2.5 model scales, TAR consistently achieves a better balance between reasoning performance and diversity preservation than existing diversity‑preserving post‑training methods, while substantially reducing the forgetting of correct base‑model solutions. Code is available at https://github.com/wenzhilics/tar.
Authors:Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson, Vincent Sitzmann
Abstract:
Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video‑to‑action or "inverse dynamics" model trained on large, human‑collected video‑action datasets, severely limiting scalability. We introduce Mirror, a self‑supervised approach that learns video‑to‑action mappings through environment interactions, without prior action data or hand‑crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video‑action pairs, incrementally self‑improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning‑from‑observation methods with learned video plans, and achieves near‑expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand‑engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self‑improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand‑crafted rewards or large collections of expert action data.
Authors:Amreen Hasheem Siraj, Alireza Javanmardi, Pragati Jaiswal, Didier Stricker, Alain Pagani
Abstract:
Object removal and semantic inpainting of missing regions remain inherently ill‑posed, as multiple plausible completions may exist for a given scene. Recent diffusion‑based methods have demonstrated strong potential for high‑quality object removal; however, they still face two key limitations. First, although they can synthesise sharp and high‑resolution images, they often struggle to complete removed regions in a semantically consistent and contextually meaningful manner, where the generated content should agree with the surrounding scene layout and object relationships rather than merely appear visually realistic. Second, their iterative denoising process makes them computationally expensive, limiting their practicality for downstream applications. We propose SERO, a semantic prediction framework for single‑step object removal. Our model predicts a hidden‑region semantic prior in embedding space, which guides a latent bridge‑matching generator toward plausible scene completion by providing both where to edit and what structure is likely to exist behind the removed object. This enables visually realistic and contextually coherent completions in one forward pass. Experiments on the RORD and OBER datasets show that SERO achieves competitive or state‑of‑the‑art performance while requiring only a single generation step.
Authors:Mani Amani, Hesam Mojtahedi, Reza Akhavian, Nikolay Atanasov, Ki Myung Brian Lee
Abstract:
Fast motion planning requires both sample‑efficient exploration and computationally efficient local steering. Conventional straight‑line steering is inexpensive but wastes samples by rejecting colliding extensions. Quadratic program controllers with control barrier function (CBF) constraints can be used to achieve effective local steering but can increase the planning time due to the expensive sequential evaluations. We develop a method to accelerate CBF steering by computing a time horizon within which CBF‑filtered velocity may be applied constantly using closed‑form kinematic bounds on collision‑clearance. Our method, FLASH‑CBF‑RRT, significantly reduces the number of evaluations compared to prior CBF‑RRT approaches because the planner can take adaptive steering step sizes that are shorter near obstacles and longer in open space. We implement this steering procedure as an extension of RRT‑Connect in OMPL. Across 689 MotionBenchMaker problems, FLASH‑CBF‑RRT matches RRT‑Connect's 100% success rate while reducing median planning time by 3.82x, and is 3.88x faster than fixed‑step CBF‑RRT. Hardware experiments on a 17‑DoF mobile manipulator demonstrate 2.2x faster planning and 4.2x fewer evaluations. Videos and results are available at: https://mamani828.github.io/FLASH‑CBF‑RRT/.
Authors:Danilo Augusto Sarti
Abstract:
lepidoptera is an open‑source R and Python package for simulation from structural causal models with heterogeneous variable types. The package is built around a typed mechanism architecture, which separates the causal graph from the generating rule at each node and exposes each rule as an inspectable, serialisable object. A pre‑built mechanism library covers all nine combinations of continuous, categorical, and ordinal parent and child types, removing the need for users to implement type‑specific sampling from scratch. A vine copula mechanism estimates the full joint dependence structure from observed data using pair‑copula constructions and the Rosenblatt conditional‑simulation algorithm, supporting over twenty copula families. Individual‑level counterfactual queries are supported through partial propagation, which freezes the random draws of unaffected nodes and replays only the downstream consequences of an intervention, thereby separating the individual treatment effect (ITE) estimand from the population‑level average treatment effect (ATE) without additional model assumptions. An extension layer allows any pre‑fitted predictive model to serve as a generative mechanism via a unified callable interface. Three worked illustrations spanning clinical trial planning, emergency‑ward capacity analysis, and epidemiological mediation analysis demonstrate the package's scope.
Authors:Magnus Sesodia, Juil Sock, Philip Torr, Christian Schroeder de Witt
Abstract:
Solving algorithmic tasks with many sequential steps typically requires scaling up transformer depth or chain‑of‑thought length, incurring unfavorable parameter and inference‑compute costs, respectively. Looped transformers, which reuse one block of layers across many iterations, are poised to address this, since they scale effective depth without adding parameters or tokens. Yet, it remains unclear what each iteration is actually computing, or whether the loop merely simulates a deeper untied stack. Here, we study directed permutation traversal, in which a model follows H hops from a starting node and reports the node it reaches, so every intermediate hop has a known ground truth. We find that a minimum effective depth is necessary to solve the task, beyond which the degree of weight tying becomes the determining factor. For instance, a looped transformer with a four‑layer block iterated eight times solves the task in every seed, while an identically‑trained untied transformer of the same effective depth never does, succeeding only with a lower learning rate or supervision of its intermediate layers. Probing and intervening on the residual stream, we find that the looped model advances through the intermediate nodes gradually, forming a staircase that adapts to its iteration budget, whereas the untied model, at that lower rate, exposes them only in its final layers. Taken together, our results recast looped transformers as iterative program learners, which allocate their iterations to successive steps of the algorithm and solve tasks that identically‑trained untied models of the same effective depth do not. Code is available at https://github.com/magnus‑sesodia/insidetheloop.
Authors:Zheng Qu
Abstract:
Topic models learn gene programs from single‑cell RNA sequencing (scRNA‑seq) data, and the same counts are then used to test which genes and donors are associated with those programs. Count splitting is the standard remedy for this reuse, but it has not been tested when an encoder reads each cell's counts. On a 104,400‑cell lupus cohort with 300 permuted null genes per run, naive tests rejected 36% of null gene‑topic pairs at a nominal 5% level, and the usual two‑way split rejected 38%, because the encoder sees the tested gene's own held‑out count. Encoding topic usage on a separate third of the counts, with beta‑binomial thinning, lowered this to 7.7% (6.7% for genes outside the model); for two other models, LDVAE and scETM, the rate was 7.0‑7.3% (naive 51‑59%). A label‑free interferon program shifts in held‑out counts in the lupus cohort and in a pre‑specified COVID‑19 cohort, and donor‑level topic usage predicts disease (AUROC 0.96) beyond cell‑type proportions (0.86 and 0.58 in the two cohorts). Calibration and prediction did not depend on the model. We do not claim better clustering, prediction or topic stability. Code, figures and result tables are public (MIT license; DOI 10.5281/zenodo.23307193).
Authors:Mustafa Umut Ozbek, Khalil El-Khatib, Li Yang
Abstract:
Machine‑learning‑based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording‑associated differences as well as attack behavior. We introduce a recording‑aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden. On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack. Same‑recording evaluation reverses the apparent modality ranking across all studied window sizes: the physical view becomes the strongest individual modality, while cyber‑only fusion shows the largest difference between cross‑recording and same‑recording AUC, reaching 0.54. Operational analysis further shows that permissive event‑overlap metrics can overstate detection when a small number of long alarm episodes cover multiple actions. An external analysis on HAI shows smaller and reference‑dependent recording sensitivity. These findings show that cross‑recording benchmark performance should be interpreted alongside same‑recording, forward, temporal, and operational evaluations.
Authors:Ella Miray Rajaonson, Jungyoon Lee, William Chau, Alan Aspuru-Guzik, Benjamin Sanchez-Lengeling, Dominique Beaini, Kirill Neklyudov, Marta Skreta
Abstract:
Designing small‑molecule drugs that selectively bind to a target protein while avoiding unintended interactions with other proteins is essential for reducing side effects that contribute substantially to clinical attrition. Although structure‑based drug design (SBDD) has advanced rapidly with deep learning, most generative models still optimize binding to a single target and disregard selectivity due to limited training data and unknown off‑target interactions. Motivated by this gap, we introduce SELECTBENCH, a comprehensive benchmark assessing the selectivity of generative SBDD models. SELECTBENCH provides three curated evaluation tasks reflecting different real‑world drug discovery scenarios: literature‑based case studies, safety screening panels used in pharmaceutical development, and large‑scale structure‑based evaluation sets. We further introduce a suite of metrics to quantify selectivity and conduct a systematic evaluation of nine representative SBDD models, including methods explicitly trained for selectivity and widely‑used inference‑time guidance techniques. Our results show that even selectivity‑trained models achieve only modest off‑target discrimination and inference‑time guidance provides limited gains, underscoring the need for method improvement. SELECTBENCH provides both a characterization of where current generative SBDD stands on selectivity and an open, extensible platform to accelerate progress. The benchmark is available at https://github.com/lovetaro/selectbench.
Authors:Peter Cho-Ho Lam, Ziyi Wang, Zirui Zhou
Abstract:
Prefix‑tuning is a parameter‑efficient fine‑tuning method for large language models (LLMs) that steers model outputs by prepending task‑specific prefix representations to the input sequence. Despite its practical success, the theoretical understanding of the expressiveness of prefixes remains incomplete. In particular, the following question remains open: Given a pretrained autoregressive Transformer, for any input‑output sequence pair (\mathttx,\mathtty), does there always exist a prefix \mathcalS such that prepending \mathcalS to \mathttx induces the model to generate the target output \mathtty? In this work, we prove that under the assumption that \mathtty is in the range of the pretrained Transformer, such a prefix \mathcalS exists and can be explicitly computed by solving multiple linear systems. We further show that this assumption is necessary, in the sense that such a prefix may not exist when the assumption fails, thereby providing a complete answer to the above question. Our analysis is constructive and the obtained theoretical results hold under fairly general assumptions on model architecture. Extensive experiments on GPT‑2 and Qwen3‑4B demonstrate that our algorithm produces prefixes with strong control over model outputs.
Authors:Dongzhihan Wang, Yi Du, Chen Wang, Xiuxian Li, Liang Xu
Abstract:
Vision‑Language Navigation (VLN) requires long‑horizon instruction‑consistent planning and short‑horizon local planning in current geometry. Video‑action models (VAMs) offer a promising way to learn predictive representations for navigation. However, existing VAMs jointly predict visual futures and local trajectories in a single process, even though they operate at different paces: future prediction remains stable across multiple updates while local trajectories must react to fresh observations. To bridge this gap, we introduce HAVANA, a dual‑system VAM that decouples predictive reasoning from reactive local planning. A slow system produces a compact language‑conditioned latent plan from observation history, while a fast system combines the latest plan with the current observation to generate local trajectories. Their asynchronous execution enables rapid adaptation without repeated future prediction, and no future video is generated during inference. Experiments on OmniNavBench demonstrate state‑of‑the‑art navigation performance over prior baselines, while real‑world deployments further show robust generalization to unseen environments across different robot embodiments. Our code and checkpoints will be available at https://github.com/Anastasiawd/HAVANA.
Authors:Youngmin Kim, Seoyeong Hwang, Sujung Hong, Seong Jae Hwang
Abstract:
Sign language production (SLP) aims to synthesize expressive signing motion from written text. Modeling signing motion presents two key temporal challenges. First, fine‑grained hand articulations demand higher temporal resolution than relatively slower body movements. Second, signing events unfold over variable durations, calling for adaptive temporal scheduling during generation. To address these challenges, we first introduce a Multi‑Rate Vector Quantizer (MRVQ), an articulation‑decoupled motion tokenizer that represents the body and both hands as separate discrete streams at distinct temporal resolutions. Building on this representation, we propose BLANC (BLock‑wise Adaptive diffusioN with Articulatory Clock), a gloss‑free SLP framework that generates variable‑duration motion blocks causally while denoising tokens in parallel within each block. At its core, a Latent Articulatory Clock (LAC) learns temporal structure from multi‑scale motion‑derived partitions to predict block durations and sequence termination during generation. We further introduce inference‑time body‑conditioned hand refinement to improve coordination across articulators. Extensive experiments on PHOENIX‑2014T and CSL‑Daily demonstrate that BLANC achieves state‑of‑the‑art performance in sign language production.
Authors:Peisong Wang, Feixue Liu, Yize Meng, Bowen Liu, Ruoyu Chen, Gaoyang Dai, Xing He, Aiqian Li, Fengchi Yuan, Aochuan Chen, Chenyi Zi, Yuhan Li, Hongchuan Zeng, Yimo Yan, Nuo Chen, Jie Fu, Zuo-Jun Max Shen, Yong-Hong KUO, Jia Li
Abstract:
Can large language model (LLM) agents deliver expert‑level operations research (OR) solutions directly from original client materials, without prior expert reformulation? Existing OR benchmarks typically begin with self‑contained descriptions or predefined optimization tasks, underrepresenting the translation of original business materials into professionally validated solutions. We introduce OR‑License, an end‑to‑end benchmark of 31 instances across 10 industrial scenarios from paid OR engagements totaling USD 3.68 million. It preserves original requirements, input data, deliverable formats, and information boundaries after de‑identification. To study information acquisition, we introduce ORSim, a controlled client simulator that discloses business facts without supplying OR expertise. Hidden evaluation checks feasibility and compares objective quality against original expert solutions accepted by clients before benchmark construction. Deployment‑Adjusted Quality (DAQ) averages expert‑relative quality across instances, assigning zero to infeasible deliveries. Across 11 frontier LLMs using coding‑agent harnesses, several exceed the expert reference in conditional objective quality, but none matches its DAQ; the strongest primary Claude Code configuration achieves 88.39% strict feasibility and a DAQ of 0.87 against the expert reference of 1. Client information acquisition extends beyond clarification into solution development and validation, with some later stages showing higher confirmed issue‑resolution proportions. Trajectory analysis identifies formulation and implementation failures, while task‑equalized research‑chain completion rates range from 24% to 45% across models on evaluable tasks. These findings motivate sustained information‑gap resolution, faithful problem formulation, and evidence‑linked self‑improvement for reliable long‑horizon agents.
Authors:Seth K. Asante
Abstract:
QRecoupling.jl is a Julia package for numerical, exact and symbolic evaluation of classical and quantum recoupling coefficients and generic finite sums of quantum factorials, with one interface across the classical limit, principal unitary levels and real or complex deformation. A compact factorial rule and its cyclotomic exponents expose factor cancellation, polynomial structure and level support. The package covers quantum Clebsch‑Gordan coefficients, 3j and 6j symbols and user‑defined sums. Its adaptive numerical evaluation is combined with exact decisions for sums that cancel to zero. Classical 3j and 6j symbols are evaluated at a speed comparable to that of WIGXJPF, which is based on prime factorization tables, at the smallest spins, and about 6 and 10 times faster at spin cutoff 500. Under real, root‑of‑unity and complex deformation, the largest relative error measured against independent high‑precision references is 3.5 × 10^‑14 for spins up to 400. Fusion matrices with 401 channels are computed about 100 times faster than by loops over their entries.
Authors:Yujie Lu, Da Yan
Abstract:
Graph‑level anomaly detectors are trained only on normal graphs, yet benchmarks may use test labels for model selection. We examine whether UB‑GOLD results reused by later papers support label‑free detection. We audit UB‑GOLD and 14 GLAD code bases, re‑run 21 detectors on UB‑GOLD's folds under selection rules with and without test labels, and evaluate sensitivity to known structural changes. Test labels select checkpoints in UB‑GOLD's shared training loop and checkpoints or hyperparameters in 8 of the 14 code bases. UB‑GOLD also evaluates its kernel baselines against misaligned labels. Across nine datasets, selecting checkpoints by test AUROC adds an average of 5.12 AUROC points [3.97, 6.24] for the 12 UB‑GOLD detectors relative to the final checkpoint and 4.13 [3.32, 4.79] relative to held‑out normal graphs. In a post hoc permutation analysis with no truly better checkpoint, picking the best of noisy test evaluations still yields a gain more than half as large as the one observed against the final checkpoint. Configurations tuned on test labels score a further 1.53 points [0.64, 2.42] above repository defaults, a gap that mixes the value of test labels with that of tuning itself. With tuned hyperparameters and label‑free checkpoint selection, the two strongest detectors retain their lead, while comparisons below them are less stable. Six of eight detectors evaluated out of sample miss size‑preserving composition changes that graphlet counts detect. We recommend reporting results selected without test labels as the main comparisons and release per‑run outputs that allow verification without retraining.
Authors:Elena Raikova
Abstract:
Isotonic regression finds the monotone sequence closest to the data in weighted least squares. It is used to calibrate probabilities and inside differentiable ranking. Its standard algorithm, pool adjacent violators, is sequential within a sequence, and torchsort, the exact GPU solver of differentiable sorting libraries, runs it with one thread per sequence, which is slow on long ones. The fit is the slope sequence of the lower convex hull of cumulative sums, and hulls of sorted points can be merged in parallel. Merging partial hulls level by level, however, copies the surviving vertices at every level, which moves Θ(n \log n) data when most points survive. We store each chunk's hull once and represent a merged hull as a list of segments of these chunk hulls, at most one per chunk; a merge then updates the list. We prove the representation correct, bound the work and depth of the merge phase, and measure a CUDA implementation on four NVIDIA GPUs against a control that copies the surviving vertices with the whole GPU at every level. On dense hulls the merge phase is 17 to 276 times faster with segment lists and the whole operator 3.9 to 19 times; on noisy data, where few vertices survive, the difference is small. In a differentiable soft rank, replacing torchsort's isotonic solver makes forward and backward 2.3 to 230 times faster on a T4, and the training step of a compact model with a Spearman loss 1.6 to 9 times.
Authors:Parsa Rahimi, Azim Dehghani Amirabad
Abstract:
Internal signatures could support monitoring, adaptive computation, and mechanistic analysis, but benchmark prompts can be separable without any shared computational distinction. We test whether a prespecified knowledge/reasoning label defines a reusable internal signature in five contemporary 7B to 31B models. We treat the task as the unit of inference probes hold out complete tasks, labels are permuted across tasks, and the best layer and stream are reselected for every permutation. Raw probes recover an eight‑versus‑eight‑subject MMLU partition in all five models (macro‑F1 0.78‑‑0.81), but prompt text predicts it nearly as well: under an identical probe configuration, the activation advantage is only 0.017‑‑0.070 macro‑F1 and is not significant. At the prespecified 0.01 level, lexical adjustment removes significance while the adjusted states still identify individual subjects, and the selected site changes across task panels and selection rules. Additionally, two released base‑training trajectories show no common direction. These pooled layerwise measures recover the task label but identify no transferable knowledge/reasoning signature at the tested sensitivity, which cannot exclude moderate effects. The resulting protocol gives future internal monitors and intervention studies a sequence of tests to pass. Project page: \urlhttps://parsa‑ra.github.io/lmbehave/.
Authors:Iori Azuma, Hiroyuki Kusuhara, Tadahaya Mizuno
Abstract:
Deep learning has achieved strong performance in computational pathology, but explaining whole‑slide image (WSI) predictions remains challenging. Existing explanation methods often highlight important regions without identifying their morphological patterns, leaving a gap between patch‑level computational evidence and slide‑level pathological interpretation. We propose ProtoGTX, a graph‑based framework that combines spatial contextualization with a flexible morphological prototype vocabulary. ProtoGTX separates the reference prototypes used to define morphological concepts from the predictive model: the reference vocabulary can be generated from training data by unsupervised clustering or supplied externally, and remains fixed during classifier training. Patch features are first contextualized over a spatial graph, after which learnable prototypes representing morphological concepts are optimized through cross‑attention. Their slide‑level activation profile is aligned with the morphological composition induced by the fixed reference prototype bank. On the CPTAC and BRACS datasets, ProtoGTX outperforms the evaluated MIL, prototype‑based, and graph‑based baselines, with particularly pronounced improvements on BRACS. Qualitative analyses further show that spatial contextualization transforms fragmented patch‑level prototype assignments into more coherent tissue regions. In BRACS, prototype maps also co‑localize with pathologist annotations and regions relevant to slide‑level prediction. These results demonstrate that ProtoGTX provides complementary semantic and spatial explanations while retaining a prototype vocabulary that can be defined independently of the predictive model. Code is available at https://github.com/mizuno‑group/ProtoGTX.
Authors:Mathias Lowes, Kurt Butler, Kristine Aavild Sørensen, Klaus F. Kofoed, Sotirios A. Tsaftaris, Rasmus R. Paulsen, Steven McDonagh
Abstract:
Counterfactual cardiac anatomy can support risk modelling and intervention planning, by predicting how a patient's heart would change under altered clinical conditions. However, current shape generative models are conditional on clinical variables and disregard causal links between these variables. Thus, they cannot answer counterfactual questions about how a given subject's anatomy would change under a different clinical profile. Modelling the heart's continuous movement over the cardiac cycle requires a representation that is itself continuous in space and time. Spatio‑temporal neural distance fields provide this while separating clinical conditioning from subject‑specific shape. We place a structural causal model (SCM) over the set of clinical variables (age, sex, smoking, BMI, heart rate, systolic blood pressure), before the condition encoding of the spatio‑temporal neural distance field which represents the left atrial complex over a cardiac cycle by signed distance fields. An intervention of the clinical variables, propagated through a SCM before the 4D counterfactual, is constructed from the neural distance field. The generated counterfactuals follow dose‑response under age interventions and transforms the male distribution to closely match the female distribution under the intervention \mathrmdo(\mathrmfemale=1). By doing inverse interventions the model also approximates the original anatomy with sub millimeter precision. To our knowledge, this is the first counterfactual generation of 4D cardiac anatomy and the first integration of an SCM with an implicit neural representation as its imaging mechanism.
Authors:Patrick Wienholt, Frederik Hauke, Christiane Kuhl, Jakob Nikolas Kather, Sven Nebelung, Daniel Truhn
Abstract:
Multimodal large language models (MLLMs) answer binary clinical questions with a yes or no token, leaving the decision threshold implicit and fixed. We derive a continuous score from a single forward pass by renormalizing the probabilities of the yes and no tokens, allowing the threshold to be adapted to clinical needs without retraining. We evaluated this approach with MedGemma and Gemma‑3 (4B and 27B) on six medical imaging datasets. The default threshold was frequently misaligned with the task: for identifying normal chest radiographs, MedGemma‑27B labeled every image as abnormal, whereas a validation‑selected threshold yielded a sensitivity of 0.945 and a specificity of 0.696. Normalization achieved the highest AUROC on four of six datasets (CheXpert: 0.819 vs. 0.777 for raw and 0.743 for text‑elicited probabilities), and a single decision required 53‑79 ms on one GPU. This simple post‑hoc method turns medical MLLMs into fast classifiers with adjustable operating points. Code: https://github.com/TruhnLab/llm‑threshold‑calibration
Authors:Shenyi Li, Ziyang Zhang, Wen Wu, Ming Liu, Chao Zhang
Abstract:
Video world models learn how visual environments evolve by predicting missing or future content. However, physical‑state information can provide useful context for predicting how environments evolve. Prior work incorporates this information through different mechanisms: domain‑specific physical laws can complicate multi‑domain training, while supervision on individual frames or frame pairs gives limited attention to how physical state evolves over time. We introduce Time‑JEPA, which fine‑tunes a video world model using low‑dimensional, temporally aligned physical time series as auxiliary supervision. Physical time series connect video representations with physical state at corresponding temporal positions, encouraging the model to preserve how physical state evolves. In multiple domains including autonomous driving, robot manipulation, and human motion, Time‑JEPA improves current‑state decoding, multi‑horizon future‑state prediction, and kinematic reconstruction over matched video‑fine‑tuned controls across both condition and state‑variable distribution shifts. Multi‑domain training provides aggregate gains over matched single‑domain training in all three tasks. External evaluations further suggest that time‑series supervision improves sensitivity to physical anomalies and the representation of physical interactions, including in unseen scenarios. Aggregate gains on a second video backbone indicate that the method can be applied to more backbones. These results suggest that physical time series help video world models better understand the physical world.
Authors:Haonan Hu, Shuhao Chen, Weisen Jiang, Lizhao Gao, James Kwok, Yu Zhang
Abstract:
Long video understanding requires VideoLLMs to answer queries about videos spanning tens of minutes to hours, but such long videos produce far more visual tokens than current VideoLLMs can process. To address this issue, keyframe selectors are proposed to select the most query‑relevant frames by computing the similarity between frame and query embeddings. Existing selectors either encode frames independently without temporal context or process chunks in isolation with ranking objectives that are difficult to optimize. To alleviate those limitations, we propose FrameScout with two core designs: (i) a streaming selector architecture that leverages a sliding‑window KV cache and successor aggregation to produce temporally‑contextualized frame embeddings; (ii) a frame‑query contrastive objective that aligns frame and query embeddings in a shared space, directly training the selector to distinguish relevant frames from irrelevant ones. Extensive experiments on three long‑video benchmarks and six base VideoLLMs demonstrate that FrameScout consistently achieves state‑of‑the‑art performance. Code is available at https://github.com/Hubert‑1102/FrameScout.
Authors:Usef Faghihi, Amir Saki, Serge Jonathan Kogbeti-Wongo
Abstract:
Geo‑vision classifies vehicle‑part damage using a fixed representation combining appearance, multiscale geometry, orientation, and sampled digital topology. Connected components, holes, Euler characteristics, and spatial arrangement complement amplitude‑sensitive pooling and a supervised tree classifier. We establish amplitude preservation, axial sign invariance, and conditional operator stability. Retrospective checkpoint evaluation on 55,303 crops from 4,916 source groups yields average precision 0.7577, exceeding 0.7458 for an ImageNet‑pretrained frozen YOLO11n comparator on identical crops. The complete added structural block, comprising topology, occupancy, and component location, improves average precision by 0.00493 beyond appearance and geometry. Geo‑vision uses no external encoder pretraining and has higher AP point estimates at every matched label budget. With approximately 25% fewer task labels, its AP point estimate exceeds full‑data YOLO, although the adjusted interval includes zero. Complete serial CPU inference requires 26.98 ms per crop versus YOLO's 11.20 ms, with peak memory 185 versus 437 MiB. Independent external validation remains necessary.
Authors:Kuizhi Liu, Junying Zhang, Xindi Ma, Xiangyu Wang, Qi Jiang, Sheng Gao, Chen Gong, Jianfeng Ma
Abstract:
Vector quantization is widely used for large‑scale data compression and approximate nearest neighbor search (ANNS). Multi‑codebook quantization (MCQ) is a practical paradigm that represents database vectors with compact codes and supports efficient retrieval through asymmetric distance computation (ADC). However, existing shallow MCQs encode vectors independently within each inverted file (IVF) cluster, without exploiting local references. Therefore, we propose Spatio‑Topological Linkage Quantization (STLQ), a topological shallow MCQ method that organizes each IVF cluster as a directed‑forest with compact codes and scaling coefficients. Its basic encoding phase first quantizes vectors independently, while the linked encoding phase re‑encodes relative residuals with respect to selected reconstructed parents, reusing local reconstruction information to obtain finer representations. The corresponding linked‑ADC preserves linear scanning complexity. STLQ further compresses the scaling coefficients and directed‑forest structure to control the additional topology overhead. Extensive experiments show that STLQ achieves higher recall than the evaluated shallow MCQs at comparable bit budgets, although at a higher offline construction cost. Meanwhile, it has a competitive recall‑QPS tradeoff against deep MCQs. The code is available at https://github.com/ighxiy/STLQ.
Authors:Hoyeon Shin, Jeongyeon Kim, Yeong Jun Koh, Yeoneung Kim, Hanul Kim
Abstract:
Vision‑Language Models(VLMs)have advanced Visual Document Retrieval (VDR) by enabling OCR‑free, multi‑vector representations. However, these models generate excessive tokens to preserve dense visual details, which severely limits retrieval scalability. Existing compression methods mitigate this overhead but rely on isolated heuristics that fail to balance token importance, semantic redundancy, and coverage capacity. We propose a visual token compression framework formulated as an Unbalanced Optimal Transport (UOT) problem. Our approach unifies these three objectives into a single alternating optimization process. Furthermore, we theoretically show that minimizing the transport cost upper‑bounds the MaxSim retrieval error, ensuring ranking stability. Extensive experiments on the ViDoRe V1 and V3 benchmarks demonstrate that our method preserves fine‑grained document semantics and improves the efficiency‑effectiveness trade‑off over state‑of‑the‑art base‑lines. Code is available at https://github.com/shhhhhyy/Unbalanced‑Optimal‑Transport‑for‑EVDR.
Authors:Ritvik Garimella, Biplav Srivastava, Amit Sheth
Abstract:
There is growing interest in automating business activities with Agentic Artificial Intelligence (AI) due to latter's seeming ease of use. Never has it been easier, or costlier, to do less with more. However, little is known about when agents are preferable to established alternatives such as local computation, Representational State Transfer (REST), the Simple Object Access Protocol (SOAP), and the Model Context Protocol (MCP), particularly when development speed, performance, and operational cost are considered. We investigate this question using a controlled mathematical task that compares seven methods on a benchmark of 1,000 arithmetic expressions where semantics of operator precedence has to be preserved. We ran this setup across a native Function Calling baseline (Python library), REST, SOAP and MCP microservice architectures, direct responses from large language models (LLMs) using a Groq cloud API and five locally deployed Ollama models, and agentic systems in which five local LLMs invoke MCP tools. The deterministic methods achieve 100% accuracy with negligible latency. Direct LLM evaluation achieves at most 92.9% accuracy at over 22,000x the latency of a native function call. With MCP tools, the best agent reaches 99.9% accuracy, but incurs nearly 1,000,000x the baseline latency. These results demonstrate that LLMs approaches are poorly suited for exact arithmetic evaluation and motivate an informed architectural decision framework (ADF) for choosing an architecture based on accuracy and throughput requirements. We recognize that an actual interaction with an agent may span both reasoning and linguistic tasks. Hence our recommendation is to explore multiple architectural choices (based on task type) rather than one‑size‑fits‑all solution (LLMs). Code is available at ‑ https://github.com/Ritvik‑G/adf/
Authors:Xianmin Ye, Zhiyuan Fan, Tianju Liu, Yuhang Wang, Zhenxing Niu, Haichang Gao
Abstract:
Large image editing models increasingly infer editing intent from both textual and visual inputs, creating an underexplored safety risk when harmful intent is distributed across modalities. In this work, we formalize this split‑intent vulnerability and propose the Split‑Intent Jailbreak Attack (SIJA), which asymmetrically distributes an unsafe editing intent by specifying the editing referent in text while concealing the safety‑critical action and specification through graphical visual semantics in an auxiliary cue. To systematically study this threat, we introduce the Split‑Intent Image Editing Safety Benchmark (SI2ESBench) for evaluating image editing models under split‑intent inputs. Extensive experiments on representative commercial and open‑source models demonstrate that SIJA effectively compromises state‑of‑the‑art image editing systems, achieving ASRs of 68.28% on GPT Image 2 and 91.38% on Qwen‑Image‑3.0‑Pro. Our results expose a mismatch between multimodal editing and safety assessment: the distributed intent can be recovered to execute the requested transformation, yet its harmfulness often remains undetected by existing safeguards. To mitigate this vulnerability, we propose editing‑intent reconstruction, a defense principle that explicitly evaluates the complete transformation implied by the joint multimodal context and substantially improves rejection of split‑intent attacks. Our findings expose a cross‑modal safety gap in modern image editing systems and highlight the need to assess the jointly inferred transformation rather than individual inputs in isolation. https://github.com/yexianmin/SIJA
Authors:Bernhard Egger, Hans Furer
Abstract:
We describe the creation of a physical artwork that was performed in synergy between an artist, a computer scientist and generative AI tools. A latent diffusion model was finetuned on photographs of the artwork of the specific artist and the generative model then generated synthetic artworks in a similar style. From the generated artworks a subset was selected to be painted again with acrylic on canvas by the original artist. In addition to the physical artifact an augmented reality component was added which brings the artwork to life with a set of generative video models. Together with this paper we release part of the artwork (training data), the code that was deployed to generate the artwork as well as the augmented reality experience for arts, research and education under https://github.com/BernhardEgger/ARTimate
Authors:Siming Zhou, Bowen Ma, Ruiran Du, Pengcheng Zheng, Sheng Zheng, Dongshen Han, Zhijie Li, Wei Dong, Fan Mo, Chaoning Zhang
Abstract:
Visual navigation requires goal directed planning informed by predictions of environment dynamics. Representative navigation world model methods propose complete action candidates before predicting consequences for selection or search updates. For each candidate, predictions primarily evaluate proposed actions rather than guide its stepwise construction. To this end, we propose InJepa, an intention first planning paradigm centered on latent states. A goal conditioned JEPA proposes latent intentions, and an action conditioned JEPA continually predicts future latent states. Both operate in a shared latent space to align their predictive representations. Within each candidate, predicted consequences feed into the next intention proposal and action inference, forming a closed loop of intention, action, and consequence. This extends the world model's role beyond evaluation to leading candidate construction through latent intentions. We evaluate InJepa through closed loop image goal navigation in Habitat MP3D. The results show effective multistep navigation through intention guided action generation and consequence feedback. Experiments across encoders further support applicability across visual representations. These findings support latent intentions as the central planning object and show that world models can directly guide plan construction through coordinated reasoning about intentions and consequences. Code is available at https://github.com/zhuxun132/InJepa.
Authors:Ruochen Zhao, Jiazheng Kang, Gezi Li, Dongying Lin, Ting Bai
Abstract:
Long video understanding requires reasoning over hours of content that far exceed the context window of multimodal large language models (MLLMs). Existing approaches compress video into fixed‑length clips, yet events are fragmented and unrelated content is merged, yielding noisy retrieval. To overcome this challenge, we propose EMMA, an Event‑driven Multimodal Memory system with Multi‑Agent retrieval, which organizes memory around semantic events in three stages: (1) Event segmentation: a dual‑stream detector adaptively locates event boundaries by fusing visual semantic drift with spoken transitions, rather than fixed windows. (2) Memory construction: built on the detected events, a three‑level hierarchy of raw video, clip‑level memory, and event‑level memory stores content at progressively higher semantic abstraction. (3) Memory retrieval: a multi‑agent framework collaboratively queries this hierarchy through query decomposition, tiered search, evidence validation, and answer generation, narrowing evidence from coarse events to precise clips. Extensive experiments on three challenging benchmarks demonstrate substantial improvements, showing the effectiveness of our event‑aligned memory system. Code, prompt templates, and reproduction scripts are available at https://github.com/BAI‑LAB/EMMA.
Authors:Jing Xiong, Jiale Xie, Haoqi Yuan, Chaoyi Xu, Zongqing Lu
Abstract:
Multi‑object grasping is a fundamental yet challenging problem, constrained by data collection costs, varying spatial layouts, and the need for broad generalization. To address these challenges, we propose MoDex, a data‑efficient framework that provides the first unified solution for universal multi‑object dexterous grasping across diverse spatial layouts. MoDex accomplishes this by formulating the task as a single‑step decision problem of learning residual corrections to a reference trajectory, and introducing a spatially adaptive dual‑strategy that performs single‑shot grasping for proximal objects and sequential interaction for separated objects. Extensive evaluations show that MoDex significantly outperforms baselines in proximal configurations, achieving 61.24% success on unseen objects, and achieves high success rates in separated layouts. Further experiments demonstrate its generalization across robot embodiments, scalability and zero‑shot sim‑to‑real deployment through a vision‑based policy trained solely on simulation data. Code and pretrained models are available at https://github.com/Aurora‑Xiong/MoDex.
Authors:Hong Wu, ChaoXuan Liu, ShuJie Wang
Abstract:
GPU acceleration has made motion planning one to two orders of magnitude faster, but it has not made the three paradigms‑‑‑sampling, search, and optimization‑‑‑comparable to one another: every GPU‑accelerated planner is built around a single paradigm, so comparing across libraries also compares their collision checkers. All three, however, bottleneck on the same GPU primitive: batched collision detection and the batched forward kinematics beneath it. PrismPlan builds that primitive once, as a shared substrate of per‑sphere collision‑‑kinematics device code that every planner consumes. Because they all execute the same device code, the backend is held fixed by construction and the paradigms can be compared head‑to‑head. We run six planners over 1800 MotionBenchMaker problems on three robots (7‑‑14 DoF, one a dual‑arm whole‑body composite) at an equal 2s budget. On the 7‑DoF Panda, sampling (pRRTC) is the most reliable paradigm at 98%, while parallel search (wPASE) follows within four points at a 3‑4 times lower median latency, and the local optimizers trail well behind when made to solve from scratch. On the 14‑DoF dual‑arm composite, sampling still solves about 90% of feasible problems, while no search or optimization planner clears a third of them. Our open‑source code is available at https://github.com/XiaoyubotMotionTeam/PrismPlan.
Authors:Md Sajid, Shreeyut Maheshwari, Akshat Mishra, Ritik Mishra, Mohammad Tanveer
Abstract:
Medical vision language models have demonstrated strong potential for medical image understanding and reasoning. However, their interpretability and reliability remain limited because medical vision language tasks require precise visual grounding and clinically coherent explanations. This is particularly challenging due to complex imaging patterns and scarce expert annotations. These challenges reduce the effectiveness of conventional reasoning methods and lead to visual hallucination, defined as the generation of text that is not grounded in the given visual input, which further limits their applicability in clinical practice. To mitigate these challenges, we propose an interpretable medical inference (I^2‑Med) framework that enhances visual faithfulness during inference without additional training to improve reasoning and explanation faithfulness in medical vision language models. During text generation, I^2‑Med uses a visual guidance module to check whether each candidate word is supported by the medical image and consistent with the current reasoning. Based on this guidance, the model's output scores are adjusted to favor visually grounded words, without changing the model parameters. Extensive experiments on the OmniMedVQA benchmark, covering eight medical imaging modalities and five clinical question types, demonstrate that our proposed I^2‑Med framework generates more clinically meaningful and visually faithful reasoning chains and reduces visual hallucination under standard medical VQA evaluation metrics. The source code is publicly available at \urlhttps://github.com/mtanveer1/l2‑med.
Authors:Weijie Lyu, Tiancheng Shen, Jason Li, Yujing Wang, Xiaoyuan Wang, Haobo Yuan, Yizhou Zhao, Ming-Hsuan Yang
Abstract:
Generated videos often lose geometric consistency under camera motion: structures warp under viewpoint change, parallax becomes inconsistent, and smooth camera movement may collapse into shot‑like transitions. We argue that this failure is largely a data problem. In web‑scale video corpora, most visible motion comes from dynamic objects captured by static or weakly moving cameras, leaving only sparse supervision for viewpoint‑induced appearance changes. To address this, we propose Video Geometry Corrector (VGC), a post‑hoc video‑to‑video model trained on synthetic paired supervision. Starting from real videos with diverse camera motion and pose annotations, we construct VG‑Pairs with a synthetic paired‑supervision data generation pipeline. Each video clip is treated as a geometry‑consistent target and paired with a distorted counterpart synthesized by a pretrained image‑to‑video inpainting model using trajectory‑adaptive keyframes. VGC learns to map the distorted input to the geometry‑consistent target without modifying the source generator, and uses optical flow as an auxiliary cue to help disentangle camera motion from object motion. Experiments on pose‑conditioned and prompt‑driven benchmarks show that VGC improves geometric consistency across multiple source generators, and scaling to a larger backbone further extends the framework toward more general video refinement.
Authors:Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
Abstract:
Pixel‑space diffusion models have recently re‑emerged as a compelling alternative to latent diffusion, avoiding the lossy latent bottleneck and separately trained autoencoder. JiT shows that a vanilla diffusion transformer can generate directly in pixel space by replacing the commonly used velocity prediction (\boldsymbolu‑prediction) with clean image prediction (\boldsymbolx_0‑prediction). In this work, we revisit this choice and introduce MFlow, an adaptive parameterization that learns to interpolate between \boldsymbolx_0‑ and \boldsymbolu‑prediction across principal directions of the data, learned end‑to‑end with diffusion training. We find that the preferred parameterization varies systematically with the data representation and model capacity ‑‑ notably, increasing model capacity shifts preference toward velocity prediction. We evaluate MFlow across ImageNet, UCF‑101 video, and large‑scale text‑to‑image generation, demonstrating competitive or improved generative performance across diverse settings. In addition, we find that vanilla pixel diffusion transformers, irrespective of parameterization, retain a persistent high‑frequency deficit relative to real images. To address this, we introduce ConvRefiner, a lightweight convolutional module trained post‑hoc with an adversarial loss while keeping the diffusion backbone frozen. With only a small amount of additional training, ConvRefiner restores the high‑frequency statistics of generated images while preserving FID. Project webpage: https://soumik‑kanad.github.io/mflow‑diffusion/.
Authors:Jianyuan Zhong, Chenwei Wu, Zhaochen Yu, Xixian Jiang, Xuewei Li, Hao Xiao, Xilin Dai, Junyang Wang, Jiaru Zou, Zhiqing Cui, Boyuan Zheng, Jing Xiong, Zhen Zhang, Qiang Gao, Michael Morley, Luis Nakayama, Qiang Xu, Liyue Shen, Tao Gui, Leo Anthony Celi, Zhenfei Yin, Yingcheng Wu, Ling Yang
Abstract:
Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence‑grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi‑agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM‑Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life‑science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self‑improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI‑27B with other research agents. Preliminary human assessments show that DiscoveryRSI‑27B scores higher than GPT‑6 Luna on hypothesis importance and whether claims are plausible and evidence‑bounded. DiscoveryRSI‑27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT‑6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human‑rated improvement along research paths.
Authors:Rama AlHamidi, Rasul Khanbayov, Erchin Serpedin, Hasan Kurban
Abstract:
A planner with a fixed budget of preventive help must decide whom to help and when. Waiting reveals risk but removes intervention opportunities as higher‑risk individuals leave first. We characterize this tradeoff under repeated noisy signals and risk‑dependent attrition: in the canonical model, a length‑τ observation prefix identifies exactly the latent risk moments m_j=\mathbb E[p^j] for j=0,\ldots,2τ‑1, buying two new moments per additional period. These identify average next‑period attrition among survivors but need not identify allocation welfare or attrition among those prioritized for treatment. Our observationally equivalent constructions establish unavoidable timing regret at any sample size for a fixed observation length, while exact certificates determine whether all compatible populations share a unique optimal date. Matched experiments separate the effects of estimation, bootstrap averaging, and regret weighting. Longer observation often resolves timing ambiguity, but the optimal date may already have passed. In our experiments, bootstrap averaging and early stopping improve decisions, while regret weighting adds no improvement in pooled mean squared regret: better statistical fit need not yield better allocation welfare. The code and artifacts are available at https://github.com/KurbanIntelligenceLab/waiting‑buys‑two‑moments.
Authors:Yuze He, Bin Tan, Zelin Gao, Yujun Shen, Yong-Jin Liu, Nan Xue, Yinghao Xu
Abstract:
Video world models that generate and explore environments at scale need a persistent 3D memory of what they have produced. Plane primitives offer a natural candidate: a compact set of oriented planes captures the dominant structural elements (e.g., walls, floors, facades) of indoor and outdoor scenes, while requiring only 7 parameters per primitive. We present PPMem, a video world model that adopts plane primitives as its persistent 3D memory. PPMem augments a video diffusion transformer with a planar prediction head and a Depth Token Fusion (DTF) module: each autoregressive chunk jointly produces RGB video and planar geometry, which merges into a persistent plane memory that conditions subsequent chunks via rendered depth. The resulting memory is three orders of magnitude more compact than per‑pixel point clouds (~100K primitives, 5MB for a full scene), geometrically explicit (metric depth, normals, exportable surfaces), cross‑view consistent (jointly predicted by the shared backbone), and adaptive (refined by the merging procedure as generation proceeds). On DL3DV and RealEstate10K, PPMem outperforms prior 3D‑grounded world models in video quality while jointly producing metric‑scale depth, surface normals, and exportable planar geometry, sustaining coherence across hundreds of frames under large viewpoint changes. Project page: https://hyzcluster.github.io/ppmem.
Authors:Maxime Baelde
Abstract:
Primal‑dual methods for constrained Markov decision processes are analyzed under a two‑timescale assumption, with non‑stationarity entering through two channels. In the additive channel, where the constraint level oscillates, the dual oscillation saturates at a floor set by the moving optimum and is amplified linearly by the timescale ratio. In the parametric channel, where the constraint sensitivity oscillates, a coherent modulation carries no resonant forcing at any depth: the dual update's stiffness modulation cancels the primal update's damping modulation. Instability instead comes from a mismatch between the constraint estimate used by the primal update and the measurement used by the dual update, producing a parametric resonance at twice the pair's natural frequency. The tolerated mismatch grows with the square root of the forcing bandwidth and collapses with the square root of the dual‑to‑primal step‑size ratio. These results hold on the tangent linear‑quadratic reduction around an active saddle, for one constraint and for K constraints under a common modulation and an isotropy condition. Driving both updates from the same estimate removes the resonant forcing but introduces a stationary offset equal to the estimator's bias. In a Lagrangian soft actor‑critic agent, the multiplier locks onto the sub‑harmonic at the timescale ratios its identified linear loop predicts and a coherent cost critic suppresses the lock‑in, while a proportional (PID) term, predicted to leave the lock‑in in place, suppresses most of it through the projection of the multiplier.
Authors:Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen, Matthias Nießner
Abstract:
Video diffusion models (VDMs) trained on internet‑scale data encode a rich prior on appearance, motion, and physics that is difficult to replicate with explicit 3D rendering and costly 3D data capture. We introduce Video Diffusion Avatars (VDAs), a method that creates photo‑realistic, multi‑view consistent avatars from a single image by fine‑tuning a VDM for precise control over camera viewpoint, facial expression, and upper‑body pose. Departing from 3D morphable models (3DMMs), VDAs use a learned expression space that lifts the expressiveness ceiling of 3DMMs, e.g., for the tongue and mouth interior. However, learned expression codes exhibit heavy viewpoint bias, as our DISTRACT benchmark reveals. We show that this bias can be removed through disentanglement, enabling their use as driving signals for multi‑view consistent avatars. Camera and upper‑body pose are controlled jointly via proxy‑mesh renderings, giving independent control over expression, pose, and viewpoint. A camera‑triplet training strategy makes the approach efficient: our models train for 34 hours on a single A100 using only public multi‑view data. VDAs outperform state‑of‑the‑art single‑image 3D avatars on novel‑view reenactment, reducing face LPIPS by 29% and improving perceptual quality (JOD) by 66% on BecomingLit. A static 3DGS reconstruction probe further confirms that our generations are the most multi‑view consistent. Built on a video prior, VDAs produce effects beyond the reach of explicit renderers, including physically plausible hair dynamics, shadows, reflections, and refractions.
Authors:Siyu Zhou, Yuqin Xie, Hao Yuan, Yingjie Zhou
Abstract:
Weakly supervised anomaly detection on tabular data is an essential task in many label‑scarce applications, since tabular data is widely encountered and it is commonly observed in practice that only a limited number of labeled anomalies are available alongside abundant unlabeled samples. Most existing methods try to learn features that capture the inherent distribution patterns in the target domain. However, such features often struggle to provide a complete description of possible abnormal characteristics and are highly susceptible to overfitting the extremely scarce labels. To address this challenge, we propose a weakly supervised anomaly detection method based on universal semantic enhancement for tabular data, which fuses the universal knowledge from a pre‑trained tabular foundation model with the domain‑specific knowledge learned on the target dataset. The two types of knowledge are naturally complementary: universal semantics describe the common patterns of tabular data across domains, while the domain‑specific features capture the inherent distributions of normal and anomalous behaviors in the target dataset. A prototype‑conditioned modulation mechanism is then designed to effectively integrate the two types of features, allowing them to reinforce each other for improved anomaly detection. Extensive experiments on public datasets show that the proposed method outperforms competitive methods when only eight labeled anomalies are available in the training procedure.
Authors:Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino, Canlin Zhang, Xiuwen Liu
Abstract:
Deep neural networks demonstrate remarkable generalization despite being highly over‑parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin‑based bounds, PAC‑Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally‑selective ray coverage in the input space. Using Generalization Intervals (GIs), we quantify direction‑specific regions around training samples where the model's predictions remain stable. Our extensive empirical studies across diverse architectures and datasets reveal that training leads to progressive expansion of ray coverage, particularly along directions with low Jacobian singular values. This mechanism helps explain a large fraction of correctly classified test inputs, adversarial examples, and even mislabeled or out‑of‑distribution samples through the geometry of ray coverage rather than instance‑level memorization. By connecting input‑space curvature to predictive behavior, our framework provides a mechanistic foundation that complements existing global theories of deep learning.
Authors:Dmytro Knopov, Nadiya Shvai
Abstract:
Claims that two training methods perform equally well often rest on a non‑significant difference between two or three runs per method. We develop a protocol, with its power theory, for certifying practical equivalence in per‑run metrics such as the AUROC (area under the receiver operating characteristic curve). It rests on a two‑level (seed x item) variance model separating between‑run dispersion from item‑level noise. Equivalence within a pre‑specified margin is tested by two one‑sided tests (TOST), equivalently by a confidence interval inside the margin; a two‑level bootstrap covers composite statistics. A closed‑form formula converts a margin and a target power into runs per method; its shifted‑t refinement is calibrated against the realised two‑level process. Simulations on a 270‑cell grid and under non‑Gaussian seed laws confirm the formula within its design‑relevant range, from five runs per method. For four published runs of a joint energy‑based model, the failure to certify equivalence was largely decided by design: with the variance calibrated to those runs, two per method certify with probability near three per cent, whereas certifying 0.01 AUROC at 80 per cent power needs twelve even under exact equality. A second case study on a thousand‑seed CIFAR‑10 zoo and the MultiBERTs checks the formula against real seed laws (within 0.05 in that range) and resolves comparisons that two or three runs leave undecided. In a two‑stage design (pilot, then formula), our experiments suggest that an upper confidence bound on the pilot variance substantially reduces the undersizing seen with its point estimate.
Authors:Manohar Vellala, Soumya Banerjee, Sandip Roy, Ross Gore, Sachin Shetty
Abstract:
Implementing Security Technical Implementation Guide (STIG) controls remains a demanding operating‑system administration task. Each natural‑language security requirement must be translated into a precise and persistent configuration change. These changes span Secure Shell (SSH) configuration, authentication policy, audit rules, and kernel parameters. Existing benchmarks do not measure whether a model can perform this task on a live system. Code benchmarks evaluate isolated programs without persistent system state, while agent benchmarks largely focus on applications rather than operating‑system configuration. In this paper, we introduce HardEval, an execution‑verified benchmark for evaluating language‑model‑assisted security hardening. HardEval is built from Red Hat Enterprise Linux 8 STIG controls that fail on an unhardened baseline system. Expert‑written reference remediations and Open Vulnerability and Assessment Language (OVAL) checks serve as hidden ground truth for grading; they are never shown to the model, and each task states only the required security outcome while withholding the implementation. The model produces a Bash script, which is executed on a live virtual machine and graded from fail to pass using the corresponding OpenSCAP/OVAL compliance check. Evaluation of fourteen frontier and open‑weight configurations reveals large and uneven capability differences across security domains. A recurring failure mode is plausible security code that does not satisfy the exact compliance condition. For four open‑weight configurations, multi‑run Wilson confidence intervals preserve the observed ordering. HardEval is publicly available.
Authors:Xikang Guan, Shaocong Yang, Yanqi Bao, Can Wang, Zheng Gu, Zhengqing Chen, Kai Chen, Jing Huo, Yang Gao
Abstract:
Physical plausibility is essential for realistic video generation. Existing methods impose physical guidance before synthesis via simulations or refined prompts, but this can limit dynamic diversity, miss errors that emerge only after generation, and require repeated regeneration when inconsistencies remain. We therefore study a generate‑diagnose‑repair paradigm that preserves the original generation capability and selectively repairs physical inconsistencies afterward, raising two key questions: what to repair and how to repair? To address them, we introduce PhysRepair, an agentic video generation framework that combines causal reasoning over the observed physical failure with prior repair experience to enable localized and adaptive correction. Specifically, Spatiotemporal Causal Constraints identify the first invalid physical state and its causally affected states, producing a repair specification that guides physically plausible local regeneration while preserving unaffected content. Experience‑augmented In‑context Learning combines the ongoing repair context with relevant experience from previous repairs, allowing the agent to adapt its repair decisions across diverse physical failures and accumulate completed trajectories for future cases. Experiments on Physics‑IQ Verified across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism demonstrate that PhysRepair consistently improves the physical plausibility of generated videos over existing methods. Our project page is available at https://xxkkxxx.github.io/PhysRepair‑Website/.
Authors:Rui Liu, Lin Gu, Hesong Li, Ying Fu
Abstract:
Infrared‑visible image fusion (IVIF) aims to combine visible and thermal infrared observations for human perception and downstream vision tasks. Visible and thermal infrared observations are physically complementary but unreliable in different ways. Existing deep IVIF methods mainly improve feature interaction and reconstruction, but often lack an explicit local rule for deciding when visible appearance should dominate and when thermal responses should contribute. To address this issue, we reformulate IVIF as local Bayesian posterior rendering, where fusion is controlled by posterior action weights rather than direct modality mixing. In this paper, we propose Proposal‑Context‑Detail Fusion (PCDFusion), a visible‑anchored framework that implements this formulation through three rendering stages. PCDFusion preserves potentially useful thermal responses, updates exposure‑degraded visible regions, and renders reliable band‑limited infrared structures into the final luminance. Extensive experiments on public IVIF benchmarks show that PCDFusion improves fusion quality, target recovery, and downstream detection and segmentation performance under challenging illumination. The code is available at https://github.com/BIT‑RuiLiu/PCDFusion.git.
Authors:Lorenzo Magnino, Antonio Marino, Michael Amir, Daigo Shishika, Amanda Prorok
Abstract:
Planning while moving faces an historical trade‑off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open‑loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state‑dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi‑Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement‑learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground‑robot navigation, agile quadrotor flight, and contact‑rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed‑budget baselines. https://lorenzomagnino.github.io/learning‑robot‑metareasoning‑review/
Authors:Juho Jung, Seongjae Kang, Jinyoung Han
Abstract:
Existing multi‑task active learning (MTL‑AL) methods predominantly rely on linear scalarization of heterogeneous objectives such as uncertainty, diversity, and gradient conflict. We argue that this entangled formulation is fundamentally limited, as linear combination fails to capture Pareto‑optimal trade‑offs under conflicting objectives. We propose a shift from combination to deconstruction and introduce a principled framework that rethinks MTL‑AL along three orthogonal dimensions. First, we identify the Conflict Paradox: although the optimization literature treats gradient conflict as interference to be suppressed, penalizing conflict when ranking candidates is measurably worse than not scoring it at all. We therefore decouple the two concerns: the acquisition function does not score conflict, and the resulting interference is resolved by gradient surgery during training. Second, we introduce Orthogonal Decomposition, separating acquisition into Informativeness and Exploration. This two‑stage process, informativeness filtering followed by coverage maximization, prevents "diversity dilution" by excluding uninformative outliers before enforcing diversity. Third, we propose Cross‑Task Mutual Information (CTMI), a covariance‑based statistic of cross‑task dependency computed from MC‑dropout task entropies. Extensive experiments on COCO, Cityscapes, and NYUv2, including comparisons against recent MTL‑AL and data‑selection baselines, demonstrate consistent improvements over state‑of‑the‑art methods, highlighting that structural rethinking of acquisition yields larger gains than incremental heuristic design. Code is available at https://github.com/Juho‑Jung/multitask_active_learning_official.
Authors:Piper Powell, Robert Franz, Amos Smith, Isaak Ihorst, Joachim Hertzberg, Martin Atzmueller
Abstract:
Research and development into artificial intelligence (AI) and machine learning (ML) applications in crop management are hindered by a lack of suitable public datasets. Whereas applications in disease and pest detection benefit from a comparatively higher amount of available data, applications like water‑deficit stress detection often have very few if any public datasets available. Moreover, most datasets that are available are single‑modality and frequently RGB‑only, leaving the early detection and enhanced performance offered by other methods, and particularly by multimodal approaches, out of reach. Datasets with accurate metadata, higher numbers of plants, and smaller intervals between data points are also scarce. To address this, we publicly release ADA 2025, a >500 GB comprehensive multimodal dataset of young sugar beet plants under healthy and water‑deficit conditions combining thermal, hyperspectral, RGB, and depth imagery along with relevant environmental and metadata and both relative water content (RWC) and leaf thickness as ground truth sources. Exemplary results showcase the dataset's validity in terms of differences detectable between stressed and healthy plants.
Authors:Manthan Gadhia, Alexander Hoyle, Karolina Stańczak
Abstract:
Post‑training is widely assumed to reduce the social biases language models inherit from pretraining, yet direct evidence for this assumption remains mixed. In this paper, we examine how gender disparities evolve across the language modeling pipeline. In particular, we study occupational gender bias in OLMo 3, a fully open language model family, across its post‑training stages by prompting each checkpoint to generate open‑ended narratives about synthetic personas given their occupation and education. We consider two settings, a gender‑given setting in which the profile states the persona's gender explicitly and a gender‑assumed setting in which it must be inferred by the model. We find that when gender is stated, post‑training increases semantic diversity for male‑coded profiles, whereas for female‑coded profiles the increase is smaller after SFT and not significant after DPO and RLVR. When gender is assumed by the model, post‑training affects narratives about female and male personas similarly. The model's own gender assumptions, however, show a consistent female‑preferential skew relative to United States labor‑force demographics, except for heavily male‑dominated occupations. Every successive post‑training stage monotonically increases the odds of attributing a female label over the base model. A provenance analysis of the supervised fine‑tuning corpus finds only a modest 7.7% female‑token surplus, too small to explain the size of these downstream effects. The project code is available under: https://github.com/manthangadhia/llm‑lifecycle‑bias.
Authors:Yan Zeng, Changlu Guo, Oskar Kristoffersen, Anders Nymark Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl
Abstract:
Visual counterfactual explanations (VCEs) should alter a classifier's prediction while remaining realistic and preserving decision‑irrelevant content. Image‑space methods provide high‑quality counterfactuals but are computationally costly, whereas latent‑space methods are more efficient but often less faithful. We identify two sources responsible for this gap. First, latent generation recovers a natural sample from a perturbed representation, whereas classifier guidance pushes it toward a target class, so their update directions may oppose. Second, VAE reconstruction can alter regions that should remain unchanged. We introduce CLEAR, a latent Flow Matching framework for VCEs. CLEAR uses Flow Matching's explicit instantaneous velocity as a reference for the generative direction to decompose classifier guidance into cooperative and conflicting components. It preserves the cooperative component and adaptively attenuates only the conflicting one according to conflict strength. We further analyze conflict frequency and strength throughout generation to reveal how classifier guidance interacts with the generative process. CLEAR also dynamically identifies editable regions, constrains latent modifications, and directly preserves factual content outside them to reduce VAE‑induced changes. Across five bidirectional tasks on two datasets, CLEAR retains latent‑space efficiency while achieving counterfactual quality comparable to or better than strong image‑space baselines, with a 2.5x speedup over the strongest baseline. Project page: https://yan‑zen9.github.io/CLEAR/
Authors:Sukhvansh Jain, Anurag Maurya, Shivam Vats, Narendra Kumar Dhar, Ravi Prakash
Abstract:
Many manipulation tasks that are effortless for dexterous human hands remain challenging for robotic systems equipped with simple, task‑specialized grippers. Industrial platforms commonly rely on end‑effectors such as suction cups or parallel‑jaw grippers, yet no single mode can reliably satisfy the diverse constraints encountered across different phases of complex manipulation tasks, including pickup, reorientation, insertion, and constrained placement. In this work, we investigate whether complementary simple grippers within a heterogeneous bimanual setup can collectively realize tasks that exceed the capability of any individual gripper alone. Given a single human demonstration specifying the desired object motion, we formulate embodiment‑aware execution as a finite‑horizon optimization over arm‑trajectory assignments. At each segment, the planner determines whether single‑arm manipulation or a coordinated dual arm handover is required, jointly reasoning over heterogeneous grasp feasibility, motion constraints, manipulability, and cross‑gripper compatibility. Exploiting this formulation, we compute globally consistent assignments with explicit switching regularization. Experiments on a dual Franka FR3 platform equipped with suction and parallel‑jaw grippers demonstrate improved feasibility and reliability in tasks that are infeasible under single‑gripper execution.
Authors:Weiyang Jin, Jiwen Yu, Sihan Xu, Yuwei Niu, Xihui Liu
Abstract:
A memory test for a video world model should say whether the scene was set, whether the asked change was carried out, and whether the last frame shows the result. Current tests fold these into one number, so a low score is easy to read as forgetting. When the change is not carried out, the hidden state transition cannot appear, so the test is not really a test of memory. We introduce MIST, an evaluation that separates the change from the memory it should leave behind. The model first sets the scene while the change stays hidden, and only later is asked to show it, with leakage before that point kept under control. This separation locates the bottleneck: the model sets the scene but does not carry out the instructed change. The same gap appears in general memory evaluations. On that basis we propose M3‑Agent (Make the Missing Move), which recursively revises the details of the input instruction so that the model successfully carries out the requested instruction. Relative to the baseline, it more than triples the MIST reveal score. On the benchmarks, WRBench's score T rises by 26.1% and MBench‑T's score U by 52.7%. The method meets the problem from the execution side: it shows what the frozen model can do once the move is stated, and it sets a stricter baseline.
Authors:Julius Erbach, Dominik Narnhofer, Bernt Schiele, Jan Eric Lenssen, Konrad Schindler
Abstract:
We introduce PONDER, a method that learns in which order an image generator should fill in content and which of its earlier predictions to revise. Masked generative models construct a sample by repeatedly filling in subsets of missing variables. The order of these partial predictions determines the context available in subsequent steps and therefore affects the quality of the final sample. Common strategies are to choose the order randomly or based on local confidence estimates. Conditioned on the partially generated sample and the generator's features, our policy chooses which positions to fill next and/or which previously filled positions to mask for later regeneration, a mechanism we call rethinking. In this way, the generation process is able to adapt to the evolving context and also to revise earlier predictions. We optimize the policy using group‑relative policy gradients, with rewards that measure constraint satisfaction, human preference, or distribution quality. For the latter, we introduce FIDgrad, an image‑level reward derived from the (first‑order) change in Fréchet Inception Distance caused by adding a generated sample to the current feature distribution. With the generators frozen, learned ordering reduces MAR‑B ImageNet FID from 4.09 to 2.78 and raises the accuracy on hard visual Sudokus from 3.9% to 56.6%, surpassing uncertainty‑based greedy ordering (38.7%). Rethinking further improves these results to 2.70 FID and 96.7% accuracy, while increasing the rethinking budget of a fixed policy yields continued improvements at inference time.
Authors:Jinheng Li, Guosheng Wang, Hongchao Du, Shangyu Wu, Qiao Li, Chun Jason Xue
Abstract:
High Bandwidth Flash (HBF) has recently emerged as a promising technology that provides terabyte‑scale accelerator‑local capacity at bandwidths approaching HBM, with a potential cost‑per‑bit advantage over HBM. However, its NAND foundation brings long latency, out‑of‑place updates, and finite write endurance directly into the accelerator memory hierarchy, making its effects on performance and lifetime difficult to predict. To study this emerging design space, we present HBFSim, an LLM‑aware event‑driven simulator for heterogeneous HBM+HBF memory systems that preserves application‑level memory semantics while modeling flash‑internal behavior. HBFSim models HBM, HBF with host‑managed flash translation, and conventional capacity tiers in one simulation driven by a calibrated serving frontend. Using HBFSim, we study decoding of Llama‑3.1‑70B and Qwen3‑235B‑A22B under three workloads that differ in their input and output lengths. HBF serves data that does not fit in HBM 11.0‑‑19.0× faster than host DRAM; HBM is better used to hold the running sequences' KV cache than as a cache for HBF; and at a published peak load of Llama‑3.1‑70B, HBF lasts 2.0 to 6.9 years, in proportion to the stacks that share the KV writes. The artifact is available at https://github.com/FlashLabAE/HBFSim and will track public HBF device data as it becomes available.
Authors:Markus Fischer, Jens Meiler, Matthias Elgeti
Abstract:
Research groups that want a language model grounded in their own literature face a choice between hosted services that receive their unpublished work and self‑hosted deployments that keep the data local but give the model no access to the corpus. Munin is an open‑source framework for a self‑hosted scientific assistant that indexes a group's corpus and its citation graph, and wraps whichever open‑weight model the group can serve in an agent harness with specialised reading, search, and computation tools. We claim that the harness, rather than the specific model, is what makes the system useful on domain literature, and we ship the evaluation suite that tests this claim with the software. On 199 paired literature questions, the harness more than doubles the accuracy of the base model, from 0.387 to 0.874 (+0.487, 95% CI [0.407, 0.568]), while naive retrieve‑then‑answer scores below the unaided model on every backbone tested. Under the harness, roughly twice as many answer claims trace to a passage the model read itself as with fixed top‑5 retrieval. When the source paper for a question is removed from the corpus, the system abstains on about nine in ten of the questions it had answered correctly on the Qwen backbones and eight in ten on gpt‑oss‑20b, and it produces no fabricated local citation across 300 fabricated references. The pattern holds on three backbones from two developers, including a 20‑billion‑parameter model that runs on a single consumer‑grade GPU. With all outbound network access disabled, both Qwen backbones still exceed the published PaperQA2 accuracy. The evaluation suite runs as a single command, so an adopting group can re‑run it on its own corpus and model.
Authors:Hongcheng Gao, Alex Zhou, Jay Zhu, Zelin Zheng, Shijia Ge, Xiping Li, Di Wu, Yazhe Wang, Jianshu Zeng, Xuan Shangguan, Yongli Wang, Yexing Wan, Lingyu He, Zhiqi Jia, Sihang Wu, Xiao He
Abstract:
Agents built on vision language models (VLMs) now compete with learned policies on robot manipulation leaderboards. We ask how far a coding agent can go when it controls a robot instead of a computer, and where it stops. Our system, HexaShell, gives the agent a thin harness: one command line that turns its commands into robot motion and makes no decisions of its own. On the robot, the agent keeps doing what it does best: it reads output, writes code and runs tests. Two agents of the same model share the work. An acting agent runs each episode. After a failure, an optimizer agent reads the full record of the episode, with every command, image and message, finds the cause, and writes or fixes small programs called tools, each with its own tests. What the agents learn stays as code, and no demonstrations or training are needed. At run time, the tools use only what a real robot also has: RGB images, camera calibration and the state of the arms. They find objects by triangulation between camera views. Because these inputs are the same in simulation and on hardware, the tools move to a real robot with a different arm after only a small change for the arm. We build the tools on ten layouts of one seed per task with Codex and GPT‑6‑Astra and freeze them; the eight Open tasks share one set of tools. In RoboDojo's official evaluation on a hidden seed, 50 episodes for each of the 42 tasks, HexaShell reaches 46.9% success and a progress score of 53.2, the state of the art on the official RoboDojo leaderboard. The limit is contact, not reasoning: memory tasks reach 81.0%, but precision tasks only 32.0%. We open source the tools evolved so far at https://github.com/HexaFuture/HexaShell.
Authors:Deming Song, Chi-Kin Lam, Kefeng Li, Hui Yu, Xilin Li, Huagang Lin
Abstract:
Resampling is the standard remedy for weight degeneracy in particle filters, but in high dimensions it distorts the spectrum of the ensemble covariance in a way that is computable from the importance weights. This paper studies the inverse problem: recovering the pre‑resampling spectrum from the observed weights and the empirical spectrum alone, and using it without biasing the filter. We first show that the forward spectral law extends to random importance weights once one conditions on the weights together with the realized offspring counts, rather than on the weights alone. We then construct a regularized free‑deconvolution estimator, prove its consistency with an explicit error budget at a fixed resolution scale, and certify model mismatch through a computable forward residual. The repaired spectrum enters the filter only as the preconditioner of a Metropolis‑adjusted rejuvenation move, so that spectral error affects mixing efficiency but not the invariant target. For the full filtering cycle, we derive a geometrically summable error recursion under stable scalar dynamics and an explicit law for the cost of abstaining from repair. Pre‑registered simulations show that the inversion reduces the spectral error by an order of magnitude, that the corrected preconditioner recovers about 90% of the oracle's sampling efficiency in an anisotropic regime, and that the cost of abstention depends on when abstentions occur rather than on how many. The repair recovers eigenvalue laws, not eigenvectors, and is certified only within an explicitly stated exogenous‑weight regime.
Authors:Liang Yue, Eric A. Vance
Abstract:
What the Web community believes about toxic conversations, such as that toxicity rises in longer and deeper threads, was measured mostly with Google's Perspective API, which is shutting down; large language models (LLMs) will make the next measurements. We find that LLM labelers mistake second‑person address for toxicity, and that this one bias manufactures laws that human labels do not show. On 1.25 million news comments with crowd labels, Claude Haiku flags 12% of the non‑toxic comments that contain no "you" and 51% of those with four or more; a single crowd rater flags 6% and 10%. Deeper replies address someone more often, so under Haiku toxicity rises with reply depth (+0.93 percentage points per level), while under the crowd's labels it slightly declines (‑0.12). Rewriting "you" in the third person removes two thirds of the false rise; flagging only as many comments as the crowd does not remove it, and showing Haiku the parent comment enlarges it. All seven LLMs we test over‑flag "you", and at the crowd's flag rate, without the parent, any false rise of the open ones vanishes at a fixed number of second‑person words; Haiku also over‑flags "you" on Reddit, Gab and Wikipedia talk pages. Agreement with human labels does not reveal the bias: we prove that a fixed Cohen's kappa leaves a law nearly unconstrained. We therefore certify the law instead of the labeler. Threadsafe uses the labeler's scores to choose which comments human raters label and to predict the others, and corrects the predictions with the human labels; linear laws are estimated without bias whatever the labeler does. In simulated audits its intervals for the laws keep nominal coverage from 250 labels on, where uniform‑sample methods such as PPI++ fall short, and at 1,000 to 8,000 labels are 0.75 times as wide as those of design‑based supervised learning (DSL); a thousand human labels rule out Haiku's rising law.
Authors:Cheng Fan, Han Wu, Tingzhang Luo, Minjing Dong, Yehui Tang, Jianyuan Guo
Abstract:
Multimodal Large Language Models (MLLMs) have significantly advanced instruction‑driven perception through the "Embedding‑as‑Mask" paradigm. However, extending this capability to Remote Sensing (RS) reasoning segmentation remains exceptionally challenging under cluttered geographical contexts and extreme scale variations, often leading to Spatial Fidelity Degradation and Target Misidentification. We identify a critical yet underexplored bottleneck: the sequential alienation of the segmentation token [SEG] from its initial visual grounding within the autoregressive generation paradigm. Produced only at the end of long reasoning chains, [SEG] relies on visual evidence that has been progressively diluted by linguistic abstraction and autoregressive noise. This information decay is particularly catastrophic for RS targets with minuscule pixel footprints, where even marginal fidelity loss leads to localization failure. To address these bottlenecks, we propose GeoReason, a novel framework designed to fortify logical deduction and preserve spatial integrity. First, we introduce an Anticipatory Prior mechanism that injects the [SEG] token directly into the initial user query, shifting it from a terminal output to a primary condition. This paradigm shift transforms the token into a persistent spatial anchor, preventing information decay during long‑form linguistic inference. Second, we enhance visual reasoning via a Location‑aware Chain‑of‑Thought (CoT) that enforces coarse‑to‑fine localization using bounding boxes, coupled with a Latent Patch‑to‑Token Alignment (PTA) Loss that explicitly aligns the latent segmentation token with physical RS textures. Extensive experiments demonstrate that GeoReason consistently outperforms state‑of‑the‑art methods on the Earthreason and RRSIS‑D benchmarks. Code is available at https://github.com/Fanc0126/GeoReason.
Authors:Alberto Foresti, Giulio Franzese, Pietro Michiardi
Abstract:
Information‑theoretic measures, such as Mutual Information (MI), play a crucial role in understanding non‑linear relationships between random variables and are widely used across scientific disciplines. Yet, their use on real‑world discrete data remains challenging. Existing methods typically rely on embedding discrete data into a continuous space and apply neural estimators originally designed for continuous distributions. This process requires careful engineering for both the embedding model and estimator architecture, but suffers from issues related to high data dimensionality. In this work, we introduce InfoSEDD, a discrete diffusion‑based approach that bridges information‑theoretic estimation and generative modeling such that they can be used to compute Kullback‑Leibler divergences. Backed by Continuous Time Markov Chains theory principles, the design of InfoSEDD is lightweight and scalable and allows seamless integration with pretrained models. We showcase the versatility of our approach through applications on motif discovery in genetic promoter data, semantic‑aware model selection in text summarization, and entropy estimation in Ising models. Finally, we construct consistency tests on real‑world textual and genomics data. Our experiments demonstrate that InfoSEDD outperforms alternatives that rely on the "embedding trick". Our results position InfoSEDD as a robust and scalable tool for information‑theoretic analysis of discrete data.
Authors:Chengyang Zhang, Zhihao Wang, Zhiyu Long, Xueqian Wang, Quanming Yao, Wei Wu, Gang Li
Abstract:
As climate change intensifies, floods are becoming more frequent and destructive, making accurate flood understanding an urgent priority. However, existing vision‑language models perform poorly on flood‑related tasks, suffering from attentional bias and inaccurate descriptions of flood magnitude. To bridge this gap, we construct the first large‑scale remote sensing flood multi‑modal benchmark that supports language‑level flood understanding and pixel‑level flood segmentation. The benchmark contains 131,200 flood image patches and corresponding 443,100 textual descriptions for flood severity and evolution. Based on this benchmark, we propose FloodVLM‑Pix, a language‑guided pixel‑level flood understanding model that includes a flood prior module and a multi‑temporal flood pixel decoder to achieve accurate flood understanding. The flood prior module automatically distills sub‑region flood spatial priors and inputs priors into FloodVLM‑Pix, solving the attentional bias problem in flood language understanding. The flood pixel decoder introduces a novel general‑specific feature fusion mechanism to address performance degradation caused by interference (e.g., clouds or mountain shadows) during flood segmentation. Experiments on FloodVL‑Pix demonstrate that FloodVLM‑Pix improves both flood description quality and pixel‑level flood segmentation performance compared with state‑of‑the‑art VLMs and flood segmentation models. Our code and data will be released at https://github.com/Haomiaoyanmeng/FloodPix.
Authors:MOSI. AI
Abstract:
Recent closed‑source image generation systems can produce diverse visual content from simple compositions to information‑dense graphics, yet open‑weight text‑to‑image models still struggle with dense text and intricate layouts. To narrow this gap, we introduce MOSS‑Image, an open generation system that combines high‑fidelity visual synthesis with intent‑aware composition. First, we pretrain a 5B single‑stream diffusion transformer (DiT) from scratch with a carefully designed recipe, enabling bilingual long‑form text rendering and precise layout control. Second, we develop an intention‑aware agentic prompt enhancement (PE) module that translates diverse user requests into detailed content and layout specifications. Together, they enable MOSS‑Image to generate text‑rich and layout‑coherent images. We further introduce LongLongTextBench (LLTB) for evaluating long‑form English and Chinese text rendering, and RealLayoutArena (RLA) for end‑to‑end layout‑rich generation. MOSS‑Image achieves state‑of‑the‑art performance among open‑weight models on LLTB, with an average text coverage of 96.3%. On RLA, MOSS‑Image ranks second in the end‑to‑end system comparison, while its backbone ranks first among generators evaluated with shared enhanced prompts. We will release the model, training recipe, and evaluation framework to support further research and promote the text‑to‑image community.
Authors:Kwangryeol Park, Seulki Lee
Abstract:
Self‑supervised representation learning has advanced time series forecasting by capturing robust features from unlabeled data. However, existing contrastive and masking‑based methods often struggle to explicitly model the underlying temporal flow, either by treating time lag as noise or by disrupting the inherent temporal dynamics essential for forecasting. To address these limitations, we propose OliO (ODE‑based Linear Transition Operator), a plug‑and‑play self‑supervised learning method that learns bidirectional temporal flows as a continuous and structurally consistent dynamical system. OliO introduces a transition operator derived from linear Ordinary Differential Equations (ODEs) to align latent representations across arbitrary time shifts. By imposing a strictly upper triangular constraint on the transition matrix, we mathematically ensure numerical stability and enforce a robust polynomial boundary that effectively captures long‑term dependencies while preventing the exponential divergence typically found in unconstrained ODE‑based models. This structural constraint induces a temporally coherent representation space that preserves the underlying flow essential for accurate forecasting. Extensive experiments across various backbone architectures and benchmarks demonstrate that OliO achieves significant performance gains, providing Mean Squared Error (MSE) reductions of up to 5.75% over the most competitive grid‑searched SOTA baselines. The code is available at https://github.com/eai‑lab/OliO
Authors:Eunju Park, Uiwon Hwang
Abstract:
Large Vision‑Language Models (LVLMs) repeatedly process hundreds of visual tokens across decoder layers, incurring substantial computation and memory costs. Existing visual‑token reduction methods often irreversibly discard tokens despite depth‑varying query relevance. Although recoverable routing allows excluded tokens to return, re‑admitted tokens retain stale hidden states after bypassing intermediate decoder transformations. Thus, token recoverability does not guarantee representation readiness at the current decoder depth. Under a fixed routing trajectory at a 64‑token budget, replacing stale re‑admitted states with current‑depth oracle representations improves average RefCOCO Acc@0.5 from 18.96 to 23.18, revealing the impact of representation staleness. We propose CatchUp, a training‑free framework combining reversible visual‑token routing with Local Transition Correction (LTC). CatchUp repeatedly re‑evaluates query‑conditioned relevance over the complete visual‑token set, preserving deferred tokens for later re‑admission. LTC approximates their skipped representation evolution by transferring reference‑to‑current hidden‑state transitions from semantically and spatially related active anchors. This correction improves alignment with current‑depth oracle representations without recomputing the skipped decoder layers. Across three referring‑expression grounding datasets and six multimodal benchmarks, CatchUp achieves the best grounding performance across all eight splits and the highest average multimodal retention among evaluated efficient methods at 64‑, 128‑, and 192‑token budgets. At a 64‑token budget, CatchUp processes 88.9% fewer active visual tokens per post‑routing decoder interval, retains 92.17% of dense‑model multimodal performance, and reduces end‑to‑end latency by 8.0%. Code is available at https://github.com/pej0918/CatchUp
Authors:Ching-Heng Cheng, Chia-Ming Lee, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
Abstract:
Single‑image refocusing changes the focal plane and depth of field of a photograph after capture. It is inherently interactive: users try many focal points and blur strengths on one image, so each edit should be fast and change only the blur. Recent generative methods, however, rerun large diffusion models for every edit. We observe that the two sub‑problems differ in kind: recovering the all‑in‑focus image is ill‑posed and needs a generative prior, whereas rendering blur is largely determined by depth and the focal plane. We therefore propose FlashReFocus, which runs diffusion once per photograph and renders every edit with a lightweight network. To address the ill‑posed restoration, a one‑step diffusion deblurrer with a decoder refiner recovers fine detail efficiently while balancing fidelity and perceptual quality. For precise control, the renderer is conditioned on an explicit defocus map and uses defocus‑bias attention, which lets pixels at similar defocus share information without interference from pixels at other depths; in‑focus content stays unchanged, and editing the map yields tilt‑shift and two focal planes. To further test generalization to unseen cameras, we collect 3CReal, a three‑camera benchmark of 89 pairs. FlashReFocus achieves the best perceptual fidelity for zero‑shot bokeh rendering on EBB! and 3CReal, refocuses 148.3x faster than GenRefocus, and takes about half a second per further edit, 462x faster.
Authors:TianLiang Ma, Ting Ma, ZhongYuan Ma
Abstract:
We present LY‑GWM‑RoboCasa‑Human300, a policy that combines a Human300‑fine‑tuned GR00T‑N1.5 action generator, an action‑conditioned graph dynamics model, and a learned binary state‑reward scorer. At inference, the policy samples four action chunks, predicts the future latent features and robot state for each, and executes the candidate with the highest predicted‑state reward logit. Predictions therefore influence the actions sent to the environment. The evaluated system uses a 16‑step prediction horizon and keeps all three components frozen during evaluation. In the local evaluation on 50 RoboCasa 1.0.1 tasks in the pretrain kitchen split, with 50 episodes per task, the system records 207/2,500 successes (8.28%), compared with 181/2,500 (7.24%) in the historical self‑trained GR00T‑N1.5 baseline. The observed difference is 26 successes, or 1.04 percentage points. The atomic group improves from 141/900 to 150/900. Separately, the RoboCasa maintainers re‑evaluated the submission and reported 354/1,500 successes (23.6%) across 50 tasks, with leaderboard rates of 48.3%, 16.3%, and 3.1% for Atomic‑Seen, Composite‑Seen, and Composite‑Unseen. The local and official results are reported separately; their difference is not attributed to a model improvement. Recorded local audits identify 212,700 changed environment‑input chunks, including 28,223 in atomic tasks. We describe the actual training stages, implemented inference mechanism, evaluation provenance, and public reproduction entrypoints. These results document an active prediction‑and‑selection system and a positive historical difference.
Authors:Guodong Du, Wanyu Lin
Abstract:
Vision‑language‑action (VLA) models can learn a wide range of robotic skills, but combining separately trained skills remains challenging. The original training data may be unavailable for joint retraining, while directly merging model weights can cause interference. We propose NestedVLA, which consolidates independently fine‑tuned VLA policies without access to their original training data. Instead of merging all updates into one checkpoint, a hypernetwork embedded in the VLA generates action‑expert weight updates for a selected task. Each task's compressed vision‑language model update is stored separately, and both updates are added to a frozen base VLA to instantiate the corresponding policy. We evaluate NestedVLA by consolidating three policies fine‑tuned with reinforcement learning on MetaWorld, ManiSkill, and CALVIN. It achieves 66.7% average success, improving on the strongest model‑merging baseline by 41.1 percentage points. It performs within 1 percentage point of per‑task fine‑tuning on ManiSkill and substantially narrows the gap on CALVIN. In this three‑policy setting, total storage is 1.5x that of a single full checkpoint. Code is available at https://github.com/WanyuGroup/NeurIPS2026‑NestedVLA.
Authors:Yifan Li, Bowen Qin, Kun Zhou, Haochen Zhao, Zikang Liu, Zheng Li, Minsheng Hao, Yuhao Cheng, Haosheng Zou, Jeff Chen, Wayne Xin Zhao
Abstract:
Agentic image generation pairs an image generator with a search agent that retrieves references from the web, extending generation beyond the generator's training data. Real‑world requests, however, often concern entities that appeared only days ago. We call this setting Image Generation in the Wild. In this setting, the web offers limited visual evidence, search results are crowded with easily confused entities, and the target is rarely shown on its own. To study this setting, we build an automated pipeline that synthesizes prompts from recent headlines and keeps only those whose real search results are highly confusing, yielding requests that are hard to solve yet natural to ask. With it, we construct Savanna, a benchmark of 500 prompts across 16 subcategories. On Savanna, even harness frameworks built on the frontier models retrieve a correct reference for only about half of the prompts. We then propose Locate‑Corroborate‑Isolate, a paradigm in which the agent first pins down the target's identity through text search, then verifies candidate images against multiple sources, and finally crops a clean reference for the generator. Trained on 15K trajectories that follow this paradigm, Ranger‑8B achieves a search score of 67.2, outperforming harnesses equipped with frontier models. It also achieves the best generation quality with every generator we test, and surpasses prior agentic methods on other image generation benchmarks. We release the pipeline, benchmark, training data and model at https://github.com/RUCAIBox/Agentic‑ImageGen‑Wild.
Authors:Jiwan Hur, Dong-Jae Lee, Dongyeun Lee, Gyojin Han, Junmo Kim
Abstract:
Classifier‑free guidance (CFG) is essential for improving sample quality in visual generation, but it incurs substantial sampling overhead. Existing guidance‑free approaches halve this cost by approximating the CFG with a single model inference, but their performance is bounded by the CFG and inherits its limitations. In this paper, we propose self‑contrastive guidance (SCG), which contrasts the model's prediction on real and self‑generated samples to obtain a direct, model‑based signal that pushes generation toward the true data manifold. Integrating SCG into the standard likelihood objective via reparameterization yields Self‑Contrastive Alignment for Likelihood Estimation (SCALE), a guidance‑free training framework that produces SCG‑enhanced predictions in a single forward pass. Across class‑conditional and text‑to‑image generation on diffusion and autoregressive models, SCALE consistently outperforms CFG‑mimicking guidance‑free baselines and improves further when combined with DPO, demonstrating that the self‑contrastive signal yields gains complementary to both CFG and preference‑based fine‑tuning.
Authors:Fabian Schmidt, Karol Fedurko, Markus Enzweiler, Abhinav Valada
Abstract:
Candidate‑based end‑to‑end driving planners reason over a finite set of trajectory hypotheses. However, for sparse planners, a limited candidate set may provide insufficient support for scene‑relevant maneuvers, while fixed downstream candidate budgets do not account for prediction uncertainty. To address these challenges, we introduce Expand‑and‑Prune, a test‑time framework that jointly adapts candidate support and capacity without updating learned parameters. The framework expands the native candidate set using semantic driving intent and temporally propagated predictions and then uses uncertainty in intermediate trajectory scores to adaptively prune the resulting candidates. Across four heterogeneous candidate‑based planners and four benchmarks, Expand‑and‑Prune consistently improves planning performance. It achieves state‑of‑the‑art results on LangAuto, increasing the Driving Score from 68.2 to 75.9, while providing further improvement on Bench2Drive, Fail2Drive, and NAVSIMv2, with particularly strong improvements in out‑of‑distribution scenarios. Moreover, in its full instantiation, 72.5% of selected trajectories originate from the newly introduced intent‑guided or temporal candidates, confirming that candidate‑space expansion provides useful hypotheses beyond those generated natively by the planner. The code is available at https://github.com/iis‑esslingen/Expand‑and‑Prune.
Authors:Uriel Dolev, Idan Schwartz, Yoav Goldberg
Abstract:
Chain‑of‑thought (CoT) controllability, the ability of reasoning models to control properties of their CoT, is consequential for both their usefulness and reliable oversight. Recent work evaluates this capability by measuring whether models follow CoT constraints given in the user prompt, and reports very low controllability across models. We argue that this low controllability may not reflect an inherent inability of reasoning models to control their CoT, but instead result from naively specifying constraints in the user prompt. To test this, we introduce Self‑Instructed CoT Controllability: a complementary evaluation that places the constraint as a self‑instruction inside the model's CoT and measures its ability to follow it. We show that, under self‑instruction, models achieve significantly higher controllability than with standard user instructions. Even a generic reminder to follow the user's constraint substantially improves controllability. Moreover, effective self‑instructions can be bootstrapped into the reasoning through a simple instruction in the user prompt, enabling improved control even without access to the reasoning channel. Finally, using self‑instruction, we demonstrate in a realistic monitoring setup that CoT controllability can be leveraged for monitor evasion, posing a risk to CoT monitorability.
Authors:Md Sabbir Hossen, Anichur Rahman, Pabon Shaha, Airin Afroj Aishi, Andrew H. Sung, Md Shohel Rana
Abstract:
Small‑to‑large language model routing can reduce inference cost by escalating difficult inputs while retaining easier ones on a smaller model. Its success, however, depends on identifying examples that benefit from escalation, and the reliability of confidence across multilingual tasks remains unclear. We present a systematic evaluation of confidence‑based routing across three model families, four languages, and three task types, covering 72 model, dataset, language, and scale configurations and 42,288 predictions. We compare global confidence routing with random routing, equal‑quota confidence routing, and an oracle upper bound. Confidence identifies beneficial escalations on the Belebele reading comprehension and SIB‑200 topic classification datasets, with mean correction‑detection AUCs of 0.612 and 0.705. At 50% escalation, it improves over random routing by 1.40‑‑4.67 percentage points on Belebele and 2.54‑‑5.63 points on SIB‑200. In contrast, the mean AUC on the MMLU‑ProX‑Lite knowledge and reasoning dataset is only 0.510, and routing provides no consistent improvement over random selection because small models often remain highly confident while incorrect. Global confidence routing also creates language escalation gaps up to 30.33 percentage points, whereas equal‑quota routing removes this disparity with little aggregate accuracy cost. These results show that confidence routing is task‑dependent and can allocate computation unevenly across languages. Code is available at https://github.com/sabbir5622r/multilingual‑llm‑routing
Authors:Wangze Xu, Chu Zhou, Mae Yamaguchi, Linrui Dai, I-Chao Shen, Takashi Ijiri, Imari Sato
Abstract:
Jointly reconstructing RGB and X‑ray provides new insights by connecting external appearance with internal structure, enabling a more complete understanding of real‑world objects than single modality. Yet existing methods train separate models per modality and suffer from redundant geometry, alignment artifacts, and loose coupling that limits cross‑modal insight discovery. We propose a unified representation in which all Gaussian primitives share a single set of geometric parameters, with only surface‑proximal primitives additionally carrying appearance attributes. We perform cross‑modal registration through clean multi‑view stereo and X‑ray geometric proxies instead of aligning two sets of optimized Gaussians, improving alignment stability. We then jointly optimize the unified representation via supervision from both modalities, complemented by a surface‑depth consistency loss derived from X‑ray geometry to guide RGB reconstruction. Experiments on the InsideOut dataset and our Flower & Insect dataset show that our framework achieves more robust cross‑modal alignment, higher RGB rendering quality without compromising X‑ray reconstruction, and a more compact representation than maintaining two independent geometries.
Authors:Haonan Han, Yukang Lin, Minliang Lin, Haoxian Tan, Jun-Yan He, Rui Yang, Zhi-Qi Cheng, Jie Hu, Guanglu Wan, Xiu Li
Abstract:
Making a conference poster is visual coding in miniature: an entire paper must be compressed onto a single page, which exercises summarization, visual design, and code implementation at once, using nothing beyond the paper itself. Yet benchmarks for this task in the context of visual coding remain scarce: existing resources are small and annotated only with layout boxes, so evaluation must fall back on VLM‑as‑judge or human checklists rather than objective render‑and‑compare, and the generation methods they support design layouts from scratch, leaving human‑crafted posters unusable as either a retrievable design library or as verified supervision. We introduce RAP (Reconstruct A Poster), a dataset and benchmark for paper‑to‑poster generation as a visual coding task. RAP contains 25K+ verified paper‑poster pairs from various machine learning conferences held between 2022 and 2026, annotated with a three‑level content taxonomy, plus a held‑out benchmark of 150 test papers each paired with its human poster. A fully automated derendering pipeline turns each poster into styled HTML/CSS of typed, refillable slots, so every sample is a paper‑poster‑markup triplet verifiable by rendering against the source poster. The corpus serves both inference and training: PosterRAG, a training‑free multi‑agent pipeline, retrieves an exemplar and refills its slots without touching layout or styling, while an SFT+RL recipe trains a dedicated model from the same triplets. An online demo is available at https://reconstruct‑a‑poster.github.io/.
Authors:Yuki Otsuka, Daisuke Oba, Naoaki Okazaki
Abstract:
Drifting Models generate a sample with a single function evaluation, making them attractive for fast parallel language generation, but their one‑step formulation does not support iterative refinement. We propose Recurrent Language Generation via Drifting (ReLaDrift), which uses a shared Drifting Model to generate an initial draft from noise and iteratively refine it in a continuous latent space. We train this recurrence by sampling the number of rounds and applying the drifting loss only at the final round, stopping gradients through preceding rounds so that only one round requires backpropagation. Unlike the one‑step Drifting Model, whose accuracy does not improve when iterated, ReLaDrift enables a trade‑off between test‑time computation and generation quality, as diffusion and flow language models do. On Sudoku Hard and GSM8K, it achieves competitive or superior performance compared with these iterative‑generation approaches across a range of inference budgets. Our code is available at https://github.com/Yuki‑Ots/ReLaDrift
Authors:Xiaoyong Li, Haolong Li, Dongliang Xu, Haoyu Li, Yifan Wang, Jingyuan Shi, Yanghua Wang, Yanbin Sun, Shidong Pan, Yue Yao
Abstract:
Agent skills have proven to be an effective technique for improving agent ability on specific tasks, including software engineering, natural science, and mathematics. During such a process, using agent skills often involves manually discovering and organizing skills from heterogeneous sources, while existing skill retrieval methods mainly focus on selecting relevant skills from established skill libraries. However, with the rapidly increasing number and variety of reusable agent skills, manually searching for relevant skills across heterogeneous sources is becoming impractical at scale. In this paper, we introduce the Code Skill Server (CSS), a large‑scale search engine for finding the most useful code generation skills for a target task. CSS consists of a skill server that incrementally collects and indexes publicly available code generation skills from multiple sources, with 993 skills included in the corpus used in this work. Extensive experiments across different retrieval methods and code generation benchmarks show that the best CSS configuration improves Pass@1 by up to 13.91% and consistently improves all evaluated models by at least 4.21% in the cross‑model evaluation. These results demonstrate that CSS can effectively search for and serve relevant skills, improving code generation quality and enhancing models' code generation capabilities. Code and the CSS‑993 corpus are available at https://github.com/haoyuli‑sdu/CodeSkillServer.
Authors:Qiang Yan
Abstract:
Many language‑based applications need a probability distribution over a small set of decisions rather than an open‑ended response. We present Chinese Decision, a family of Chinese‑language decision models that adapts pretrained causal language models to this interface. Each question is compiled into a shared instruction‑and‑candidate prompt; a single forward pass reads the final‑position logits of the candidate letters and normalizes them over the valid options, without autoregressive answer generation. The interface covers Boolean judgments, categorical choices, and ordinal scores. We report three completed members built on Qwen3.5 0.8B, 2B, and 4B backbones, each trained for one epoch on the same 13.048 million decisions and released with a validation‑fitted temperature and a Q8_0 quantization. On the full 307,900‑decision CJ‑Bench, the 4B model reaches 85.97% overall accuracy, 75.69% general‑domain accuracy, and 84.00% average accuracy across medicine, law, and finance with a single checkpoint, above every reported baseline in each domain, at 1.00% expected calibration error. Between the 2B and 4B members, which share data, objective, and minimum batch, parent‑clustered paired bootstrap intervals show gains in all four domains, largest in the two low‑resource domains (+8.5 and +7.1 points), whereas the 2B gain over 0.8B in law is not statistically resolved. Analyses of the stored predictions further show that a single global temperature lowers pooled calibration error while worsening some domains, that larger members rank their own errors better for selective prediction, and that merging and quantization each change 0.4‑1.1% of top‑1 predictions while leaving aggregate metrics nearly unchanged. All comparisons rest on single training runs and imported baselines.
Authors:Semra Guven, Tobias Gårdhus, Andreas Bjerre-Nielsen, Anna Rogers, Hjalmar Bang Carlsen
Abstract:
Large language model chatbots have been proposed as AI‑interviewers for collecting qualitative interview data at scale, potentially bridging social science's longstanding divide between the breadth of surveys and the depth of qualitative interviews. This paper provides the first direct empirical comparison of data quality in AI‑led and human‑led interviews. In a randomized experiment (N = 41), university students were interviewed via synchronous text chat by either trained non‑expert interviewers or a locally hosted AI system; we evaluate exposure, specificity, and relevance. Median interview duration was similar across conditions, but AI‑interviewers asked more than twice as many questions per interview and elicited roughly 65 percent more respondent text per interview. Per‑question responses were longer with human interviewers, though not after multiple‑testing correction; only AI‑condition participants skipped questions. Responses were almost uniformly on‑topic in both conditions; for specificity we detect no difference, but the design cannot exclude one of up to three quarters of a point. These results were obtained with a small, locally hosted open‑weight model (7B parameters), which addresses concerns about data security, GDPR compliance, and scientific reproducibility. Our findings support the viability of AI‑interviewing for large‑N qualitative data collection.
Authors:Donghoon Kim, Seunghyuk Oh, Kevin Galim, Minseo Kim, Minjae Lee, Younghyun Kim, Wooseok Seo, Hyung Il Koo, Wonjun Kang
Abstract:
Supervised fine‑tuning on agent trajectories is an effective way to improve coding agents, but selecting useful training examples requires more than identifying successful runs. A resolved patch may follow inefficient reasoning or unnecessary tool use, while a failed run may still contain useful intermediate steps. Training pipelines therefore rely on rules and qualitative assessments to filter trajectories, yet designing these filters typically requires substantial human expertise, and validating improvements through repeated training and evaluation is expensive. We introduce MetaCurate, a training‑free framework that evolves executable trajectory curation filters using pairwise LLM judgments. Disagreements between the filter's rankings and the judge's preferences provide targeted feedback for jointly revising its rules, rubric, and aggregation policy. The resulting filter scores trajectories independently, so pairwise comparisons are required only during search. Starting from a blank filter, MetaCurate autonomously discovers a curation strategy that achieves competitive downstream performance without task‑specific handwritten criteria. We validate downstream utility through controlled SFT experiments on our uncurated trajectory pool and the Nemotron v3 SWE dataset, with random selection and hand‑authored filtering as baselines. In our experiments, MetaCurate achieves the highest mean SWE‑Verified score among the compared methods, including hand‑authored rule‑based filtering and a training‑loop baseline whose search costs 3.5x as much.
Authors:Junyang Shu, Bingqing Wei, Jishuo Li, Zheqi He, Yongtao Wang
Abstract:
Embodied heuristic learning must balance strategic adaptation with the preservation of previously successful behavior. Local repairs may leave structural failures unresolved, whereas broader revisions can introduce regressions that are costly to detect through repeated environment interaction. We introduce SCOPE (Selective Certification Of Program Edits), which uses failed and successful executions as complementary evidence for program revision. Frontier‑guided program search annotates a shared diagnostic graph with execution evidence to distinguish supported progress from unresolved checks. Selective behavioral certification uses stateful API replay to restore recorded responses and mutations, checking whether edits preserve selected historical successes. Diagnosis and certification feedback, supplemented by sparse environment diagnostics, guide consecutive repairs across configurations. SCOPE outperforms the strongest baseline in both simulation and real‑world tasks. On LIBERO‑PRO and Robosuite, it achieves mean success rates of 80.66% and 84.48%, with respective gains of 6.51 and 3.48 percentage points. Our code is available at https://github.com/VDIGPKU/SCOPE.
Authors:Liyun Yan, Haoyang Li, Yang Zhang, Jialing Zhu, Zhanxiang Cao, Zihao Zhou, Keqi Zhu, Yizhi Chen, Huang Chuxuan, Shengcheng Fu, Zhong Jiangwei, Qiao Sun, Chen Zhang, Yue Gao
Abstract:
Humanoid control requires coordinating whole‑body posture with position and heading in the world frame. Mainstream controllers imitate trajectories in the root frame, but struggle to regain the target world pose after displacement. We introduce BeJEPA, a goal‑conditioned Behavior World Model learned through unsupervised reinforcement learning from human motion priors and robot interaction. BeJEPA learns a shared representation of body configuration and world placement through a joint‑embedding predictive architecture (JEPA). A Dynamic Predictor predicts the next state embedding under an action, while a Successor Predictor summarizes a goal‑conditioned policy's expected discounted future state embeddings in a single vector. During joint training, Dynamic grounds the initial representation in observed physical transitions, while Successor increasingly organizes it around long‑term behavior. Their composition guides policy improvement through one physical prediction and one behavioral summary, without long imagined rollouts. In 20‑second motion‑tracking experiments, BeJEPA with online action refinement using Dynamic outperforms all evaluated baselines in global body‑position error, reducing it on AMASS by 61.7% relative to BFM‑Zero with world‑pose inputs and 58.7% relative to SONIC. Physical deployment on a Unitree G1 demonstrates joint attainment of posture, world position, and heading goals, recovery from the ground, and closed‑loop return after disturbances and goal changes. Project page: https://ylyem9x.github.io/BeJEPA‑project.
Authors:Zijun Dou, Zhen Yang, Xijun Liu, Dongzhuoran Zhou, Aohan Zeng, Jie Tang
Abstract:
Real‑world frontend development requires agents to compose reproduction, generation, modification, and repair over an evolving application state. Learning a general policy for this process requires translating heterogeneous execution feedback into task‑aligned rewards and effective credit assignment across long sequences of interdependent actions. We introduce WebCoder, a general frontend development agent trained with artifact‑grounded agentic reinforcement learning. Our key idea is to assess diverse development activities through their effects on a shared executable artifact and its satisfaction of task requirements. We build Webdev‑World, an executable environment that integrates code editing, application execution, browser inspection, and on‑demand skills while preserving project state across interactions. An independent verifier evaluates the final application against task requirements to provide terminal rewards, while changes in intermediate artifact progress redistribute the resulting rollout advantage across development segments. Context compaction and message‑tree sample construction support training on trajectories beyond a single context window while avoiding repeated loss computation on shared generated history. We further introduce Webdev‑Bench to evaluate composed development requirements within a single project. On Webdev‑Bench, WebCoder‑27B raises requirement satisfaction from 64.8 for its Qwen3.8‑27B base model to 72.8, exceeding Claude‑Opus‑4.8 (65.1) and GPT‑5.4 (59.6); on Vision2Web, it improves the average visual and functional score from 41.1 to 62.4, on par with Claude‑Opus‑4.8. Ablation studies confirm the contributions of artifact‑grounded credit assignment and long‑horizon training.
Authors:Benhao Huang, Chufan Shi, Junlin Chen, Shicheng Wen, Zhengzhong Liu, Eric Xing, Xuezhe Ma
Abstract:
Looped language models reuse physical Transformer blocks across logical depth, but established architectures entangle recurrence placement, input injection, and recurrent state organization. We isolate these axes in models trained from scratch at matched parameter scale, logical depth, and token budgets, and then test whether the resulting design principles transfer to resident‑ and active‑parameter‑matched Mixture‑of‑Experts (MoE) models. The prelude‑recurrent block‑coda sandwich architecture consistently improves performance on context extraction and reasoning‑intensive tasks. Input injection primarily enhances knowledge retrieval and context utilization, whereas random initialization and separate H/L recurrent states provide mixed or negative returns. Taken together, these results favor the Huginn‑style design: at the 730M‑parameter and 336B‑token scale, the complete Huginn architecture outperforms Ouro across all ten benchmarks. The advantages of Huginn over Ouro transfer to MoE models. At the 8BA0.8B (32BA3B when unrolled) scale trained on 500B tokens, Huginn outperforms Ouro on nearly all benchmarks spanning knowledge, reasoning, and coding, while also exhibiting more balanced expert routing. Under matched training and inference FLOPs, the Huginn‑style model uses 75% fewer resident parameters than a 112‑layer feedforward MoE baseline. Despite its smaller physical parameters, it outperforms the feedforward baseline on DROP and GSM8K, matches it on MATH500, and remains competitive on the other benchmarks.
Authors:Yuanwei Wu, Lingdong Huang, Yunlong Mao, Kanye Ye Wang, Jingyu Hua, Sheng Zhong
Abstract:
Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Existing optimization‑based attacks share a single objective: maximize a token‑level proxy of victim compliance‑‑‑such as the likelihood of an affirmative prefix‑‑‑computed on the prompt rather than the decoded reply. This proxy cannot distinguish a cautious hedge from a fully actionable harmful answer, leaving the search blind to what the model actually says. We propose ECHO (Evaluation‑Conditioned Harmfulness Optimization), which replaces the compliance proxy with response‑level, continuous harmfulness scores from HarmSig, a 1.7B regressor grounded in actual harmful capability. ECHO couples HarmSig with five word‑level perturbation operators and Keyword‑Anchored Beam Search with global top‑m memory, under a strict black‑box threat model. Across five open‑weight victims on AdvBench, ECHO achieves up to 0.898 StrongREJECT mean score and 89.23% HarmBench success. Our code is publicly available at https://github.com/STANWYW/echo.
Authors:Chulhoon Yoon, Kyungri Park, Jinyeong Yun, Daeyun Jeong, Sukmin Yun, Woohwan Jung
Abstract:
Recent advances in multimodal foundation models have significantly improved the ability to operate real‑world software through graphical user interfaces (GUIs) using visual observations and natural language instructions. However, maintaining strong performance on unseen applications remains challenging, as practical tasks often require long‑horizon procedural knowledge and application‑specific operation patterns. In addition, constructing executable tasks and collecting corresponding trajectories for training still heavily rely on human‑annotated data, incurring substantial cost. In this paper, we propose MAGE (Manual‑augmented computer use AGent Execution and learning), a framework that leverages user manuals as a scalable source of procedural knowledge to improve both inference and training. During execution, MAGE retrieves task‑relevant manual content and uses a planner to generate explicit procedural guidance for CUAs. To further account for the discrepancies between manuals and actual GUI environments, such as version differences or user settings, MAGE also adapts the agent policy through environment interaction. During training, MAGE converts manuals into executable task‑plan pairs and uses the plans to guide environment rollouts toward task‑relevant states; the resulting trajectories are then used to optimize the policy without human‑authored demonstrations. This unified design enables effective utilization of manuals for both task execution and training, reducing reliance on human‑authored training data. Experimental results across six applications demonstrate improvements in task success rates over strong baselines, highlighting the effectiveness of manual‑guided planning and policy adaptation in new software environments. We provide the code and reproduction scripts at https://github.com/ku‑autoai/MAGE.
Authors:Qi Si, Xuyang Liu, Penglei Wang, Shuo Su, Limei Han, Mengzhu Li, Xin Guo, Yuan Qi, Yuan Cheng
Abstract:
Designing RNA sequences that fold into specific 3D structures is a fundamental challenge in therapeutics. Current RNA inverse folding methods typically condition on 3D backbone coordinates but the generation is limited to 1D discrete sequences. This ignores the continuous side‑chain orientations, which are essential for overall structural stability. To address this issue, we explicitly introduce side‑chain geometry into the RNA generative process. Unlike co‑design frameworks that diffuse across separate raw spaces, we propose Geometric Optimized Latent Diffusion (GOLD), which project sequence and structure into a unified latent manifold. GOLD employs a novel GeoVAE to elegantly integrate discrete RNA sequences with continuous side‑chain angles. By mapping these modalities into a single, smooth latent manifold, GOLD naturally facilitates a synergistic co‑diffusion process. To enhance structural validity, we further introduce a training‑free, gradient‑guided sampling mechanism. By decoding the generated sequence probabilities and side‑chain geometries into structures during the sampling process, we compute analytical gradients from physical constraints. This dynamic feedback actively guides the generated sequence to a physically stable state. Extensive experiments validate the superiority of GOLD, achieving state‑of‑the‑art performance across sequence recovery, secondary structure validity, and 3D foldability metrics. Our code and data are publicly available at https://github.com/darkflash03/GOLD.
Authors:Haoming Li, Defu Cao, Xu Hu, Yingtao Luo, Yiyang Wang, Xianghe Xu, Jiashu He, Wanting Jiang, Yifan Zhang, Yunhan Zhao, Siyang Wu, Chen Yang, Haixin Wang, Zheyuan Deng, Taoran Li, Jingwen Zhang, Zhejian Peng, Shushu Liang, Yue Zhao, Bingzhe Li, Feng Chen, Yifan Wang, Jintao Huang, Yijun Wang, Yuexing Hao, Xiaomin Li
Abstract:
Computer‑use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 tasks across 11 domains fixed and varies only who asks. A persona agent, playing one of 44 personas in four groups, rewrites each instruction in that user's own words and answers the CUA's questions, while the CUA sees only the dialogue, never the profile or the original instruction. Across nine backbones (3,330 runs), mean task completion falls by 36% relative to the original requests. The losses have two sources: in 59.9% of cells that lose points, the agent engages with what the user asked for and still gets it wrong, and 11.2% are stops the user requested that the checker scores as failures. Even when a persona request is fully completed, the user gives the top satisfaction rating in only 62.7% of cells, against 84.1% for the original requests. CUAs should therefore be evaluated not only on task completion but also on whether they meet the requirements and constraints users express. Code: https://github.com/MatrAIx‑ai/PersonaCUABench. Project page: https://matraix.ai/research/persona‑cua.html
Authors:Yiyang Lin, Chenhui Wang, Tianliang Yao, Pengli Zhu, Shuaibo Li, Zijun Sun, Yiming Deng, Yixuan Yuan
Abstract:
MRI field‑strength enhancement seeks to combine the accessibility of low‑field scanners with the image quality of high‑field scanners. Yet, existing methods remain prone to anatomical hallucinations, as uniform processing leaves difficult brain regions insufficiently examined and scarce paired data limit their learning of how tissue contrast and anatomical visibility change across field strengths. To address these challenges, we propose UniField++, a two‑agent framework that unifies enhancement tasks across multiple field‑strength transitions, including 64mT‑to‑3T and 3T‑to‑7T. Unified learning allows both the VLM‑based planner and the flow‑matching‑based enhancer to draw on field‑dependent image characteristics observed across tasks, rather than relying on a single transition. Trained with radiologists' annotations of hallucination‑prone regions and corresponding correction plans, the planner predicts regional risk and formulates targeted restoration strategies. Guided by these plans, the enhancer performs loop reasoning along the flow‑matching trajectory, repeatedly processing high‑risk regions for additional refinement. Together, cross‑transition learning and risk‑guided refinement support anatomically faithful enhancement across field strengths. Across the evaluated tasks, UniField++ improves PSNR by an average of 2.74 dB and SSIM by 14.05 percentage points over comparison methods, with corresponding gains of 3.97 dB and 13.92 percentage points over the ablation baseline. It also maintains enhancement quality on an external cohort with an unseen field strength and improves downstream whole‑brain segmentation Macro Dice by an average of 0.1457 over the original low‑field images. Code is available at https://github.com/CUHK‑AIM‑Group/UniField_plusplus.
Authors:Jiayuan Wang, Tongyao Pang
Abstract:
Reinforcement learning improves diffusion models using rewards on generated images, but repeated denoising and gradient estimation make training expensive. We introduce \method, an efficient method that constructs useful update directions by regressing centered rewards on centered final denoised images. It requires neither reward derivatives nor adjoint backpropagation through the sampling trajectory. A unified first‑variation formulation identifies the value‑gradient target with or without a trajectory‑KL penalty. A population analysis under Gaussian noising explains the regression geometry, while a subspace analysis motivates clean‑image coordinates. The implementation adapts rollout bandwidth and query weights to reward continuity: local rollouts suit smooth rewards, whereas broader groups provide informative contrasts for discrete rewards. Trained jointly on five rewards, one SD3.5‑Medium model matches or exceeds DiffusionNFT's joint model on seven of eight metrics, reaching GenEval 0.94, OCR 0.92, and PickScore 23.82 at about one fifth of its estimated training compute. In single‑reward training, \method reaches matched reward thresholds 2.1‑‑6.7× faster than reproduced DiffusionNFT on identical hardware. Ablations support the clean‑image estimator, direct direction transfer, and reward‑dependent rollout choices. Code is available at https://github.com/jy‑wang‑tsinghua/DIFFUSIONRVGE.
Authors:Byunghoo Park, Jayeon Yi, Minje Kim
Abstract:
Spectral losses for differentiable digital signal processing (DDSP) commonly compress magnitudes, for example by using log‑magnitude terms. Such compression mimics human perception, but whether it helps the optimization is less clear, and is usually found only by comparing trained models. In this work, we show that whether compression helps depends on the system's informative region: the region of its dynamic range that carries the information pointing toward the target. To indicate where this region lies without having to train, we introduce concordance, which measures how well a loss orders candidate parameter settings by their distance from the target. We then compare uncompressed, compressed, and combined losses by fitting a modal plate reverb and a subtractive synthesizer model to target sounds. A third system, selected for its low‑energy informative region, tests the relationship in the opposite direction. In every system, the loss that matches the informative region performs best: the uncompressed one where the region lies at high energies, and the compressed one where it lies low. Surprisingly, the common combination of both log‑magnitude and linear‑magnitude terms performs best in none of the systems. Code and audio examples are available at https://github.com/superstem000/compression‑analysis
Authors:Haoming Li, Dingyi Kang, Xu Hu, Jinxin Yang, Zhu Xu, Chenru Wang, Yi Ge, Miao Yin, Guanpeng Li, Feng Chen, Bingzhe Li
Abstract:
Large language model (LLM) agents increasingly rely on persistent memory to support long‑term reasoning and personalized interactions. However, information collected from external sources can introduce security risks when untrusted claims (i.e., atomic statements) are later treated as authorized preferences, policies, or permissions. Existing defenses typically rely on content‑based filtering or coarse‑grained provenance tracking, which either fail to prevent unauthorized memory use or block legitimate information, sacrificing memory utility. To address these limitations, we propose SAFE‑Mem, a fine‑grained memory governance framework that improves security while preserving legitimate memory use. First, SAFE‑Mem annotates each claim with three governance labels (i.e., kind, authority, and scope) to capture its provenance and permitted use. Second, governance‑aware retrieval matches query requirements with claim permissions to block unauthorized influence while retaining legitimate information. Third, a governance‑enabled memory graph connects source sentences, claims, and entities to support provenance tracking and efficient retrieval. Across four LLM backbones on MPBench and LoCoMo, SAFE‑Mem attains the lowest attack success among defenses that retain legitimate memory, at least 77% below the strongest of them, and the highest answer accuracy of all evaluated defenses, improving it by up to 15 points. The code is available at https://github.com/LVenum/SAFE‑Mem.
Authors:Khuyen Pham, Tran Nguyen Le, Edoardo Fazzari, Cesare Stefanini
Abstract:
Vision‑language‑action (VLA) models have shown strong promise for generalist robotic manipulation, but their inference cost remains a major bottleneck for real‑time control. This limitation is especially pronounced in flow‑matching VLAs such as π_0.5, where the action expert is evaluated repeatedly during iterative denoising. We introduce MambaAct, an efficient Mamba action expert for flow‑matching VLA policies. MambaAct retains the pretrained visual‑language prefix encoder of π_0.5, while replacing its transformer‑based action expert with a compact Mamba‑based action expert. Within this expert, Mamba serves as the action mixer that models dependencies across the noisy continuous‑action sequence. To preserve visual‑language conditioning without repeated token‑level cross‑attention, MambaAct uses a lightweight Context Compressor to map Vision‑language model (VLM) prefix tokens into a compact conditioning signal that gates and modulates each Mamba action block. MambaAct uses a 10× smaller action expert and reduces flow‑matching denoising latency by 64.8% on LIBERO and 58.8% in real‑world deployment, while improving mean LIBERO success by 1.08 percentage points across three independent fine‑tuning seeds and maintaining gains on LIBERO‑Plus and real‑world benchmarks. Our code and model weights are publicly available at https://github.com/giakhuyendihoc/MambaAct.
Authors:Meng Wang, Wu Zeng, Mei Li, Chao He
Abstract:
Poor illumination and occlusion obscure clothing cues in indoor work sites, while diverse clothing styles complicate workwear compliance judgments. Although vision‑language models (VLMs) support localization and semantic reasoning, their predictions often lack explicit supporting evidence, and a single confidence score cannot distinguish visual uncertainty from ambiguity in compliance judgments. We propose DICL‑Qwen3VL, a decoupled interpretable VLM that integrates structured workwear detection, evidence generation, and separate visual and reasoning confidence estimates. First, low‑rank adaptation (LoRA) adapts Qwen3‑VL to localize visible upper‑body clothing and identify workwear. Second, decoupled calibration separates visual observations from compliance reasoning and assigns a confidence score to each. Token entropy and sensitivity to image perturbations provide supervision for visual confidence, linking confidence estimation to the generated observations and their dependence on image content. Finally, group relative policy optimization (GRPO) jointly optimizes detection accuracy, output validity, and confidence calibration. Experiments on the company‑collected Indoor Construction Site Workwear Dataset (ICSWD) and the web‑collected Indoor Construction Workwear Dataset (WICWD) yield mAP@50 values of 0.730 and 0.753, respectively, outperforming the evaluated LoRA‑adapted VLM baselines, including Qwen3‑VL. The framework also provides explicit evidence and separate confidence scores to support manual review in industrial safety monitoring. WICWD is publicly available at https://github.com/DaErwen1/WICWD.
Authors:Lijia Liu, Kyohei Atarashi, Jiyi Li, Koh Takeuchi, Shunji Umetani, Hisashi Kashima
Abstract:
Formulating optimization problems from raw inputs of problem descriptions and data is time‑consuming and costly. While Large Language Models (LLMs) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete. This diverges from reality, where data‑centric reasoning is required to extract, compute, and predict parameters from noisy sources. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi‑stage reasoning inherent in practical optimization workflows. NLData2Opt decomposes problems into problem descriptions and data, and requires models to (1) extract relevant parameters from data containing misleading attributes, (2) compute missing parameters derived by transforming existing data attributes, and (3) predict unavailable future values from historical observations. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows. NLData2Opt is available at https://github.com/sumarivyn/NLData2Opt.
Authors:Haoming Liu, Junhao Zhu, Shaoyi Zheng, Yuanhe Guo, Jinnuo Liu, Shenji Wan, Hongyi Wen
Abstract:
Classifier‑free guidance (CFG) is a standard component of conditional diffusion models, and guided generation performance has become the de facto measure of state‑of‑the‑art models. Recent work on latent diffusion models seeks to learn latent spaces that are easier to model with diffusion. However, under CFG, generation additionally depends on how conditions are arranged in the latent space. We study this arrangement under discrete class conditions, termed \emphclass geometry, and investigate what class geometry allows a latent space to work well with CFG. Under a Gaussian class model, we show that shifting variance from intra‑ to inter‑class at fixed total variance makes classes more distinguishable under noise, and derive a noisy posterior objective whose per‑sample descent direction is proportional to the oracle CFG correction. Based on the analysis, we propose Prototype Regularization (PR), a lightweight surrogate for the intractable noisy posterior objective, which applies a cosine prototype classifier to spatially pooled latents. With a controlled training budget (20 epochs for VAE, 80 epochs for DiT), adding PR to an LDM baseline reduces guided gFID on ImageNet from 3.31 to 2.28, surpassing a DINOv3‑aligned VA‑VAE (2.44) with negligible overhead. We further demonstrate that PR generalizes to pseudo‑labels derived from clustering, continuous text‑embedding conditions, and video latents. Together, our findings highlight class geometry as a critical factor in guided generation and take a first step toward CFG‑compatible representation learning. Our project page is available at: https://maps‑research.github.io/cfg‑geometry/.
Authors:Zesheng Shi, Xue Li, Chenrui Dai, Jing Li
Abstract:
LLMs over external data face a per‑instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Existing defenses suppress or trust all external instructions, collapsing the safety‑utility frontier and forcing a choice between over‑refusal and over‑compliance. We argue the missing capability is per‑instruction credit attribution and formalize it as External‑Data Instruction Classification, an identify‑classify‑act decision over each embedded directive. We propose EDIC, a decomposed reward framework for per‑instruction compliance. A scene‑grounded pipeline injects mixed beneficial and harmful instructions with counterfactual filtering, while a generative judge is initialized by SFT and refined with GRPO to attribute compliance at the instruction level. The judge then produces a reward that separately encourages beneficial compliance and penalizes harmful compliance for RL training. Across two backbones and five benchmarks, EDIC improves both safety and utility over the strongest prior defense, lifting harmful‑rejection by up to 7.7% and Arena‑Hard‑Auto utility by 12.5% on Qwen3‑4B. The code repository is hosted at https://github.com/yingyingxia666/EDIC.
Authors:Bowen Ding, Qixin Wang, Lexing Ruan, Weiyi Lu, Qiwei Ge, Ruishan Fang, Tao Lin
Abstract:
LLM agents are increasingly integrated into real‑world works, yet they must handle diverse domain‑specific tools and complex multi‑sub‑task problems that sequential tool invocation resolves inefficiently and with severe context bloat. The Main‑Agent Delegation and Sub‑Agent Execution (MAD‑SAE) paradigm addresses these issues by delegating sub‑tasks to specialized sub‑agents for parallel execution in isolated contexts, but its delegation and orchestration quality remains unquantified. We introduce MS‑Eval, a unified evaluation kit comprising an MCP‑intensive benchmark, a controlled execution harness, tailored delegation and orchestration metrics, and the MSQ index for comprehensive assessment. Evaluating four MAD‑SAE variants and eight frontier models reveals that (1) sub‑agent specialization (Spec) is the best practical practice; (2) open‑source models are largely competitive with few exceptions; and (3) MAD‑SAE generally raises cost with contingent payoff, benefiting most open‑source models in both performance and efficiency while leaving closed‑source models with higher cost and degraded performance. Our case study further traces these gains to better single‑agent solutions, effective context offloading, and genuine sub‑task parallelism, whereas unnecessary or redundant delegation turns the same cost into pure overhead. Codes: https://github.com/LINs‑lab/MS‑Eval
Authors:Noé Mosca, Mohamed Rayan Barhdadi, Muhammad Hamza, Joshua Yeh, Suryodaya Bikram Shahi, Zhiyan Li, Selim-Antoine Lali, Weiyue Li, Hasan Kurban, Erchin Serpedin, Gang Luo, Yue Zhang, Minghan Li, Mengyu Wang
Abstract:
Blindness and low vision limit many people's ability to complete daily tasks, motivating AI assistants that provide useful guidance. Yet, existing benchmarks do not adequately test whether vision models can provide effective everyday assistance. In particular, i) existing benchmarks primarily rely on visual and textual inputs, with limited support for other modalities that are increasingly available in egocentric wearable devices such as gaze, inertial measurements, spatial trajectories, and others; and ii) existing evaluations emphasize scene understanding and question answering, with limited coverage of goal‑conditioned guidance for blind and low‑vision users across everyday tasks. To bridge this gap, we introduce VISTA (Visually Impaired Scene and Task Annotations), a multimodal egocentric benchmark for goal‑oriented assistance for blind and low‑vision users. VISTA contains ~1,000 egocentric recordings captured with Project Aria glasses; each recording provides synchronized sensor streams, six of which serve as model inputs, and three different types of annotation, covering ten task categories (TC), from object localization to hazard detection. We evaluate 14 representative vision‑language models zero‑shot on VISTA and design a lightweight multimodal model baseline that integrates all six modalities. Experimental results show that, although existing models can often describe egocentric scenes, they remain limited in generating goal‑directed assistive guidance. VISTA thus provides a benchmark for evaluating multimodal models that could improve everyday accessibility.
Authors:Louis Robinson
Abstract:
Standard autoregressive language models generate text sequentially, but revision requires referring back to tokens in the input context or those already generated. Causal Edit Serialization (CES) is introduced as a framework for expressing such revisions within the same causal token stream. Special pointer tokens are paired with predicted source addresses, allowing insertion, deletion, replacement, copying and movement to be localized explicitly. A lightweight localization head and dual‑index rotary position encoding support both address prediction and conditioning on previously emitted pointers, while retaining a standard causal attention mask. A pretraining procedure combines ordinary language modeling with serialized repairs derived from random corruptions, model‑generated edits and repeated text. Across model scales and editing baselines, CES models demonstrate complete applied edits with modestly higher clean‑text bits per byte than clean autoregressive models. Supervised and reinforcement‑learning post‑training provide a grammatical‑error‑correction proof of concept, producing more beneficial than harmful changes under reference‑blind model judging. Correct repair programs also improve subsequent text prediction while the original context and its cached states are retained. CES provides a common interface for generation and localized revision, with potential extensions to iterative refinement and additional pointer‑addressed transformations.
Authors:Zewei Deng, Mohamed Seif, Liyan Xie
Abstract:
Language‑model watermarks can help identify model‑generated text, encode provenance information, and detect post‑generation edits. A broad class of logit‑based watermarks increases the logits of a designated token set at each decoding step. Because a base model's native probability mass on this set varies across contexts, models, and watermark rules, the same logit bias can yield different adherence rates and increases in perplexity (PPL). We introduce watermarkability to characterize the compatibility between a model and the token sets requested by a watermark. At a target mass τ, we quantify this property through a watermarkability cost: the expected minimum KL divergence from the base‑model distribution required to assign probability at least τ to the requested set. A lower cost means that the model is more watermarkable for that request distribution. To reduce this cost, we propose a two‑stage method. In the first stage, we train a target‑set‑conditioned low‑rank adapter on unions drawn from a 16‑way vocabulary partition to increase the requested set's probability mass. In the second stage, a closed‑form controller applies the smallest remaining uniform logit bias needed to reach the target mass. We evaluate the method on six watermark schedules using OPT‑125M and Qwen3‑8B. Across the 12 model‑schedule pairs, the pre‑adaptation watermarkability cost correlates with the log‑PPL increase under a near‑hard fixed‑bias baseline (pooled Pearson r=0.929). For every schedule, at least 99.80% of sampled tokens fall in the target set requested at their decoding step. At this reliability, the adapter and controller reduce average per‑schedule prompt‑conditional PPL by 46.3% on OPT‑125M and 5.2% on Qwen3‑8B relative to the fixed‑bias baseline.
Authors:Zhenwei Tang, Jintao Liu, Di Bai, Xidong Wu, Xinwu Cheng
Abstract:
Reasoning rerankers write a rationale before ranking a candidate slate, and existing systems read item metadata to do so, even though generative recommenders increasingly represent items as compact Semantic IDs (SIDs). Ranking from SIDs alone has two appeals: SID prompts are far shorter than the matched item text, and although item codes are fixed from content, the backbone learns their token embeddings in part from user‑item interaction sequences, so the tokens can carry behavioral signal that item text lacks. That training targets next‑item generation, and whether it supports ranking a supplied slate from SIDs alone is untested. We build ReRankID, a controlled testbed in which models reason and rank from a history and slate shown only as SIDs, varying item rendering, history length, and slate size one at a time. On this testbed, such models often copy the input order instead of reranking, and readable metadata does not repair the failure. We compare supervised fine‑tuning, rejection‑sampled self‑training, on‑policy distillation, and verifier‑based reinforcement learning as post‑training methods and find that they elicit the missing behavior. The strongest post‑trained model, the 8B model trained with a graded ranking reward, gives the best SID‑only ranking and matches strong text‑reading LLMs whose prompts are about 20x longer. It ranks above random with no history at all, where a caption‑reading reference does not, and it keeps its gains at long histories, where text prompts pass 50k tokens and the caption‑reading reference stops improving. In a paired rationale audit, this model argues trade‑offs better than that reference but describes items less accurately. The common post‑training methods we test thus elicit ranking from SIDs but not faithful item descriptions, which calls for post‑training that improves the reasoning itself, not only the ranking it produces.
Authors:Ilia Zaznov
Abstract:
Reusing a Transformer block across depth increases computation without replicating its parameters, but successive passes have no dedicated state through which positions can exchange compact summaries of their causal prefix. We introduce the causal latent workspace Transformer: an anchored, four‑round shared‑depth architecture in which each position writes gated vectors into prefix‑specific latent slots and retrieves a content‑addressed summary before the next shared block application. This path complements causal self‑attention while the original input embedding remains accessible at every round. Across three byte‑level corpora, three seeds, and two matched parameter budgets, the proposed model achieves the best mean held‑out perplexity in five of six settings against five Transformer controls. At 101.71M parameters it leads on WikiText‑103 raw, enwik8, and TinyStories, reaching perplexities 2.7820, 2.7843, and 1.6823; these improve upon the strongest non‑workspace control on each corpus by 1.06%, 0.73%, and 1.01%. All nine seed‑labelled comparisons at this budget favor the workspace. At 0.918M parameters it leads on WikiText‑103 and TinyStories, whereas a conditioned loop is stronger on enwik8. Architectural ablations show that adding the workspace improves upon the anchored loop in all six settings and support refreshing its causal summaries at each round. Together, these results identify a causal, content‑addressable prefix workspace as an effective structural augmentation of looped Transformers.
Authors:Lieqi Liu, Rui gao, Jia-Chen Gu, Wenbo Hu, Zhaobin Mo, Ahmadreza Moradipari, Nejib Ammar, Wei Wang, Nanyun Peng
Abstract:
In autonomous driving, perceiving surrounding objects is not enough: failing to understand how they move and interact over time can lead to dangerous decisions. Evaluating vision‑language models (VLMs) therefore requires testing whether they connect past spatial observations to future consequences. Yet existing benchmarks often assess edge cases through basic perception questions or test temporal understanding in simple scenarios, leaving complex spatiotemporal reasoning underexplored. We introduce STRIDE (SpatioTemporal Reasoning In Driving Edge Cases), an object‑centric benchmark for long‑tail, safety‑critical driving scenarios. STRIDE covers six task families: spatial perception, spatial understanding, temporal memory, temporal extrapolation, trajectory prediction, and scene‑context awareness. It pairs multiple‑choice questions probing individual spatiotemporal judgments with related free‑form visual question answering (VQA) requiring detailed scene explanations and justifications of driving decisions. We prioritize scenes with unusual object behaviors and complex traffic interactions, generate reference answers from structured dataset annotations through automated procedures, and use limited language‑model assistance and human verification to ensure annotation quality. Across 12 proprietary, open‑weight, and driving‑specialized models, even the best performer, GPT‑6‑Astra, achieves only 43.4% average multiple‑choice accuracy, versus an approximately 20% random‑choice baseline. Further analysis reveals that strong performance in describing driving scenes does not necessarily translate to accurate safety‑critical judgments, and that models struggle more with reconstructing object motion than recalling past positions. STRIDE thus exposes spatiotemporal driving limitations that aggregate VQA scores can conceal.
Authors:Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Arun Innanje, Terrence Chen, Ziyan Wu
Abstract:
N‑dimensional splatting extends 3D Gaussian Splatting with view direction and time to model view‑dependent and dynamic effects. Each primitive is lifted into a joint distribution over position and conditioning variables, then conditionally sliced at each rendering query. N‑DGS and UBS share this step across Gaussian and Beta kernels: recovering the conditional mean, opacity, and covariance requires per‑primitive matrix inversions and regression products in every forward pass. We introduce direct conditional parameterization, replacing this covariance‑derived readout with explicit per‑effect parameters: a Cholesky precision factor for opacity and a factorized query‑to‑position regression operator with learnable per‑dimension coupling. Unrestricted conditional coordinates recover a valid joint covariance; our factorization adds spatial‑scale and coupling priors. Applied to both kernels, the parameterization yields dGS (Gaussian) and dBS (Beta). Across five static and dynamic benchmarks, they match or exceed their covariance‑derived counterparts on quality, with gains up to +1.26 dB in the main MCMC evaluation. Production slicing is 6.9‑7.7 times faster than on‑the‑fly covariance‑derived conditioning; complete‑render profiling gives 1.25‑1.91 times speedups on four trained scenes. Direct conditioning provides a structured, kernel‑agnostic alternative for N‑dimensional splatting. Project page: https://gaozhongpai.github.io/dgs/.
Authors:Gabe Schulman, Lucius E. J. Bynum, Yoav Wald
Abstract:
Causal foundation models (CFMs) show promising results for treatment‑effect estimation, with unit‑level estimates that are competitive on standard benchmarks. Many causal analyses, however, inform decisions through effects averaged over groups of units, such as the full sample, the treated units, or a subgroup, and for such decisions a CFM is useful only if its credible intervals are calibrated. We examine these intervals and find that their coverage degrades, both in simple synthetic settings and on standard benchmarks. On ACIC 2016, for example, temperature‑calibrated 95% intervals from CausalPFN fall below nominal coverage beyond groups of 30 out of 4802 units, and intervals for the full‑sample average contain the true value in only 20% of datasets. We trace this problem to how these models are trained and how their intervals are formed. They output a predictive distribution for one unit at a time and, at inference time, combine the unit‑level variances as if the errors on different units were independent. We show formally that posterior uncertainty over the data‑generating process makes these errors correlated. As a result, the combined variance is too small by a factor that grows with the size of the group. Empirically, this factor predicts where coverage fails and by how much. We then show that two neural process models, which make joint predictions for a group of units, account for this correlation and substantially improve coverage on synthetic tasks and, as heads on a frozen CausalPFN, on all six benchmarks.
Authors:Nitin Rai, Won Suk Lee, Hongyoung Leon, Heping Zhu
Abstract:
Precise pedicel‑targeted grasping in strawberry harvesting remains a critical bottleneck due to the pedicel's small diameter (1.3‑3 mm), color similarity with surrounding foliage, and precise force control required for clean abscission zone detachment. As a market‑grade crop, strawberry aesthetic value makes bruise‑free pedicel detachment essential over direct fruit grasping. Classical approaches struggle to unify vision models, motion planning, and contact manipulation into a robust framework, particularly in occluded canopy environments. Therefore, this work presents a visuomotor imitation learning pipeline for pedicel‑targeted strawberry harvesting that unifies perception, motion planning, and contact manipulation from collecting human teleoperation demonstrations, training an Action Chunking with Transformers (ACT) policy with a ResNet‑50 visual backbone, and deploying the learned policy autonomously on a Jetson AGX Orin edge platform at 15 Hz. The trained policy achieved consistent loss convergence from 2.84 to 0.091 over 100K gradient steps on 130 demonstrations. We have also established a reproducible open‑source pipeline with publicly released dataset and model weights for the agricultural robotics community. Full dataset is available at, https://huggingface.co/datasets/nitindominicrai/strawberry_pedicel_grasp_teleoperation and trained policy at, https://huggingface.co/nitindominicrai/act_strawberry_pedicel. To replicate this work, all code, configurations, and deployment instructions are publicly available at, https://github.com/nitin‑dominic/autonomous_strawberry_pedicel_harvesting.
Authors:Yuanzhe Liu, Bastian Lipka, Avraham Shinnar, Jatin Ganhotra, Yao Ma, Martin Hirzel
Abstract:
Benchmarks drive AI progress, but are hampered by task under‑specification and over‑specification. Task under‑specification causes guesswork, where solutions have to make things up, or worse, resort to contamination or reward hacking. Task over‑specification causes spoilers, where solutions can bypass some of the harder steps the benchmark was supposed to measure. Since future‑looking benchmarks should be difficult, over‑specification reduces a benchmark's shelflife of community adoption. Repository‑level coding benchmarks have helped the field progress but experience unnecessary churn due to both over‑ and under‑specification. For instance, SWE‑bench Pro, FEA‑bench, Doc2Feat‑bench, and Multi‑SWE‑bench all tackle under‑specification by augmenting task statements with hints derived from ground truth. We show that leaking ground‑truth information into the task statement causes over‑specification in some of these benchmarks. We experiment with various specification levels, trying to thread the needle between spoilers and guesswork. Based on these results, we recommend techniques for deriving parsimonious hints. We will release our hints so the community can obtain more reliable signals from popular benchmarks.
Authors:Edrees Al-Edreesi, Ghaleb Al-Gaphari, Muneer Alsurori, Mohammed Edrees, Fouad Al-Hamdani
Abstract:
Multi‑human personalization must account for identity changes in people outside the nominal target of a generative operation. We characterize this behavior in UniPortrait, without introducing a new preservation algorithm. Two sequential studies use 119 accepted two‑person baseline scenes and three fixed output seeds, measuring continuous reference‑based facial similarity with InsightFace \buffalo. In Study~1, source‑coupled single‑ID target‑only generation (A1) produces negative mean transitions for both the protected person (‑0.567904) and target (‑0.236904), indicating broad perturbation rather than established successful repair with collateral loss. In Study~2, native dual‑ID generation (A2) yields more positive paired contrasts: protected +0.512918 (95% case‑cluster CI [0.471434,0.550259]) and target +0.158853 ([0.118903,0.197523]), across 287 jointly measurable pairs from 111 cases. Shared baseline similarity cancels algebraically in these paired contrasts, leaving a matched POST comparison. Validation designed after the primary result freeze finds directional agreement with an independent identity evaluator and blinded human judgments; within‑\buffalo\ nuisance models provide supportive sensitivity evidence. The findings characterize intact native regimes, whose references, routing, adapters, and token counts co‑vary. They do not establish component causality, categorical identity preservation, successful localized editing, or generalization beyond the evaluated implementation and measurable cohort.
Authors:Xiaoyan Cong, Tianyu Wang, Hyeonho Jeong, Zhifei Zhang, Yuqian Zhou, Yijun Li, Shaoteng Liu, Zhe Lin, Srinath Sridhar, Soo Ye Kim
Abstract:
Creative intent in video editing can be expressed through text instructions (e.g., "remove the people"), image references (e.g., subject appearance), video references (e.g., camera movement), or a combination of the above. We introduce OmniXRef, a unified framework for multimodal multi‑reference video editing that allows users to combine variable numbers of text, image, and video references to control diverse attributes of a source video. We first devise a unified input design with a reference‑bank interface and introduce R^2‑RoPE to support generalization to more references. Then our model, OmniXRef‑Model, uses a single‑stream diffusion transformer to jointly process the source video and multimodal context from an MLLM in an interleaved sequence. To address the scarcity of training data for video‑reference editing, we construct OmniXRef‑Data using synthesis pipelines for different editing tasks and combinations of references. We use curriculum learning to progressively increase task complexity and resolution. To support systematic evaluation of video as a control modality, we introduce OmniXRef‑Bench, a benchmark for video‑reference attribute transfer across Camera, Motion, Content, Style, and Visual Effects (VFX). Comprehensive evaluations show that our OmniXRef‑Model outperforms task‑specific baselines and commercial models.
Authors:Mingda Zhang, Yun Huang, Qiang Huang, Yanjin Li, Rui Qian, Qichao Zhao, Tiesunlong Shen, Na Zhao, Jian Wang, Xiaoying Tang
Abstract:
In recent years, agentic orchestration has been widely applied to solve complex human tasks. However, existing orchestration learning still faces key challenges, including hand‑designed curricula, advantage collapse when a task's rollouts all reach the same outcome, and unreliable evaluation signals that judge each skill by one noisy score. To address these challenges, we propose R^2Orch, a new paradigm of Recursive Self‑Play Orchestration: an inner loop of self‑play that improves the orchestrator on self‑chosen tasks, and an outer loop that improves how the inner loop improves. To support this paradigm, we introduce Recursive Learnability, a training objective that rewards a Proposer for choosing tasks whose rollouts keep reaching different outcomes as skills are withdrawn. Built on this objective, we further propose Skill Withdrawal, in which the outer loop withdraws each skill once a controlled evaluation within self‑play shows that the orchestrator performs as well without it. Moreover, the two loops form one cycle: failures are distilled into skills, skills into parameters, and withdrawals lead the Proposer to new failures, so the orchestrator moves toward zero‑skill deployment. Experimental results on twelve datasets show that R^2Orch consistently outperforms baselines across question answering, math, code, and interactive tasks. Our code is available at https://github.com/beita6969/R2Orch.
Authors:Qianwei Wang, Ce Zhang, Zhanpeng Luo, Pengpeng Su, Shuo Yang, Hao Xu
Abstract:
Spatial memory enables robots to reuse past observations for localization, navigation, and question answering. Designing such memory involves jointly deciding what to retain and how to retrieve it for diverse semantic, metric, and temporal needs. We investigate whether coding agents can automate this design process through iterative implementation and evaluation. Under a fixed development budget, a Designer Agent writes and revises code for memory construction and retrieval using feedback from a fixed Query Agent. The resulting implementations are frozen before building memories from held‑out observation histories, without access to downstream questions, and evaluated across five complementary task tracks using the same Query Agent. Across three independent design searches, Codex / GPT‑6 Astra averages 44.23 on the equally weighted five‑track score, versus 29.32 for ReMEmbR, the strongest evaluated baseline supporting all five tracks, a 50.9% relative improvement. Qualitative analysis relates performance differences to evidence retention and retrieval, while identifying grounding and evidence‑use failures. These results demonstrate the potential of coding agents to automate spatial memory design, while revealing task‑dependent trade‑offs in performance and computational cost. Project page: https://ba‑fa‑3344.github.io/.
Authors:Nuzhat Khan, Ab Al-Hadi Ab Rahman, Mohamad Anuar Kamaruddin, Muhammad Paend Bakht, Sana Fatima, Farhad Khodadadi, Ibrahim Yousef Alshareef
Abstract:
Distinguishing visually similar characters is a persistent challenge for optical character recognition (OCR) systems, causing errors in critical applications of text digitization. While preprocessing techniques improve image quality and word‑level post‑processing corrects spelling errors, neither addresses character‑level visual ambiguity, an intrinsic property of certain character pairs that limits even state‑of‑the‑art OCR models. We present the Character Verification Model (CVM), a character‑level post‑processing framework that verifies and corrects ambiguous characters in OCR outputs using specialized binary classifiers trained on visually confusable pairs such as 'O'/'0', 'Z'/'2', and 'S'/'5'. CVM operates independently of dictionary validation, making it suitable for non‑lexical data including serial numbers, alphanumeric codes, and license plates. Experiments on three OCR engines (Tesseract, EasyOCR, TrOCR) and ten fonts with low to high complexity demonstrate accuracy improvements of 3.95‑14.81 percentage points, with an 87.3% correction rate for visually similar character errors and a false‑correction rate of only 2.3%. The modular architecture integrates with existing OCR pipelines without modification to underlying recognition models.
Authors:Nicholas Saban, Charvi Bannur, Mariya Vasileva, Darshan Deshpande
Abstract:
Text‑based world models have recently shown potential to simulate complex RL environment state transitions for a diverse set of use cases. However, no standardized benchmark exists to compare, study and improve these models. Through this paper, we attempt to solve this problem by building GlobeBench, a novel benchmark with 1,632 tasks across five separate domains and nineteen simulation skills. The tasks check whether models can predict the environment's response to a specific action given the full agent trajectory by understanding earlier state changes, following environment rules, and predicting the effects of actions on both related and unrelated objects. We compare the predicted environment response with an execution‑derived reference, using a hybrid architecture with both deterministic gates and a semantic judge. Among 20 general‑purpose LLMs and specialized text world models, GPT‑6 Astra achieves the highest task success rate at 49.57%. We find that model accuracy decreases as actions affect a larger portion of the environment. Specifically, when the agent's changes must propagate to dependent objects, task success drops to an average of 5.1% across all 20 models. Although model strengths differ across tasks, 631 tasks remain unsolved by any model, suggesting a significant gap in general LLM capabilities as a world model.
Authors:Yinyi Luo, Haoyue Bai, Yi Ge, Nilton Cesar Rojas Vales, Marios Savvides, Jindong Wang
Abstract:
Unified multimodal models (UMMs) can integrate understanding and generation within a single backbone. However, it is commonly known that these two capabilities can cause conflicting representations, leading to a central question: how representation and computation should be shared across functionalities? Existing UMMs often rely on a largely fixed structure, where different inputs and functionalities follow the same depth and parameter‑sharing pattern. We argue that this assumption is too restrictive: more effective unification requires sharing the computation that supports cross‑functional communication while specializing the capacity that becomes task‑ or input‑specific. We introduce FlexUMM, a flexible UMM framework that dynamically decides when to preserve shared computation, when to route specialized capacity, and when to stop further processing. Evidence shows that this flexible allocation improves UMMs along two complementary axes. It improves accuracy by reducing harmful competition between functionalities, and it improves efficiency by avoiding unnecessary computation once an input representation is sufficiently mature. Beyond simple architectural unification from the pre‑trained models, FlexUMM works as a model‑agnostic, functional plug‑in to make existing UMMs more unified, stronger, and efficient. Code is available at: https://github.com/AIFrontierLab/TorchUMM/tree/main/src/umm/post_training/flexumm.
Authors:Yupeng Zhou, I-Chia Chang, Ziran Wang, Bin Yao
Abstract:
Friction uncertainty is a significant challenge for autonomous vehicle control, undermining feasibility and performance. This paper proposes a novel two‑layer control framework to achieve vehicle motion control under large friction uncertainty by combining optimization‑based planning with indirect adaptive robust control (IARC). The upper layer solves a constrained optimal control problem to generate a feasible trajectory that satisfies all the constraints in a time‑optimal manner along the desired path. The lower layer, IARC, adapts the unknown parameters related to friction uncertainty and combines model compensation with robust feedback to ensure transient and steady‑state tracking performance. An estimation monitor is designed to prevent poor learning when data indicate a model mismatch, while a smooth parameter reset reduces the estimated tire capacity and limits abrupt tightening of constraints. The updated parameters are fed back into the model and constraints in the planner, enabling trajectory replanning as friction conditions change. Simulation results indicate that the proposed method achieves accurate tracking of the desired trajectory and online friction adaptation under large friction uncertainty while ensuring time‑optimal vehicle motion. The simulation results demos are available at https://github.com/TedZhou0603/CO‑IARC.
Authors:Narayanan Palghat Parameswaran, Dong-Ki Kim, Jay Patrikar, Santiago Gutierrez, Harshad Zade, Shayegan Omidshafiei, Harmish Khambhaita
Abstract:
Quadruped parkour policies handle canonical obstacles well, but their training curricula often miss the irregular, compound geometry of industrial and disaster‑response sites, where agile locomotion matters most. Such environments pose a hard generalization problem: even training directly on reconstructed geometry from the target site does not reliably yield policies that traverse it. Hand‑designed curricula span only a few terrain families, and recent LLM‑generated curricula offer little control over which policy weaknesses they target. We propose FCTS, a failure‑conditioned terrain curriculum framework that uses localized traversal failures to automatically generate the terrains a policy should train on next. Each failure's location, termination mode, robot state, and surrounding terrain geometry condition this generation, and Monte Carlo Tree Search explores alternative curriculum trajectories under a fixed training budget. On terrains built from scanned real‑world construction sites, FCTS raises success rate from 28.4% to 85.8% on a medium‑difficulty course and from 14.9% to 44.7% on a hard course relative to the strongest parkour or LLM‑curriculum baselines, and maintains higher progress on extreme terrains where full traversal is rare. Hardware experiments on a quadruped robot confirm that the learned FCTS policies transfer to the real world.
Authors:Jae Joong Lee, Bedrich Benes
Abstract:
FOR‑species20K is the largest public LiDAR benchmark for tree‑species classification, pooling 25 independently collected terrestrial, mobile, and airborne laser‑scanning datasets across 33 species. We introduce a multi‑resolution, multi‑view ConvNeXtV2 model that improves official hidden‑test accuracy from 79.5% to 85.0%, ranking first at the time of submission. We then examine what this score measures. Because the hidden test is species‑balanced, its overall accuracy is more comparable to a macro metric than to accuracy under the public set's long‑tailed distribution. Moreover, the official random split places every source dataset in both training and evaluation. A logistic probe recovers source identity from out‑of‑fold model probabilities with 69% accuracy, while geometric features identify the source above 90% within two widely shared species; 18 of the 33 species occur in only one source. Under a source‑species‑cell holdout, evaluated on 5,566 trees from eight eligible species that retain the same LiDAR‑type training support in other sources, the unweighted species mean falls from 82% to 36%, and tree‑weighted accuracy falls from 95% to 60%. Count‑matched retraining and experiments with two alternative backbones indicate that this gap is not explained by reduced training support alone, while an external PureForest experiment shows a smaller but consistent grouped‑split effect. These results establish the 85% leaderboard score as strong in‑distribution performance, but not as an estimate of source‑disjoint generalization, motivating balanced metrics and source‑aware evaluation. The code is available at: https://github.com/JaeLee18/FOR20K‑lidar‑tree‑species
Authors:Ayush Bansal, Aldo Pareja, Sean Klein, Serge Didenko, Artem Lukoianov
Abstract:
Recent advances in AI make Recursive Self‑Improvement (RSI) increasingly plausible. Large Language Models (LLMs) can already propose, select, and run Machine Learning (ML) experiments. However, the space of possible design choices in ML is vast. An LLM that proposes an experiment needs a reliable intuition of whether it is worth running given the timing constraints of long training runs. In this work, we measure such intuition in current frontier LLMs by evaluating how accurately they predict the impact of self‑generated ML experiment ideas before these are put into training runs. To this end, we construct a dataset of 288 LLM‑proposed ideas on eight MLE‑bench tasks. We then prompt five frontier LLMs (GPT‑6 Astra, Claude Opus 5.5, Claude Fable 5.1, Kimi K3, and GLM‑5.3) to predict the impact of each idea and compare their predictions with the actual outcomes after training. We report four main findings. First, LLM predictions are substantially better than random chance but far from accurate: even the best model, Claude Opus 5.5, explains only half of the variance in outcomes (R^2 = 0.49). Second, we notice that all tested models systematically underestimate the magnitude of the changes. Third, prediction quality varies widely across tasks. Finally, the choice of the coding harness that actually implements the ideas has little effect on outcomes, and we find no evidence of self‑preference. Higher reasoning settings and additional context lead to modest gains. Overall, our results show that current LLMs have useful but limited intuition about which ML ideas will work. We release our dataset to support further study.
Authors:Akash Kumar, Karthik Sankaranarayanan
Abstract:
Can a network keep learning while its training cross‑entropy stays nearly flat at a positive level? We show that it can. On modular addition, a standard testbed for grokking, ReLU networks trained with supported‑Polar hidden updates, normalized output‑layer updates and weight decay can keep 100% training accuracy while their hidden features change and held‑out accuracy rises from zero to above 90%, at relatively flat training cross‑entropy. Correct classification need not minimize cross‑entropy, which still depends on prediction confidence. We prove this coexistence for one‑hidden‑layer networks in two finite‑window theorems from constructed initializations: one gives a constant gain in the Fourier fraction of the hidden features, and the other holds at arbitrarily low positive loss, where a readout refitted on the training data generalizes before the network. Experiments from random initialization show the same coexistence on a window fixed in advance, and in deeper networks held‑out accuracy also rises over post‑hoc windows with a net increase in training cross‑entropy. With a last‑layer rate set per training fraction and head decay scaled with the prime, deeper networks at p=47 and 61 meet a criterion on post‑hoc windows, fixed in advance, in nearly all fresh runs. A one‑block transformer trained with the same updates and a slowly decaying output gain keeps every training pair correct at every logged checkpoint while its training cross‑entropy rises and its held‑out accuracy approaches one; with two blocks this holds through the held‑out rise, after which training pairs begin to flip. A plateau in training loss need not be a plateau in learning. Code: https://github.com/akashkumar‑d/flat‑loss‑evolving‑features.
Authors:Kaan Baydemir, Sajjad Baghaee, Ilkay Ulusoy
Abstract:
Channel State Information (CSI) captures fine‑grained changes in Wi‑Fi channel propagation and provides a foundation for device‑free sensing applications such as activity recognition, localization, and occupancy detection. The development of such applications depends on an acquisition system that can handle high‑rate CSI streams while remaining extensible to the requirements of individual experiments. VesperCSI is an open‑source, GPL‑3.0‑licensed redesign of FeitCSI for CSI acquisition with Intel AX200 and AX210 Wi‑Fi interfaces. It targets high‑rate acquisition on amd64 and aarch64 Linux hosts using fixed‑capacity preallocated object pools and separate capture and consumer threads. A loadable plugin interface supports experiment‑specific monitoring, storage, and processing. We evaluate VesperCSI and FeitCSI on Raspberry Pi CM4 receivers in a burst‑traffic software crossover. At a 600‑MHz all‑core frequency ceiling and a target rate of 2500 frames/s, VesperCSI achieves mean sequence retention of 99.017‑99.087% in 10,000‑frame bursts, compared with 43.972‑46.553% for FeitCSI. This ordering persists after exchanging the userspace programs between two receivers while retaining their respective drivers. The comparison therefore characterizes the tested software‑and‑driver packages rather than isolating userspace effects. These results support VesperCSI as a reusable acquisition component for edge‑hosted Wi‑Fi sensing under the evaluated workload and platform conditions.
Authors:Tianyi Yang, Thomas Beckers
Abstract:
Gaussian process distributed port‑Hamiltonian systems (GP‑dPHS) enable data‑driven modeling of distributed dynamical systems while preserving their physical structure. However, standard stationary kernels can be inefficient for spatially heterogeneous systems because they neither exploit known input invariances nor reflect structured spatial dependencies. We address this limitation with a structure‑aware kernel design that removes irrelevant state coordinates and decouples state and spatial dependencies through a tensor‑product covariance. This modular construction allows qualitative knowledge of spatial heterogeneity to be incorporated by selecting an appropriate spatial kernel without requiring an analytical description of the underlying profile. Numerical experiments on a nonlinear string with periodic, smoothly varying, and semi‑piecewise stiffness profiles demonstrate that structural decoupling consistently improves reconstruction accuracy, while profile‑matched spatial kernels yield substantially larger gains in force‑field and trajectory reconstruction.
Authors:Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev, Iuliia Tsvetkova, Egor Cherepanov, Nikita Kachaev, Artem Latyshev, Aleksandr I. Panov, Alexey K. Kovalev
Abstract:
Long‑horizon, multi‑stage Vision‑Language‑Action (VLA) policies often require visual information that is no longer present in the current observation. Retaining the complete observation history is computationally expensive and highly redundant, motivating selective visual memory. We introduce RePlanMem‑VLA (Replanning‑Guided Keyframe Memory for VLA), which turns a policy's own plan revisions into an online memory‑write signal. Under receding‑horizon execution, consecutive action chunks predict some of the same future actions. RePlanMem‑VLA treats unusually large disagreement between these temporally aligned predictions as evidence of a task‑relevant event. We first compare 35 disagreement measures across single‑task autoregressive decoder‑only and Flow‑Matching Transformers trained from scratch, then validate the selected signals on pretrained OpenVLA‑OFT and π_0.5 policies over MIKASA‑Robo‑VLA tasks. The resulting detectors achieve ROC‑AUC of 0.981‑0.997. We then use the selected signals in RePlanMem‑VLA to trigger updates to a bounded visual memory, with a robot‑masked write filter suppressing redundant frames. In the evaluated memory‑retention experiments, RePlanMem‑VLA preserves task‑relevant observations until the later decision point in 100% of episodes. Real‑robot replay data further show that event selection and redundancy suppression extend beyond simulation. These results establish plan revision as a policy‑internal signal for controlling what a VLA remembers. Website: https://vika787.github.io/RePlanMem/.
Authors:Eliezer Shlomi, Eliron Rahimi, Avi Mendelson, Chaim Baskin
Abstract:
Diffusion language models (DLMs) promise parallel text generation, but in practice face an unresolved speed‑quality trade‑off. Exact decoding fills in only one masked position per model pass, while faster multi‑token decoding schemes are lossy. Speculative decoding is the standard lossless acceleration for autoregressive models. However, it does not transfer directly to DLMs, which must choose both what to generate and where. We show that this order is the missing piece: the model's own predictions from the previous pass already give the right token at 92‑100% of positions, but not which position comes next. Based on this observation, we introduce Speculating Positions Order Trajectory (SPOT), which learns to predict this order. A 0.62M‑parameter model reads only the target's output distributions and proposes a tree of candidate continuations, and the target verifies all of them in a single pass by checking whether each equals its own next state. On the verification side, our analysis shows that prior block‑diffusion verification approaches either assume T=0 or a sequential proposal interface. To support tree‑based speculation at any temperature, we propose using coupled‑noise match verification for diffusion blocks, following shared‑Gumbel‑noise coupling approaches. Our comprehensive experiments show that SPOT cuts target passes by up to 6.8x and reaches up to 3.2x the throughput of standard decoding, beating the calibrated and training‑free drafting baselines in every task and budget at T=0 and T=1. In addition, we show that the learned drafter transfers across models, demonstrating successful transfer across six different DLMs. Project page: https://eliezer318.github.io/SPOT‑dLLM/
Authors:Adib Sakhawat, Syed Rifat Raiyan
Abstract:
Decision models, non‑generative scorers that report a probability over caller‑supplied options and emit no text, are increasingly deployed as routers, classifiers and guardrails. Because they generate nothing, rationales and chain‑of‑thought do not apply. We audit three: an encoder scorer, a decoder scorer and a closed API, entirely through interventions on their input, using expert‑annotated Bengali idioms carrying a gold figurative and a gold literal gloss. Dropping an idiom's own literal gloss into the option list draws every arm toward it, taking 65% to 96% of each arm's remaining Bengali errors. A model‑by‑language matrix over one shared item set shows how far this depends on the test language: every arm is 1.9 to 6.4 times more captured in Bengali than in English on the same paired entries, the commercial model's apparent resistance is a property of its test language, and capability does not order the failure. Calibration degrades in all three arms, and on the open arms confidence inverts, so gating on it is worse than gating at random. On a semantics‑free option list, where accuracy is at chance, the best‑calibrated arm still reports mean confidence 0.82. Code and data: https://github.com/sakhadib/literal‑lock‑in.
Authors:Shreeyans Arora, Utsav Avaiya, Zera Lyngkhoi, Hem Gosalia, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
Abstract:
Evaluating language models and agents requires combining benchmark accuracy, judge‑based assessment, runtime performance, tool‑use behavior, and retrieval quality. Differences in interfaces, metric semantics, and trace formats complicate these comparisons and can hide failures in agentic workflows. We present AuditKIT, a source‑available evaluation library that records these measurements in one result representation, with direction‑aware metric contracts, configuration fingerprints, and extensible metrics and backends. For agents, it provides turn‑aware tool‑call matching, separates parallel from sequential execution, and distinguishes verified task outcomes from trajectory‑level diagnostics. Missing or unobservable evidence is marked unscored rather than counted as failure. For retrieval‑augmented generation, it combines ranked retrieval metrics, judged grounding, and deterministic evidence checks. Seven case studies spanning model comparison, serving runtimes, retrieval, tool calling, and interoperability show how answer extraction, runtime behavior, and metric selection change evaluation outcomes.
Authors:Markus Kramer, Jan Wachenfeld, Oskar von Stryk
Abstract:
Reliable 3D person detection is important for rescue robotics to give first responders situational awareness. Yet most multimodal 3D detection research targets automotive scenarios in which pedestrians mostly appear in a narrow range of upright, near‑road poses. It is therefore unclear how much of the reported performance transfers to rescue settings, as a person may be lying, partially occluded, or in unstructured indoor and outdoor environments. To investigate this gap, we introduce TriSAR‑3D, a multimodal person detection dataset recorded with a high‑resolution 128‑channel LiDAR, an RGB camera, and a thermal camera. It captures varied poses in indoor as well as outdoor locations at different times of day to study their influence on 3D object detector performance. Furthermore, we propose a novel trimodal 3D detection pipeline, PaintFusion, that fuses PointPainting‑based evidence using a simple MLP. We show that the derived method reaches 65.4 AP@50 and thereby outperforms strong LiDAR‑only (CenterPoint, 53.4), RGB+LiDAR (BEVFusion, 60.5), and adapted trimodal (Tri‑BEVFusion, 60.7) baselines on the dataset. At the same time, it is substantially faster than other fusion‑based methods such as BEVFusion, with a per‑frame latency of 51.6 ms compared to 101 ms. Additionally, we test how the performance of different 3D object detectors generalizes to unseen pose categories. Removing lying instances from training data collapses the lying‑pose AP@50 to near zero across all evaluated detectors. The results indicate that dataset diversity exerts a large effect on 3D object detection performance. The dataset and code are available at https://github.com/MarkusKramer1/3D‑Trimodal‑People‑Detection.
Authors:Xinwei Ju, Zhehua Mao, Tianyi Song, Danail Stoyanov, Sophia Bano, Francisco Vasconcelos
Abstract:
Gaussian‑splatting Simultaneous Localization and Mapping (SLAM) augments conventional SLAM with a dense, renderable 3D scene representation, enabling high‑quality reconstruction and novel‑view synthesis. However, its localization, reconstruction, and rendering performance depends strongly on which observations are retained for mapping. Existing systems commonly rely on fixed temporal intervals or simple pose‑ and overlap‑based criteria for keyframe selection. In colonoscopy, non‑uniform camera motion, repetitive anatomy, specularities, and visual artifacts can make these criteria unreliable, leading to either redundant keyframes or missed informative viewpoints. We present KFS^4LAM (KeyFrame Selection with Structure‑aware mapping for endoscopic Gaussian Splatting SLAM), which couples motion‑guided adaptive keyframe selection with structurally supervised mapping and restricted temporal deformation. Accumulated motion guides adaptive keyframe selection, with complementary cues suppressing redundant or unreliable candidates. Density‑aware mapping with anatomical contour supervision (colon folds) emphasizes structurally informative regions, while restricted temporal deformation accommodates local tissue dynamics without unconstrained geometric drift. We evaluate KFS^4LAM against Gaussian‑SLAM baselines on C3VD and C3VD‑v2, together with cross‑backbone and component ablations. KFS^4LAM achieves strong overall performance across trajectory, rendering, and depth metrics, with pronounced gains on more challenging C3VD‑v2 dataset. In the matched comparison with SplaTAM on C3VD‑v2, it reduces rotational relative pose error by 20.3% and depth RMSE by 38.4%, while improving PSNR by 0.78 dB. Cross‑backbone experiments further show that the framework is not tied to a single underlying SLAM architecture. Code and visualization results are available at https://xinweiju.github.io/KFSLAM/.
Authors:J. P. Naiman
Abstract:
Figures are essential for scientific communication, and the field of Visual Question Answering (VQA) aims to test the figure understanding capabilities of large multimodal models (LMMs). Here, we investigate the accuracy of several low‑cost LMMs on domain‑specific VQA tasks for astronomical images of the sky. As instrument for this task we build a domain‑specific VQA dataset comprising 2001 partially‑synthetic figures divided evenly between contour plots, real telescope images drawn from NASA's SkyView, and "fake" sky images which share the axes and angular scales of the real images but draw their colors from a gaussian mixture model, alongside 47357 questions. Each figure is released as pristine and with realistic page‑aging degradations mimicking the historical astronomical corpus. Testing three low‑cost proprietary LMMs, we find models frequently fail to answer domain‑specific questions at high accuracy. An initial comparison with 104 human‑verified, astrometrically tagged figures from historical journal publications shows similar median errors in angular extent. All code is available on GitHub: https://github.com/jnaiman/SkyImagesWASP2026 and dataset is available on HuggingFace: https://huggingface.co/datasets/ReadingTimeMachine/astro_sky_image_vqa.
Authors:Tianyu Wu
Abstract:
Automated discovery systems propose scientific explanations and test which to accept. Does passing a test establish the intended claim? We develop four executable operations for changing physical descriptions: identifying redundancies (L1), coarse‑graining dynamics (L2), transferring structure (L3), and inferring missing quantities from conservation imbalances (L4). We evaluate these operations and three interpretability settings through null acceptance (NAR), positive acceptance (PAR), and consequence agreement (CAR). A reduction mapping every state to zero predicts its own future perfectly. It passes closure on 96% of 129 public chaotic systems despite retaining no dynamical variation. An independent observable‑preservation test rejects it throughout. On controlled benchmarks, the combined criteria accept 24/24 genuine reductions and reject 72/72 nulls. The other operations distinguish symmetries from redundancies, validate transfers, and reject spurious conservation imbalances. In language and ESM‑2 protein models, linear CKA remains unchanged when a proposed correspondence is randomly rotated, whereas localized interventions distinguish genuine mappings from tested nulls. In sparse autoencoders, enrichment rejects all 16 tested nulls, yet independent ablation supports 45% of evaluable enriched features, compared with 0.5% of matched controls. These results reveal two acceptance gaps: descriptive tests can accept scientifically empty candidates, and accepted candidates can lack independently verified consequences. Reliable automated discovery requires evaluating both what a test accepts and the scientific consequences it establishes. Code and data: https://real‑project‑aether.github.io/project‑aether/.
Authors:Yizhe Zhang, Xujia Wang, Songqi Tang, Xujie Tang, Leo Jiahao Jiang, Jiacheng Wang, Bowen Qu, JinBo Li, Qin Wang, Ji Qi, Lei Hou, Bin Xu, Juanzi Li
Abstract:
Spatio‑Temporal Video Grounding (STVG) aims to jointly localize a target in space and time given a natural‑language query. Despite continued advances, existing STVG benchmarks remain largely centered on localizing targets through instance‑specific descriptions, leaving them largely decoupled from long‑form reasoning increasingly emphasized in multimodal large language models (MLLMs). In this paper, we introduce Event‑STVG, a task that shifts STVG from locating what is described toward event‑level reasoning by unifying long‑form context, repeated‑event reasoning, event localization, role inference, and multi‑target grounding. To facilitate research on Event‑STVG, we construct a large‑scale benchmark comprising 4,016 annotated event occurrences across soccer and seven additional domains. Experiments with state‑of‑the‑art MLLMs reveal substantial difficulty, with the strongest model achieving only 5.4% m‑vIoU at the 10‑minute horizon. Across domains, we consistently observe confusion among repeated same‑type events, spatial grounding bottlenecks, and rare success in jointly grounding all required roles. Event‑STVG establishes a new challenge for visual understanding in MLLMs, calling for future models that can jointly reason about complex events across space and time. Code and annotations are available at https://github.com/THU‑KEG/EventSTVG
Authors:Ryan Wyton, Bacha Rehman
Abstract:
Generative models now synthesise video that is hard to distinguish from genuine footage, and learned detectors that key on a generator's artefacts degrade sharply on unseen models. This paper develops a forensic framework that authenticates video by a signal that generation does not control: the electric network frequency (ENF), the mains‑driven hum in audio and flicker in video. A unified two‑channel detector extracts the ENF from audio and from rolling‑shutter video through one second harmonic estimator and a shared network; a synthetic‑injection training method, driven by an Ornstein‑Uhlenbeck drift model and anchored to real recordings, overcomes the scarcity of labelled ENF data; a reference‑matching channel verifies a claimed time of recording and recovers an unknown one; and a two‑axis agentic orchestrator fuses recording authenticity with content‑manipulation evidence under an asymmetric rule, escalating expensive detectors only when needed. A central finding reframes the task: an ENF‑band flicker is present in generated video at a prominence indistinguishable from real footage, so its presence is not proof of authenticity, and authentication must rest on consistency with an independent reference rather than on presence. On held‑out real recordings the detector accepts genuine audio and video at 93% and 100% against naive baselines of 53% and 15%; claimed‑time authentication reaches an area under the curve of 1.0; and the agentic cascade matches a flat ensemble while cutting wall‑clock time by 54% and frames decoded by 64%, without loss of accuracy.
Authors:Ziheng Zhang, Wei Zhang, Yunyi Zhao, Zuming Liu, Wei Zhou, Boon Ping Gan
Abstract:
Flexible job‑shop scheduling becomes resource‑coupled when a scarce vehicle fleet moves jobs between stations, because the schedule sets the transport waits. BOLT (bound‑oriented learning for transport‑coupled scheduling) builds a lower bound from chain, machine‑capacity, and fleet‑capacity relaxations. The bound equals the makespan at termination, so its per‑step changes form a reward that sums to a constant minus the makespan. Because it never exceeds the attainable makespan, each candidate's bound increase is a proven rise of the makespan floor, which the policy reads as a price. The root bound certifies a completed schedule's suboptimality gap without a solver. On industry‑motivated 10‑module prefabricated‑construction instances, explicit transport coupling outperforms a transport‑penalty policy in every tested cell where loaded moves occupy at least 66% of fleet time and in none at or below 56%. Across five zero‑shot 50‑ and 80‑module production batches, BOLT outperforms the best dispatching‑rule pair on four and the penalty policy on the three with long moves. In the three‑vehicle 80‑module batch, its mean makespan is 13.0% below the best pair and 4.2% below an hour‑budget constraint‑programming solver whose proven gap averages 34%. BOLT decides in 2.3‑5.9 ms per event on one processor core and re‑plans a disruption in seconds.
Authors:Ming Lu, Junqi Shi, Zhan Ma
Abstract:
End‑to‑end learned image coding (LIC) has achieved substantial improvements in rate‑‑distortion (R‑‑D) performance, yet practical deployment requires addressing a broader set of system‑level constraints. This article provides a deployment‑oriented review of LIC, with particular attention to the transition from high‑performance neural compression models to practical codecs. We first summarize the basic end‑to‑end coding framework, including learned transforms, quantization, entropy modeling, and R‑‑D optimization. We then review practical extensions that are increasingly important for deployment, including variable‑rate coding and rate control, integer‑precision inference and cross‑platform consistency, robustness and communication resilience, lightweight and hardware‑aware implementation, and standardization and interoperability. To study these issues under a common setting, we further introduce TinyLIC as an open benchmark and reproducible case study for rate‑‑distortion‑‑complexity analysis. Experimental results illustrate the trade‑offs among compression efficiency, computational complexity, numerical consistency, and robustness. In particular, TinyLIC with approximately 10 KMACs/pixel decoding complexity substantially outperforms the H.265/HEVC image‑profile anchor on Kodak, while the 50 KMACs/pixel configuration achieves performance comparable to the H.266/VVC image‑profile anchor on several test sets. The review highlights practical constraints and design trade‑offs that remain central to the deployment and interoperability of learned image codecs. Materials are accessible at https://njuvision.github.io/TinyLIC.
Authors:Fubin Yan, Ruxuan Dong, Ruoxuan Shi, Duolei Wang, Kenneth W. Shum
Abstract:
Large language models have recently demonstrated an ability to generate substantial mathematical proofs and formalizations, including machine‑checked arguments for major open problems and classical theorems. These developments highlight a growing workflow in which AI systems search for proofs while proof assistants certify their correctness. Within information theory, however, formalized libraries remain limited, and existing formalized theorems focus almost exclusively on Shannon's foundational lossless theorems. This paper presents the first Lean formalization of core components of rate‑distortion theory, a central framework for lossy compression and modern machine‑learning analysis. We identify the alignment challenges that arise when translating information‑theoretic definitions, such as distortion measures, typicality and achievability, into a proof‑assistant setting, and we develop a reusable library of definitions and lemmas faithful to textbook conventions. Our formalization provides a foundation for future AI‑assisted proofs in information theory and enables subsequent mechanized developments to build on a coherent, verified infrastructure rather than generating long proofs from scratch.
Authors:Zhe Zhou
Abstract:
When a fact becomes harder to elicit from a language model, has the model lost it, or has it merely become less accessible? Parameter‑updating test‑time continual learning agents are usually evaluated by current output accuracy; we ask whether that measurement captures how prior experience shapes later updates. We test whether learning a specific factual binding produces a later relearning advantage beyond familiarity with its entity, answer vocabulary, format, or query slot. In Qwen2.5‑1.5B, we compare three matched, counterbalanced histories: LEARNED receives the correct binding, CONTROL receives matched exposure without it, and WRONG receives a competing relation‑valid answer in the same slot. After an identical rehearsal‑free continual fine‑tuning stream, LEARNED reacquires the target faster than CONTROL at a prespecified checkpoint of partial forgetting. WRONG shows no positive advantage over CONTROL, while its comparison with LEARNED provides slot‑matched corroboration, subject to possible competitor interference. The advantage remains positive after adjustment for measured pre‑relearning behavior. In a secondary descriptive conversion using an unpaired never‑seen pool, prior learning reduces the optimizer updates required for restoration by about 18%. Current behavioral accuracy can therefore underestimate the lasting effect of previously acquired experience. Code is available at https://github.com/zhezhou1106/relearning‑savings
Authors:Faridoun Mehri, Hinrich Schütze
Abstract:
Not all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond their own recall, from inert facts, which can be recalled but do not support such generalization in a given task. We study this distinction by constructing new relations from existing ones. Take the relation that maps a place to its continent (Paris ‑> Europe) and assign each of its answers an arbitrary new label (Europe ‑> crab, Asia ‑> zebra). The new relation then maps Paris ‑> crab. We fine‑tune the model on the new relation for a subset of subjects, without telling it the original relation or the relabeling rule, and test whether it assigns the corresponding labels to held‑out subjects that share an answer in the original relation. Having learned Paris ‑> crab, does it answer Munich ‑> crab? Across eleven language models and 70 relations, how linearly readable the original relation is in pretrained representations strongly predicts this generalization. Within each model, the two correlate across relations at Pearson r = 0.87 to 0.92. This association persists when controlling, separately and jointly, for the number of fine‑tuning subjects and answers, the model's accuracy on the original relation, and how often the subjects occur in pretraining text. It is also robust to re‑randomizing the label assignment and fine‑tuning seed, and to replacing the animal labels with labels from thirteen other semantic categories. When we map a fine‑tuned model's wrong label back to the original answer it was assigned to, Munich ‑> zebra becomes Munich ‑> Asia. Answers recovered this way are more often among the linear probe's top answers for the subject than among the pretrained model's own. Whether a new relation generalizes depends less on whether its facts can be recalled than on how readily their shared structure can be read out linearly.
Authors:Francesco Alemanno
Abstract:
In Hebbian learning from examples, a Hopfield network retrieves archetypes seen only through noisy examples. We study it in the exponential Hopfield model: one softmax attention step with the unlabelled examples as keys. A signal‑to‑noise analysis gives the overlap with an unseen archetype in closed form: a majority‑rule law in which an attention entropy replaces the dataset entropy. At high temperature the archetype is lost in the crosstalk; at low temperature the softmax freezes on the closest training example, as in a random energy model, and the layer memorizes. In between, in a window widening with the square root of the network size, the layer pools the examples of the correct class and retrieves the unseen archetype as well as their majority vote allows, at any fixed load. On structured datasets, such as MNIST and fashion‑MNIST, the best accuracy lies beyond the freezing.
Authors:Jian Gu, Hongyu Zhang, Chunyang Chen, Aldeida Aleti
Abstract:
The vocabulary of a language model is its native semantics, and the embedding and unembedding fix the meaning of a state at the two ends of the residual stream. Between them, existing methods reuse a fixed lens or treat states as a point cloud, and neither reads a state as the model does. We argue that every hidden state has a native reference frame, induced by the layers that remain and adaptive to the context. Lexical reference frames approximate it in a model‑native and context‑aware way, reading a state through the remaining layers with a given number of rounds of attention between positions and a typical attention output for the rest, fitting nothing to the model's output. No lens with fixed directions per layer can reach a mean cosine with the native frame's lexical readouts above the square root of their coherence, 0.30 and 0.35 at mid‑depth of SmolLM3‑3B and OLMo3‑7B. LexFrame[1], of order 1, reaches 0.97 and 0.97, and its readings are on average within 0.16 and 0.15 nats of the model's output. Its coefficient differences attribute predictions more faithfully than attribution patching, and its lexical basis steers predictions while sparing other tokens. On average, its readings settle on the model's top token over the final 0.82 and 0.81 of the layers, those of lenses over at most 0.11. These frames thus make every hidden state readable, attributable and steerable.
Authors:Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu, Yang Sui, Yangming Shi
Abstract:
Vision‑Language‑Action (VLA) models present a promising paradigm for general‑purpose robot manipulation, but closed‑loop deployment is often limited by inference efficiency: at each timestep, high‑resolution, often multi‑view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision‑critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control‑critical tokens. We introduce FastVLA, a training‑free, task‑aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task‑to‑vision attention and adaptively sets the compression rate from the importance‑distribution uniformity, measured by an inverse‑Simpson metric with online sliding‑window calibration. We evaluate FastVLA on OpenVLA, OpenVLA‑OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM‑backbone FLOPs by 35‑59% and action latency by 9‑32% (a 1.09×‑1.46× speedup) while maintaining task success, achieving a comparable or better efficiency‑performance trade‑off than existing plug‑and‑play token compression/pruning/reuse baselines. On the real robot, it cuts action‑prediction latency by 20% (1.25× speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.
Authors:Joran Michiels, Anis Ismail, Maarten De Vos, Alejandro Sifrim
Abstract:
Existing feature attribution methods that account for feature dependencies do not scale to high dimensions. We propose causal integrated gradients (CIG) and causal expected gradients (CEG), which extend integrated gradients to incorporate prior causal structure among input features while retaining its computational efficiency and, under conditions on the causal graph, its axiomatic guarantees. In many settings, some causal structure between features is already known, for example, that a transcription factor causally regulates a target gene's expression. Yet existing gradient‑based methods cannot exploit this structure, instead crediting a feature's downstream effects over its true upstream cause, such as ranking the target gene above the transcription factor that regulates it. Our key idea is to embed this causal structure by routing the integration path along the causal graph. The method thereby decomposes each feature's attribution into a direct effect and effects mediated through its causal descendants. We evaluate CIG in three settings of increasing realism. First, on a synthetic benchmark with a known causal model, CIG's attributions best match the ground‑truth causal effect from Pearl's do‑calculus, matching the accuracy of causal Shapley values at a fraction of the computational cost. Second, on simulated single‑cell gene expression data with a known gene regulatory network (Dyngen), CIG most accurately recovers the transcription factors driving the simulation. Third, on real interferon‑β stimulation data, where only an approximate causal graph is available, CIG better prioritizes the transcription factors driving the measured response than DeepSHAP and integrated gradients. Together, these results show that incorporating causal structure into gradient‑based attributions recovers upstream drivers that existing methods miss, without sacrificing scalability.
Authors:Venki Parsuram, Michelle Wang, Jin-Hyuk Hong, Pulkit Agrawal
Abstract:
We propose Compliant GMT (C‑GMT), a modular framework for full‑body compliant responses in general motion tracking. As humanoid robots move into real world deployment, they not only need to perform any motion but also respond safely and robustly to interaction forces. Existing general motion tracking policies can imitate diverse motions but are trained to be stiff, leading to unexpected and unsafe behaviors under disturbance. Prior work in compliant humanoid control trades off compliance and generality. There exist examples of full‑body compliant responses, but these are heavily curated to single‑reference policies. On the other hand, general motion trackers (GMT) are trained to rigidly imitate massive motion capture datasets. Our key insight is to leverage the generality of pretrained GMTs for motion tracking while introducing compliance from a residual adapter that conditions on a history of proprioception. This enables our approach, C‑GMT, to achieve both general motion tracking and whole‑body compliant responses. Compared to state‑of‑the‑art general motion trackers and single‑motion compliant policies, C‑GMT demonstrates competitive performance in both motion tracking accuracy and compliance adherence. We further demonstrate improved safety, robustness, and error tolerance in simulation and real‑world G1 experiments.
Authors:Nicolás Vera Zúñiga
Abstract:
Byte‑level language models such as the Byte Latent Transformer (BLT) run their large global model once per patch and start a patch where a small model's next‑byte entropy is high. This rule rarely starts a patch at in‑line computed results, such as the number after "=" in a worked solution, and accuracy on them collapses under tight patch budgets. We propose novelty patching, motivated by LZ77‑style compression: start a patch where the upcoming bytes are unlikely to repeat the recent context. The rule is a lookup table over the previous two bytes, fitted in seconds on raw text with no labels, reference model or losses. In BLT‑style models trained from scratch on a 1.9 GB math corpus at a matched ~11% patch rate (3 seeds each), it has no clear effect at 1M parameters, raises final‑answer accuracy from 47% to 72% at 12M, and at 50M raises computed‑result accuracy from 21% to 57% and final‑answer accuracy from 60% to 90%; a table fitted on the raw pretraining corpus keeps the gain. Outside math the advantage shrinks: novelty lowers bits per byte on prose and ties entropy on source code and on synthetic execution traces. An analysis of where boundaries fall explains the pattern. Computed results have ordinary surprisal for the patcher's model, so surprisal‑seeking rules pass over them: entropy concentrates patch starts on high‑surprisal bytes most strongly of all rules (boundary enrichment 1.61) and is the worst on computed results, while the best rules sit near chance enrichment. Novelty works where new content begins at segment starts close to the computation, as in written math; a learned chunker (H‑Net) finds such structure on its own, in math and in code. We also report what did not work: typed uncertainty as a training rule, switching rules after training, and a context‑aware novelty predictor.
Authors:Madhav Iyengar, Andreas Geiger, Hilde Kühne
Abstract:
Recent work shows that depth recurrence can be retrofitted into pretrained language models. In this work, we adapt this paradigm for visual understanding, using depth recurrence to reduce the parameter footprint of a pretrained vision‑language model (VLM). To this end, we prune layers from the language backbone of the VLM and loop over a subset of the remaining layers. At each step of the loop, a linear adapter merges the block's output with the token representations from the preceding layers and projects the fused representation back into the block. As the pruning impacts the original model performance, we propose a three‑stage recovery training: first, we leverage a language‑only pretraining to recover the performance of the language backbone, followed by an alignment of the vision‑language projector, and a full fine‑tuning of the model via visual instruction tuning. In the first and the last stage, we distill the outputs of the pretrained non‑recurrent model into the recurrent model, which further improves the recovery. We evaluate the proposed approach on the TinyLLaVA‑1.4B backbone without introducing any new vision‑language data during retrofitting. Our experiments demonstrate that the retrofitted recurrent model outperforms the unpruned baseline trained with the same recovery data while storing 23.3% fewer language parameters. We further show that the proposed retrofitting recurrence outperforms the setup of converting a recurrent LM into a VLM on the same data. Additionally, our ablations show how design choices, such as the adapter between recurrences and a larger recurrent block, lead to the improved performance of our approach. Finally, we find that our proposed model gains on reasoning‑oriented benchmarks, whereas its performance on basic visual understanding benchmarks remains on par with the baseline.
Authors:Xiuhui Zhang, Ning Gao, Xingyu Jiang, Yihui Chen, Yuheng Pan, Mohan Zhang, Yue Deng
Abstract:
Traditional reinforcement learning (RL) algorithms face significant limitations in handling long‑term tasks with sparse rewards. Recent advancements have leveraged large language models (LLMs) to enhance RL by utilizing their world knowledge for task planning and reward generation. However, planning‑based approaches often depend on pre‑defined skill libraries and fail to optimize low‑level control policies, while reward‑based methods require extensive human feedback or exhaustive searching due to the complexity of tasks. In this paper, we propose the Progress Reward Model for RL (PRM4RL), a novel framework that integrates task planning and dense reward to enhance RL. For high‑level planning, a complex task is decomposed into a series of simple manageable subtasks, with a subtask‑oriented, fine‑grained progress function designed to monitor task execution progress. For low‑level reward generation, inspired by potential‑based reward shaping, we use the progress function to construct a Progress Reward Model (PRM), providing theoretically grounded optimality and convergence guarantees, thereby enabling effective policy optimization. Experimental results on robotics control tasks demonstrate that our approach outperforms both LLM‑based planning and reward methods, achieving state‑of‑the‑art performance. Code is available at https://github.com/deng‑ai‑lab/PRM4RL.
Authors:Omer Imran
Abstract:
Radio technosignature searches rely on machine learning detectors to shrink the candidate pool before a human looks at it. They are validated on held‑out data from the training distribution, which excludes the cases where they are most likely to fail: a technosignature has unknown morphology, and terrestrial interference is unbounded in variety. I trained a standard competition detector once, froze it, and probed it with 15,000 injections across ten morphologies and five amplitudes, placed into real Green Bank backgrounds. Two directions were tested: genuine signals with morphologies other than a straight narrowband line, and interference resembling the training data. Recall did not follow the predicted ordering, and it fell as signals brightened, even for a needle of the kind the detector was trained on (from 0.590 to 0.090). One interference morphology reached a false positive rate of 0.627, comparable to the detector's own baseline recall of 0.5525 on genuine signals. Gradient‑weighted Class Activation Mapping (Grad‑CAM) showed attention along the injected path in one missed family by a pre‑specified test, and visually in a second. A pre‑specified, time‑averaged ON‑OFF filter removed false alarms and genuine detections at the same rate. The detector reads only the ON panels. Since the only evidence distinguishing a genuine candidate from terrestrial interference lies in the discarded OFF panels, it cannot tell if a signal originates from the target star; it can only tell if it resembles the training data. This explains the false alarm directly; the missed signals reflect what the detector learned to recognise, which reading the OFF panels would not by itself change.
Authors:Shubham Subhnil, Yannis Belkhiter, Erik Miehling, Adam Dahlgren Lindström, Elizabeth M. Daly
Abstract:
Verifying the final answer of a long reasoning trace may show that the trace is wrong but it does not distinguish three cases. The problem may lie beyond the model's capability, another attempt would succeed, or one step in an otherwise recoverable trace made the trace fail. Standard hidden‑state probes are trained on final‑answer correctness, a label that conflates these cases. Monte Carlo rollouts distinguish the cases and locate the failing step at additional inference cost. We test the hidden states of 4B and 8B reasoning models for the failure type and the failing step and compare each probe with the strongest baseline that uses no hidden states. On incorrect 8B traces, the confidence probe, a calibrated correctness probe at a midpoint cutoff, misses 81% of cases in which a single step makes the trace unrecoverable and 47% of cases in which another attempt would succeed. A probe with a different training design over the same states reverses the ordering of these two kinds of failure, so either ordering can be read from those states. At both scales, no single tested probe locates the failing step more accurately than a rule based only on trace length, and at 4B a probe trained to predict that step fails to outperform this rule in any of 10 runs on identical traces. The confidence probe therefore indicates that an answer is wrong, but its failure‑type ordering depends on its training design.
Authors:Niaz Mohaiman Abtahi, Evan Maksud, Reyana Shahid
Abstract:
Arabic calligraphy is a high‑stakes cultural visual domain for Muslim communities, yet VLM evaluations rarely test its fine‑grained script traditions. We audit 12 VLMs on a cleaned 1,195‑image six‑way benchmark using numeric, visual‑cue, and Diwani‑first prompts. In this closed‑set setting, the best numeric‑prompt accuracy is 60.9% for closed/API models and 47.2% for open‑weight models. The main failure is default‑label collapse: every numeric run is dominated by a single label taking 33‑63% of predictions, against an 18.8% majority‑class baseline. For 8 of 12 models the default is Thuluth; GPT‑4o labels 52.9% of all images as Thuluth although it is only 18.1% of the set. Prompting helps unevenly: visual cues move accuracy by more than 10 points for some models but hurt others, and foregrounding Diwani often relocates the default rather than removing it. Arabic calligraphy recognition is therefore a compact, culturally grounded stress test for VLM robustness and for measuring which visual traditions become legible to automated systems.
Authors:Niaz Mohaiman Abtahi, Reyana Shahid, Evan Maksud
Abstract:
Vision‑language models (VLMs) are increasingly used to interpret religious, educational, and culturally sensitive visual content, yet their reliability on citation verification remains underexplored. We introduce Visual Sanad, a controlled benchmark for testing whether VLMs falsely accept incorrect Qur'anic verse attributions when the same quote‑reference pair is rendered under different visual framings. The benchmark contains 2,400 rendered images built from 100 English Qur'an verses, four factual states per verse, and six visual conditions ranging from plain text to decorative and authority‑framed layouts. All quote‑reference pairs were source‑checked and human‑audited before rendering. Across 24 full model‑prompt runs, we observe a sharp capability split. Proprietary API‑served VLMs range from near‑perfect citation verification to moderate false acceptance, whereas recent open‑weight VLMs often show weak discrimination and default to VALID on most images. De‑biased prompting can reduce false acceptance, but often by shifting the decision threshold toward INVALID rather than by producing strong citation discrimination. Visual Sanad therefore exposes a culturally grounded multimodal factuality failure: fluent VLMs may treat displayed religious citations as self‑authenticating unless they have strong citation‑verification ability.
Authors:Zhao Yang, Zezhong Qian, Yinan Shi, Longjun Liu, Yawei Jueluo
Abstract:
Autonomous driving planning requires selecting trajectories that remain safe, rule‑compliant, and comfortable under complex multi‑agent interactions, yet robust closed‑loop decision making in planning‑critical scenes remains challenging. Existing diffusion‑ and RL‑based planners often rely on generic rewards and use relational structure mainly for representation, limiting how planning‑critical rollouts shape policy improvement. We present GRID‑Planner, a graph‑‑reward integrated diffusion planning framework that aligns generative planning with structured relational signals. Specifically, GRID‑Planner combines graph‑aware importance weighting, a time‑indexed relational reward surrogate, and an advantage‑weighted diffusion objective with on‑policy self‑distillation, forming a rollout‑return‑advantage‑update loop without explicit likelihoods. Experiments on nuPlan and interPlan show that, compared with the diffusion‑only baseline, GRID‑Planner reduces collision rate, lane deviation, and jerk by 35.8%, 60.8%, and 27.4%, respectively, and improves the planning score by 3.4%. Project website: https://github.com/yangzhaojason/GRID‑Planner.
Authors:Jon Škerlj, Sami Haddadin
Abstract:
Continuous physical human‑robot interaction requires robots to rapidly adapt their control behaviour without interrupting task execution. However, existing ROS 2 control architectures rely on stop‑to‑switch mechanisms that introduce substantial delays and interrupt execution. We present an open‑source, sub‑millisecond reflex engine based on an enhanced ROS 2‑compatible multimode controller that enables deterministic switching between control modalities and integrates reflex behaviours for mitigating contact hazards and kinematic limit violations. Experiments on a Franka Robotics manipulator demonstrate switching delays below 0.1 ms, more than four orders of magnitude faster than conventional ros2_control, while maintaining continuous operation during safety‑critical events. In a multi‑stage manipulation task, the proposed framework reduced execution time by 3.5% and peak trajectory jerk by more than threefold. By eliminating runtime middleware overhead, controller transitions are implemented as lock‑free pointer operations, enabling reflex‑level responsiveness within an open‑source robotics framework.
Authors:Wenzhe Wang
Abstract:
Hyperspectral images (HSIs) are often corrupted by Gaussian noise, impulses and stripes whose intensities vary across bands. The representative coefficient correlated total variation (RCCTV) model restores HSIs efficiently, but it lacks Gaussian and stripe terms, and its noise‑aware extensions require the noise level and a stripe budget to be specified in advance. This article proposes a self‑calibrated RCCTV (SC‑RCCTV) model, in which the noise parameters are estimated adaptively during the iterations. A band‑wise dense term and line‑constant column and row stripe terms are introduced into RCCTV. The noise level of each band is initialized by a stripe‑invariant median absolute deviation estimator and refined online from the residual, which turns the separation of sparse and dense noise into a band‑adaptive Huber loss, and the stripe thresholds follow from a significance test whose false‑alarm rate can be predicted analytically. No parameter is tuned for a specific image, and every subproblem of the alternating direction method of multipliers (ADMM) solver is solved in closed form. Under line stripes, SC‑RCCTV outperforms the parameter‑free FastHyMix by 8.17 dB on native‑size cubes, and, without any noise information, it exceeds S3TTV, which is given the true noise parameters, by 1.52 dB while running more than 1,900 times faster. On average, it stays within 0.01 dB of the same model given the true noise levels, and its online calibration improves on a two‑stage estimate‑then‑denoise scheme. Experiments on real Indian Pines and Urban data further delineate its scope for stripes that do not span entire lines.
Authors:Weiting Liu, Zhiyu Pan, Shuhong Wang, Zhenquan Yao, Jungang Li, Li Zhao, Jian Jiang, Jianfeng Feng, Ai Han, Wenlian Lu
Abstract:
Context compression in long‑running applications of large language models (LLMs) must preserve goal‑relevant information, often before specific downstream questions are known. We introduce GACC‑Bench, a benchmark for Goal‑Adaptive Context Compression that evaluates how reliably LLMs adapt information retention to specified high‑level goals. For each source context, we derive one or more high‑level goals by grouping source‑grounded questions according to shared downstream objectives. The benchmark contains 500 goals and 2,571 questions over 281 document and agent‑trace contexts from seven collections spanning diverse domains, with all instances reviewed by human experts. The compressor receives a source context, a goal, and a context‑length budget, with the associated evaluation questions withheld. A fixed reader answers these questions individually using the same compressed context. Our evaluation of ten LLM compressors reveals substantial gaps to full‑context performance even for the best‑performing model, particularly in supporting correct answers to every question associated with a goal. Compared with goal‑agnostic compression, target‑goal conditioning consistently improves accuracy, whereas mismatched‑goal conditioning reduces it. GACC‑Bench is available at https://github.com/weiting24/GACC‑Bench.
Authors:Povilas Daniušis
Abstract:
We propose the Graph Dirichlet Energy Causal Inference (GDECI) criterion for distinguishing cause from effect in bivariate settings. For a candidate direction X\to Y, we construct a kernel similarity graph from X and treat Y as a graph signal. The score is the Dirichlet energy of this signal, induced by the unnormalized graph Laplacian. The reverse score swaps the variables, and the lower‑energy direction is inferred as causal. The method requires neither regression fitting nor independence testing. After the empirical probability integral transform, its decisions are invariant under strictly monotone transformations of either variable, and the graph operations reduce to convolutions, allowing exact evaluation in O(n\log n) time and O(n) memory. At the population level, GDECI compares degree‑weighted Bayes squared prediction errors plus a non‑negative smoothness term of the regression function. As the bandwidth tends to zero, the weights become uniform and the smoothness term vanishes, reducing the criterion to a RECI‑type comparison of expected conditional variances in copula space. Empirically, we compare GDECI with QCCD, GRCI, CAM, IGCI (two variants), RESIT, LOCI, QPE‑k, and RECI (two variants), in accuracy on 1899 cause‑‑effect pairs and in inference speed. GDECI ranked fifth of eleven methods in macro‑averaged accuracy on 12 synthetic suites (0.776). On the real‑world Tuebingen pairs, it ranked third in unweighted accuracy (0.684) and fifth in weighted accuracy (0.678). It was also the fastest of the compared methods up to n\approx10^4. The accompanying code repository: https://github.com/povidanius/gdeci.
Authors:Yiqun Zhao, Binbin Huang, Haobin Duan, Zibo Zhao, Shenghua Gao
Abstract:
Generating 3D assets with explicit parts requires two properties at once: geometric detail inside each part and structural coherence across the whole object. Recent image‑conditioned 3D generation models adopt a structured 3D representation that anchors tokens to explicit spatial positions, capturing both properties, but their output remains a single fused mesh. We ask how a part‑decomposed generation model can inherit both properties of these holistic priors, and answer with STRUCTPARTS, a framework whose core technical contribution is a dual‑frame coordinate interleaving mechanism: each shape latent is interpreted under both a part‑local and an object‑global reference frame, and processed through a global‑local inter‑leaved transformer that simultaneously reuses the prior's geometric detail at the part scale and its structural coherence at the whole‑object scale. As its structural input, STRUCTPARTS uses a segmented mesh scaffold, a coarse mesh paired with a face‑level segmentation mask kept entirely within the prior's native 3D representation. With only lightweight finetuning, both intra‑part detail and inter‑part coherence emerge from the same pretrained weights. On PartObjaverse‑Tiny, STRUCTPARTS matches or surpasses prior part‑decomposed models at a fraction of their training cost, while naturally supporting part‑mesh refinement and part‑level editing.
Authors:Mathias Lowes, Kristine Aavild Sørensen, Maxime Sermesant, Rasmus R. Paulsen
Abstract:
Implicit Neural Representations (INRs) have recently gained popularity for their ability to model functions using simple and lightweight networks. These methods have also proven effective in deformable image registration. In this work, we extend INRs for deformable image registration to the domain of temporal image registration, focusing on periodic temporal image sequences. Our approach, Temporal‑IDIR, optimizes a single INR to model deformations across all frames in a temporal image sequence simultaneously, allowing for self‑regularization through its own deformation predictions. To achieve this, we introduce a temporal consistency loss that penalizes discrepancies between direct source‑to‑target transformations and those traversing intermediate frames. We evaluate our framework on the DIR‑LAB dataset, using the target registration error (TRE) between annotated and moved landmarks as the metric. Here, we achieve a TRE of 1.03 mm, outperforming other INR‑based registration methods. Additionally, our framework supports smooth interpolation between time frames by estimating deformations between the given input frames. Code is publicly available at https://github.com/MMLowes/Temporal_INR.
Authors:Cristian Curaba, Stefano Grimaz
Abstract:
In a network of networks, a dependency link runs from a supplier to the node it serves and binds the supplier's failure to the node's failure. Indices computed from the graph topology leave aside the service each edge carries, but in real interdependent systems that service decides how failure propagates: a pumping station requires electricity and telemetry, and losing either stops it; a district fed by either of two reservoirs survives the loss of one. Building on this observation, we label each node of the network with the service it provides. We then measure how far this qualitative input can change the node's vitality ranking: by fixing the graph topology we show that changing only the labels moves the Mean Operativity Loss from 0.023 to 0.114, while a ranking computed from the unlabelled graph topology is blind to the difference. The links give a node its suppliers; the labels group those suppliers into alternatives within one service and joint requirements across services. These labels should belong to the description of an interdependent system alongside the links themselves for a fidelity vitality measure
Authors:Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Sen Cui, Faguo Wu, Shiji Zhou
Abstract:
Large language model (LLM) agents have demonstrated strong capabilities in complex tasks, including code generation, tool use, and multi‑hop question answering. Existing self‑improvement methods enhance agent performance through model optimization, harness evolution, and executable artifact refinement, while recursive self‑improvement (RSI) further extends optimization to the improvement mechanisms themselves. However, adapting self‑improvement workflows to evolving performance bottlenecks while improving workflow orchestration through execution feedback remains a key challenge. We propose RSIFlow, a Recursive Self‑Improvement framework that integrates adaptive workflow orchestration with meta‑agent capability evolution. RSIFlow diagnoses performance bottlenecks from task‑agent trajectories and environmental feedback to dynamically select component‑specific improvement workflows. The meta‑agent then distills workflow outcomes into component‑specific skills and cross‑component principles, evolving its own harness to improve subsequent workflow orchestration. Experiments across six benchmarks and two backbone models show that RSIFlow achieves the SOTA average performance on all benchmarks. Ablation studies further demonstrate the contributions of adaptive workflow orchestration and meta‑harness evolution. Code is available at https://github.com/1HUAN1/RSI‑Flow.
Authors:Aleksandr Evgrashin, Carolin Schmitt, Timo Lübbing, Ashutosh Jha, Buket Naz Zeren, Sophie K. Bauer, Johannes Kopf-Beck, Anton K. G. Marx, Anamaria Semm, Gökce Ergün, Senya Polikovsky
Abstract:
Understanding human communication is a long‑standing goal in psychological research. People express themselves through many verbal and nonverbal channels: posture, gesture, gaze, facial expression, and speech. Measurement and annotation of these signals has traditionally required time‑consuming manual work by trained human labellers. The Nonverbal Interpersonal Communication Exploration (NICE) Toolbox is a free, open‑source Python framework that automatically measures these signals by combining existing state‑of‑the‑art computer vision and speech‑processing methods into a single software package. It includes support for multi‑camera systems, interchangeable detection algorithms, 2D and 3D visualization of detector outputs, evaluation on the user's custom data, and export to third‑party software. With the release of this tool, we aim to make data exploration of interpersonal communication signals scalable, reproducible, and accessible for psychological research.
Authors:Xin Zhang, Robby T. Tan
Abstract:
Multi‑modal semantic segmentation benefits from complementary sensors, but this benefit is fragile when some modalities are degraded or missing at test time. Many existing methods rely on dedicated fusion modules or multi‑branch cross‑modal encoders, and the fusion often becomes unreliable when modalities are absent, leading to substantial performance drops. We propose FuseAdapt, a parameter‑efficient framework that performs fusion within the adaptation space of a frozen vision foundation model (VFM), exploring whether VFM priors can serve as an effective shared backbone for multi‑modal segmentation. Instead of attaching a standalone fusion network, FuseAdapt assigns each modality a lightweight adaptation branch and performs \emphadaptation‑space fusion by keeping Q/V low‑rank adaptation private to each modality and routing only FFN low‑rank residual activations across modalities inside the frozen backbone. The pretrained VFM remains fixed while modality‑specific corrections and explicit cross‑modal interaction are learned through the adaptation paths. To handle missing modalities, we further introduce Predictive Modality Modeling (PMM), a direct representation‑level objective for cross‑modal redundancy. During training, PMM masks a target modality and predicts its embedding from the visible ones, supervised by a teacher target computed under full‑modality input. PMM is used only during training and adds no inference‑time overhead. Across three multi‑modal segmentation benchmarks (MCubeS, DELIVER, MUSES), FuseAdapt consistently improves both modality‑complete and modality‑incomplete segmentation with a small trainable budget, yielding up to +3.01 mIoU under full modalities and +12.56 mIoU under missing modalities over prior state‑of‑the‑art. Project page: https://devinxzhang.github.io/FuseAdapt‑Page/.
Authors:Kehao Zhou, Yixuan Li, Yuxuan Jiang, Fan Zhang
Abstract:
Recent advances in generative artificial intelligence have accelerated video generation technologies. However, AI‑generated videos (AIGVs) remain susceptible to various distortions, highlighting the need for dedicated video quality assessment methods. Content semantics play an important role in AIGV quality assessment, with source prompts typically used as textual references. However, these prompts may not accurately reflect the generated video content, resulting in a discrepancy between the intended textual description and the generated result. To address this limitation, this paper proposes VGCap, an AIGV quality assessment framework guided by visually grounded captions that align better with generated content. Specifically, we introduce two forms of captions: a global caption obtained by minimally revising the source prompt based on multi‑frame visual evidence, and prompt‑independent segment captions that describe localized temporal content. Together, these captions provide complementary textual evidence for assessing spatial quality, temporal quality, and text‑video alignment. Experimental results on the LGVQ and FETV datasets show that VGCap outperforms representative baselines across all three dimensions. Further ablation studies on caption sources and temporal components confirm the effectiveness of visually grounded captions for each quality dimension, while highlighting the benefits of segment‑level features for temporal quality prediction. The source of VGCap will be available at https://github.com/lolo123‑123/VGCap.
Authors:Owen Walters, Lihu Chen, Francesca Toni
Abstract:
Web search allows LLMs to access up‑to‑date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available. To study this problem, we construct temporal facts whose values changed after the models' training cutoffs. We evaluate one proprietary model under deployed web search and six open‑weight model configurations under controlled retrieval. We find that models often answer from search snippets without opening the source pages, which is associated with higher rates of outdated answers and unfaithful citations. Our controlled experiments and layer‑wise analysis reveal a gap between the availability of evidence and its effective use. Even when retrieved evidence influences internal predictions, models may still favour outdated information in their final answers. We hope these findings will help advance research on the reliable use of retrieved evidence in search‑augmented LLMs.
Authors:Yuanjian Xu, Jianing Hao, Tianze Sun, Yongyao Duan, Sam Chen, Jinyi Han, Wanbo Zhang, Yuxuan Sun, Guang Zhang, Zhong Li
Abstract:
Data strategy plays a central role in training large language models (LLMs). It governs the selection, weighting, and scheduling of training data, thereby determining the capabilities a model can acquire from a fixed data budget. We treat high‑quality data as having high training‑dependent utility, which depends on the target objective, the model, and its training state rather than on the text alone. Research in this area now spans pretraining and post‑training, but related ideas use inconsistent terminology and evaluation protocols. This work reviews data strategies across the training lifecycle and formulates them as policies that allocate a training budget over candidate examples. We organise methods by their decision granularity, model dependence, and temporal adaptivity, then compare their computational requirements and empirical support. Fine‑grained or state‑dependent policies offer more control but incur additional scoring, search, or rollout costs. Evidence for transfer across model scales and comparable end‑to‑end cost remains limited. A continuously updated list of papers, along with their public implementations, is available at https://github.com/xuyj233/data_strategy_survey.
Authors:Chanwoo Kim, Tackgeun You, Hwasup Lim
Abstract:
We introduce Edit4D, an editable text‑to‑4D scene generation framework that constructs 4D scenes from separate background and dynamic character representations. Recent advances in generative models have enabled text‑driven dynamic scene synthesis, but editable 4D scene generation remains challenging due to the need to preserve spatial geometry and temporal dynamics. Existing approaches typically generate holistic scene videos, where the background and dynamic characters are coupled, making object‑level editing and flexible camera control difficult. Edit4D addresses these limitations through a stage‑specific generation pipeline. Given a user instruction, Edit4D first decomposes it into stage‑specific prompts for background, dynamic character, and scene‑character composition. Using these prompts, we synthesize multi‑view scene images with text‑to‑image and camera‑conditioned generative priors and reconstruct a background 3D Gaussian splatting representation. We then generate a character image, animate it with an image‑conditioned video prior, reconstruct a 4D mesh, and bind Gaussian primitives to obtain a character representation. Finally, both components are composed through text‑guided segmentation and contact‑aware optimization. A global scene refinement is then performed to improve harmonization and rendering consistency. By integrating stage‑specific components, Edit4D enables controllable camera motion, flexible object‑level editing, and the practical construction of editable 4D scenes. Project page: https://chanwo0kim.github.io/Edit4D/
Authors:Zhanli Li, Zeyuan Xu, Chaoxu Pang, Zeyuan Liu, Anyang Ji, Te Sun, Kongcheng Zhang, Wenbo Xu, Xubin Yue, Shipeng Luo, Junjie Zhao, Yingfan Hua, Jingqing Ruan, Zhongda Su, Feng Hong, Zhizhao Zeng, Ke Zeng, Xunliang Cai, Hanjia Ye
Abstract:
Data agents use code to process heterogeneous files and derive analytical insights. Scaling their training faces costly environment curation, privacy and intellectual‑property restrictions, and difficulty verifying open‑ended findings without known generative truth. Our core insight is to specify the world behind the files before generating the files themselves. We introduce Data World, a multi‑agent engine that synthesizes environments, tasks, and rubrics from explicit mechanisms. Independent audits check target recoverability from observed files; rubrics support evaluation and RL rewards. Human preference‑based curation for question realism selects 100 tasks across 50 held‑out worlds and 16 domains for DataWorld‑Bench. The engine also yields 2,135 training tasks from 1,037 disjoint worlds. Evaluation of ten frontier models shows that the benchmark remains challenging, particularly for insight discovery, with a consistent computation‑insight gap. RL‑zero training on the synthesized tasks produces DataWorld‑35B‑A3B from Qwen3.6‑35B‑A3B without trajectory distillation. Under matched evaluation, it improves over its base by 7.06 points on DataWorld‑Bench, 14.28 on DAComp, 20.90 on DABstep‑Hard, and 12.13 on JobBench, a broader occupational benchmark.
Authors:Asanobu Kitamoto, Victor Le Louarn, Noelia Falczuk, Ngoc Anh Tong, Anja Veleva
Abstract:
This paper presents the Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years, designed to benchmark machine learning models on long‑term spatio‑temporal data. The latest addition to Dataset V3 is multispectral satellite data, enabling multi‑channel models. We investigate data‑splitting strategies and present benchmarking results for the Northern and Southern Hemispheres. Experimental results show that, under the current dataset construction and experimental setting, multi‑channel inputs do not clearly outperform single‑channel inputs for pressure regression. Among the evaluated model architectures, image types, and spectral channels, ResNet‑18, full‑resolution images, and Channel 1 generally provide the best single‑channel performance for pressure regression. Performance also differs between the WP and AU datasets by regression target, with lower RMSE on WP for pressure regression and on AU for wind regression. These results highlight the importance of considering dataset‑specific and task‑specific characteristics when interpreting benchmark performance. Additionally, we analyzed systematic bias in intensity regression, which leads to underestimation of intensity for strong typhoons. We tested imbalance‑aware weighting strategies, including inverse‑density weighting inspired by BalancedMSE, and showed that these approaches are not sufficient to remove the bias. The dataset is publicly available at http://agora.ex.nii.ac.jp/digital‑typhoon/dataset/ and https://github.com/kitamoto‑lab/digital‑typhoon/.
Authors:Florian Mai, David Kaczér
Abstract:
Finetuning a language model on narrowly misaligned data can make it broadly misaligned. Interleaving generic safety data is the most effective known in‑training defense, but the misalignment returns as soon as a prompt resembles the finetuning context. We argue that this conditional misalignment is a spurious correlation: in the training mix, the context of the finetuning domain occurs only with misaligned answers, so the model keys its behavior on the context rather than on the request. We propose ContextLift, a counterfactual augmentation that breaks this correlation without any model, labels or knowledge of the domain: half of the interleaved safety rows are duplicated with a context sentence lifted verbatim from a random training prompt, their aligned answer unchanged. On Qwen2.5‑7B finetuned on bad security, legal and medical advice, generic interleaving leaves 18.9%, 41.9% and 58.2% misaligned answers under a domain cue; ContextLift reduces this to 2.6%, 6.1% and 12.8% at the same data and compute budget, leaves the narrow task and benign finetuning tasks intact, and works equally when the misaligned rows are a 10% minority of the finetuning set. Code: https://github.com/davidkaczer/contextlift
Authors:Yuanjian Xu, Jinyi Han, Jianing Hao, Yuxuan Sun, Fanyang Lu, Richeng Xuan, Guang Zhang, Zhichao Hu
Abstract:
Real‑world tasks are often complex and noisy, posing substantial challenges for LLM agents. Yet existing evaluations lack a systematic framework for jointly characterizing potential perturbation targets and types. We use this two‑dimensional view to conduct a unified study that quantifies the robustness of current frontier models, traces the mechanisms behind perturbation‑induced failures, and draws implications for robustness‑oriented training. Through controlled evaluations on APEX‑Agents and Toolathlon, we find that perturbations consistently degrade performance, while the dominant vulnerabilities vary across task environments. Trajectory‑level analysis reveals that agents rarely recognize altered inputs and often propagate corrupted information without verification. A focused cost analysis further shows that successful perturbed trajectories can require substantially more computation. These findings motivate training objectives that cover diverse forms of noise and reward verification, uncertainty awareness, and recovery. We provide a rigorous benchmark‑construction pipeline. Code and data will be released at https://github.com/xuyj233/AgentPerturb.
Authors:Zewen Wu, Ruizhe Liu, Pei Zhou, Feng Chen, Yanchao Yang
Abstract:
Recent Vision‑Language Models (VLMs) enable automated preference labeling for robotic manipulation, yet their deployment remains impractical since each query incurs substantial inference cost, while VLMs frequently abstain on visually challenging comparisons. We propose SPEAR, a framework that transforms VLMs into efficient preference oracles via two synergistic mechanisms. First, we design distinguishability‑aware query selection to reduce abstention rates by targeting only informative and distinguishable comparisons. Second, we deploy an adaptive reward transition that bootstraps exploration with VLM‑synthesized guidance before preference‑based refinement. Experiments on the manipulation benchmarks demonstrate that SPEAR achieves 88.9% average success, outperforming baselines by 23.3 percentage points, while significantly reducing abstention and wall‑clock training time on contact‑rich tasks. SPEAR learns three Mobile ALOHA manipulation tasks fully online on real hardware, demonstrating end‑to‑end real‑world applicability. By addressing query efficiency and exploration challenges, SPEAR establishes VLMs as practical preference oracles for scalable robot learning without manual reward engineering.
Authors:Sibaek Lee, Kyeongsu Kang, Seongbo Ha, Yewon Jeon, Joonyeol Choi, Daegeol Ko, Jaehyuk Hur, Taemin Choi, Minsu Kim, Hyungjoo Ahn, Wonsok Yoo, Hyeonwoo Yu
Abstract:
Distilling vision‑language features into 3D Gaussian Splatting yields a language map that can be queried with arbitrary text. In deployment, however, a small set of user‑specified targets must be reliably located, which current distilled maps often miss, while open‑vocabulary queryability for other words must remain intact. We propose a training‑free, target‑aware refinement of an already‑distilled language map. Guided by a text‑promptable segmenter, the visual embedding inside each detected target region is blended with the target text embedding, a convex combination whose coefficient sets the amount of injected prior, while non‑target regions are left untouched. Injecting too much overwrites appearance cues and general queryability, so we seek, for each target, the smallest coefficient that still lets it win, and we obtain every coefficient in closed form, leaving no hyperparameters to tune. We further identify a failure mode of injection in language‑goal localization. Text embeddings share a common component, and injecting them makes the standard cosine read‑out query‑agnostic, which a centering step in the read‑out removes. Across three benchmarks, the refinement consistently improves open‑vocabulary segmentation, roughly doubling the base mIoU on Replica, and localizes language goals more reliably than the base map. It also transfers as a plug‑in to off‑the‑shelf field‑based and object‑level language maps and supports a real‑world demonstration in which a legged robot navigates to natural‑language goals. Project page and code: https://lab‑of‑ai‑and‑robotics.github.io/target‑aware‑langmap/
Authors:Islam Mesabah, Sary Elmansoury, Gerrit Großmann, Peter Neigel, Sebastian J. Vollmer
Abstract:
Decision trees can make vision‑language model (VLM) classification interpretable by replacing a single prediction with a sequence of root‑to‑leaf questions. However, the accuracy cost of this greedy, non‑backtracking procedure remains unclear. We introduce MYOPIA, a benchmark comprising four LLM‑generated, human‑refined decision trees for GTSRB, MNIST, CIFAR‑10, and MSCOCO, together with matched prompts, diagnostic protocols, and an evaluation harness. Across GPT‑4o, Qwen‑VL Max, and LLaMA‑3.2 11B Vision, tree traversal underperforms zero‑shot prompting in all 12 dataset‑‑model pairs; paired bootstrap analyses consistently favor zero‑shot. Our diagnostics characterize interacting failure modes associated with this gap. Models often recall tree‑encoded class attributes more accurately than they ground the same attributes in images: the mean label‑‑image accuracy gaps are 9.5, 13.8, and 12.6 points for GPT‑4o, Qwen‑VL Max, and LLaMA, respectively. Incorrect local decisions then irreversibly exclude the correct class, while answer extraction introduces additional model‑dependent loss. A node‑level audit of the four trees finds that split entropy is weakly but consistently associated with lower node accuracy, whereas depth, branching factor, and within‑node grounding differences are less stable. MYOPIA therefore provides a framework for diagnosing hierarchical visual classification and motivates testing tree designs that prioritize perceptual answerability and recovery from early errors. Code is available at https://github.com/islammesabah/VLM_Classification.
Authors:Zhiyuan Wang, Yuezhi Zhou, Ziqi Zhang, Zhu Zhang, Zhixing Tan, Yongheng Deng
Abstract:
On‑device deployment of large language models (LLMs) is increasingly adopted due to latency and privacy requirements, but exposes proprietary models to reverse engineering and intellectual property theft. Trusted Execution Environments (TEEs) provide hardware‑enforced isolation for on‑device inference, yet most TEE‑based parameter obfuscation relies on unilateral matrix transformations that obfuscate only the output side. We identify a vulnerability in this design, termed the Solution Determinacy Property (SDP): the obfuscation secret is uniquely determined, exposing the structure of the true key and enabling attackers to break the protection. Under our attacks, representative methods GroupCover and ArrowCloak are reduced to negligible protection, with attack accuracy reaching over 2.4 times the full‑shielding black‑box baseline and recovering near‑complete model functionality. To address this vulnerability, we propose BiShield‑TEE, a bilateral framework that induces Ambiguous Solutions through joint obfuscation of input and output dimensions, hiding the true key among many equally valid candidates. Experiments across diverse models and datasets show that BiShield‑TEE confines attack accuracy to the full‑shielding black‑box baseline, while achieving higher inference efficiency than prior methods on real hardware. Our code is available at https://github.com/laduiw/BiShield‑TEE.
Authors:Sergio Burdisso, Esaú Villatoro-Tello, Dairazalia Sánchez-Cortés, Yanis Labrak, Jordi Luque, Milos Cernak, Petr Motlicek
Abstract:
Large language models (LLMs) deployed as conversational agents are increasingly expected to follow procedural logic. While factual hallucination has received extensive attention, a different failure mode has been overlooked: procedural hallucination, in which an agent produces individually fluent and factually plausible utterances while deviating from the intended conversational process. We introduce DialFork, the first benchmark to measure this failure mode on raw dialogue text. DialFork is built by forking conformant dialogues at controlled turn positions and injecting one of four hallucination types, yielding 36,278 dialogues across 24 tasks in 12 domains; 30 human annotators validate the construction. We define three evaluation tasks that characterize procedural deviation at three granularities: how far a set of dialogues deviates from conformant ones (monitoring deployed agents), how far a single dialogue does (procedural retrieval), and at which turn a dialogue first goes off‑track (real‑time guardrails). For the first two, we propose dialog‑flow‑aware metrics, FlowKL and FlowScore, that compare dialogues by how they unfold rather than their content. At the dialogue level, FlowScore significantly outperforms all baselines across seven encoders, showing that semantic similarity is no substitute for procedural similarity. We release DialFork under open licenses at https://github.com/idiap/dialfork, hoping the community will measure conversational process alongside content.
Authors:Vasileios Perifanis, Nikolaos Pavlidis
Abstract:
Federated learning (FL) is increasingly used to adapt a pretrained, centrally fine‑tuned model to data that remain on user devices, beyond training a model from scratch. In this report, prepared for the 4th Federated Machine Learning International Summer School, we study the privacy and integrity of this approach with a reproducible pipeline. An ImageNet‑pretrained MobileNetV4‑Conv‑Small model is fine‑tuned centrally on ten Quick, Draw! classes (97.50% test accuracy) and then adapted with head‑only FedAvg over ten non‑IID clients, which raises client training accuracy from 90.34% to 91.49% while preserving global accuracy. We evaluate four adversarial scenarios. First, from a single one‑step client update, an honest‑but‑curious server recovers the complete private label set, but its reconstructions follow class prototypes rather than the private drawings (mean SSIM 0.130). Second, from a one‑round FedAvg aggregate, label‑count recovery falls from 87.5% with one client to 75.0% with five and ten clients, below the 80.0% of an uninformed uniform guess. Third, under FedAvg, a backdoor injected by 30%, 50%, and 100% of the clients reaches an attack success rate (ASR) of 21.9%, 31.3%, and 54.7% at a cost of at most 0.8 percentage points of clean accuracy, whereas label flipping is effective only when most of the source‑class data are relabeled. Fourth, FLTrust with a root set retained from centralized fine‑tuning lowers the backdoor ASR to 1.8% and 4.0% at 30% and 50% malicious clients and the label‑flipping ASR from 12.6% to 0.8% without measurable clean‑accuracy loss, but provides no protection when every client is malicious. The report closes with a live session in which participants draw, fine‑tune, and poison the model through Indigma's FedMaestro orchestrator. Code and executed notebooks, runnable on Google Colab, are at https://github.com/Indigma‑Innovations/fl‑summer‑school.
Authors:Zhiyuan Ma, Jiabao Wei, Tianyu Wang, Hanqing Wang, Jiadong Pan, Haowen Xu, Liangliang Zhao, Ruixun Liu, Yuhan Wang, Yuwei Zhang, Liang Feng, Junhao Xiao, Kaiyan Zhang, Xuekai Zhu, Daizong Liu, Biqing Qi, Yihao Liu, Ganqu Cui, Ning Ding, Linfeng Zhang, Jianjun Li, Kun He, Xiang Bai, Bowen Zhou
Abstract:
Diffusion models (DMs) have steadily demonstrated outstanding performance in high‑fidelity, controllable generation across diverse scenarios, following the "pretraining‑finetuning" paradigm established in LLMs. Despite these advances, conventional DMs still face critical limitations in sophisticated reasoning, autonomous exploration of generation pathways, and adaptability to complex, human‑aligned objectives. Inspired by RLHF's success in LLMs, RL has recently emerged as a pivotal post‑training mechanism to iteratively refine DMs through reward‑driven trajectory optimization. This paper presents the first comprehensive review of RL‑driven DMs and an empirical analysis of how RL reshapes the diffusion process towards experience‑centric, goal‑directed generation. We propose a novel technical panorama elucidating RL's multifaceted roles in diffusion‑based language, multimodal, robotic, and scientific models, with a granular focus on diffusion trajectory optimization and reward engineering. Through extensive experiments on multiple task‑specific benchmarks, we systematically evaluate representative methods to intuitively highlight the advantages of different RL post‑training strategies. Furthermore, we reveal and dissect three crucial bottlenecks that currently hinder the real‑world deployment of RL‑driven DMs: reward design fragility, exploration‑exploitation trade‑off in high‑dimensional generation spaces, and training instability under sparse rewards. Based on these perspectives, we provide a principled roadmap for the integration of RL into DMs and outline promising future research directions for robust, interpretable RL‑diffusion systems. We also construct a public repository to continuously track the latest developments in this rapidly evolving field: https://github.com/MAIR‑Lab‑HUST/Awesome‑RL‑for‑Diffusion‑Models.
Authors:Zejian Kang, Yifu Luo, Boyang Shen, Jingxuan Li, Zekai Chen, Zekun Shi, Yongxue Xu, Zhenyu Hou, Bingqian Wu, Shaolei Zhang, Wentao Yang, Xiangru Huang
Abstract:
Language‑conditioned humanoid motion generation enables users to create and refine robot movements through natural language. The central challenge is to connect an underspecified request to concrete motion structure that can subsequently be revised relative to an accepted reference. We present TRACE, a generation and editing framework for the Unitree G1 humanoid robot built around typed, measurable kinematic quantities. Our insight is to ground generation conditions in paired training motions and editing goals in the current motion. A language model learns to predict support, root, and hand events from motion‑derived supervision; their time‑aligned encodings complement the original text in a diffusion generator. Gravity‑referenced states and trajectory‑‑contact objectives improve kinematic consistency. For editing, typed requests define body‑‑time scopes and source‑relative goals. A separate reference‑conditioned diffusion editor uses these goals and revised events to realize changes, while sampling‑time projection preserves protected state coordinates. Quantitative evaluation and human assessment show improved semantic alignment and motion quality over the compared methods. Multi‑turn editing evaluations demonstrate more reliable execution of successive relative requests under shared preservation constraints. Code is available at https://github.com/Multimodal‑Perception‑Action/TRACE.
Authors:Syed Rifat Raiyan, Md. Tariquzzaman, Md Farhan Ishmam, Tenghao Huang
Abstract:
Every story has two orders: the order in which its events happen and the order in which they are told. A story is nonlinear when the two differ. We introduce a benchmark of 83 published works that measures whether language models can tell nonlinear stories. For each work, we annotate one list of events in both orders, together with a story graph of causal dependencies, character knowledge, and the questions the telling raises and resolves. Given events in chronological order, all 19 models tell them, on average, more chronologically than the published works do; asked for a specific structure, they land within the human range about half the time. We then compare a model's stories of the same events under different conditions, using both an automated judge and 71 human readers. Both prefer the chronological story a model writes by default to the nonlinear one it writes on request, which readers find harder to follow in 81% of decisive judgments. Readers also prefer stories written from the story graph to stories constrained to the published telling order (72% of decisive judgments), a difference the judge does not detect. Our results suggest that following an instruction to reorder a story is not yet the same as telling it well out of order.
Authors:Wei Zeng, Ye Wang
Abstract:
Transcribing expressive piano performances into scores requires interpreting individual note timings within an underlying metrical structure. However, score attributes are associated with notes, whereas beats may occur without coincident note onsets. We introduce DuET, a Dual‑Encoder Transformer that jointly learns note‑level performance MIDI‑to‑score (PM2S) transcription and frame‑level beat and downbeat tracking. Bidirectional cross‑attention connects note and frame encoders, while an autoregressive decoder alternates between their representations to predict structured score attributes. We further introduce TimeRoPE and Time‑Pitch Rotary Positional Encoding (TPRoPE) to establish a shared performance‑time reference and explicitly encode relative temporal and pitch relationships. Evaluations on ASAP, a classical piano dataset, and our curated Gangqinpu dataset, which spans multiple genres with a focus on popular piano music, demonstrate improvements in score timing and notated duration. Compared with the strongest baseline on each dataset, DuET reduces MUSTER mean error from 6.56% to 3.64% on Gangqinpu and from 11.30% to 11.22% on ASAP. Ablations support the benefits of frame‑level context, time‑based positional encoding, and decoder‑history masking. Code, weights, and demos are available at https://github.com/wei‑zeng98/DuET.
Authors:Hui Yang, Ziyu Li, Xuan Liu, Anders Jarfors, He Tan, Jiaoyan Chen
Abstract:
Ontologies are widely used to represent and organize complex knowledge in domains such as biomedicine and healthcare. However, constructing reliable ontologies remains challenging, as class hierarchies, whether manually or automatically constructed, may contain subtle subsumption errors that require careful refinement. We study this problem in CASTON, a casting‑domain ontology, and present a lightweight but scalable refinement workflow that combines a fine‑tuned ontology embedding model with Large Language Models (LLMs). The embedding model is used to extract a limited number of candidate axioms for refinement together with possible replacement candidates, and an LLM then returns repair suggestions. In a small expert evaluation, the default configuration using a CASTON‑fine‑tuned OnT embedding model and Qwen3‑8B produced reliable high‑confidence repairs: all six high‑confidence replacements were accepted by both domain experts. The implementation is available at https://github.com/HuiYang1997/OntRefine.
Authors:Revanth Reddy Lingala, K. Gangadhara Rao, Padigela Srujan, Saarthak Manocha, Vainavi Venkat
Abstract:
Data encrypted today with classical public‑key algorithms could be decrypted once large quantum computers exist. Cloud object stores keep data for years, so the client‑side encryption that protects it must move to post‑quantum algorithms such as ML‑KEM, which adds a key‑encapsulation step and a larger ciphertext to every stored object. So far, the cost of ML‑KEM has been measured for the algorithm on its own and for securing network connections, but not for encrypting each object in a storage system that serves many clients at once. It has therefore been unclear whether ML‑KEM slows such a system, under which conditions, and why. We address this problem by measuring per‑object ML‑KEM against classical key exchange on an S3‑compatible object store, with two client implementations, two processor types and up to 500 concurrent workers, using pre‑registered experiments and statistical equivalence tests. ML‑KEM reduced throughput only when the client's processor was at or near full load, with one unexplained exception, and then by 4.0 to 4.9% in our main client. Controlled experiments on one processor type traced most of that loss to the way the software library was called, and a second, native client showed that the size of the loss depends on the implementation. The lasting cost is storage: 1,056 more bytes per object than with classical key exchange. The results cover two clients, one single‑node store and one cloud provider.
Authors:Zixuan Wang, Yuqi Liu, Yuxin Chen, Sengqiao Yang, Jinhui Ye, Pengguang Chen, Changsheng Lu, Shu Liu, Bei Yu, Jiaya Jia
Abstract:
While generalist vision‑language‑action (VLA) models have enabled diverse open‑world manipulation, real‑world robotic assistance requires two critical capabilities they currently lack: personalization and reliable execution. First, executing an instruction like "make me a sandwich" requires grounding subtask decomposition in persistent user preferences rather than relying on a single generic solution. Second, reliable deployment demands explainable safety assurance that proactively prevents hazards and gracefully recovers from occasional physical failures. To address these gaps, we present CarryOn, a dual‑loop harness designed to augment a frozen VLA policy. The planning loop maintains persistent user‑level and scene‑level context that is continually grounded against new observations and revised through interaction, enabling personalized task selection and execution grounding across conversations and changing scenes. The execution loop actively supervises policy rollouts, shielding actions from predicted hazardous contact. When execution fails, the system invokes reusable control‑based recovery skills that restore task progress and are refined through reflection on earlier failures. On a real robot and in simulation, CarryOn dispatches preference‑consistent subtasks across users and scenes, reduces the violation rate of an already safety‑finetuned VLA from 52.6% to 6.1%, and increases recovery success from 13% to 80% when refined skills are reused.
Authors:Fryderyk Kögl, Anna Reithmeir, Daniel M. Lang, Emily Chan, Rickmer Braren, Julia A. Schnabel
Abstract:
We adapt the diffeomorphic coarse‑to‑fine LapIRN architecture for longitudinal whole‑body PSMA PET/CT registration in the Learn2Reg 2026 PSMAReg challenge. The method combines affine pre‑registration, multi‑modal deformable registration, and test‑time instance optimization. We further introduce PET biomarker preservation losses for metabolic tumor volume (MTV) and total lesion glycolysis (TLG), including Jacobian‑based constraints, and a per‑structure rigidity regularization for skeletal structures. The submitted configuration achieved 81.0% DSC and 6.17 mm HD95, with MTV and TLG errors of 3.3% and 3.3%, respectively, on our evaluation protocol. Code is available at https://github.com/koegl/psmareg.
Authors:Ruibin Ju, Ruifeng Cai, Min Xu, Lin Wang
Abstract:
Aerial vision‑language navigation (VLN) agents built on large vision‑language models are expensive to query. Executing a complete predicted action chunk before replanning amortizes this cost, reducing model inference from 195 to 42 ms per executed waypoint for our policy. It also moves the states at which the policy is queried away from expert demonstrations, which supervise neither recovery from these states nor termination at the learner's own arrival. On OpenFly test‑seen, no‑stop failures account for 47.78% of full‑chunk behavior cloning (BC) outcomes. We present X‑Fly, a staged training framework for full‑chunk aerial VLN. Recovery DAgger labels learner‑visited decision states with smooth chunks that reconnect to future expert segments and with STOP at the first in‑region state. Flow‑GRPO then jointly refines the chunked flow controller and a Bernoulli STOP policy from same‑task closed‑loop outcomes, supplemented by outcome‑verified stopping supervision. With eight controls per decision, X‑Fly reaches 74.18% SR on test‑seen and 51.73% on test‑unseen, compared with 16.45% and 18.43% for full‑chunk BC and 32.04% and 29.46% for DreamFly.
Authors:Skylar Zhai, Haonan Wang, Zhining Liu, Linxin Song, Xuwei Ding, Jingcheng Liang, Jian Kang
Abstract:
Language models are increasingly trained against several verifiable rewards at once, such as the correctness and the format of a tool call, or the individual constraints of a complex instruction. Reward‑decoupled normalization (GDPO) normalizes each reward separately so that different reward combinations keep distinct advantages, but it still spreads every reward's advantage evenly over the whole response, although each verifier checks only a few positions. We propose MRTPO, which assigns the advantage of each reward to the tokens relevant to it, using the information its verifier checks, without auxiliary models or token‑level annotations. For each reward, the policy replays its own response with and without that reward's verifier information, and the per‑token Jensen‑Shannon distance between the two next‑token distributions measures each token's relevance and reassigns the advantage. On text‑to‑SQL, multi‑constraint instruction following and tool calling, MRTPO converges faster and achieves better performance than the compared methods.
Authors:Shukesh Reddy, Abhijit Das
Abstract:
Existing deepfake detection models are built upon manipu‑ lated images that are tampered with either exclusively fully swapped or fully synthesised faces, which leads us to an open question: Can off‑the‑shelf deepfake models detect manipulation effectively while tampering is confined to a local facial sub‑region? To fill this gap, we introduce RLDF (Region‑Level Deepfake), the first dataset for region‑level deepfake evaluation. Employing the proposed region‑aware partial swap synthesis model, we generate over 2M par‑ tial face‑swap images by using eight face‑swapping meth‑ ods. Altogether, we considered seven face regions, among which there are four discrete regions (eye, nose, mouth and eyebrow) and four composite facial regions (eye and nose; eye, nose and mouth; eye, nose, mouth and eye‑ brow; and full face). In addition, we define three evalu‑ ation protocols and benchmark 12 state‑of‑the‑art detectors across the five popular standard cross‑dataset evaluations, DF40, and the proposed RLDF, providing a comprehen‑ sive audit of where and why existing detectors underper‑ formed. Our evaluation reveals that existing detectors fail to generalize consistently across partial facial regions, re‑ sulting in an overall drop of 10.5% AUC and 10% accu‑ racy under region‑level evaluation. Further, we propose a mask‑guided cross‑attention model as a baseline approach that can effectively detect region‑level manipulation. The proposed dataset, code, and checkpoints are available at https://github.com/RLDF‑Dataset/RLDF.
Authors:Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji
Abstract:
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow and costly. Most of these calls end in a multiple‑choice decision, such as which tool to call next. Decision models such as Jev answer these questions directly, at a fraction of an LLM call's time and price. We ask how a harness can delegate these decisions to a decision model with little loss of success. To evaluate decision models in agentic tasks, we build AgentDecisionBench, which tests them from isolated questions up to control of a running agent, on public benchmarks and our DecisionGym environment. The decision model most accurate on isolated questions solved only 69.5% of agent tasks, and without a fitted confidence threshold, delegating tool choices lowered success by 19.0 points. Building on these findings, we then design JevAgentHarness, which fits this threshold for each choice on the agent's own trajectories and lets the LLM decide below it. Because existing decision models never see the states an agent reaches after its own mistakes, we train AgentDecision‑4B, an open 4B model, on such states with Jev's answers as labels. On tasks unlike its training data, the agent solved 91.5% of them with AgentDecision‑4B and 82.0% with Jev. On tasks like their training data, 4B models trained this way made agents 81.1‑89.0% cheaper and up to 2.5 times faster, without losing 3 or more points of task success. The models, data, and harness will be released at https://github.com/JevAgentHarness/JevAgentHarness.
Authors:Arnab Bhattacharjee, Wayes Tushar, Tapan Kumar Saha
Abstract:
Learning effective representations from battery operational history is essential for predictive health management and control‑oriented applications. Existing approaches rely on response‑dependent engineered features or short early‑life windows, limiting generalisability and precluding use in prospective settings where future responses are unavailable. We propose NOCE‑Net, a deep nested sequence model that learns control‑enabling representations directly from raw cyclic operational or telemetric data. Our data pipeline, re‑BatteryData, standardises heterogeneous aging datasets into semantically equivalent Full Equivalent Cycles (FECs) defined by absolute charge throughput, enabling end‑to‑end learning across diverse operational regimes. NOCE‑Net combines multi‑patch transformer encoders for intra‑cycle dynamics with a GRU backbone for inter‑cycle state evolution, operating on approximately 700 batteries across ten public datasets spanning over 300 operational profiles. In zero‑shot evaluation on unseen battery chemistries and usage patterns, NOCE‑Net achieves 13.7% MAPE in voltage response prediction, outperforming strong long‑sequence baselines by over 43% while using 40 × fewer parameters. Shape and component diagnostics of the learned hidden states reveal that the GRU's principal direction of evolution achieves Spearman |ρ| \geq 0.95 with the capacity fade trajectory on average across all datasets with monotone degradation profiles, including zero‑shot test batteries, providing evidence that the representations encode degradation dynamics transferable to downstream tasks beyond battery response modelling. Code is available at https://github.com/BeArnab96/NOCE‑Net.
Authors:Kehong Gong, Guanli Hou, Xiaoxu Zheng, Qi Wang, Mingxi Xu, Anbang Zhai, Xiaoyu He, Mingyuan Zhang, Michael Bi Mi
Abstract:
Vision‑language‑action (VLA) policies commonly place a multi‑billion‑parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast‑VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi‑layer key‑value task memory. During execution, a lightweight online actor fuses learned time‑ and arm‑indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re‑running the foundation model. Fast‑VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO‑Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9‑17.1x and requires 1.22 GiB of post‑prefill online resident memory. Task‑memory swaps further show that the actor relies on episode‑matched task memory. These results show that task context can be established once while closed‑loop control remains responsive throughout execution. Project page: https://garfield‑kh.github.io/FastVLA
Authors:Vedant Pandhare, Piyush Dhoka
Abstract:
Block Influence (BI) ranks Transformer blocks for depth pruning from their input‑output cosine change, but the ranking depends on the calibration distribution. We study whether English calibration transfers to Indian languages in Qwen2.5‑1.5B‑Instruct, Llama‑3.2‑1B, and Gemma‑2‑2B. English, Hindi, Marathi, and Tamil use disjoint Wikipedia articles for calibration and evaluation; every selector receives the same 150‑chunk budget, and Tamil is excluded from multilingual aggregation to test cross‑script transfer. Across all three models, late‑block BI is 20.7‑55.5% higher for the three Indian languages than for English, and every predeclared late‑minus‑middle contrast has a 95% bootstrap interval above zero. This mismatch changes the low‑BI blocks selected for removal despite high global rank correlations. Averaging BI over equal English, Hindi, and Marathi budgets mitigates the resulting perplexity damage for Qwen at six removed blocks and for Gemma at all tested depths, including held‑out Tamil. The mitigation is not universal: on Llama, balancing improves Hindi and Marathi at two removed blocks but severely harms Tamil, while larger pruning budgets select the same blocks and collapse perplexity regardless of calibration. Rank‑mean and minimax‑rank aggregators do not dominate the raw mean. These results support a diagnostic claim rather than a universal pruning rule: English‑only BI systematically misses late‑layer activity for Indian‑language text, but multilingual calibration must still be validated per architecture and pruning budget.
Authors:Qixin Sun, WenLong Gao, Xiansheng Chen, Zhongyuan Wang, Si Liu, Pengwei Wang
Abstract:
Robotic manipulation often requires remembering interactions that are no longer visible. We introduce Latent Transition Memory (LTM), which stores history as compact latent transitions between consecutive observations. Instantiated on π_0.5 as LTM‑π, our method uses a frozen latent action encoder, a lightweight adapter, and causal KV caching for incremental inference without memory‑specific semantic annotations or external reasoning‑model calls. With DreamDojo transitions, LTM‑π achieves 58.54% success on RoboMME and 80.5% on LIBERO‑Mem, performing best among baselines without external reasoning models or memory‑specific annotations. Ablations support the contribution of transition content beyond temporal cues and the efficiency of delayed memory admission. Scaling RoboMME demonstrations tenfold improves success from 57.67% to 87.50%, while a Cup Swap diagnostic highlights the role of process supervision in temporal composition. Under a 200‑token memory budget, success remains 84.00%, versus 87.50% without compression. Finally, real‑robot counting success improves from 11.25% to 82.50%. Together, these results demonstrate the effectiveness, scalability, and efficiency of latent transition memory, highlighting its capacity for complex temporal information, bounded‑context operation, and real‑world robotic control.
Authors:Silin Chen, Zihan Zhang, Xiaodong Gu, Jiale Huang, Yuling Shi, Han Li, Zhao Shan, Chengcheng Wan, Zhensu Sun, Haibing Guan
Abstract:
As coding agents become increasingly capable, robust evaluation requires a broader and more diverse supply of verifiable software engineering (SWE) tasks. However, real‑world tasks are constrained by the natural frequency of issue reporting and the substantial cost of manual curation. Furthermore, existing synthetic approaches rely primarily on the pretrained knowledge of large models coupled with heuristic prompting, often failing to capture the authentic, complex mechanics of software failures. To exponentially scale SWE task production, we introduce SWE‑at‑Scale, a framework that automatically synthesizes realistic issues by reverse‑engineering real‑world software fixes. Our key insight is that historical fixes encapsulate deep mechanistic knowledge of bug formation: inverting a fix transforms repair supervision into bug‑induction supervision grounded in actual development scenarios. SWE‑at‑Scale transfers these learned bug patterns to novel code contexts, repeatedly sampling diverse bug typologies to construct executable SWE tasks. Extensive experiments show that SWE‑at‑Scale generates more diverse and realistic SWE tasks than current synthesis baselines while posing greater challenges to state‑of‑the‑art coding agents. Notably, SWE‑at‑Scale can generate distinct bug types from a single code context, with bug diversity scaling robustly even up to 200 generations per file. Overall, SWE‑at‑Scale turns finite real‑world fixes into reusable supervision for scalable synthesis of realistic, diverse, and challenging SWE tasks.
Authors:Utsav Poudel, Anuj Nepal, Vyas Viswanath, Subramaniyaswamy Vairavasundaram
Abstract:
Resting‑state functional connectivity is widely used in machine learning‑based classification of neuropsychiatric disorders. Quantum machine learning models have been proposed for functional connectivity data, but few studies have evaluated them under leave‑site‑out validation or against matched classical models. We compared six quantum model families with classical ones using identical correlation‑derived features across four public cohorts (4,308 subjects). Leave‑site‑out validation was used for the Autism Brain Imaging Data Exchange (ABIDE I), ADHD‑200 (attention‑deficit/hyperactivity disorder), and REST‑meta‑MDD (major depressive disorder) cohorts, while the UCLA Consortium for Neuropsychiatric Phenomics (UCLA‑CNP) was evaluated using stratified 10‑fold cross‑validation. The quantum kernel was statistically indistinguishable from a matched classical kernel across all four cohorts, with the average accuracy difference within 1.4 percentage points at 95% confidence. On ABIDE I, a hybrid quantum convolutional network achieved similar accuracy to its classical counterpart (0.645 vs. 0.641), while a trainable quantum kernel exceeded its classical counterpart by 0.064 in area under the curve (AUC). Quantum convolutional networks also achieved higher AUC than their matched classical networks on this cohort and showed smaller train‑test gaps on all four cohorts, without an accuracy gain. Overall, most quantum models matched their classical counterparts, but none consistently outperformed them. This study, among the first quantum evaluations of ABIDE I under leave‑site‑out validation, provides a feature‑matched benchmark of quantum and classical models comprising 284 matched comparisons. We further analyse fidelity‑based graph construction and account for the observed quantum‑classical parity through effective‑dimension and eigenspectrum analyses.
Authors:Zhuowen Liang, Xiaotian Lin, Yuyu Luo, Haixun Wang, Nan Tang
Abstract:
Multimodal long‑document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision‑language models (VLMs) can overlook sparse evidence and struggle with cross‑modal reasoning in a single pass. Existing document agents further introduce multi‑step interaction, but some still rely on predefined workflows or coarse‑grained actions, limiting the flexibility and granularity of document reasoning. We introduce DocPilot, an agentic document framework that supports expressive orchestration and makes complex document solving learnable by small language models (SLMs). DocPilot orchestrates evidence seeking through page‑level navigation and fine‑grained evidence localization, and integrates local visual inspection, code‑based numerical reasoning, and cross‑modal reasoning for multimodal understanding. To transfer these capabilities to small models, we propose curriculum‑based agentic training that progresses from single‑ability learning to adaptive orchestration through supervision from data‑grounded reasoning traces and executable interaction trajectories. Across MMLongBench‑Doc, DocBench, FinRAGBench‑V, and MADQA, DocPilot achieves strong gains with frontier VLM backbones, while DocPilot‑4B delivers competitive performance against substantially larger models, demonstrating strong effectiveness and generalization across diverse long‑document reasoning settings. The code is available at https://github.com/HKUSTDial/DocPilot.
Authors:Max Rosenblattl, Nikhil Krishnaswamy, Leon Nissen, Manuel Milling, Nicolas Zumarraga, Ning Wang, Thomas Kaar, Robert Jakob, Björn W. Schuller, Elgar Fleisch, Patrick Langer, Paul Schmiedmayer
Abstract:
Consumer phones and wearables increasingly collect physiological time‑series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preserving temporal structure. Cloud‑hosted time‑series language models avoid smartphone limits, but they repeatedly move sensitive physiology off the device and add latency and per‑query compute cost. Mobile model alternatives that aggregate time‑series data into statistical features or serialize raw samples as text either compress temporal structure or exhaust the context and memory budgets of small on‑device models. We present PocketTSLM, a mobile‑native deployment of OpenTSLM SoftPrompt, the soft‑prompt variant of the OpenTSLM time‑series language model. PocketTSLM runs fully on a consumer smartphone by injecting encoded time‑series windows as embeddings instead of tokenizing time series as text. A numerical parity protocol verifies identical greedy‑decoded token sequences between the Swift port and a Python reference using Apple's MLX machine‑learning framework. Answers also match the PyTorch reference, although rationale wording can differ. An on‑device context‑economics benchmark compares the soft‑prompt path with a text‑serialized baseline on the same backbone. In our benchmark format, soft‑prompting uses approximately 12 times fewer time‑series tokens and reduces prompt‑prefill cost. PocketTSLM processes 30‑second Sleep‑EDF electroencephalography clips for sleep staging and a 1,000‑sample (10 s) 12‑lead electrocardiogram clip from ECG‑QA. In comparison, text‑serialized baselines exceed the phone's context or memory budget at longer time‑series windows. All inference runs entirely on the device, with no health data transmitted during inference.
Authors:Qi Lu, Jiarun Li, Ce Zhang, Miao Pan, Yi Pan, Xu Wang, Wenqi Zhang, Xuhong Zhang
Abstract:
Vision‑language‑action (VLA) models transfer semantic knowledge acquired through vision‑language pretraining to robot control, achieving strong performance across various benchmarks. However, most VLAs directly map observations to demonstrated actions without explicit supervision of intermediate physical state changes, making them prone to overfitting observation‑action correspondences. Some methods introduce intermediate reasoning or future‑state prediction in language, visual, or latent spaces to guide action generation, but do not explicitly supervise the physical state transition itself. We therefore propose VLA‑Δ, a general framework that uses physical changes between current and future states as explicit learning targets. To support this objective, we construct a large‑scale pretraining dataset and derive physical state‑transition annotations covering task‑relevant semantics and visual motion changes, providing more direct supervision for continuous action generation. During inference, the model generates actions directly from current observations without additional inference overhead. VLA‑Δ achieves performance competitive with the state of the art on LIBERO and RoboTwin, and outperforms π_0.5 in real‑robot experiments. Our code is available at https://github.com/ZJU‑OmniAI/VLA‑Delta
Authors:Junru Wu, Jingting Zheng, Yuqi Ren, Deyi Xiong
Abstract:
Current value alignment evaluations for large language models (LLMs) heavily rely on output compliance, overlooking whether LLMs maintain value consistency when alternating between generator and evaluator roles. Such consistency is increasingly important because modern LLM agents and alignment pipelines often rely on the same LLM to both generate actions and evaluate their acceptability. To mitigate this limitation, we propose VALCON, a role‑driven data construction framework that operationalizes value consistency into three benchmark suites (VAB‑Consistency, VAB‑Boundary, VAB‑Challenge) via dual‑role variation. By systematically pairing an LLM's generative behavior with its evaluative judgments under identical principles, VALCON synthesizes target evaluation data that exposes hidden value discrepancies. Experiments on 28 LLMs reveal pervasive cross‑role inconsistency: models with highly compliant outputs frequently fail to align their judgments with their own generations. To understand and address this gap, we perform mechanistic probing that uncovers a role‑dependent representational decoupling associated with this behavioral asymmetry. Guided by these insights, we propose a representation‑directed intervention that effectively mitigates consistency failures. Our work establishes value consistency as a distinct alignment dimension and provides a complete evaluation‑analysis‑mitigation pipeline. Code and data are available at https://github.com/tjunlp‑lab/VALCON.
Authors:Minhyeok Roh, Minseo Lee, Jun Young Jeong, Byeonghyeon Lee, Lucas Yunkyu Lee, Jaesik Park, Eunbyung Park
Abstract:
Dynamic Gaussian representations have made high‑quality, real‑time volumetric video increasingly practical, yet challenging motion can still break their reconstruction: small fast‑moving objects may disappear, and fine textures in highly dynamic regions are often lost. In particular, simply training longer or using more Gaussians may still fail to recover small, fast‑moving objects. We argue that the key bottleneck is not representational capacity itself, but where and when Gaussian primitives are allocated. Existing allocation strategies rely on current Gaussian locations and often overlook the spatiotemporal destinations of relocated Gaussians. To address these limitations, we introduce two complementary strategies. First, Seed Injection identifies under‑reconstructed dynamic regions using temporal appearance changes and reconstruction errors, and initializes new Gaussians at geometrically plausible 3D locations through multi‑view observations, independent of existing Gaussian locations. Second, Temporal Biased Relocation uses time‑resolved optimization signals to guide the spatiotemporal placement of relocated Gaussians. Both strategies maintain a fixed Gaussian budget without modifying the underlying representation. Experiments across multiple dynamic‑scene datasets demonstrate improved reconstruction quality, particularly in highly dynamic regions, recovering small, rapidly moving objects and fine‑grained appearance details.
Authors:Donghwan Lee, Sanghyun Jo, Sihyun Kim, Ziseok Lee, Kyungsu Kim
Abstract:
Training‑free object removal commonly redirects self‑attention away from the target region, but on few‑step distilled diffusion models this often leaves object‑shaped residuals. We identify a temporal mismatch: a structured prediction forms inside the mask at the first forward, and the short trajectory offers few later chances to revise it, while attention rerouting acts mainly in later steps. Across four distilled backbones, magnitude‑matched single‑forward interventions show that background‑directed state correction affects the final result more than attention rerouting. We therefore propose ReState, which corrects the masked clean prediction toward background evidence retrieved from the input using a noise‑level‑aware weight that vanishes at the final forward. ReState retains object‑excluding attention as a complementary constraint and adds no denoiser evaluations. Across two paired benchmarks and two few‑step backbones, ReState achieves the best masked PSNR, SSIM, and LPIPS among training‑free methods, improving PSNR by 3.17 dB on average over the strongest baseline. On OBER‑Test, it also outperforms the 50‑step attention‑forcing reference on all three metrics at 6.7x lower latency. The same configuration transfers to two additional distilled backbones without retuning. Project page: https://tdrlee.github.io/ReState
Authors:Xuanyi Liu, Ruiqi Li, Siwei Ma
Abstract:
Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers object interaction requirements from the prompt and jointly plans object structure, properties, and rules, favoring compact representations that support those requirements. The world evolves under the planned rules; optional user actions steer it, while text requests can edit properties or extend capabilities with unaffected state preserved. Object texture memory retains reusable appearance only for whitebox‑defined objects and adapts it as those objects change. The program advances world state, while a generative renderer combines current structure with remembered appearance. By organizing construction, intervention, and memory around the same objects, Mallea conditions subsequent observations on the latest state and compatible appearance. Project page: https://lxycopper.github.io/Mallea.
Authors:Max Rosenblattl, Milana Gurbanova, Karl Aurel Deck, Narayan Schuetz, Thomas Kaar, Manuel Milling, Nicolas Zumarraga, Ning Wang, Kelly W. Zhang, Anders Johnson, Robert Jakob, Björn W. Schuller, Elgar Fleisch, Daniel Seung Kim, Allan Lawrie, Patrick Langer, Maxwell A. Xu, Euan Ashley, Paul Schmiedmayer
Abstract:
Time‑series data generated continuously across industrial, environmental, and especially health domains produce multimodal signals whose scale and heterogeneity make their processing and interpretation particularly challenging. Time‑Series Language Models (TSLMs) promise scalable interpretation of these signals, yet remain bottlenecked by a fundamental lack of text‑annotated time series pairs that are signal‑grounded, reproducible, and extensible across modalities and schemas. Existing paired corpora are either closed or limited in scale and modality coverage. Annotation with large language models could scale them up, but its outputs cannot be regenerated exactly, which prevents others from re‑deriving, extending, or auditing the resulting corpus. To address this, we release TimeCap, an open deterministic captioning pipeline with three reusable extractor tiers and a cross‑channel synthesis layer that operates directly on the signal with no language model in the loop, extensible to new schemas by writing channel definitions alone. This positions TimeCap as a reproducible mechanism for future dataset generation rather than a one‑off release. Applying TimeCap to OpenMHC, the MyHeart Counts wearable dataset, yields MHCap, the largest open caption layer for multimodal wearable time‑series data. MHCap spans 19 Apple HealthKit channels and 7,005,763 captions over 908,944 person‑days. To showcase the impact of the corpus, we train OpenTSLM, a time‑series language model, on MHCap and benchmark on health‑outcome prediction. Caption‑augmented training (OpenTSLM‑Cap) outperforms a label‑only baseline on 29 of 32 per‑user binary evaluations across 8 tasks.
Authors:Zane Durante, Paul Tang, Keshigeyan Chandrasegaran, Amit Kaushal, Shrinidhi Kowshika Lakshmikanth, Feolu Kolawole, Yu Jun Zheng, Yifan Geng, Shobhit Agarwal, Kevin Schulman, Juan Carlos Niebles, Ehsan Adeli, Arnold Milstein, Li Fei-Fei
Abstract:
Continuous video and clinical records provide complementary information for understanding how patient behavior evolves over a hospital stay. A central challenge is how to represent behavior across the entire stay while preserving access to fine‑grained visual and clinical evidence. We introduce TRACER, a training‑free multimodal harness that constructs minute‑level action records from video and combines them with clinical records into hourly reports. The harness equips an LLM agent with tools to query this hierarchy, inspect relevant video, and retrieve clinical records. To evaluate TRACER, we construct two benchmarks. The first contains 397 annotated one‑minute clips spanning 15 atomic actions; the second poses 144 clinician‑informed multiple‑choice questions, grounded in 264 hours of video and clinical records from two ICU patients. TRACER achieves up to 84.8% accuracy for action detection and 61.0% QA accuracy versus 49.3% for the strongest baseline using the same backbone. Across 280 patients spanning approximately 60,000 camera‑hours, TRACER identifies significantly less purposeful activity in patients with a positive delirium assessment during their stay than in those without (p < 0.0001). Our results show that combining continuous video with clinical records can support understanding of patient behavior over time and across patient cohorts without training on protected health data.
Authors:Tianhao Ma, Wei Wang, Yivan Zhang, Dong-Dong Wu, Takashi Ishida, Gang Niu, Masashi Sugiyama
Abstract:
Learning from label proportions (LLP) is a weakly supervised setting where training data are grouped into bags with only bag‑level label proportions observed, and the goal is to learn a classifier that predicts labels for individual instances. Recently, a large body of methods has emerged; however, most of them rely on the assumption that instance‑label pairs are sampled i.i.d. and randomly assembled into bags, an assumption often violated in real‑world scenarios. Our experimental results indicate that these methods perform suboptimally under non‑random settings. In this work, we adopt a more realistic assumption: bags are sampled i.i.d., and instance labels are conditionally independent given the instances. Under this setting, we show that the label counts follow a Poisson multinomial distribution. Motivated by this observation, we propose LLP via moment matching (LLP‑MM), a simple yet effective approach which leverages multi‑order factorial moments as training objectives, encouraging the classifier predictions to match these moments and thereby more fully exploit the underlying statistical structure of the data. Extensive experiments on benchmark datasets under various bag construction strategies demonstrate the effectiveness of our approach while maintaining high computational efficiency. Our code is available at https://github.com/TianhaoMa5/LLP‑MM.
Authors:Haotian Ma, Ziye Mei, Frederic Sala
Abstract:
Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order. Under independent feature subset sampling, population SAR is exactly the normalized energy of interactions above that order and bounds coalition ordering error for the optimal low order approximation. To distinguish compositionality associated with the SAE representation from that induced by downstream computation, we compare learned SAE coordinates with sum preserving random rotations of the same selected subspace and find lower first order residuals for the learned coordinates across all 18 model and behavior settings. The coordinates are nevertheless not fully additive: pairwise terms improve held out coalition ordering in every setting and outperform parameter matched models using randomly selected third order terms. In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606. Together, these results show that SAE coordinates support comparatively compositional behavioral descriptions while faithful joint explanations and interventions still require interactions beyond individual feature effects. Code is available at https://github.com/OpenSAE/SAE‑Behavioral‑Compositionality.
Authors:Tianyu Chen, Xinyang Liu, Ruyi An, Zhendong Wang, Mingyuan Zhou
Abstract:
Large language models (LLMs) often make discrete decisions, such as flagging phishing or escalating to a human. System‑1 models, such as Jev and the OpenAI Decisions API, return probabilities over options in a single forward pass, while System‑2 approaches generate reasoning steps. Which decisions can System‑1 models handle, and which require generative reasoning? We introduce System1‑Arena, a benchmark that gives all models the same decision requests and scores their probabilities. It contains 92k questions across 105 tasks from 268 public sources, in four groups: text classification and knowledge, applied decisions, evaluation and control, and agent and action. It also includes 138 closed‑loop environments, from web shopping and SWE‑bench coding control to robotics, games, and driving; in real‑time mode, the world evolves while a model decides. We evaluate 30 models: 7 closed LLMs, 13 open LLMs read at the answer token, and 10 System‑1 models. The largest System‑1 models match or exceed fast closed LLMs in non‑agentic accuracy, with better‑calibrated probabilities at one‑quarter of the latency. A reasoning LLM is the most accurate, at about 70x the cost of a fast LLM. In closed loops, large open LLMs succeed most often (up to 32.9%); small System‑1 models rarely succeed.
Authors:Ziye Mei, Haotian Ma
Abstract:
Sparse autoencoders (SAEs) decompose language model activations into sparse features, but prompt transformations designed to preserve the underlying request can redistribute activity across them. A feature change observed on one transformed prompt need not recur, and even a recurring change need not affect a downstream computation. We introduce the redistribution sensitivity score (RSS), which identifies large feature responses that recur in a consistent direction, and evaluate the resulting supports on variants excluded from selection. Across 27 pretrained SAEs spanning eight language models, the tested prompt transformations produce a reproducible redistribution profile on these excluded variants. We then separate recurring representation change from signed readout contribution and decision failure. For a fixed linear readout, an exact decomposition shows how response magnitude, readout coefficients, directional alignment, and the clean margin determine downstream consequences. In two frozen safety readouts, combining RSS support erosion with the full clean margin yields recurrent failure AUROCs of 0.935 and 0.958 using median supports of 20 and 19 features, compared with 0.965 and 0.964 using full erosion. RSS therefore provides a way to identify recurring sparse representation changes and quantify their contribution to a downstream readout without equating representation change with failure. Code is available at https://github.com/OpenSAE/Redistribution‑Sensitivity‑Score
Authors:Songlin Liu, Linwei Chen, Li Zhang, Jun Hu, Ying Fu
Abstract:
A few well‑annotated datasets for land‑cover semantic segmentation have recently been introduced to advance the field of earth observation technologies. However, these datasets overlook the significant diversity among geographic areas with different climates, which can greatly impact and diversify land cover. Consequently, this leads to a domain gap in remote sensing images and severe performance degradation of the segmentation models. To enhance land‑cover semantic segmentation with improved generalization ability, we conducted the first investigation into the impact of climate on this task. In this paper, we present a unique large‑scale Climate‑Aware Satellite Images Dataset (CASID) specifically designed for domain adaptive land‑cover semantic segmentation. It consists of 980 satellite images with a size of 5000 x 5000 pixels, collected from 30 different regions around Asia, covering over 24,500 square kilometers. These images are gathered from four distinct climate zones, namely temperate monsoon, subtropical monsoon, tropical monsoon, and tropical rainforest. It includes four sub‑datasets/domains, each representing one of the aforementioned climate zones. This characteristic makes CASID the first climate‑aware land‑cover semantic segmentation dataset with multiple domains. Additionally, we provide a comprehensive analysis of the samples from the four climate zones, emphasizing differences in global image features, image texture, category distribution, spectral value, and object shape. These analyses offer valuable insights for subsequent research in this field. Moreover, we conduct extensive experiments to evaluate the latest semantic segmentation and unsupervised domain adaptation methods on the CASID dataset. These results serve as a robust baseline for future research endeavors. Our dataset will be made publicly available at https://github.com/Linwei‑Chen/CASID.
Authors:Hanyue Chen, Chang Liu, Miaomiao Zhang
Abstract:
Active model learning is a crucial technique for automatically constructing formal models of black‑box timed systems. However, existing learning approaches typically construct a single monolithic automaton for a timed system with multiple synchronous components, obscuring its component structure and hindering component‑level analysis. We propose a compositional learning approach that constructs a deterministic real‑time automaton (RTA) for each component with access only to the target system as a whole. Upon termination, the learned RTAs form a network that exhibits exactly the same timed behaviors as the target system. Within the respective resource limits, our implementation terminates successfully on 213 of the 240 (88.8%) randomly generated benchmark instances and all six adapted case‑study benchmarks, and requires less learning time than monolithic approaches on some instances.
Authors:Esther Xin
Abstract:
Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data‑loss costs of cleaning policies remain underexamined. We audit sixteen open RLVR math corpora comprising 1,673,514 corpus‑specific training items against six benchmarks containing 1,949 questions. We evaluate eight policies based on exact, lexical, and semantic matching using 594 independently and blindly dual‑reviewed training‑benchmark pairs. Calibration and held‑out sets have disjoint benchmark questions and training‑example families. A separate review of all 83 candidate matches for 80 sampled training items measures item‑level false removal. At cosine threshold 0.80, 52 of 59 sampled removals (88%) were confirmed false removals; this is not a demonstrated population rate. No semantic threshold in our preregistered grid (0.80‑0.98) meets the 90% conservative‑precision target on calibration. The existing judge‑adjudicated policy achieves 98.1% resolved precision and 98.2% candidate‑frame recall on held‑out corpora. It has zero confirmed false removals among six sampled removals, with one unresolved case; this does not establish zero risk. Within the evaluated candidate frame, this policy offers a more favorable precision‑recall balance than the semantic‑threshold alternatives tested. Recall does not measure contamination outside the candidate frame. We release reviewed labels, per‑policy decisions, and coverage accounting to support evaluation of cleaning reliability and cost rather than overlap counts alone.
Authors:Zidong Jia, Guofu Xie, Xiao Zhang
Abstract:
Recommender systems often need to balance multiple conflicting objectives. Controllable Multi‑Objective Recommendation (CMOR) enables dynamic adjustment of objective trade‑offs at inference time without retraining, adapting to shifting user or system requirements. Existing CMOR approaches typically rely on output‑level reranking or parameter‑generating modules, which either limit the scope of feasible trade‑offs or incur substantial training and inference cost. Model merging provides a promising alternative by reusing multiple objective‑specific models via parameter combination. However, its direct application to recommender systems is challenging due to the lack of a shared linear regime and cross‑objective interference. To address these issues, we propose RecSoup, a lightweight two‑stage approach for effective CMOR via specialized model merging. RecSoup first mitigates parameter interference across objectives through a pruning strategy that jointly considers objective‑specific importance and cross‑objective relevance. It then performs adaptive parameter merging via a learnable preference‑to‑coefficient mapping, allowing dynamic trade‑off control at inference time without joint retraining or additional generative modules. Theoretically, we prove that RecSoup outperforms the parameter generation paradigm in effectiveness and efficiency, as it can fully navigate the Pareto front without smoothness constraints. Extensive experiments on multiple benchmarks demonstrate that RecSoup achieves improved Pareto front quality and effective controllability while delivers 165‑‑228x faster inference‑time latency than parameter generation approaches. Code is available at https://github.com/35798642/RECSOUP.git.
Authors:Dohyun Lee, Minho Park, Jeonghoon Park, Dongyoon Hwang, Junha Hyung, Sungwon Hwang, Byungkun Lee, Jaegul Choo
Abstract:
Proactive robots should anticipate user needs and provide assistance before being explicitly asked. A straightforward approach is to infer human intentions from commonsense knowledge, but the same observed behavior can imply different needs across users, potentially leading to unwanted assistance. We observe that everyday routines in the real world exhibit recurring patterns, so learning these patterns can reduce ambiguity in proactive decisions. Based on this, we propose a memory‑based agentic framework in which a VLM task predictor reads a Procedural Assistance Memory (PAM) alongside the current observation to decide when to assist and what task to perform, and a VLA executor performs the predicted task. PAM is structured as a finite‑state machine over activity phases, per‑state assistance rules, and event‑driven transitions; a reflection VLM revises PAM from each day's interaction trace without weight updates or retraining. In our 5‑day desk‑assistance evaluation, memory‑free baselines remain conservative, with limited recall on proactive tasks. Our system improves consistently with accumulated experience, achieving the highest precision, recall, and F1 and reducing first‑fire delay from approximately 5 seconds on Day 1 to 1 second by Day 5. Our approach also adapts to changes in the user's routine through continued memory reflection, recovering from memory‑‑routine misalignment after a single memory update. The project page is available at https://davian‑robotics.github.io/PAM/
Authors:Michael Tian-Yue Liu, Anne Harrington, Trevor Darrell, Yutong Bai
Abstract:
Rubrics as Rewards (RaR) extends reinforcement learning to non‑verifiable domains by using an LLM judge to score responses against a structured rubric. Prior work has focused on improving the rubric or strengthening the single judge, with comparatively little attention to how different single or multiple judges might be used during RaR training. We first show that judge identity alone produces a substantial spread in scores when training Qwen3‑4B‑Base on RaR‑Medicine and RaR‑Science across five frontier and open‑weight judges. Building on this, we propose RaRE (Rubrics as Rewards with Ensembles), a novel judge ensemble strategy for RaR with two mechanisms: a soft‑Likert reward that uses judge logprobs to weight more confident judges more heavily, and a drop‑on‑disagreement mask that excludes high‑disagreement rollouts from the GRPO update. We find that this design is critical: while a naive average of judge scores fails to outperform the best single judge, RaRE outperforms every single‑judge RaR variant on both HealthBench and GPQA Diamond, with per‑theme analysis showing that it inherits the complementary strengths of its constituent judges. Our findings establish judge ensembling as a compelling axis for improving RaR.
Authors:Hiroki Fukui
Abstract:
Let m\ge5 be odd and q=2^m. For every odd r with 3\le r<m and \gcd(r,m)=1 we give the reduced polynomial P_m,r=\mathrmred((T_r^3)^2^m‑2), T_r=\sum_i<rX^2^i, explicitly: it has coefficients in \mathbbF_2 and exactly 3r‑2 terms, it is an APN permutation of \mathbbF_q, and its formal derivative 1+X^2^r‑2+X^2^m‑2 has no zero on \mathbbF_q and is two‑to‑one. We determine the ramification of P_m,r as a map of the projective line for all admissible (m,r): there are 2^m‑r finite ramification points, all wild, with index and different exponent 2^r‑2, all over the single finite branch value 1. The 0/1 lift of P_m,r satisfies an identity in \mathbbZ[X] that evaluates it over every Galois ring \mathrmGR(2^k,m) with m‑1 squarings, one general multiplication and r+5 additions or subtractions. For every quadratic optimal APN permutation p of \mathbbF_q (in particular P_m,r), every k\ge2 and every coefficient lift F of p to \mathrmGR(2^k,m), we determine the differential rows of F exactly in the unit directions whose residue lies outside the kernel \0,ρ\ of the linear part of p' (they are the residue rows of p; ρ=1 for P_m,r) and in the directions in 2^k‑1\mathrmGR(2^k,m)\setminus\0\ (every nonzero entry equals 2q^k‑1). We also give a self‑contained proof of the Li‑‑Wang criterion for a Gold function plus a linearised polynomial, which explains why the inner map of P_m,r is not a monomial, and record a two‑word, two‑layer transition that a simple linear layer forbids at the residue level.
Authors:Hiroki Fukui
Abstract:
The formal derivative of the reduced polynomial representative of a map on a binary finite field depends on the coordinates, although its differential uniformity does not. This matters for lifting permutations to Galois rings: a polynomial over a Galois ring of characteristic a power of two greater than two permutes the ring exactly when its reduction permutes the residue field and the derivative of the reduction has no zero there. We prove that for every odd n\ge5, every quadratic almost perfect nonlinear permutation Q of \mathbbF_2^n admits invertible binary‑linear maps L and A such that the derivative of the reduced representative of L\circ Q\circ A has no zero and each of its nonempty fibres has exactly two elements. The range is exact: this fails for n=3, as computed earlier, and for even n there are no such permutations. For n\ge9, a linearization and a projectively injective cofactor map reduce the problem to counting binary matrices: more than 5% of the invertible inputs A succeed, more than half for n\ge15, without any classification. For n=5,7 we use two independent classifications and finite certificates. Consequently every such permutation is linearly equivalent to a counterexample to the reduced critical‑point conjecture of Bartoli and Stănică that lifts to permutation polynomials of optimal differential uniformity over every Galois ring with this residue field. For odd n\ge9 the existence proof is checked in the Lean 4 proof assistant.
Authors:Huachao Zhu, Junjielong Xu, Sicheng Li, Liang Yang, Mingtao Chen, Zhichao Hu, Li Jiang, Pinjia He
Abstract:
On‑screen text can describe what a video shows, or it can frame what the video is supposed to mean. Current benchmarks treat both alike, testing whether a model can read the text rather than whether it should trust it. We study claim deference, multimodal large language models' tendency to let an on‑screen claim override conflicting visual evidence. We introduce VisualTrust, a counterfactual benchmark with 5,063 cases across 14 claim‑fact classes. Each case pairs a question with three versions of the same video: one with no claim, one with an accurate claim, and one with a misleading claim that alters a single grounded fact. The option the misleading claim supports is the trap answer. Across seven open‑weight and four proprietary models, mean accuracy falls from 90.1% without a claim to 51.9% with a misleading claim. Among initially correct responses, 42.8% flip to the trap answer and only 0.6% become other errors. Strong no‑claim accuracy does not ensure resistance. We also introduce DEAR (Decoupled Evidence Arbitration and Reading), a training‑free agent for video question answering that separates evidence arbitration from reading. An independent check reports conflicting evidence, a selector weighs it without a fixed modality priority and blinds what it withholds, and the reader answers from the resulting video in a fresh context. On 500 videos across three proprietary models, DEAR raises accuracy under misleading claims from 47.2‑‑64.0% to 81.8‑‑85.2% and reduces trap‑answer selection from 34.4‑‑50.8% to 11.6‑‑15.0%, while preserving no‑claim accuracy.
Authors:Guo-Wei Wong, Meng Chang Chen
Abstract:
Agentic software invokes tools, launches processes, accesses files, and communicates with local or remote services. Application traces describe logical operations, whereas operating‑system telemetry records machine activity without necessarily identifying the responsible agent action. Relating these evidence planes is a reconstruction problem: a plausible but unsupported association can assign an effect to the wrong principal. We present ExecWeave, a local‑first system that materializes provider semantics and independently observed runtime events into a typed execution graph. The implementation retains separate source artifacts, distinguishes logical occurrences from repeated evidence, and records attribution, inference, and causal‑status metadata. An optional command‑to‑process correlator emits a non‑causal inferred bridge only when its bounded search yields one compatible observed process; uniqueness within this observed candidate set is not a guarantee of causal correctness. OS‑observed file and network effects acquire a semantic tool or agent principal only through an observed process plus a justified command‑to‑process association; session‑correlated file‑system events do not by themselves provide that principal. Source code is available at https://github.com/Irish‑kw/ExecWeave.
Authors:Yingzhuo Deng, Chong Li, Bingxin Liu, Yi Shu, Tianyu Peng, Can Xie, Siyao Song, Shuo Ren, Jiajun Zhang
Abstract:
Spoken Dialogue Models (SDMs) typically share a speech‑token space for generation and synthesis, leading to long autoregressive sequences and costly inference. Lower‑rate speech tokenizers shorten sequences but may introduce compression‑quality trade‑offs and costly replacement without breaking this coupling. To address this limitation, we propose CoSTA, a compatibility‑preserving post‑hoc framework that decouples generation and synthesis token spaces while retaining the original speech tokenizer and decoder. CoSTA extends the generation token space with compressed representations via mean‑initialized vocabulary extension, pronunciation adaptation, and dialogue adaptation, followed by deterministic expansion back to the original speech‑token space before synthesis. On bilingual speech‑dialogue benchmarks, CoSTA achieves approximately 1.7× expanded speech‑token generation speed, lower full‑response latency, and higher throughput while preserving response quality. Analyses show more sustained interaction under limited budgets and a practical trade‑off among compression, decoding efficiency, and quality. Further experiments demonstrate cross‑architecture transfer and complementarity to low‑rate tokenization.
Authors:Siddhesh Kulkarni
Abstract:
We characterize the multiplicative half‑Cauchy process through its log‑scale random‑walk representation, establishing shrinkage, expansion, and critical regimes, almost‑sure summability, local rebounds, and effective dimension. We also derive a semi‑analytic calibration rule linking interpretable prior probability statements to its scale parameter.
Authors:Woocheol Shin, Taeyoung Yun, Jiwon Sung, Hyeongyu Kang, Jeongmin Park, Jaewoo Lee, Kyuil Sim, Dohyun Kim, Yutong He, Jinkyoo Park
Abstract:
Few‑step generative models produce high‑quality samples in one or a few evaluations. However, steering them toward desired properties remains difficult, especially when only black‑box reward evaluations are available. While one could apply approaches developed for iterative diffusion and flow‑based models, such as direct RL fine‑tuning or test‑time scaling, we find that neither transfers well to the few‑step regime: direct fine‑tuning suffers from severe reward over‑optimization, and test‑time scaling sacrifices the efficiency advantage of few‑step generation. We propose Noise Policy Optimization (NPO), which explicitly targets the alignment problem for few‑step generative models. The key idea is to learn an amortized flow policy over the initial noise space of the generator. The trained policy transports Gaussian noise to a reward‑tilted noise distribution, whose pushforward distribution through the frozen generator is exactly the reward‑tilted output distribution. By keeping the generator frozen, we can effectively utilize the knowledge encoded in the pretrained model, yielding a better reward‑quality trade‑off than direct fine‑tuning. The amortized policy moves the search cost into training, so inference adds only a lightweight noise sampling step on top of the pretrained model. To improve sample efficiency during training, we use ELBO‑based likelihood estimation that requires only sampled endpoints rather than simulated trajectories. Across image, language, and protein structure generation, NPO consistently achieves a better reward‑quality trade‑off than direct fine‑tuning, and matches test‑time search methods with 16 to 549 times lower inference cost.
Authors:Haotian Ma, Ruqi Yang, Philip Townsend
Abstract:
Post‑hoc image explanations usually show where a model looks using spatial heatmaps or indicate which channels matter using global channel scores. These separate views are limiting for multispectral data because channel relevance is often spatially localized and physically meaningful. We introduce Spectral‑Spatial Interpretation (SSI), a full‑grid attribution method over position‑channel players (g,c). For each spatial position, SSI fixes the observed surrounding context, defines a conditional game over the channels at that position, and applies the Shapley value to this local game. The result is a full spectral‑spatial attribution matrix rather than a single heatmap or a global channel ranking. SSI has a clear local interpretation. It satisfies Shapley fairness within each position, and aggregating its channel attributions exactly recovers the position‑level contextual contrast obtained by replacing that position with the baseline. SSI also differs from context‑averaged Owen‑style hierarchical accounting, which uses the same position‑channel grouping but averages over external spatial coalitions. On HLS Burn Scar, SSI outperforms an MC‑Owen estimator across retained attribution metrics while using only about 6% of its evaluation budget. Across CUB‑200, EuroSAT, and HLS Burn Scar under both per‑channel mean and zero baselines, SSI achieves the best joint deletion and insertion AUC over dense, flat, and rank‑1 baselines, yields the strongest spectral hit rates and channel deletion, and leads spatial localization on both annotated datasets. Code is available at https://github.com/AtheavenMa/Spectral‑Spatial‑Interpretation.
Authors:Zichuan Xu, Yuyang Dong, Makoto Onizuka, Chuan Xiao
Abstract:
The success score of a tool agent reflects both its model and the budgets of the harness that runs it. We present ToolBottleneckBench, a benchmark that measures tool‑agent performance under controlled pressures and fixed harness budgets. Synthetic enterprise tasks stay fixed while one pressure varies: the registered tool catalog, the visible tool menu, the certified workflow, or the encoding of returned records. An exact grader records the first unmet obligation of each failed episode and the harness limit, if any, that ended it. Across 21 model and harness configurations, the visible menu is the only one of the three pressures that act on the model with a large penalty, and this penalty tracks truncation at the completion cap. Raising the cap sixteenfold removes most of the penalty in the truncation‑heavy configurations. Most failures end at a harness limit, and the failure record identifies the assigned pressure no more reliably than a count of tool calls. The repairs that help change which tools reach the model or who issues the calls, and only two repairs, neither improving model behavior, beat every configuration switch. We recommend reporting pressure effects with the harness budgets under which they were measured. We release the benchmark at https://github.com/zichuanxu/tool‑bottleneck‑bench.
Authors:Yue Su, Bokai Lin, Xinkai Wang, Chubin Zhang, Xiangnan Wu, Kuan Zhang, Weiyang Jin, Jiwen Yu, Yiming Li, Cewu Lu, Lixin Yang, Xihui Liu
Abstract:
World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video‑action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action‑conditioned world modeling. We use LIBERO, LIBERO‑Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real‑world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi‑view world‑prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.
Authors:Arshia Abolghasemi, Seyed Reza Tavakoli, Parsa Naderi, Nazanin Mirzaei, Behrad Jahani, Saba Yousefzadeh, Tahereh Javaheri, Mohammad Hosein Rohban, Gholamali Aminian, Ali Habibnia
Abstract:
Score‑based diffusion methods usually perturb data with Brownian Gaussian noise, corresponding to a light‑tailed forward stochastic differential equation (SDE). For market time series this SDE can be a poor abstraction: limit order book (LOB) order flow is heavy‑tailed, bursty, and occasionally jump‑like. We study STOCHLOB, a training‑only score‑matching framework that changes the forward diffusion corruption law while leaving the deployed predictor deterministic. Instead of Brownian‑only Gaussian corruption, the noising process uses two non‑Gaussian SDE‑inspired alternatives: jump‑Lévy noise through a compound‑Poisson variance mixture, and clipped α‑stable noise through a positive‑stable Gaussian scale mixture. These corruptions inject finite jumps and heavy‑tailed residuals into training without requiring reverse‑time diffusion sampling at inference. The resulting score target has a useful interpretation: its radial factor h_t(r) is the posterior expected corruption precision, so large residuals are weighted according to the latent corruption scale that most plausibly generated them. On chronologically held‑out Coinbase LOB data, the proposed variants remain competitive on uncorrupted BTC and altcoin test horizons and improve several realized tail‑window stress diagnostics. An exploratory Feishu A‑share transfer study further tests whether the probability outputs remain usable under transaction costs and trading‑rule constraints. Code and configuration files are available at https://github.com/ArshiAbolghasemi/Penny.
Authors:Sai Varun Kodathala, Prashanth Pollishetty, Jaylen Cargill
Abstract:
AI agents now pause for approval before they send money, delete files or email outsiders, and that approval is usually treated as a second factor. We argue it is one only when compromising the agent cannot produce it and when it names exactly what it approves, and we test both conditions. We define twelve attacks on the approval path, including decoy recipients, amount cancellation, card padding, label spoofing, enrollment squatting and threshold splitting, and run them against the Agent Payments Protocol (AP2), the Microsoft action‑bound approval design and Model Context Protocol (MCP) elicitation. An audit of ten agent runtimes finds no third‑party verifiable approval record, no flood limits where measurable, and post‑approval argument changes in four of eight runtimes with measurable binding. As a reference defense we build ASU (Agent Step‑Up), a gateway that renders the approval card from a canonical effect descriptor, has a passkey on the phone of the user sign its digest, and executes only the stored call. Tamarin proves its binding, single‑use and key‑level non‑repudiation properties under three stated hypotheses (53 lemmas). Across 27,108 scripted trials and 609 AgentDojo attack pairs, ASU stops substitution, replay and grant stretching, and the trusted card cuts forged‑narration attack success from 0.54 to 0.03. With 15 current language models as automated approvers, the trusted card raises the attack catch rate from 0.30 to 0.98, at the cost of refusing 16 percent of harmless near misses, mostly due to a placeholder artifact in our stimuli. A second device adds nothing measurable over a trusted on‑host confirmation in these experiments, flooding still succeeds 0.45 to 0.60 of the time against a modeled approver, and approval cannot stop effects that policy already allows. Code, data and proofs: https://github.com/Sports‑Vision‑Inc/agent‑step‑up
Authors:Dingjie Song, Zhiling Yan, Ziqian Zhang, Yuxuan Zhang, Lichao Sun
Abstract:
Peer review is under strain from the growth of submissions, and review comments written by humans or AI can misstate what a paper says. We introduce ReviewFact, a benchmark that labels such factual errors in review comments under one definition across NLP, machine learning, and the natural sciences, and that measures cost and latency alongside accuracy. A frontier LLM checks comments well but must read the whole paper for each check, while cheap decision models, which score whether a statement holds in a text, fall far short on whole comments that mix several implicit assertions. PremiseCheck is a framework that combines the two. An LLM rewrites each comment as explicit premises without reading the paper, and a decision model checks each premise against the paper. When each comment is checked on its own, PremiseCheck reaches accuracy comparable to GPT‑5.5 across all three fields at about half its cost, and on NLP reviews at a quarter of its cost and less than half its latency.
Authors:Zhaoqian Xue, Shaoze Zhou, Xuexin Li, Junting Mu, Haozheng Liu, Zhongyin Wang, Yichun He, Renkai Ma, Hao Zhou, Sarah Bratt, Siyuan Ma, Chen Chen, Lingyao Li
Abstract:
Systematic literature reviews follow the PRISMA process of screening scholarly works and extracting evidence from included studies. Deep research tools based on large language models (LLMs) can search and summarize the literature but do not record which studies are excluded or where extracted values come from. We develop a conversational multi‑agent system called ReviewPilot that automates the PRISMA workflow from paper screening to information extraction. Researchers can tailor criteria and schemas through dialogue, and a project memory keeps their configurations and revisions. Each screening decision comes with its reason and each extracted value with its source. On epidemiological expert reviews, ReviewPilot outperforms a single‑prompt LLM and AgentSLR across three LLMs, with screening macro F1 up to 0.776 and extraction F1 up to 0.788. Case studies in biomedicine and human‑computer interaction show close agreement with human annotators, suggesting that ReviewPilot can transfer beyond epidemiology. Code is available at https://github.com/ZhaoqianXue/ReviewPilot.
Authors:Matteo He, William F. Shen, Alex Iacob, Andrej Jovanovic, Xinchi Qiu, Nicholas D. Lane
Abstract:
When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes through the unembedding matrix, or readout, which is itself still being learned during pretraining. We show that a model's hidden states can encode task information before its own readout expresses it, so timelines read from outputs conflate what the model encodes with what its readout expresses. To track the readout directly, we introduce parameter‑trajectory crosscoding, which fits one sparse dictionary to every token's readout row at every checkpoint. Unlike crosscoders on activations, it needs only checkpoint weights. We use it to locate a narrow early window, 0.4‑1.1% of the way through Pythia's pretraining, in which the readout reorganizes and WordNet word categories become decodable from its rows. By then, linear probes on the hidden states already decode task information that the model's own outputs do not yet show. To test whether such gaps depend on how fast the readout learns, we pretrain small models from scratch that differ only in the readout's learning rate. We find that raising it makes the readout develop earlier and the gap close sooner, with little effect on final loss. Together, these results imply that early in training, where many developmental transitions are dated, a behavioral date can reflect when the readout learns to express task information and not when the model learns it.
Authors:Yuchen Zhang
Abstract:
Large language model (LLM) agents for health can accumulate reusable textual skills from task feedback, such as calculation criteria, measurement time points, and formula units. Deciding which skills to retain is consequential: useful rules improve later answers, while faulty generalizations propagate errors. We propose CausalEvo, a method for causal skill self‑evolution that estimates the relative effects of deliberately changed skill sets while holding the model weights, feedback cases, and rule count fixed. At each evolution step, a frozen LLM proposes candidate rules, which are pooled with the incumbent skills so that the current skill set remains a feasible choice. Paired skill replacements against shared background rules are scored on the same feedback cases; least squares converts the score differences into relative skill values, and the top‑ranked rules are retained, favoring incumbents on ties. A decision‑loss bound separates candidate support, approximation, and estimation errors. On six MedCalc‑Bench‑Verified tasks with 120 held‑out cases, CausalEvo achieves 60.00% accuracy, compared with 55.83% for SkillOpt and 51.67% without skills. Across the recorded training configurations, which use different optimization budgets, it consumes 55.6% fewer training tokens than SkillOpt. Gains concentrate in particular calculations alongside regressions on others, and paired accuracy intervals include zero. CausalEvo turns task feedback into controlled decisions about skill retention and replacement. Project repository: https://github.com/txbrook/CausalEvo
Authors:Ziheng Zhang, Neha Hulkund, Colin Lee, Zheda Mai, Hui Shen, Ying Ding, Tanya Berger-Wolf, Sara Beery, Wei-Lun Chao, Jianyang Gu
Abstract:
Data selection reallocates training support across visual concepts, including objects, attributes, color, texture, and backgrounds. Different allocation strategies may produce varied downstream model behavior, especially for tail concepts supported by only a few samples. Discarding them can hurt test‑time robustness, while overemphasizing samples lying on category boundaries may introduce unstable training signals. This work explicitly investigates this relationship across fine‑grained, spurious‑correlation, and general image classification, as well as multimodal pretraining. Specifically, we use sparse autoencoders (SAEs) to extract concept activations from individual images. During selection, we accumulate the dataset‑level concept support for each added sample. Applying diminishing returns to this distribution down‑weights well‑represented concepts and shifts the focus to the distribution tail. We then incorporate local support to find samples that reliably strengthen under‑supported concepts. Empirical results show consistent improvements over strong data selection baselines across all benchmarks. We also reveal the connection between concepts our method enhances and the induced behavioral changes, including improved robustness against out‑of‑distribution and spurious contexts. This work establishes concept support, beyond individual sample importance, as a useful principle for data‑efficient and robust learning.
Authors:Yiyang Cai, Tatsuya Kamijo, Ninghao Lu, Zijin Hu, Sherry Yang
Abstract:
Passive human videos of everyday activity offer robots access to a massive amount of human experience without requiring a demonstration for each task. However, using this experience requires recovering knowledge from unstructured video recordings and adapting it to a different body (i.e., the robot). To overcome these challenges, we present mobile code‑as‑policy (MoCaP), an agentic framework that turns passive human video recordings into guidance for an end‑to‑end mobile manipulation policy. MoCaP extracts spatial memory and cues of how people prepare to interact with objects, which inform navigation and the search for a manipulation‑ready configuration. Guided by passive human experience and current observations, MoCaP uses a closed‑loop code‑as‑policy to coordinate planning and tool use primitives to solve motions under the robot's geometry and kinematic constraints. On a real mobile manipulator, MoCaP achieves 80% success on each of two mobile manipulation tasks in an indoor environment whereas existing approaches cannot succeed at the task. Video‑derived spatial memory improves object‑navigation success from 70% to 100%, and human approach prior raises the rate of finding a reachable base pose from 0% to 100% at an unfavorable start location. Videos and code are available on the project website: https://mocap‑7.github.io/MoCAP‑page/.
Authors:Frank F. Yang, Kefei Yan, Yiren Wang, Yiyang Wang
Abstract:
Improving the reasoning capabilities of vision‑language models (VLMs) often relies on post‑training with supervision from stronger models. Two common strategies are supervised fine‑tuning (SFT), which trains on teacher‑generated demonstrations, and on‑policy distillation (OPD), which provides teacher feedback on trajectories generated by the student itself. While OPD can be more data‑efficient by supervising student‑generated trajectories, it is unclear how the teacher's supervision at each state affects the student's learned behavior. To study this question, we conduct a controlled comparison of OPD and filtered SFT across data budgets and tasks where the teacher exhibits different levels of reliability, and analyze how these differences shape student accuracy and completion behavior. Across the two tasks, we find that OPD is effective only when the teacher remains reliable on student‑generated trajectories. On ChartQA, where the teacher completes 99% of its solutions, OPD trained on only 10% of the data comes within 1.4 accuracy points of full‑data SFT. In contrast, on Geometry3K, where the teacher fails to complete 22% of its solutions, OPD underperforms SFT by 7.9 points, with most of the gap explained by incomplete responses. More strikingly, starting OPD from an SFT student reduces its completion rate from 76% to 62%, effectively erasing the completion behavior learned through SFT. Filtering teacher demonstrations for completeness alone recovers much of SFT's advantage. These results show that on‑policy supervision can transfer not only teacher knowledge but also persistent failure modes on student‑visited states. Code is available at https://github.com/ffy208/opd‑inherits‑teacher‑vlm.
Authors:Zhivar Sourati, Marianne Menglin Liu, Rongguang Wang, Zheng Wang, Yazhe Hu, Lan Guo, Yuying Wang, Mengqing Guo, Bhavesh Dnyaneshwar Gore, Shubham Pathak, Ramapriya Ranganath, Sandeep Sharma, Tao Sheng, Sujith Ravi, Dan Roth
Abstract:
Existing long‑document QA benchmarks rarely combine large‑scale human‑authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD‑QA, a large‑scale benchmark for Visual Evidence grounding and question answering in Long Documents. It contains 15,000 fully human‑authored question‑answer pairs across 5,859 real‑world PDFs totaling 372,445 pages, balancing chart, table, and complex‑layout tasks. Human annotations identify supporting pages and evidence elements and describe cited visual content, enabling separate retrieval and interpretation evaluation. We evaluate nine vision‑language models under four evidence settings and seven retrieval‑augmented and agentic systems. Our analysis identifies bottlenecks in evidence localization and interpretation. Retrieval‑based methods substantially underperform the oracle setting with ground‑truth supporting pages; full‑document access does not close the gap. Localization alone is insufficient: oracle accuracy peaks at 81.7%, highlighting persistent visual interpretation and reasoning challenges. Enriching inputs with our visual‑element annotations narrows the performance gap by 17‑24%, demonstrating value beyond evaluation and motivating data enrichment for multimodal document QA. Our code is available at https://github.com/oracle‑samples/veld‑qa.
Authors:Saurabh Kumar
Abstract:
Client diversity is often cited as the Lightning Network's defence against a bug in any one implementation, yet LND runs 80% of public nodes (84% counting one operator's fleet of about 2,000 tiny nodes) and holds 84% of public channel capacity, a capacity share that has barely moved since 2023. We test what diversity buys with a natural experiment. On 26 August 2026, amid security fixes in every implementation, Core Lightning (CLN) maintainers told operators who could not upgrade to disconnect their nodes. Using 805 daily gossip snapshots, 132,360 on‑chain channel closes and a client classifier validated against self‑identifying aliases and against the wallet that builds each node's funding transactions, we find that 22% of CLN nodes went dark overnight, ten times the usual rate, mostly Tor‑reachable ones; a third were still dark five weeks later. The rest of the network barely noticed. Reachability among all node pairs fell by 7.8 percentage points, a loss exceeded in 4 of 94 ordinary two‑day windows and half that of a Tor denial‑of‑service wave in July, while among CLN pairs it fell by 24.5 points, more than in any ordinary window. Calibrated failure scenarios explain the containment: the blast radius depends on whether a failure reaches the hubs, not on how many nodes run the failing software. Losing ACINQ's Eclair node alone costs the surviving network four times the reachability lost with all 619 routable CLN nodes, and minority clients' reliance on LND for connectivity is what their size implies; CLN nodes in fact peer with each other more than chance. Diversity statistics that count nodes mislead; the quantity to track is which software the hubs run.
Authors:Dingjie Song, Zhiling Yan, Yuxuan Zhang, Lichao Sun
Abstract:
Decision models answer typed questions about an input with one probability per option, computed in a single forward pass without generating text. Like the fast System 1 of dual‑process theory, they answer at once, but their depth is fixed at training time, so they spend the same computation on easy and hard questions. Models that reason in generated text can think longer, but they pay for every extra step in decoded tokens. We present LEV, a decision model that thinks deeper by rerunning its layers in a loop instead of generating tokens. LEV builds on a language model pretrained as a loop, reads the options after every loop step and trains all steps jointly, so one checkpoint can answer at any depth. On three looped backbones, accuracy rises with depth on the same weights, mostly on hard questions, and a 2.6B LEV goes from 63.2% at depth one to 76.2% at depth four on JevBench. On‑demand depth, set per question by a small stopper with a threshold fitted on unlabelled requests, saves 15% to 24% of the computation at a cost of under one accuracy point, level with or slightly above a fixed mix of depths. On a locked test set, the 1.4B and 3B models are 5.4 to 8.8 points more accurate than dense (non‑looped) models and cascades of the same latency below 50 ms.
Authors:Guanghui Min, Chen Chen
Abstract:
Long‑horizon LLM agents rely on context compaction: when the interaction history outgrows the context window, a compressor replaces it with a natural‑language summary. We show that these summaries contain more than a record of what happened. Compaction prompts ask for decisions and next steps, so each summary mixes facts established by the environment with reasoning written by the compressor, and the agent cannot tell the two apart afterwards. In an analysis of a widely used structured compressor, 83% of failed runs trace back to an action that carries out a specific summary entry, most of which no environment output supports. We propose Source‑Judged Compaction (SJC), a training‑free post‑processing step that scores each summary entry for whether it has a source in the environment outputs or the previous summary, and removes the entries that do not. It changes neither the agent nor the compressor and adds one inexpensive judge request per compaction. The judge is Jev, a non‑generative decision model that answers the source question with a calibrated probability instead of text. On AppWorld and OfficeBench with two agent models, SJC improves cross‑run reliability over the underlying compressor by 2.1%‑8.9% without increasing the agent's token usage.
Authors:Abdul Rehman Antall, Baidya Nath Saha
Abstract:
Typology‑aware tokenizers are motivated by the claim that segmentation which respects morpheme boundaries preserves information that frequency‑driven segmentation destroys. Studies routinely establish that it helps while assuming the boundaries must be linguistically correct. We separate them. Tagalog marks voice with infixes (s‑um‑ulat 'wrote', s‑in‑ulat 'was written'), so the root is not contiguous and no sequence of BPE merges can recover it. We build MorphAustro‑Tok, a tokenizer that emits the root as a contiguous unit followed by a positional infix token, and compare it against a corpus‑matched BPE baseline and, at two of three fertility levels, against a scrambled‑morpheme control that preserves morph count, invertibility and per‑type consistency while randomising the pseudo‑infix. Across 33 runs, MorphAustro‑Tok improves Tagalog machine translation over matched BPE by 0.84, 1.49 and 3.40 mean sentence‑level chrF++ points. Token count does not explain the gain, since it grows as segmentation coarsens, and neither does capacity: at the coarsest level the scrambled arm carries more parameters and scores lower. Scrambling the analysis moves the gain by less than between‑seed dispersion at both controlled levels, so we detect no contribution from morphological correctness at this scale.
Authors:Beth Pearson, Michael Wray, Martha Lewis
Abstract:
Negation is a fundamental aspect of language understanding and it is important vision‑language models (VLMs) are able to process it effectively. We investigate how VLMs process negation by analysing attention patterns of different models. We draw comparisons between human gaze and attention in models when processing affirmative vs negated statements. We compare object level model attention to human eye‑fixations in both binary and multary settings, evaluating with both synthetic and realistic datasets. We find models generally do not follow human‑like attention patterns when resolving negation, suggesting they rely on mechanisms outside of attention. While layerwise analysis shows that PaliGemma2‑3B shifts object attention to the alternative under negation, our headwise ablations suggest this is a byproduct rather than a negation processing mechanism. Additionally, we find that LLaVA‑1.5‑7B displays a spatial bias towards left‑side objects. Our evaluation of models on downstream tasks shows that all models have a discrepancy between affirmative and negated questions highlighting the need to better understand the underlying processing of negation to close this performance gap. Code available at https://github.com/otmive/negation_attention
Authors:Dmytro Babarytskyi
Abstract:
A mesh of interceptors defending a fixed site must spread over the predicted arrival point of an approaching object. This is a recurring task in multi‑agent robotics: N agents, each covering a disc of radius r, spread over a probability density so that a point drawn from it falls within some agent's disc. We give closed forms for the quantities that bound performance before any policy is chosen, and validate each by Monte Carlo. The uncertainty of a point predicted by extrapolating a noisy linear track follows from least‑squares variance propagation plus a delta‑method term for the uncertain arrival time, agreeing with simulation to within 1.8%. The coverage probability of a packed mesh admits a closed form accurate to 0.007 absolute, with a volumetric analogue. A naive area ratio misses a ceiling: while agents stay at least 2r apart, no mesh exceeds 90.69% coverage in the plane or 74.05% in a volume. This ceiling belongs to the separation constraint, not to the problem ‑ permitting overlap substitutes Kershner's thinnest‑covering density and removes it, the better regime above a crossover at p=0.80. Hexagonal packing is optimal among five arrangements; the centroidal‑Voronoi configuration coverage control supplies reaches only half its coverage at small budgets. The Gaussian assumption is conservative under heavy tails and anisotropy but fragile under bias: a one‑sigma aim‑point offset costs 14 points of coverage. A repositioning team must also choose a commit time, and the admissible window can be empty. Against the applicable ceiling, analytic placement attains 85% of what the configuration permits while centralised PPO averages 15% over three trainer seeds, with a paired bootstrap against a null policy containing zero at every seed. We argue such ceilings should be computed before any learning experiment and quoted alongside every result.
Authors:Marcel Friedrichs, Addie Jordon, Juri Kolčák, Daniel Merkle
Abstract:
Graph transformations model changing systems, with applications in software, biology and chemistry, among others. We introduce MolFoundry, a graph transformation framework, and present a workflow for observable analysis that tracks the dynamics of local patterns through rule composition. Analysing rule compositions avoids the need to enumerate a potentially infinite state space. Building on rule‑algebraic stochastic rewriting, MolFoundry's kernel composes rules with nested application conditions, including missing‑edge constraints. Given rules, rates and query patterns, the algorithms iteratively identify auxiliary pattern counts and search for closure. Using exact counting identities supplied by graph‑family backends, successful searches yield closed ordinary differential equations for average counts or, with justified individual jumps, finite event models describing joint changes and rates. We evaluate conditional composition and automated discovery of auxiliary observables against published calculations and independent rule application. Applications to aldol addition, copolymerization and chlorination show how the framework yields quantitative predictions directly from molecular graph rules.
Authors:Xuanchang Zhang, Tong Zhang
Abstract:
Reinforcement learning with verifiable rewards often trains language models on responses generated by a different, possibly stale behavior policy. Correcting this mismatch with learner‑to‑behavior importance weights can introduce high variance for long responses. We introduce Gibbs‑Gap Policy Optimization (GGPO), which minimizes a log‑mean‑exp contrast of KL‑adjusted rewards among responses to the same prompt, without learner‑to‑behavior importance weights. We derive GGPO from KL‑regularized policy optimization: on policy, its population loss is exactly the KL‑regularized optimality gap, and under suitable coverage the learner's single‑response expected reward is at least the behavior policy's best‑of‑K reward, minus terms for the raw finite‑group loss, regularization, and finite training data. We characterize reference refresh through exact Gibbs‑target composition and a finite‑data final‑policy reward bound under local reference control and tokenwise behavior mismatch. This refresh is empirically important: in an ablation, fixed‑reference GGPO scores 32.96 on the five‑benchmark problem‑weighted average, versus 46.58 with the default refresh schedule. Experiments on mathematical reasoning further show that GGPO outperforms GRPO and AGRO under training‑inference mismatch and on fixed offline data, while maintaining its performance as controlled policy staleness grows.
Authors:Mengfei Li, Yi Zhang, Xingtong Ge, Lunjie Zhu, Fangyu Lin, Peng Li, Guanglu Song, Yu Liu, Wei Xue, Wenhan Luo, Yike Guo
Abstract:
Generative gameplay requires more than immediate responses to player actions.Actions should have persistent consequences, while the generated world should act according to its own state and rules, shaping the player's situation and subsequent choices. Video generation captures local dynamics, but limited history, partially observed game state, and the need for contextual behavior decisions make such interactions difficult to sustain through audiovisual prediction alone. We introduce \rig (Reciprocally Interactive Generation), a framework for reciprocal interaction in generative gameplay. \rig combines persistent state and configurable rules, contextual interaction reasoning, and action‑conditioned audiovisual generation. Together, these components make interaction consequences and actor behavior jointly determine generated content while maintaining consistency between game state and visible outcomes. We construct a Xonotic gameplay dataset to train an audiovisual generator for learning dynamics and interactions. Further post‑training enables \rigarena, an interactive generative first‑person shooter (FPS) prototype. Experiments demonstrate more reliable combat outcomes and reciprocal interaction between the player and the generated world. Project page: https://murphylmf.github.io/RIGen/.
Authors:Giovana Kerche Bonás, Thales Sales Almeida, Marcelo da Silva Reis, Rodrigo Nogueira, Marcos Meideiros Raimundo
Abstract:
Recent literature shows that unsafe tool calls can often accompany textual refusals in language models. To investigate this hidden vulnerability, we introduce PRISM, a framework for evaluating the transfer of activation interventions across precision, languages, and scales. Our findings demonstrate that leveraging lower‑resource settings (specifically quantized NF4 labels and cross‑lingual transfer) offers a robust pathway to expose and improve model alignment. Notably, using behavioral labels from an NF4 copy enables stable intervention estimation even with very few scenarios. Furthermore, these alignment improvements transfer effectively across languages: five‑language NF4‑supervised directions successfully transfer to held‑out Arabic, reducing unsafe violations in Llama‑3.1‑8B by 21.95 percentage points without target‑language tuning. By mapping paired benign activations, we also successfully transfer these safety interventions to much larger models (up to 70B parameters). Ultimately, PRISM highlights how auxiliary supervision from compressed models and multilingual data can uncover and mitigate hazardous tool‑calling behaviors across different model families. We release all experimental code and supporting artifacts at https://github.com/GiKerche/prism, alongside an interactive explorer of recorded model responses and experimental results at https://gikerche.github.io/prism/.
Authors:Sujie Hu, Huaisong Zhang, Pujing Yang, Qian Wang, Xiaomin Li, Xinhang Leng, Chen Li, Jing Lyu, Xiu Li, Ling Pan
Abstract:
Recent reinforcement learning approaches have improved visual text generation, yet generated images still suffer from incorrect wording and malformed glyphs. We identify two coupled limitations: recognition‑based rewards often overlook visible glyph defects, while whole‑image scalar feedback obscures the uneven quality of individual text regions. These limitations motivate our central principle: Blame errant words, not the canvas, to effectively preserve local quality distinctions. First, we introduce \mathbfTextDART, a defect‑aware region‑targeted reward model that localizes individual text regions and independently evaluates their semantic correctness and glyph quality, providing fine‑grained, region‑precise feedback. Building on TextDART, we propose \mathbfSCANNER, which turns these regional judgments into spatially differentiated policy feedback. It normalizes rewards for each text target across sampled images and casts the resulting regional advantages onto the corresponding latent tokens. Extensive experiments on existing benchmarks and our newly established PolyText‑Bench demonstrate that our approach achieves state‑of‑the‑art performance in text accuracy and glyph fidelity. Project page: https://sujiehu.github.io/textdart‑scanner
Authors:Jianyu Zhang, Léon Bottou
Abstract:
The closed world asks AI for specialized competence and lets it manufacture the iid data its theory assumes: a fixed criterion licenses collecting examples until the task is covered. The open world asks for versatility ‑‑‑ taking up tasks a machine was not built for, from few examples and with little prior from a human designer ‑‑‑ and offers data that is grouped, not shuffled. Its tasks are not known in advance, so a machine must do inference‑time learning (1st requirement). Modeling tasks as sharing latent factors and decomposing the training objective, we derive two more: a rich representation, retaining more than iid generalization requires, and a predictive disentanglement that aligns inputs and outputs into matched low‑dimensional groups. The analysis further predicts the need for a consistent inner learner, and the benefit of long contexts spanning related tasks. We test these claims on language models up to 8B, asking whether a model has a property (qualitative) rather than how much it wins (quantitative). Models that meet the requirements improve as data from related tasks accumulates; the rest degrade, and neither scale nor layer‑type interleaving repairs this.
Authors:Bolin Shen, Yushun Dong
Abstract:
In LLM‑based multi‑agent systems (MAS), a local mistake can propagate as other agents build on it, turning a correctable error into a failed task. Diagnosing completed runs or repeatedly re‑executing them to search for corrections does not provide a learned policy for intervening while execution is still in progress. We present Agent Runtime Inspection and Amendment (ARIA), a framework for learning when and how to repair ongoing multi‑agent executions. Our key intuition is to turn execution‑verified repairs into reusable supervision, moving repair search into data construction and training so that a learned model can intervene at runtime without repeatedly testing alternatives. First, we collect over 24,000 successful and failed trajectories across four multi‑agent frameworks, six benchmarks, and eight host models. From these trajectories, we construct the ARIA dataset, to our knowledge the first large‑scale dataset pairing failed multi‑agent trajectories with execution‑verified repairs. It contains 3,953 verified repair pairs, each specifying what went wrong, where to intervene, and how execution should continue. Then, we develop the ARIA framework for MAS‑agnostic runtime inspection and amendment, designed to integrate with any MAS that exposes execution trajectories and supports continuation replacement. Using verified repairs as training examples, we teach a compact model to recognize when intervention is needed and how to correct an ongoing run. The aim is to learn repair strategies that apply to related failures in new configurations, rather than merely reproduce individual corrections. Finally, the trained ARIA model demonstrates strong repair performance in both post‑hoc and runtime settings, with generalization to unseen frameworks and host models. Our code is available at https://github.com/LabRAI/ARIA.
Authors:Avdhesh Singh Chouhan, Jack Perry
Abstract:
Automating organizational work means answering many small typed questions about one piece of state (does this email need a reply, how urgent is it, who owns it) with probabilities good enough to act on without a human. The open typed‑decision model we compare against, like a standard cross‑encoder, answers each question with a separate pass over the input. We present Verdict‑MQ, an open encoder that reads a case once and answers all of its questions: given identical retained state tokens after truncation (verified on every reported case), a structured attention mask makes the state encoding independent of the questions, keeps questions from influencing each other, and makes option order irrelevant by construction. On the typed‑decisions benchmark, a 395M Verdict‑MQ matches the soft gold labels significantly better than the released 421M Laya specialist on all five fresh seeds (KL 0.080 vs 0.117, Brier 0.045 vs 0.061), an advantage that survives per‑system temperature scaling; its higher accuracy (0.801 vs 0.767) lies past the benchmark's own saturation point, so we read that margin as closer agreement with the benchmark's labelling teacher. On RACE, BoolQ and MultiRC, against a matched fine‑tuned baseline it wins on MultiRC and loses on RACE and on BoolQ, where its training is unstable; we measure zero‑shot references ourselves, including a closed general system (Jev) that beats every fine‑tuned encoder. A controlled comparison measures what the shared state costs: giving each question its own question‑aware copy of the state gains 0.9‑3.5 points on RACE and MultiRC at 1.4‑2.5x the per‑case latency, and nothing measurable on typed decisions, with the invariance guarantees unchanged.
Authors:Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson, Kevin Leach
Abstract:
LLM‑based agents increasingly execute long‑horizon, end‑to‑end workflows involving shell commands, code modification, dependency installation, and interaction with remote services. This growing autonomy expands their exposure to indirect prompt injection (IPI), where adversarial instructions embedded in external content are interpreted as legitimate task directives. Existing evaluations, however, often focus on isolated malicious strings, single‑step tool calls, or narrowly scoped agent configurations. We introduce INFUSE, an end‑to‑end benchmark for evaluating IPI throughout the lifecycle of long‑horizon agent workflows. INFUSE comprises 40 realistic tasks and a structured taxonomy of 56 attack scenarios spanning five threat categories. Attacks are injected through realistic interaction surfaces encountered during task execution, extending evaluation beyond attacks placed only in initial prompts or isolated tool interactions. We evaluate 19 LLMs from the GPT, Claude, Gemini, Llama, Gemma, and Qwen families, producing 41,824 adversarial trajectories (26,220 without defenses) and 720 benign reference runs. Across the benchmark, mean attack success reaches 19.6% under LLM‑based evaluation and 15.4% under heuristic verification, with per‑model ASR‑L ranging from 3.2% to 41.9%. Existing guard models provide limited protection, while prompt‑level instructional hardening reduces mean attack success from 58.2% to 14.3%, revealing substantial but model‑dependent security‑utility trade‑offs. Our code is available at https://github.com/judydnguyen/infuse.
Authors:Yuxuan Chen, Yang Xiao, Jack Sun, Chak Sun, Brick Li, Sam
Abstract:
Full‑duplex spoken language models decide at every frame whether to take the floor, defining a conversational personality ‑‑ yet each set of weights encodes exactly one, which human speakers bracket rather than share. Existing controls mostly act from outside; none identifies the quantity that sets how much the model speaks. We identify it: a single late‑layer residual‑stream direction, the initiative vector, extracted from the model's own conversations without training or annotation. The same direction reads the decision, anticipating onsets up to 800 ms ahead, and shifts it: scaled during generation, it moves speech ratio from 6‑12% to 42‑61% (no steering: 21‑30%), consistently across conversations, weights, and corpora. A short reference recording suffices to transfer a speaker's turn‑taking style to the model, and the transfer is reference‑specific: swapping in another speaker's reference worsens the match in 93‑98% of cases, whereas the best uniform dose is indistinguishable from no steering. Turn‑taking style is not frozen into the weights but a direction that can be read, shifted, and calibrated per person. Audio examples are available at https://elpsykongloo.github.io/initiative‑vector/.
Authors:Wenhao Yan, Haodong Li, Xiaotong Li, Yaxin Luo, Zhengrong Yue, Jialv Zou, Xu Huang, Haojin Yang, Nan Jia, Yan Feng, Peng Pei, Xunliang Cai
Abstract:
Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four‑level hierarchy and discover that higher‑level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi‑step world state prediction, a high‑level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT‑Image‑2.5 System scoring 72.8 and demonstrating early‑stage visual state prediction capabilities on real‑world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open‑source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning‑heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real‑world states.
Authors:Dongyue Lu, Guian Fang, Hur Lim, Yonggang Wu, Yanrong Wang, Hongshan Chen, Shengju Qian, Yue Liao, Wei Chow, Lingdong Kong, Tianxin Huang, Yu Yang, Hongji Yang, Junchao Huang, Ao Liang, Yihao Wang, Zihan Wang, Rong Li, Hanlin Chen, Xin Wang, Wei Tsang Ooi, Mike Zheng Shou, Shuicheng Yan
Abstract:
Video generators serve as interactive worlds for exploration and simulation, yet realistic and visually appealing frames do not necessarily depict a consistent 3D world. Existing benchmarks assess selected geometric properties or use vision‑language models to judge memory and consistency, leaving systematic 3D evaluation underexplored. We introduce Orbia, the first benchmark dedicated to a systematic and comprehensive evaluation of 3D consistency in interactive world models. Through controlled exploration and revisit, we organize the evaluation into three dimensions: input preservation, generated‑content persistence, and 3D self‑consistency. For input preservation, registered RGB‑D references and annotated anchors support appearance, geometry, and spatial stability measurements. For generated‑content persistence, we compare revisits to regions absent from the input image with their first generated observations to test whether models preserve what they create. For 3D self‑consistency, cross‑view depth and feature correspondence, regional scale agreement, and held‑out Gaussian rendering test whether generated views describe the same 3D scene. Minute‑long exploration and revisits further test long‑term consistency. Our construction and annotation pipeline brings together public datasets, self‑captured scenes, and simulation, yielding 720 instances across indoor and outdoor environments and six exploration‑and‑revisit protocols. Evaluation spans 27 text‑, action‑, and camera‑conditioned models through native‑interface adapters. Our Orbia benchmark provides a systematic basis for diagnosing 3D inconsistency and developing more persistent video world models.
Authors:Jie Yu, Yuxi Sun, Zhaoxu Li, Chunxiao Liu, Xudong Jiang
Abstract:
Recent latent reasoning methods move beyond discrete Chain‑of‑Thought (CoT) by replacing sampled tokens with continuous soft embeddings or hidden states, enabling richer intermediate representations and preserving alternative reasoning paths. However, the need for latent reasoning varies across decoding steps: confident predictions favor discrete commitment, whereas ambiguous steps benefit from preserving multiple plausible alternatives. Existing approaches typically handle this variation through either global soft activation or hard switching based on uncertainty thresholds, treating latent intervention as an all‑or‑nothing decision rather than continuously adapting its strength. To address this limitation, we propose Competitive Token Selection (CoTS), a training‑free framework that reframes latent intervention from binary mode selection into continuous allocation at the token level. CoTS uses the probability competition between the two most likely candidates to dynamically balance discrete commitment and soft exploration at each reasoning step. The second candidate represents the strongest alternative to the model's preferred continuation, so the separation between the two directly reflects the stability of the current discrete choice. When one candidate clearly dominates, CoTS keeps the representation close to the sampled discrete token to preserve a stable reasoning trajectory; as the competition strengthens, it increases the contribution of the soft representation to retain plausible alternatives. This competition‑based allocation requires neither manually tuned switching thresholds nor mixing coefficients, and introduces no additional forward passes. Experiments across six reasoning benchmarks demonstrate that CoTS consistently outperforms standard CoT and state‑of‑the‑art training‑free latent reasoning methods. Our code is available at https://github.com/OkinoYu/CoTSReasoning.
Authors:Minjong Cheon
Abstract:
Global streamflow forecasting supports flood preparedness, yet data‑driven models trained on reanalysis lose skill when driven by operational forecasts, increasingly so with lead time. AIFL addresses this shift with a two‑stage transfer‑learning recipe and a deterministic LSTM, but it feeds one forecast lead time into the whole input window and gives no uncertainty. We ask how far this open recipe can be extended without changing its recurrent core. KNOU‑Flood keeps a single‑layer LSTM and the two‑stage training and changes the information around it: a longer input window (424 days), lead‑aware forecast inputs, GraphCast as a second forecast source next to ECMWF HRES, and a latent‑noise ensemble trained with the fair CRPS. We compare against a re‑implementation of the AIFL recipe (its code is not public) under identical data, stations, splits and metrics. Our contributions are threefold. First, over three seeds and 2,099 test basins (2021‑2023), KNOU‑Flood reaches a median KGE' of 0.65 at LT1 and 0.48 at LT10 against 0.62 and 0.34, with significant gains at all ten lead times under two scoring protocols, also against a 1024‑unit baseline and on held‑out basins (one seed each). Second, identical weights give an LT10 KGE' of 0.18 or 0.34 depending on how forecasts are fed, so much of the lead‑time decay reported for AIFL is an input‑protocol effect; the probabilistic head contributes most to the gain, followed by the raw GraphCast channels, the longer window and mixed fine‑tuning. Third, the ensembles reach lower CRPS than probabilistic versions of the baseline, are nearly calibrated with inference‑time dropout, and detect 1.5‑ and 2‑year floods better than all baselines; the benefit for rarer floods depends on the seed, and an exploratory tail‑weighted variant improves 10‑year and larger events. KNOU‑Flood is a controlled extension of an open recipe, not a new architecture.
Authors:Zahra Dehghani, Pablo Piantanida, Mohammadhadi Shateri
Abstract:
Sampling from continuous flow‑based generators such as flow matching models requires numerical ODE or SDE integration, making inference cost depend on the number of function evaluations (NFEs). In practice, the sampling budget is inherited from prior work or selected through expensive FID‑versus‑NFE sweeps, using FID to assess image‑generation quality, requiring sample generation and evaluation at multiple budgets. This makes sampling‑budget selection dependent on repeated empirical evaluation. We therefore ask a complementary question to solver design and timestep allocation: for a fixed pretrained model and solver, what numerical resolution is sufficient? We introduce FLOWBUDGET, a training‑free estimator of solver‑relative sampling complexity. Using simulation‑free interpolation probes of the learned dynamics, FLOWBUDGET normalizes the leading one‑step discretization error by the motion, estimates its coefficient from interpolation probes, and maps a numerical tolerance to a step budget using the solver order. We derive estimators for numerical solvers in flow~matching sampling, namely Euler and Heun for ODE sampling and additive‑noise Euler‑‑Maruyama for SDE sampling, and connect the ODE coefficients to terminal integration error. Experiments on SiT, FlowDCN and Rectified Flow models, spanning flow‑model architectures and formulations, test whether FLOWBUDGET can estimate the NFE budget without generating samples at multiple budgets or performing an FID‑versus‑NFE sweep. The code is available at https://github.com/Yasaman‑dt/FlowBudget
Authors:Alden Do Rosario, Kirollos Atef
Abstract:
Retrieval‑augmented generation asks one question at volume: does this evidence support this claim? The answer is one bit, yet most pipelines obtain it from a text generator and pay generation prices for the prose wrapped around it. We compare a non‑generative decision model, which returns a typed value from a declared enumeration together with a native probability and no text, against a frontier LLM configured as an engineer would deploy it for binary classification. Both answer the same 495 human‑labelled claims from LLM‑AggreFact, so every comparison is within‑item. We report three things. First, the two systems are statistically indistinguishable on the metric this task is conventionally scored by, a paired difference of ‑0.56 points, Jev minus Astra, at a 187‑fold cost difference. Second, that tie carries almost no information about deployment: the systems disagree on 15.2% of claims, and the cheaper one admits 23.2% of the claims annotators marked unsupported against the frontier model's 14.1%. A component whose job is to catch unsupported claims should be judged on that rate, and the conventional headline hides it. Third, because the errors differ they compose: routing the claims the cheap model is confident about and escalating the rest reaches 74.10 at USD 2.92 per thousand claims against 73.83 at USD 8.93, with the threshold chosen out of fold. But a matched random‑escalation control at in‑sample thresholds, spending the same budget by coin flip, does as well, so what buys the saving is the escalation share and not the probability that picks what to escalate. Measured against the cheap model it is built from, the cascade costs 61‑fold more for 0.83 points, 95% CI [‑3.36, 4.86], and it is the more permissive guard of the two. Equal accuracy conceals unequal errors. All predictions, prompts, pricing and code are released.
Authors:Thomas Serval
Abstract:
Long‑horizon failures of language models are usually measured as end‑task success or as a horizon length. We instead score, step by step, three components of a 400‑step ledger task in which every step is easy in isolation and the model's own answers are re‑injected into its context: recognising a record already seen 5 to 300 steps earlier, processing the current record locally, and writing a counter incremented at every step. On Qwen3‑8B (non‑thinking, 20 seeds, replicated on a second machine and backend), with every step scored component by component, two results hold. First, duplicate‑recognition errors rise late in the episode ‑‑ from 0.0019 to 0.011 per step at risk across step 200 ‑‑ while the counter makes no error in 8,000 steps and local processing is descriptively stable; the rise combines long‑distance recall errors and rising false alarms. Second, a block of irrelevant content inserted before each step makes the counter fail in 20 of 20 seeds on each machine, against none without it, while its effect on local processing is much smaller and not distinguishable from none. Three pre‑registered follow‑ups probe the counter. Its value depends on the value present in the response history: a corrupted value persists without correction in every seed, and an exact external reminder, repeated before each message, re‑aligns it within one to two steps. Under the irrelevant block, no counter error was observed when an exact compact reminder of the state, supplied externally by an oracle, preceded each message (0 of 20 seeds with an error, against 15 without it), and this reminder outperformed a length‑matched neutral text; whether a single reminder suffices was not conclusive, and the absence of damage to the other components was not established. An aggregate or exclusive error classification can hide the difference we report.
Authors:Anna Mei, Michael R. Douglas
Abstract:
We introduce a novel method for verifying Lean formalizations that goes beyond LLM‑judged faithfulness. Unlike most verification methods, which rely on LLMs or trained models, our method combines LLM‑based semantic review with mechanical numerical and vacuity checks, providing checkable feedback and detecting errors missed by LLM judges. We apply this method, along with our Extract‑Verify‑Proof autoformalization pipeline, to construct the Formalized Library of Mathematical Functions (FLMF), a partial formalization of the NIST Digital Library of Mathematical Functions (DLMF). On three labeled datasets, our method achieves the lowest accepted‑error rate among the evaluated methods, measured as the fraction of incorrect statements accepted by the verification method. Expert review of the results also identifies translation and source defects in existing datasets. Code available at https://github.com/annamei1/FLMF‑Ch7ErrFunc‑Release.git.
Authors:Fares B. Mehouachi, Saif Eddin Jabari
Abstract:
Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate‑value pairs (x, u(x)). We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear explicitly in the solution operator, suggesting that providing derivative features should reduce the network's implicit differentiation burden. We prove this intuition: providing derivative features improves approximation bounds from O(W^‑(s‑m)/d) to O(W^‑s/d), where W is parameter count, s is input regularity, m is the PDE order, and d is spatial dimension. Our central finding, however, is a surprise: the optimal derivative order β^ is strictly less than the PDE order m. This gap arises from a bias‑variance tradeoff in spectral space that we characterize in closed form. Learning β from data achieves automatic spectral regularization. We introduce \partial‑NO (del‑NO), for derivative‑augmented neural operators, an architecture‑agnostic augmentation that provides learnable fractional derivative features to any neural operator backbone. Across benchmark problems and architectures, \partial‑NO consistently improves prediction accuracy, with learned orders that reflect a representation of known physics modulated by noise and finite‑sample constraints.
Authors:Ren Zhuang, Ben Wang, Shuifa Sun
Abstract:
Preference optimization ranks final answers and trajectory RL maximizes aggregate returns, yet neither credits the local change that turns a flawed reasoning state into a better one. We introduce Lifted State Policy Optimization (LSPO), which establishes this finite, cost‑adjusted transition as the primary unit of learning. Each textual answer is augmented with a learned continuous coordinate, forming a lifted state that an energy function evaluates and orders. A shaped reward drives energy descent net of edit and step costs, and an internalization loss absorbs selected low‑energy answers into the direct generator. On Qwen3.5‑27B, the resulting single‑pass policy reaches 71.8% aggregate accuracy across mathematics, science, code, and logic benchmarks, matching test‑time reflection at 71.9% with 28% lower latency and 25.8% fewer tokens. The pattern holds across dense and mixture‑of‑experts backbones, and matched controls confirm that transition credit and the lifted coordinate each contribute, as does answer internalization. Our code is available at https://github.com/Venomrko/lspo.
Authors:Yongzhen Hu, Hao Yu, Jiawei Wang, Xianming Zeng, Jianyun Xu, Xingliang Liu, Ruizhen Hu, Xiaowei Zhou, Hujun Bao, Sida Peng
Abstract:
Feed‑forward Gaussian reconstruction builds 3D scenes directly from input images for novel view rendering, but learning Gaussian attributes from predicted geometry for high‑quality rendering under large viewpoint changes remains challenging. We analyze limitations in Gaussian attribute learning and introduce InfiniSplat v2, which improves rendering quality through architectural and training changes. Our method decouples geometry and attribute prediction with a frozen geometry branch, aligns predicted Gaussians using ground‑truth camera poses during training, and adopts large‑angle target view supervision. Across six datasets, the feed‑forward outputs of the same trained model achieve state‑of‑the‑art mean LPIPS and DISTS among the evaluated methods in both pose‑conditioned and pose‑free settings. In the pose‑conditioned setting, mean LPIPS and DISTS decrease by 25.8% and 32.9%, respectively, compared with the strongest evaluated baselines. We further find that opacity learned with only image reconstruction supervision suppresses geometrically inconsistent points. Project page: https://zju3dv.github.io/InfiniSplat2/
Authors:Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
Abstract:
Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading‑order prediction, and image captioning. This separation leaves scientific figures, captions, and visual semantics weakly connected, and fails to measure whether a model can produce a single page‑level parse that is both structurally faithful and semantically accessible. We introduce Read‑Parse‑Describe, a unified dataset for document parsing with visual element description generation. Given a page image, models are required to output an ordered list of layout elements containing bounding boxes, semantic labels, reading‑order indices, recognized textual content, and alternative text descriptions for visual elements. The benchmark is built from 2,402 documents, comprising 18,163 continuous pages, 125,218 layout elements, and 11,338 visual instances. Besides, we further propose a plug‑and‑play space‑awareness enhancer that reuses multi‑layer visual features and injects grid‑aware spatial guidance into vision‑language tokens for layout‑aware generation. Experiments across end‑to‑end VLMs and pipeline parsers show that existing systems still struggle to jointly recover reading order and visual semantics, while our enhanced model achieves strong joint performance. These results highlight the need to move beyond isolated document parsing modules toward unified models that jointly read, parse, and describe complex documents. The new dataset and code will be made publicly available at https://github.com/yufanchen96/RPD
Authors:Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung Han
Abstract:
Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This has motivated extensive research on one‑step generative modeling primarily built on diffusion and flow, though existing approaches often rely on complex training machinery. In this work, we introduce Priming Models, a simple two‑stage framework for training one‑step generators. Motivated by the difficulty of distributional training from scratch, Priming Models are first trained on a reconstruction objective, e.g., pixel‑space denoising at a fixed noise level, before undergoing feature‑space distributional post‑training. Our central scientific finding is that this simple reconstruction stage is already enough to entirely alleviate the difficulty of distributional training. Empirically, across class‑conditional image generation, text‑to‑image generation, and world modeling, Priming Models significantly advance the Pareto frontier of training compute vs. generation quality. On ImageNet at 256 × 256 resolution, we achieve a state‑of‑the‑art FID of 1.19 among one‑step generators using approximately one‑tenth the training compute of prior methods with comparable FID. We open‑source our codebase at https://github.com/JaeyeonKim01/priming.
Authors:Yinzhen Huang, Jiahe Xie, Yitong Sun, Ruochen Zhang, Xingxing Wei
Abstract:
Multimodal Large Language Models (MLLMs) are increasingly deployed in safety‑critical domains, yet their susceptibility to backdoor attacks remains a formidable challenge. While existing defenses have evolved from visual‑centric patch cues to unsupervised internal attention statistics, they are fundamentally constrained by a reliance on static snapshots and attack‑specific heuristics. Given the intricate interplay between dual modalities, the vast design space for multimodal triggers allows diverse threats, such as textual injections, sub‑perceptual noises, and coordinated cross‑modal triggers, to easily circumvent such rigid priors, particularly when attack types are unknown in real‑world applications. In this paper, we rethink backdoor detection by shifting the diagnostic paradigm from static observations to dynamic output evolution. We observe that a successful attack inevitably hijacks the model's decoding trajectory, skewing internal modal contributions during generation while maintaining an abnormal output invariance despite input perturbations. To capture this behavior, we propose EVO‑Detect, a unified detection framework that monitors the dynamic evolution of internal signals during early decoding steps through two complementary channels: Attention Flow and Logits Flow. The attention branch measures how image‑text attention allocation evolves during early generation, whereas the logits branch quantifies the sensitivity of the output distribution to modal perturbations through interventional ablations. We then convert these trajectories into compact sample‑level metrics and normalize them against negative‑control signatures so that natural perturbations are less likely to be mistaken for backdoor evidence. Extensive evaluations show that our approach substantially outperforms state‑of‑the‑art baselines such as BYE and TCAP across diverse attack families.
Authors:Sam Gijsen, Moritz Seiler, Kerstin Ritter
Abstract:
Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concern whether one intervention changes the effect of another. Such interactions can be of primary interest while explaining little of the total variation, allowing models to appear accurate overall without recovering the relevant scientific effect. It remains unclear how these models learn interactions and when these successfully extrapolate to unseen conditions. We therefore study interaction recovery in flow‑matching models using a physiological time‑series simulator, drug dose‑response data, combinatorial cell perturbations, and molecular conformer generations, where we separate recovery from hallucinated interaction structure. In the controlled setting, individual effects are learned much earlier than interactions and models hallucinate interactions where none exist. Across settings, we find qualitatively different generalization problems, with successful prediction depending on whether condition structure continues from observed to held‑out targets. When it does, interactions are recoverable. If not, models can generate plausible responses while missing the interaction. We furthermore find that hallucinations increase as targets become less identifiable, and that successful interaction recovery in output space can fail under nonlinear readouts. Finally, we show that aggregate fidelity metrics can conceal whether models learn and extrapolate interactions.
Authors:Dai Shi, Andi Han, Xiaoyu Li, Luke Thompson, José Miguel Hernández-Lobato
Abstract:
The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on aligning representations via their neighborhoods, spanning trees, and distances. We show that these views can differ because the alignment criteria behind them sit at different points of one lattice, and we formalize this explanation as the Lattice Representation Hypothesis (LRH). Under the LRH, the convergence claimed by the PRH is agreement at the top of the lattice, whereas its evidence comes from neighbor overlap, a criterion near the bottom. The Aristotelian Representation Hypothesis (ARH) restricts convergence to local neighborhoods because calibration keeps the growth of neighbor overlap with model size but largely removes that of centered kernel alignment. We show that this removal can come from calibrating a maximum over layers, so global convergence remains open. Back into Plato's Cave (BPC) infers coarse shared structure from the collapse of neighbor overlap on larger evaluation sets, whereas we trace it at least partly to a fixed number of neighbors covering shorter distances. We show that the spanning‑tree overlap used by Structure over Geometry (SOG) mostly repeats one‑nearest‑neighbor overlap. The LRH also extends to independently trained models, whose training procedure and objective decide the criteria met reliably across runs. We prove this claim for the training procedure in a linear autoencoder and for the objective in a finite contrastive Gibbs model. Experiments on dozens of pretrained vision and language models and thousands of runs of both theoretical models support these conclusions. We hope this work paves the way toward a general theory of representation hypotheses, with a framework to analyze them and compare their evidence. Project page: https://eethanshi.github.io/LRH
Authors:Zewen Jiang, Yahan Li, Chenlu Guo, Diana Inkpen, Yuan Wu, Yi Chang
Abstract:
Large language models (LLMs) have demonstrated multilingual capabilities across diverse language generation and understanding tasks. Yet effective code‑switching requires more than multilingual proficiency: models must decide when, where, and why to switch languages in ways that fit the surrounding context. This context‑sensitive ability remains insufficiently evaluated. We introduce CSWBench, a benchmark for evaluating context‑sensitive code‑switching in creative writing through two complementary settings: Focused Narrative and Extended Narrative. We further propose CSWEval, a theory‑informed framework with two complementary levels: event‑level evaluation of Licensing, Contextual Appropriateness, and Functional Gain, and whole‑text evaluation of task‑specific quality‑‑‑overall literary and narrative quality in Focused Narrative and quality emphasizing multilingual organization in Extended Narrative. Evaluating twelve LLMs, we find that models exhibit meaningful but uneven code‑switching competence across narrative contexts. Model differences are particularly pronounced in multilingual literary integration, while multilingual‑focused dimensions show stronger difficulty effects overall than the general narrative dimensions; narratively salient switches are associated with stronger contextual and functional realization; and event‑level CS quality is moderately associated with whole‑text performance.
Authors:Mohsen Iranmanesh, Sina Moradi Sabet, Sina Marefat, Ali Javidi Ghasr, Maxime Newman Nereyabagabo, Kousha Amouzesh, Jaivir Malhi, Vinci Velasco, Alex Chung, Mohammad A. Tayebi
Abstract:
Large language models are increasingly deployed as autonomous agents for automated program repair, patching software vulnerabilities through multi‑step reasoning and tool use. Yet these systems typically validate a patch against the same evidence used to construct it ‑‑ a static alert, sanitizer trace, published exploit, or test suite. A patch that blocks that one piece of evidence can therefore look successful while remaining vulnerable to the same vulnerability with some variation. We present P2Patch, a generative adversarial repair system that treats each patch as a new attack target. Starting from a normalized vulnerability finding, an exploitation agent constructs a reproducing proof of vulnerability (PoV), a patching agent repairs the program, and the exploitation agent then attacks the patched program for a distinct same‑vulnerability exploit. Every successful exploit becomes a permanent repair requirement, and the final patch must block all PoVs accumulated during the patching process, with construction‑time PoVs kept strictly separate from held‑out evaluation oracles. Across 101 C/C++ and Java CVE subjects, this adversarial loop surfaces a distinct confirmed exploit in 41% of runs after the patch has already blocked its original construction‑time PoV. Against single‑pass repair from the same finding using Claude Code, P2Patch raises held‑out fix‑closed coverage from 78.1% to 84.7% and beyond‑upstream coverage from 42.1% to 62.5%. Under the same held‑out oracles, it also certifies more attack paths as blocked than recent state‑of‑the‑art agentic repair systems on overlapping subjects. These results show that adversarial testing and independent security oracles expose repair gaps that one‑shot validation misses.
Authors:Huailiang Ma, Yuxuan Li, Ziyi Zhao, Xinkai Wang, Mingyu Li, Xinyu Guo, Lifeng Zhu, Aiguo Song, Cewu Lu, Jianhua Sun
Abstract:
Vision‑language‑action (VLA) models have emerged as a powerful paradigm for robotic manipulation, enabling robots to generate actions from visual observations and language instructions. However, most existing VLA models still operate on image‑level visual representations inherited from pretrained vision‑language backbones, while lacking fine‑grained object understanding for manipulation. As a result, their robustness can degrade under out‑of‑distribution object layouts, clutter, and background variations. To address these limitations, we introduce ObjDream, an object‑centric VLA framework centered on object‑level understanding and future object state prediction. ObjDream uses a spatio‑temporal object encoder to organize visual observations into temporal object slots, where persistent object identity and dynamic object state are represented through separate channels. These structured object representations are inserted into the token sequence of the vision‑language model (VLM), where identity‑guided routing modulates VLM attention to the corresponding object tokens. In addition, an action‑conditioned object dynamics module predicts compact future object states relevant to manipulation during training, explicitly supervising object state evolution under robot actions without requiring future prediction at inference. Experiments on LIBERO, LIBERO‑Plus, and RoboTwin 2.0 show that ObjDream improves generalization under diverse distribution shifts and scene perturbations. Real‑world robot experiments further demonstrate that ObjDream transfers to physical manipulation settings with novel object arrangements and background changes.
Authors:Yule Sheng, Yue Yuan, Shan Shen, Daying Sun
Abstract:
Routability prediction enables rapid evaluation of placement quality without routing, providing effective feedback for routability‑driven placement optimization. Recent machine learning methods have achieved efficient routability prediction through data‑driven approach. However, their prediction performance highly depends on the capability of input representations to capture essential circuit information. To address the limitations of existing circuit representations in simultaneously preserving logical topology, physical layout and routing resource information, this paper proposes HGRP, a heterogeneous graph neural network for routability prediction. HGRP models the cells, nets, and routing grids as distinct types of nodes and explicitly represents topological connectivity, spatial mapping, and grid adjacency, thereby constructing a heterogeneous graph representation that preserves multi‑level circuit semantics. Based on the representation, a heterogeneous message passing mechanism is developed to enable information interaction and feature fusion among distinct types of nodes, effectively learning the complex correlations between circuit structures and routing behaviors. Experimental results on the CircuitNet dataset show that HGRP improves F1‑score by 18.69% relative improvements compared with the state‑of‑the‑art in the congestion prediction task. The source code is publicly available at https://github.com/ShenShan123/HGPR.
Authors:Davide Baldelli, Robyn Kozierok, Mirko Bronzi, Damiano Fornasiere
Abstract:
Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot control what their activations encode. Humans can learn, through neurofeedback, to control the firing of even single neurons, and recent studies investigated whether this ability extends to language models, reaching conflicting conclusions. No experimental design so far disentangles competence at the task from genuine access to one's own activations. We fill this gap with in‑context neurofeedback sessions, in which a "subject" model is asked to maximize (or minimize) a score derived from its activations, shown back to it at every turn. In our first experiment, the sentences are generated either by the subject or by a second "writer" model, while the score always derives from the activations of the subject. We test ten open‑weight models from five families, with sizes from 9B to 32B. Some models move their own scores considerably, but no more than a writer of comparable competence moves them. Is this because the task requires no privileged access, or because the subject controls its activations while processing the sentences, whoever wrote them? To answer this, we prefill a fixed sentence as the subject's reply at every turn and ask it to increase or decrease the score. Three out of ten models show small but consistent control effects. We further ask whether models can control their activations to the point of fooling an activation monitor. We make models repeat sentences a monitor would flag, such as a deceptive answer for a deception monitor, and ask them to raise or lower the monitor's score. No model learns to lower it, but one model can raise it. In‑context control of a model's own activations might not yet be a threat to activation monitors, but it exists in a few models and needs to be tracked as models evolve.
Authors:Shijun Shi, Zhe Liu, Zhi Hou, Yinan Chen, Tao Zhang, Kai Hu, Yong Liu, Jiangning Zhang
Abstract:
World Action Models (WAMs) typically adapt video foundation models to robotic control through joint video and action prediction. However, task‑level instructions may leave the next physical interaction underspecified, requiring semantic grounding and state‑dependent motion selection. To this end, we introduce Atomic‑WAM, an understand‑then‑execute framework that connects task reasoning to action generation through atomic‑motion descriptions. Specifically, a vision‑language reasoner describes the next local motion from the task instruction and visual context, guiding a WAM to predict future video and an action chunk. Furthermore, an automatic annotation pipeline curates temporally aligned, contact‑enriched supervision from multi‑view DROID demonstrations. To make atomic‑motion predictions executable by the WAM, we post‑train the reasoner with Execution‑Grounded GRPO (EG‑GRPO). With the WAM fixed, EG‑GRPO adapts the reasoner through rewards for motion appropriateness and execution success. Finally, Atomic‑WAM achieves 39.6% task success under default instructions and 31.6% under vague instructions in a real‑to‑sim evaluation on RoboLab. Project page: https://ssj9596.github.io/Atomic‑WAM‑project
Authors:Mohammad Yousuf, Arif Md Tareque Habib
Abstract:
News outlets in poorer countries cover the world much as the leading outlets of rich countries do. Two explanations compete. One says they depend on Western news agencies for lack of resources. The other says they choose to copy prestigious outlets or whoever moves first. Observation cannot tell these apart. This study measured how closely 18 national outlets in Bangladesh, Kenya, South Africa, India, Malaysia and Nigeria followed four core outlets in how much attention they gave, which angle they took and when they published, on the wars in Ukraine and Gaza and two UN climate conferences in 2022‑2024. It then ran a simulation in which language models acted as the outlets' editors and dependence, imitation and herding were switched on and off. Of nine tests, one held: outlets that relied more on wire copy published sooner after the core did. Tier and local stake showed no effect distinguishable from none. The herding test was supported and the imitation test was not. The simulation did not decide between the mechanisms, because the dependence rule, as fitted, reduced to publishing wire copy. Wire copy sets the clock of the periphery's foreign news. Whether it sets the agenda stays open.
Authors:Hongyu Ding, Kanyu Xu, Mingjia Chen, Zixuan Chen, Jieqi Shi, Jing Huo, Yifan Zhang, Yang Gao
Abstract:
Rapid progress in AI agents and robotics makes it a matter of the near future that humans are served by a team of robots. Such a team will receive several goals at once, each specified in whatever form is convenient for the human, such as a category name, a photo, or a spoken description. Existing benchmarks study multi‑agent, multi‑modal, and multi‑object goal navigation in isolation, and a benchmark that places all three in one episode is missing. To fill this gap, we introduce HM3D‑M3. A team of K agents starts at distributed locations in a photorealistic indoor scene and searches for an unordered set of M object goals, each given as an open‑vocabulary category, an instance image, or an instance description. HM3D‑M3 provides 1,000 validation episodes over 3,926 goals from 379 open‑vocabulary categories, 100K observation‑assignment training samples, and two team‑level metrics, goal‑level SR and makespan‑normalized SPL. We further contribute two baselines. HiGA is a trained hierarchical goal assigner on top of a frozen open‑vocabulary navigation policy, and LaViRA‑M3 is a training‑free team of VLM agents with shared mapping and progress analysis. Adapted multi‑robot and zero‑shot methods reach 16‑‑27% SR, while our baselines reach 38‑‑41% with complementary strengths and distinct failure modes. Furthermore, we deploy LaViRA‑M3 on a heterogeneous three‑robot team. Project page: https://robo‑lavira.github.io/hm3d‑m3/
Authors:Biplab Poudel, Betserai Albert Magarire, Anish Kumar Shah, Jiacheng Xie, Dong Xu
Abstract:
Accurate nuclei segmentation underpins quantitative computational pathology, yet remains challenging due to morphological variability, weak boundaries, nuclear crowding, and pronounced domain shifts arising from differences in tissue preparation, staining, and acquisition. While deep learning and recent foundation models have improved segmentation performance under matched distributions with the original training data, their generalization across heterogeneous histopathology domains remains limited. We introduce GDBA‑Net, a parameter‑efficient nuclei segmentation framework built around the proposed Gated Dual‑Branch Adapter (GDBA). GDBA‑Net adapts a frozen SAM2 encoder by integrating pretrained semantic representations with explicit frequency‑derived structural priors. The complementary representations are independently transformed through Grid MLP branches and dynamically integrated using sample‑adaptive channel‑wise gating, selectively emphasizing structural cues relevant to nuclear delineation. Despite using only 4.3 million trainable parameters, GDBA‑Net achieves state‑of‑the‑art in‑dataset performance and consistently superior cross‑dataset generalization across six diverse nuclei segmentation benchmarks under fully supervised, cross‑dataset, and five‑shot adaptation protocols, outperforming competing CNN‑, transformer‑, and foundation model based methods. Comprehensive ablations confirm that these gains arise from the coordinated design of semantic‑frequency complementarity and adaptive gating rather than increased model capacity. By uniting transferable semantic representations with explicit structural priors, GDBA‑Net provides a robust, annotation‑efficient solution to domain variability in nuclei segmentation. https://github.com/biplabpoudel25/GDBA_Net
Authors:Hang Ren, Peidong Wang, Yichen Gao, Wen Zhang, Jiayang Yu, Xiaocui Yang, Shi Feng
Abstract:
In recent years, with advances in multimodal interaction technologies and the growing demand for intelligent emotional support, Multimodal Emotional Support Conversation (MESC) has emerged as an important task. However, existing systems still suffer from shallow responses and inappropriate support strategies due to weak semantic feedback, cross‑dimension reward compensation, and coarse reasoning supervision. To address these challenges, we propose FACES, a fine‑grained gated semantic alignment framework for MESC. Specifically, FACES integrates multi‑source rewards to construct multidimensional semantic and functional alignment signals, employs a non‑compensatory gated fusion mechanism to reduce cross‑dimension reward offset, and leverages structured Chain‑of‑Thought (CoT) anchors to perform token‑level credit assignment. Under a controlled response‑length protocol, FACES improves the average MLLM‑based evaluation score from 3.61 to 4.04 and achieves an 80.22% pairwise win rate over SOTA. Human evaluation under the same protocol yields a 75.3% preference rate. Results on three external datasets further support transferability. The code and dataset are available at https://github.com/OCT‑ARR/arr.
Authors:Tongwei Deng, Jinyu Zhu, Yue Cao, Chenxu Wang, Yanzhen Zou, Bing Xie
Abstract:
Linters are widely used in code maintenance and real‑time IDE editing to surface lint alerts related to performance, style, error‑prone, and other patterns in source files. However, existing code benchmarks do not explicitly evaluate linter‑guided file refactoring, where models must reduce lint alerts in source files with correct initial functionality while satisfying functionality preservation. We introduce LintEval, a multilingual benchmark for linter‑guided file refactoring across C++, Python, and Java, comprising 13,061 programs with correct initial functionality and 171,662 lint alerts extracted using language‑specific linters. LintEval jointly evaluates functionality preservation, lint alert reduction, new alert avoidance, and edit minimality. Experiments reveal a persistent trade‑off: closed‑source commercial models reach up to 90.4% Reduction@P but achieve only 67.1%‑68.3% Pass@1, while several open‑source small models show low or negative Reduction@P. A controlled fine‑tuning study further shows that supervision format matters: refactoring‑oriented SFT raises Qwen2.5‑Coder‑7B Reduction@P from 24.7% to 76.0%. LintEval enables controlled evaluation of models that improve source files under functionality preservation.
Authors:Maosheng Qin, Qi Zhang, Mingxuan Xia, Jinglei Chen, Jiexiang Wang, Sheng Guo, Cheng Peng, Haobo Wang
Abstract:
Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. Crucially, in harness‑based personal assistance, procedures or skills are inherently user‑conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness‑based benchmark for persistent personal assistance across daily‑life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single‑type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation‑related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual‑maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint‑state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location‑aware GRPO and dynamic residual‑routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. Our code is publicly available at https://github.com/QMMMS/PraxisMemBench.
Authors:Ziyan Liu, Jiting Liu, Weihong Zhong, Wenda Yu, Yeqiu Chen, Jason Li, Xiaojiang Peng, Shuyuan Hu, Kai Wang, Bo Zhao
Abstract:
Progress toward general intelligence in the physical world requires scaling robot learning across diverse environments and interactions. However, existing robot policies use shared parameters that remain fixed across interaction contexts. When different operations favor competing parameter adjustments, fitting their mixture can leave individual operations insufficiently adapted. Our atomic‑task diagnostics reveal competing update directions in both vision‑language and action parameters, with source‑improving updates increasing loss on other tasks. This motivates MorphAct, a contextual parameterization framework that dynamically adjusts robot policy parameters as interaction contexts change. Two coupled generators produce context‑dependent low‑rank updates for the vision‑language and action modules. Their coupling follows the policy's computational dependency: vision‑language updates shape the representations that condition action‑side parameter generation. Both generators are trained jointly through the native action objective, without operation labels or expert‑adapter targets. MorphAct improves performance across three VLA backbones on LIBERO and Meta‑World, with gains of up to 16.30%. In real‑world evaluation, it achieves gains of up to 14% over π_0.5. Ablations support the benefits of contextual parameter generation and representation coupling.
Authors:Timo Klein, Thomas Lang, Andrii Shkabrii, Lukas Miklautz, Yllka Velaj, Sebastian Tschiatschek
Abstract:
Scaling self‑supervised contrastive reinforcement learning (CRL) to 1000‑layer Euclidean neural networks has enabled state‑of‑the‑art goal‑reaching performance, yet wall‑clock time scales linearly with depth. We find that changing the geometry achieves stronger performance with far shallower networks. This matters in complex navigation environments, where walls can make route distance deviate substantially from Euclidean distance. In a U‑shaped maze, we show that the longer the route, the more it deviates from Euclidean distance. Hyperbolic CRL addresses this mismatch by replacing Euclidean distances in CRL's contrastive objective with geodesic distances in hyperbolic space. On hard navigation tasks, inductive bias beats scale: depth 64 hyperbolic agents outperform 1024‑layer‑deep Euclidean networks while training more than 12 × faster and using about 13 × fewer parameters. We observe that hyperbolic agents behave qualitatively better in hard humanoid mazes, where they follow corridors to reach goals, whereas Euclidean agents try straight‑line shortcuts over walls. Our results show that matching the representation geometry to the task can substitute for substantial network depth. Code is at https://github.com/Probabilistic‑and‑Interactive‑ML/HypGCRL .
Authors:Arjun Boya, Charan Sai Bandaru, Masthanaiah Pillimetla, Kavana Chidanandappa
Abstract:
The number of attention heads in a transformer is usually set by convention or grid search. We test a simple heuristic for setting it in advance: estimate the intrinsic dimension of token representations without labels, and use roughly that many heads. We estimate intrinsic dimension with a nested‑dropout VAE trained on DistilBERT embeddings of WikiText‑2, choosing the KL weight at the knee of the VAE's rate‑distortion curve rather than by hand. This gives k^=9.2 (95% CI [8.6, 9.8]). In a small 4‑layer transformer trained from scratch, 10 heads and 32 heads are statistically equivalent on SST‑2 and on MRPC (paired TOST over 10 seeds, \pm1 point margin), with 69% fewer attention parameters for the 10‑head model. Training 10 heads from the start also slightly outperforms training 32 heads and pruning to 10 (0.6 points, p=0.008). The evidence is narrow: two classification tasks, one small architecture, no pretrained models, and no formal link between intrinsic dimension and head count. We present the result as a practical heuristic worth testing at larger scale, not as a general principle.
Authors:Yuanfan Li, Junyi Chen, Qi Zhou, Rui Yang, ZhangLiujie, Weihang Chen
Abstract:
Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT‑5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5‑27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5‑27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object‑first choice in dots3‑note‑prev but not detectably in Qwen3.5‑27B. Adding a general "people or animals before property" rule increases subject choice by 19‑38 percentage points, whereas factual reminders favor the object by 0.7‑5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10‑15 points. We also examine whether post‑training introduces this pattern. Base models already show Protection Mode; post‑training changes justifications more than choices.
Authors:Amartya Banerjee, Xingyu Xu, Caroline Moosmüller, Harlin Lee
Abstract:
Energy‑based models can represent distributions over discrete structures such as molecular graphs, but an energy that is accurate at equilibrium need not yield accurate samples when the sampler stops after a fixed number of steps. We introduce finite‑horizon energy learning, which adapts the acceptance energy of a Metropolis‑‑Hastings sampler to the distribution it returns after a prescribed number of steps, with initialization, proposals, and annealing schedule held fixed. For graph generation, a frozen pretrained model supplies initialization and proposals, and a learned energy governs acceptance. A training‑only discriminator scores completed samples, and coupled continuations of the accepted and rejected branches of a proposal give an acceptance‑gradient estimator that is unbiased for a fixed discriminator under stated conditions, without equilibrated negative samples or differentiation through discrete states. Generation uses the learned energy in the original single‑trajectory sampler, without discriminator evaluations or reranking. On twelve pairwise binary targets with exactly evaluated terminal laws, learned energies have lower median terminal total variation than the equilibrium‑correct energy in every tested short‑horizon setting, and than its numerically tuned scalar multiple in most settings. On MOSES, a learned two‑coefficient combination of pretrained Graph Energy Matching (GEM) potentials lowers FCD by 3.7‑‑10.5% relative to scale‑calibrated GEM at matched sampling steps, with validity gains at the shortest budget and a sub‑point validity trade‑off at higher budgets.
Authors:Taha Sabir, Alberto Montero Molina, Thinh Quang Ly, Veronika Dmitrenko, Mohammad Yaqub, Tausifa Jan Saleem
Abstract:
Adapting a single vision‑language model to medical image parsing requires training choices that reflect the demands of each task. We study task‑specific adaptation with Qwen3.5‑9B for FLARE 2026 Task 3, covering six task families and seven imaging modalities. Our system couples a shared, frozen 4‑bit backbone with LoRA adapters selected from the input question. We customize adapter ranks and prompts, rebalance rare classification targets, and supplement regression training with dermatology classification examples. On the fixed internal holdout, the task‑specific adapters achieve a dataset‑macro balanced accuracy of 0.7791 for classification, a dataset‑macro micro‑F1 of 0.6286 for multi‑label classification, a counting mean absolute error (MAE) of 11.73, a dataset‑macro detection F1 of 0.8914, a dataset‑macro regression MAE of 9.78, and a report‑generation CRIMSON score of 0.8204. On the organizer's 1,703‑example hidden‑validation set, the submitted system achieves 0.8496 classification balanced accuracy, 0.5338 sample‑averaged multi‑label F1, and 14.5950 regression MAE. The final 30.1 GB container averages 3.1 seconds per case, including model loading, and produces identical outputs across repeated verification runs. The benefits vary across tasks, supporting the use of task‑specific adapter configurations and image budgets within a shared medical vision‑language model. Code is available at https://github.com/TahaSabir0/FLARE26‑Hexapus.
Authors:Ridong Zhang, Shuxian Liang, Chaoqun Wan, Zhen Huang, Xian-Sheng Hua
Abstract:
As GUI agents are increasingly deployed for mobile task automation, their reliability depends not only on final task outcomes, but also on whether they reach these outcomes through stable interaction trajectories. In automation workflows, unnecessary actions, detours, and execution errors can add noise to collected data, introduce irreversible side effects, or undermine trust, even when the agent eventually completes the task. It is therefore important to systematically diagnose the execution stability, including whether an agent deviates from a feasible path, enters a loop, becomes stagnant, or recovers from unstable actions. Existing benchmarks, however, primarily assess final task success or a small set of intermediate milestones, leaving step‑level execution behavior unclear. To bridge this gap, we introduce GraceBench, an online evaluation framework that represents the reachable interaction space of mobile applications as UI Transition Graphs (UTGs), providing a structural reference for step‑level execution diagnosis. To support reliable alignment between dynamic UI observations and graph states, GraceBench uses video‑grounded trajectory evaluation over this graph‑aligned representation, assessing process stability beyond final success with metrics such as deviation rate, loop burden rate, and recovery ability. GraceBench contains 169 tasks spanning 10 applications and constructs 10 UTGs with 2,909 canonical GUI states and 3,116 observed transitions. Across 3,718 trajectories, GraceBench shows that outcome scores hide execution instability: GPT‑5.6 Sol achieves the highest success rate (SR) but the lowest recovery ability, 51.24% of successful runs still contain a deviation or loop, and trajectories with the same success rate (SR) and completion rate (CR) can differ substantially in recovery behavior. Code and data will be released on https://iaie‑physicalai.github.io/GraceBench.
Authors:Fiorenzo Parascandolo, Enver Sangineto, Xiaoyi Yu, Wenhui Tan, Jipei He, Ruihua Song, Rita Cucchiara
Abstract:
Majority Voting (MV) is a simple and widely used strategy which improves the reasoning accuracy of Large Language Models (LLMs) by allocating additional inference‑time computation. Specifically, in MV multiple Chain‑of‑Thoughts (CoTs) are independently sampled and the most frequent final answer is selected. In this work, we propose Length‑Conditioned Majority Voting (LCMV), a Test‑Time Training approach which improves MV leveraging an empirical observation that has recently been established in several studies: longer CoTs tend to be associated with a lower probability of correctness. We formulate this inductive bias as a length‑conditioned prior probability, which is modeled by a simple logistic function. Moreover, we adapt this prior at test time using only unlabeled testing data and without fine‑tuning the LLM. Finally, the adapted prior is used to weight the vote of each sampled answer. Extensive experiments across multiple LLMs and reasoning benchmarks show that LCMV consistently improves over both standard MV and existing length‑based and confidence‑based aggregation strategies, while retaining the simplicity and model‑agnostic nature of MV. Our code is available at: https://github.com/aimagelab/LCMV.
Authors:Amartya Banerjee, Harlin Lee, Caroline Moosmüller
Abstract:
Population dynamics are often observed only as unpaired snapshots, which leave the evolution between measurements underdetermined. Second‑order regularization can constrain it, but a penalty on the accelerations of sampled particles is a prior on joint particle paths, and different particle representations of the same marginal curve can have different acceleration costs. We therefore introduce Neural Wasserstein Splines (NWSP), a sample‑based method that learns a continuous population curve under a Wasserstein smoothness prior. A neural network maps latent variables to the control points of a cubic spline in time, so samples at any time come from the spline basis without numerical integration. We project the generator's velocity onto gradients of a finite dictionary of scalar potentials and differentiate this measure‑dependent projection in time, obtaining a finite‑dimensional surrogate for the intrinsic Wasserstein acceleration, trained jointly with distributional fit losses. For a fixed dictionary and ridge parameter, we prove that the population penalty depends only on the marginal curve and bound its sensitivity to moment‑estimation errors. Learned potentials (NWSP‑N) sharpen acceleration estimates. On various synthetic and single‑cell benchmarks, NWSP is highly competitive with transport and flow‑based methods, ranking in the top two in 11 of 16 settings. The penalty improves on an unregularized spline in 13 of 16 settings and, while competitive with a particle‑acceleration penalty, adds provable representation invariance.
Authors:Qiuyu Chen, Qi Wang, Liang Xu, Xin Jin, Baao Xie, Wenjun Zeng
Abstract:
Mathematical research drives scientific discovery, and recent advances in AI for mathematics have shown that language‑model agents can solve challenging problems and develop substantial mathematical proofs. Yet much of today's AI‑for‑mathematics work centers on obtaining a final answer. This emphasis leaves the exploration behind a result difficult for researchers to understand, while long‑running investigations face fragmented context and the risk of unsupported claims propagating across reasoning steps. We present Euler, a human‑centered mathematical research system built around a shared, structured representation of the research process. Euler organizes mathematical statements, assumptions, proof dependencies, alternative approaches, failed attempts, and verification evidence into connected research graphs. This representation allows researchers to trace how results are developed and supplies agents with persistent, task‑relevant knowledge for planning, collaboration, and verification across sessions. Researchers can also select a graph node linked to a saved team state, revisit the agents' earlier reasoning, and explore alternative directions through continued interaction. We illustrate Euler's capabilities through three mathematical research case studies and describe the implemented system and its software validation.
Authors:Minghao Liu, Tutian Tang
Abstract:
Recovering 3D human motion from sparse wearable sensors is challenging due to the inherent noise, occlusion, and instability of real‑world tracking signals. Existing approaches are restricted to fixed sensor configurations and, when built on diffusion models, are computationally prohibitive for real‑time deployment. We propose OmniPoser, a universal framework that recovers full‑body SMPL motion from a flexible combination of heterogeneous inputs, such as 3D keypoints, sparse IMU rotations, head‑mounted 6‑DoF poses, or any subset thereof, within a single model. OmniPoser introduces a cross‑modal masking mechanism paired with a Masked Conditioning Encoder, enabling one architecture to incorporate diverse sensor configurations through a masked design. A decoupled dual‑stream generative backbone generates complete motion via Conditional Flow Matching, requiring only a single ODE integration step at inference. Across three settings and seven benchmarks, including in‑the‑wild Nymeria and Ego‑Exo4D datasets, OmniPoser performs on par with state‑of‑the‑art methods while running at over 700 FPS. The project page is available at https://jamesbridgewater51.github.io/omniposer‑eccv/.
Authors:Boyuan Sun, Deshui Miao, Wenzhao Gao, Shaoyong Jia, Shaohui Jiao, Chi Liu, Yan Shu, Derek Li, Bryan Dai, Shengsheng Qian, Qibin Hou
Abstract:
We propose DeltaStreamer, a novel agentic framework for streaming video understanding inspired by how humans observe, remember, and investigate. Streaming video understanding requires a system to retain evidence before knowing which details future queries will need. Existing approaches often obscure event transitions and omit relevant details when compressing video history, and cannot recover missing visual evidence. DeltaStreamer addresses these limitations by combining change‑aware memory with query‑driven planning and targeted evidence gathering with replay. Before a query arrives, it continuously observes the stream and explicitly records visual changes in structured memory, preserving how events unfold rather than only what is present. When a query arrives, a query‑driven planner identifies the required evidence and guides memory retrieval and targeted replay. By revisiting historical frames, the agent can recover visual details omitted during observation and ground its answer in evidence beyond the retained descriptions. This connects continuous observation with active investigation tailored to each question. Experiments on OVO‑Bench and StreamingBench demonstrate improvements over representative streaming video understanding methods, with overall scores of 66.07% and 59.53%, respectively. Project page: https://bbbbchan.github.io/DeltaStreamer/
Authors:Qisen Ma, Yixiang Chen, Yuan Xu, Peiyan Li, Jiabing Yang, Kai Wang, Jianhua Yang, Jianlou Si, Jun Huang, Jing Liu, Nianfeng Liu, Yan Huang, Liang Wang
Abstract:
Integrating world models into Vision‑Language‑Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynamics beyond reactive observation‑to‑action mapping. However, existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration. To disentangle these factors, we categorize existing efforts into four representative paradigms, namely Implicit Future Alignment, Explicit Future Query, MoT Future Expert, and External World Predictor. We instantiate each within a unified framework and systematically evaluate these paradigms with diverse world representations and policy backbones across in‑domain, out‑of‑domain, long‑horizon, dynamic, few‑shot, and real‑world settings. Our study yields four main findings: 1) world representation effectiveness is paradigm‑dependent, whereas 2D semantic representations offer robust, cross‑paradigm gains; 2) no single paradigm is universally optimal, yet the decoupled MoT Future Expert offers competitive overall performance, particularly in OOD robustness; 3) world modeling consistently benefits policies built on standard VLMs, while decoupled designs exhibit greater compatibility with pretrained VLAs; and 4) world modeling improves performance over action‑only baselines across multiple settings, with notable gains on long‑horizon and dynamic tasks. Project page and video demonstrations are available at https://dylanmqs.github.io/WM‑in‑VLAs/.
Authors:Siwei Meng, Xuerou Cai, Rui Hu, Ping Liu
Abstract:
Text‑to‑video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt and scene. Existing benchmarks largely rely on holistic plausibility scores or broad quality dimensions, leaving it unclear which physical behavior is being tested and why a model fails. We introduce PhyMetric, a scene‑level diagnostic benchmark that recasts physical plausibility evaluation as verifiable, scene‑grounded question answering. PhyMetric contains 1,115 scenes spanning 50 physical categories across six domains, paired with 8,920 answer‑balanced binary questions over 12 evaluation dimensions. Each scene is associated with eight targeted yes/no questions and a structured physical prior specifying the governing law, expected temporal sequence, observable signatures, and common failure modes. At evaluation time, a vision‑language model (VLM) judge answers each question from question‑aware sampled frames conditioned on the prior, and correctness is hierarchically aggregated into scene, dimension, physical category, and model scores. Experiments on four open‑source T2V models and three VLM judges show that scene‑specific binary QA is a critical component of the protocol: when each targeted question is replaced by a single generic plausibility query, evaluator accuracy drops to the chance level. Structured physical priors offer a complementary signal that improves human alignment of the resulting scores, with effects that vary across VLM judges of different capacity. These findings establish PhyMetric as a benchmark that turns physical plausibility evaluation from opaque holistic scoring into interpretable, scene‑level inspection. Our project page is available at: https://mengsiwei.github.io/PhyMetric/.
Authors:My Tra Nguyen Huynh, Viet Cuong Ta
Abstract:
Active domain adaptation researches focus on alleviating the annotation cost, which comprises source data and target data labelling, as well as enhancing the performance of unsupervised domain adaptation. However, optimizing the adaptation model based on the average accuracy can lead to the drop in performance of some difficult categories. To overcome this issue, a point‑wise selection strategy built upon the influence function can be used to improve domain adaptation performance. In our work, we first address the sensitive and redundant aspects of point‑wise selection, which can lead to overfitting of the acquisition function. We thus introduce a region‑based approach that ensures the improvement of the adaptation model by maximizing a positive manifold coverage. An uncertainty score is further introduced to the selection mechanism to learn a better decision boundary for the target domain. In comprehensive experiments, our proposed method surpasses the first category‑aware active domain adaptation baseline as well as existing active domain adaptation methods across multiple domain adaptation benchmarks. Our implementation is available at: https://github.com/nhtm2702/WHO IS NEXT.
Authors:Zhenyu Zhang, Zhichao Cao
Abstract:
An agent that drives an LLM sends the model the same conversation again and again, and each turn adds only a little text at the end. Inference engines reuse the KV cache of the shared prefix, so only the new tokens pass through the model. However, tokenization has no such shortcut. The CPU front end of the serving engine converts every request back into token IDs from its first character, which lengthens the time to first token (TTFT) and consumes CPU on every turn. We present Delta Tokenizer, a CPU tokenization front end that tokenizes only the delta between a request and the previous request of the same session. It retains the earlier token sequence, encodes again only the appended text and the short tail whose tokens the append could affect, and joins the two only at a position the tokenizer's splitting rules themselves cannot move, which keeps the result exact. For long requests with no state to reuse, it cuts the text where practical tokenizers already keep neighboring pieces independent, encodes the pieces on several cores at once, and concatenates the outputs into the reference encoding, exactly. A shared CPU pool schedules these chunks together with the short incremental updates, so an update runs between two chunks rather than behind an entire long job. The incremental path shortens the tokenization of a session turn by roughly 40x relative to the Hugging Face reference tokenizer, and on 16 cores the chunk‑parallel path encodes a four‑million‑character request 8.2x faster than a serial encode and 1.6‑2.1x faster than LoPT. Inside SGLang, Delta Tokenizer reduces the median TTFT through the chat interface by 59% at 100K tokens and brings the P99 TTFT of agent sessions down 9.5x under load and 23x when long requests tie up the front end. Every output we checked matches the reference, among them 41.8 billion documents encoded by its fast encoder.
Authors:Lorenzo Venturoli, Nedko Savov, Xi Wang, Marc Pollefeys, Luc Van Gool
Abstract:
Video world models aim to simulate complex environments with realistic appearance and dynamics. Maintaining consistency over long interactions requires tracking past actions and understanding how those actions affect the environment. Video world models can only generate partial observations of the entire world at a time. Remembering past observations alone may not be sufficient in every scenario, as they may no longer reflect the world after a sequence of actions. We introduce a paradigm that decouples a persistent, updatable abstract state from the visual state. The abstract state keeps track of the entities' changes as the world evolves, and is used to condition the visual state, which represents the current view. We propose StateCraft, a novel architecture that pairs a language model with a video diffusion model to evolve the dynamic world. We evaluate our approach on generative interfaces, which pose a substantial challenge because a single action may change parts of the state that are not currently visible. To support training and evaluation, we develop a data collection pipeline that produces informative, densely annotated trajectories at scale without human or agent supervision, together with a benchmark measuring generation quality, memory, and controllability. On this benchmark, our approach improves surpasses the state‑of‑the‑art baselines. We demonstrate that StateCraft maintains high self‑consistency and strong generation quality over long trajectories. Because the abstract state is explicit, it can be edited directly, and generation follows edits to states never observed in the data. Finally, a proof‑of‑concept 3D experiment, in which actions change the illumination of an unobserved room, shows that the paradigm can also be applied to spatial environments. Project page: https://insait‑institute.github.io/StateCraft/.
Authors:Zhaoxiang Feng, Lanxiang Hu, Haoran Yuan, Yu-Yang Qian, Jihui Yang, Peng Zhao, Yibo Zhu, Tajana Rosing, Hao Zhang
Abstract:
Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co‑optimized with algorithm‑system co‑design, and present JetSpec++, which retains a strong learned draft model while hiding its cost almost entirely. JetSpec++ combines three components: intra‑block self refinement, which trains the draft model to iteratively correct its own parallel predictions within a block of tokens; inter‑block foresight drafting, which begins the next block before the current one is verified; and pipelined drafting and verification, which overlaps the two stages so drafting's marginal wall‑clock cost approaches zero. In comparison with state‑of‑the‑art speculative decoding methods, JetSpec++ attains a wall‑clock speedup of up to 7.45× over autoregressive (AR) decoding and 1.54× over DSpark, with up to 1,361 tokens per second for a single request on GB200, and an average acceptance length of 8.94 across seven benchmarks, while preserving the target model's output distribution. Our code and models are available at https://github.com/hao‑ai‑lab/JetSpec.
Authors:Linhao Zhong, Bozhen Fang, Xuehao Wang, Duochao Shi, Zekai Luo, Guangming Yao, Yi Yuan, Hao Chen, Jingdong Chen, Chunhua Shen
Abstract:
Self‑distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseline scores under differing experimental conditions, provide few direct comparisons among contemporaneous methods, and give limited attention to training stability and computational cost. These limitations obscure the source of reported gains and the practical strengths and weaknesses of competing methods. Costly agent training further motivates a common basis for comparison. To address these gaps, we compare ten representative methods against GRPO on ALFWorld, ScienceWorld, Search‑QA, and WebShop across three model sizes. Our study comprises 132 training runs totaling 12,124.4 recorded H800 GPU‑hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. Beyond final task performance, we examine training progress, stability, and measured computational cost. Our evaluation yields three key findings. First, gains over GRPO remain limited in scope, with no method consistently outperforming it across settings. Second, strong final performance can mask unstable training. Third, additional training cost does not consistently translate into better performance. We establish this benchmark to support fair, comprehensive comparison and guide future research toward reliable and cost‑effective improvements in agent training. Project page: https://aim‑uofa.github.io/Self‑Distillation‑AgentRL/
Authors:Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang, Qifeng Li, Junjie Li, Zirui Wang, Yukang Feng, Yu Hong, Shaofeng Zhang, Wenlong Liao, Tao He, Juyong Zhang, Junchi Yan
Abstract:
Generating LiDAR point clouds from multi‑view camera images is a valuable task with applications in controllable data synthesis, cross‑modal simulation and various perception tasks. However, most existing methods are not specifically designed for this task, and the quality of their generated LiDAR remains limited. Moreover, these methods typically focus only on semantic cues in images or generate LiDAR from a monocular camera image. In this work, we introduce LiFi (LiDAR From images), a dedicated framework for generating visually and geometrically aligned LiDAR scenes from multi‑view camera images. LiFi guides the latent diffusion process through two complementary branches: geometry and semantics. The geometry branch constructs scene features from images through a reverse sampling algorithm and an uncertainty‑aware geometric encoder combined with DepthAnything3, whereas the semantic branch provides high‑level semantic representations via a cross‑view interaction module. We further introduce a dual‑branch balanced classifier‑free guidance (CFG) strategy, which enhances the model's conditional generation capability while preserving the independent learning and collaborative controllability of the two branches. Extensive experiments demonstrate that LiFi outperforms state‑of‑the‑art methods in generating high‑fidelity LiDAR scenes. The code will be publicly available at: https://github.com/szzhou88/LiFi.
Authors:Ianis Hammani, Claire Boyer
Abstract:
Score‑based generative models driven by α‑stable Lévy processes have been proposed as an alternative to Brownian diffusion models, with empirical evidence suggesting improved sampling and better modeling of heavy‑tailed data. Yet theoretical guarantees for these models remain limited. We develop a non‑asymptotic convergence analysis for scale‑preserving α‑stable generative dynamics, covering both compactly supported and power‑law targets. In p‑Wasserstein distance, we quantify the effects of early stopping, initialization, score approximation, and time discretization, and show exponential decay of the initialization error. A comparison with Brownian dynamics reveals that global Wasserstein convergence alone does not fundamentally distinguish the two models: both initialization errors decay exponentially with the forward horizon on the common class of heavy‑tailed targets. A sharp separation emerges instead in the tails. At any positive generation time, stable dynamics produces power‑law tails with index α, whereas Brownian dynamics remains light‑tailed. For a target with tail index ν, asymptotically correct extreme‑quantile scaling is possible precisely when α=ν. These results identify tail‑index matching as a principled mechanism behind the ability of stable generative models to capture extreme events.
Authors:Hanlin Liu, Zichen Zhou, Jiacheng Zhang, Liuyuan Na, Jin Jin, Guangyan Chen, Yufeng Yue, Gen Li, Fangqiang Ding
Abstract:
Computationally lightweight manipulation policies predict executable actions with low inference latency, but prediction errors can limit task success. Iterative generative policies can achieve higher task success rates, but repeated network evaluations increase inference latency. Refining action proposals from computationally lightweight policies can combine these strengths, but the amount of refinement needed varies across states. We introduce RAIN (Refine Actions If Needed), a framework that predicts action errors at different refinement levels to decide whether and how much to refine each executable proposal, before performing any refinement. We train an ensemble to predict these errors by comparing action proposals and their refined versions with expert actions from offline demonstrations, without requiring explicit labels of refinement needs. Calibration balances uncertainty, computation, and temporal consistency. We evaluate RAIN on eight ManiSkill tasks and four real‑robot tasks. In simulation, our policy achieves 66.6% success, approaching the best evaluated diffusion‑policy success rate with 5.71 times lower policy latency. On the real robot, success increases from 55.0% with Action Chunking with Transformers (ACT) to 72.5%, with comparable average successful completion time. Matched controls show that dynamic routing maintains the strongest observed fixed‑refinement success level with 26.6% fewer refiner evaluations. Project page: https://rain‑policy.github.io/.
Authors:Julia Barth, Armin Sommer, Kunvar Thaman, Jannik Schilling
Abstract:
Goal misgeneralization is a common failure mode in which an agent behaves competently out of distribution but does not pursue the intended goal. However, we still have a weak understanding of what happens to the intended goal in misgeneralizing agents. Across three Procgen environments and five robot tasks, we find three distinct forms of policy‑goal misalignment: (1) functional target substitution, where behavior follows an alternative goal, (2) a policy shortcut, where behavior follows a learned action strategy, and (3) cue competition between intended and proxy goal cues. We distinguish these regimes by tracing where goal information remains available internally, how strongly it influences the policy, and what role it plays in action selection, and find that the goal is lost at different depths. Where the goal reaches the decision, as in cue competition, steering redirects the policy to the goal's location, raising intended‑goal reach from 16.0% to 95.3%. Where it reaches the policy but not the decision, only new downstream wiring recovers goal reach, from 31.6% to 84.3% in the environment with the action shortcut. Where it is inert, neither works. Our results show that goal misgeneralization is not a single failure of goal representation, but distinct failures of policy alignment at different depths, tightly connected to which recovery works.
Authors:Hao-Jen Chien, Wei-Lun Chao, Yu-Lun Liu
Abstract:
Monocular world‑space human recovery methods excel on current benchmarks, but these rarely include fast camera motion. We introduce the K‑Pop stress test, broadcast dance videos that combine fast, large camera motion with dense multi‑person choreography. Leading methods produce plausible per‑frame meshes on these videos, yet their world‑space trajectories drift far from the performers' stage positions. Through a systematic diagnosis, we show that the camera and human estimates each remain self‑consistent, and that the drift appears when they are combined at mismatched scales. Each is recovered at its own scale, so a mismatch between the two can turn camera motion into human drift that grows with camera displacement and becomes severe in broadcast dance videos. We derive a closed‑form correction that estimates this mismatch and applies one scale to all performers, removing the drift while keeping their relative positions in one shared world frame. It requires no retraining and works with existing backends. Combined with physical constraints, the correction yields stable world‑space reconstruction for choreography modeling, scene editing, data generation, and virtual reshooting. Project page: https://chien90190.github.io/worldhmr‑fastcam‑website/
Authors:Haoxin Liu, Deqing Fu, Lucheng Fu, Yiqiao Jin, Rushi Qiang, Harshavardhan Kamarthi, B. Aditya Prakash
Abstract:
Time‑series foundation models provide strong zero‑shot forecast, but deployment for real‑world applications still requires task‑specific designs for model inputs and outputs. Direct inference can leave domain knowledge and recurring forecast errors unaddressed, while tailored pipelines require substantial human expertise and effort. We introduce TimeRSI, a self‑improving agentic deployment system around frozen forecasting models. A coding agent translates task descriptions into executable pipeline changes. It tests each revision on historical forecasting windows and uses observed errors to guide subsequent revisions. During deployment, an adaptive selector chooses among the resulting pipelines using completed forecasts under similar conditions. Across 198 tasks on TIME and fev‑bench, TimeRSI outperforms all submissions on six point and probabilistic forecasting metrics. Controlled comparisons support the value of feedback‑guided revision over non‑iterative pipeline search. Further experiments verify the generalization with different coding agents and forecasting models. Code is available at https://github.com/haoxin1998/TimeRSI.
Authors:Shun Inadumi, Lee Sangmyeong, Koichiro Yoshino
Abstract:
Interpreting an ambiguous utterance in real‑world interactions often requires context beyond a single utterance. When the utterance refers to objects in a visual scene, it is necessary to maintain and update the set of plausible referents as context accumulates, without assuming that the available information identifies a unique object. However, prior work on interactive visual grounding, which localizes objects referred to by linguistic expressions, has focused on identifying a unique object once sufficient dialogue context becomes available. To address this gap, we propose Candidate Set Visual Grounding (CSVG), a task for estimating the set of objects consistent with the given dialogue context. We further introduce CandVG, a benchmark that evaluates how the candidate set changes across dialogue turns. Our evaluation of vision‑language models shows that stronger visual grounding capability does not necessarily yield better CSVG performance. In particular, models have difficulty predicting the complete candidate set, owing to challenges in candidate generation and premature commitment to a small set of candidates in early turns. Nevertheless, we find that this limitation can be partially mitigated in the evaluated large‑scale models using instructions designed for CSVG.
Authors:Lizhi Xiong, Jun Li, Ziqiang Li, Weiwei Jiang, Zhangjie Fu
Abstract:
Recent advances in diffusion models have enabled high‑fidelity, identity‑preserving image generation for personalized applications such as digital avatars and virtual try‑on systems. However, their reliance on sensitive facial reference images raises growing privacy concerns. Existing defense mechanisms are primarily designed for training‑based personalization and struggle to generalize to emerging training‑free approaches, due to fundamental differences in their identity integration paradigms. To bridge this gap, we propose IDGuardian‑the first generalizable and model‑agnostic identity protection framework capable of defending against both training‑based and training‑free personalization methods. IDGuardian abstracts the personalization process into two critical stages: identity extraction and identity injection. It then introduces crafted adversarial perturbations to simultaneously disrupt both stages. Specifically, it degrades the identity features extracted by external encoders and establishes an adversarial conceptual bridge that misdirects the generative trajectory away from the target identity. Extensive experiments show that IDGuardian effectively protects identity across various personalization pipelines and model architectures, while remaining robust to post‑processing, adaptive attacks, and cross‑dataset generalization. Our code is available at https://github.com/OpenAscent‑L/IDGuardian.git.
Authors:Md Rayhanul Masud, S M Maksudul Alam, Mohammad Jaminur Islam
Abstract:
Large language models (LLMs) increasingly help users arrange appointments, plan journeys, and allocate limited resources. These tasks require models to satisfy constraints on time, cost, capacity, and order, or determine that no feasible solution exists. Bangla is the world's seventh most widely spoken language, and existing benchmarks cover many language and reasoning tasks. However, benchmarks for evaluating planning and decision‑making in Bangla under explicit constraints remain lacking. We introduce BANGLAPLAN, a benchmark of 1,000 problems across ten task families grounded in everyday life and institutions in Bangladesh. It includes feasible and infeasible problems in four language registers: everyday, formal, colloquial, and Bangla‑‑English mixed. Answers cover eight structured types, including schedules, routes, allocations, and ordered steps, in a common JSON format. We construct the benchmark through language‑model‑assisted drafting and independent human validation. We evaluate 11 open‑weight and proprietary model configurations using exact match, separately reporting feasible‑case accuracy, infeasibility detection, and format errors. We find a large performance gap: GPT‑6.1 Sol achieves 98.7% accuracy and Qwen3.5‑27B with thinking reaches 80.3%, while direct‑answer configurations score at most 30.9%. Seven of eleven configurations perform below the always‑``infeasible'' baseline. Our findings also suggest that harder problems primarily reduce plan‑construction accuracy, while infeasibility detection remains more robust. BANGLAPLAN provides a reproducible basis for evaluating planning and decision‑making in Bangla. BANGLAPLAN is available at: https://github.com/mrayhanulmasud/banglaplan.
Authors:Fanyao Li, Yuxi Liu, Xiaohao Liu, He Chang, Zhaoxin Wang, Zhulin Tao
Abstract:
Low‑rank adaptation (LoRA) enables parameter‑efficient fine‑tuning by learning low‑rank weight updates. Increasing rank expands adaptation capacity, but does not guarantee that all learned directions improve downstream performance. In a diagnostic experiment with rank‑16 LoRA, we show that retaining only the top singular components achieves the best performance, whereas adding further components degrades it. Motivated by this, we propose Residual‑Gradient Adaptation (ReGRA), a two‑stage method for making more effective use of a fixed rank budget. After the first stage, ReGRA estimates task gradients on a calibration set and removes their projections onto the two‑sided subspace induced by the first‑stage update. It then aggregates the residual‑gradient directions to initialize a second‑stage adapter without changing the first‑stage update. An orthogonality regularizer encourages the two updates to capture complementary directions. Both updates can be merged into the pretrained weights, incurring no additional inference overhead. Across diverse benchmark sets, ReGRA achieves stronger performance than alternatives with the same total rank and even matches LoRA adapters with twice the rank. Ablation studies and further analysis support the contribution of its core design choices. Code is available at https://github.com/Fanyao‑Li/ReGRA.
Authors:Chenhao Yuan, Bohan Yu, Yinhao Xu, Chao Yin, Chenxi Zhou, Zhaoxuan Wang, Shuwen Xu, Shaoping Huang, Haolin Ren, Jiaxiang Liu, Yao Xu, Jun Zhao, Kang Liu
Abstract:
Large language models (LLMs) increasingly tackle complex real‑world tasks in specialized domains. However, identifying and correctly applying domain‑specific rules remains challenging when scenario information is distributed across multiple files. Existing rule‑reasoning benchmarks typically represent scenarios using only a few factual statements, simplifying the identification and application of relevant rules. To address this gap, this paper introduces RuleTrek, a multi‑domain benchmark for agentic rule reasoning in complex scenarios. Its rule‑grounded construction pipeline extracts structured rule graphs, generates and incrementally composes rule‑specific sub‑scenarios while ensuring consistency and solvability. To systematically evaluate agentic rule reasoning in complex scenarios, RuleTrek assesses three capabilities in interactive workspaces: Rule Identification (RI), Rule Deduction (RD), and End‑to‑End Rule Reasoning (E2E). Experiments on 11 frontier LLMs show that the highest RI and RD scores are only 62.47% and 60.43%, respectively, while E2E scores range from 26.97% to 55.29%. These results demonstrate that current LLM agents face substantial challenges in rule reasoning over complex scenarios. We make our code and dataset publicly available at \hrefhttps://github.com/yuan247/RuleTrekRuleTrek.
Authors:Xin Li, Mengbing Liu, Hao Jiang, Xin Gao, Annan Wang, Xingwei Qu, Yichi Zhang, Chau Yuen
Abstract:
Multi‑agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate through text, while recent approaches also explore exchanging internal model states. However, sharing these states across architectures often requires learned translators or assumptions about model compatibility. We introduce SYNC (Symbol‑Indexed Neural Communication), a word‑indexed interface that connects heterogeneous models without fine‑tuning or a learned translator. From descriptions only it can see, the sender selects words from a shared list and sends their indices for each object, without seeing the questions. The receiver writes the described facts into its hidden states using its own word directions, vectors fitted once from its activations. We test selective use by swapping one attribute between two objects and requiring correct answers about both changed and unchanged attributes before and after the swap, four answers in total. We choose an early write layer on four development receivers and then fix it. With three different senders and nine newer receivers from five families, four‑answer success on new scenes reaches 72.0‑98.3%, up to 96 percentage points above middle‑layer writing, and the two largest receivers match plain text. A task‑free probe predicted in advance which depths each new receiver can use (45 of 45 cases). Without retuning, messages listing every fact keep 89.6‑100% success under new layouts, and sender messages reach 23.9‑64.0% on a landmark task with multi‑token symbols, below text but from 0% without a message. These results show that shared word indices and receiver‑local directions can support selective fact use across model families without additional training.
Authors:Ziyang Zhang, Wenyi Yu, Wen Wu, Xuenan Xu, Chao Zhang
Abstract:
Large audio‑language models (LALMs) struggle to parse paralinguistics (e.g., emotion, pace, pitch) in speech that combines intense emotion with complex acoustic environments. Strengthening this ability calls for training data that reflects such conditions. However, existing corpora rely heavily on studio‑recorded speech, offering limited emotional and acoustic diversity. To address this gap, we introduce TVPara, a paralinguistic captioning dataset built from TV series. Speech in TV series often carries diverse emotional expression and co‑occurs with scene‑specific background sound, providing expressive emotion and complex acoustic environments at scale. Constructed via LALM pre‑annotation and human correction, TVPara includes two tasks: Single‑Utterance (~492h) and Multi‑Utterance (~213h) Paralinguistic Captioning. Fine‑tuning open‑source LALMs on TVPara yields an average relative improvement of 31.26% in paralinguistic understanding under complex acoustic environments. The gains further generalize to an unseen TV series and to cleaner, cross‑domain speech.
Authors:Bin Zheng, Zheng Chen
Abstract:
Typed decision models need an interface that can use additional reasoning while retaining candidate outputs. We train a shared candidate‑answer reader on a frozen Qwen3.5‑4B backbone to unify direct decisions with on‑demand, interruptible reasoning. The reader returns distributions from empty and partial reasoning prefixes. A benefit gate starts reasoning, a natural‑state trigger requests explicit reads, and an entropy rule accepts acquired answers and cancels generation. Measurements distinguish answer‑query work, generated output, and pipeline time. Across 768 questions from ProofWriter, LogiQA 2.0, and ARC‑Challenge, accuracy rises from 70.96% for direct reading to 84.38%. Relative to 512‑token terminal reading, accuracy improves by 7.29 percentage points with 2.25% more mean output work. Relative to 1,024‑token terminal reading, pipeline time falls by 46.30% with an accuracy difference of ‑1.17 points. Exact‑prefix controls isolate the saving from interruption, while read‑scheduling controls distinguish reduced observation work from shorter response time.
Authors:Dikai Li, Weijia Wang, Yuan-Gen Wang
Abstract:
Visual recognition models usually process a fixed view under a fixed computation budget, although images often benefit from different scales and amounts of spatial evidence. Active vision can adapt by observing a coarse scene and then selecting informative fixations, but the public AdaptiveNN‑DeiT‑S pipeline still fixes both field of view and observation count. We extend this fixed‑step recognizer with a controllable observation interface that adapts both the field of view and the number of fixations to the input. Specifically, the model predicts a per‑fixation scale that sets its field of view, refines each selected crop with optional scale‑gated foveation and a residual high‑resolution detail‑token branch, and uses label‑grounded stop heads for input‑dependent early termination. Experiments across fine‑grained benchmarks, full ImageNet‑1K, and ImageNet‑A/R‑derived classification splits show that the interface remains competitive with fixed‑step references while exposing a controllable observation budget. The same checkpoint enables threshold‑controlled accuracy‑compute trade‑offs without per‑threshold retraining. On sequential ImageNet‑1K, the learned stop rule outperforms fixed‑step truncation of the same checkpoint, sustaining 81.9% top‑1 using 2.45 of the four focus steps and exceeding three‑step truncation at lower cost. On each benchmark, a variant combining scale control and detail‑token fusion achieves the highest accuracy among the tested component‑added non‑exit variants. Our code is available at https://github.com/GZHU‑DVL/LI‑Dikai.
Authors:Jia Chen, Ruyang Zhao, Jun Bai, Chao Huang, Saishang Zhong, Xinrong Hu
Abstract:
Single‑image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin‑shell geometry, open boundaries, and layered structures. Existing datasets and evaluation protocols provide limited support for systematic evaluation in this setting. We introduce MagicGarment, a benchmark for single‑image standalone 3D garment reconstruction with 1,518 professionally constructed assets across 17 composite garment types, aligned multimodal annotations, and a unified evaluation protocol. We further establish a garment‑aware reference baseline by adapting a pretrained image‑to‑3D backbone at two levels: Grounded Semantic Refinement (GSR) provides garment‑focused visual conditioning, while Flow‑Time‑Route LoRA (FTR‑LoRA) adapts the frozen 3D generative backbone using flow‑timestep, token‑routing, and expert‑load information. Experiments demonstrate MagicGarment's benchmark utility and the effectiveness of garment‑aware adaptation. Project page: https://1019538157.github.io/MagicGarment/.
Authors:Xuan Han, Zinuo Jin, Tianlin Li, Yunfan Lou, Xin Liu, Yuchen Xue, Yuanpeng He, Boxian Yao, Shuhan Zhan, Ruihai Wu, Zhibo Pang
Abstract:
Force feedback provides vision‑language‑action (VLA) policies with physical observations that can guide action selection and execution. However, differences in tasks, demonstrations, and control settings across existing systems hinder reliable comparisons of force integration designs. We introduce ForceBench, a simulation benchmark for evaluating six‑axis wrist‑force integration under a unified protocol. ForceBench combines eight tasks spanning physical‑property inference, resistance‑dependent interaction, and contact‑rich assembly with 1,600 synchronized demonstrations and task‑specific success criteria. We compare one no‑force baseline and six force integration designs on a shared π_0.5 backbone. Prompt Fusion achieves the highest observed mean simulation success of 35.31%, compared with 23.25% for the baseline, and improves success rates across all three task categories. Predictive and temporal gains in simulation concentrate on physical‑property inference, while both temporal designs underperform the baseline on assembly. Real‑world evaluation of eight systems across five tasks supports the main qualitative trends: Prompt Fusion and Dual‑Path Fusion achieve 63.00% mean success, compared with 40.00% for the baseline, but only Prompt Fusion improves every task. These findings support ForceBench's practical value for identifying the benefits and limitations of force integration across manipulation demands. The project page is available at https://forcebench‑platform.github.io/.
Authors:Zixuan Wang, Jia-Ning Li, Yinzhou Tang, Yanzhao Yu, Biao Wu, Meng Fang, Xin Li, Yiwu Zhong
Abstract:
Recent vision‑language‑action (VLA) models have shown strong performance on local robot manipulation, yet their success does not readily translate to long‑horizon mobile manipulation. In our experiments, a pretrained subtask VLA achieves 80% success rate on atomic manipulation tasks but only 19% when these skills are composed into mobile manipulation. We argue that this gap stems from failures across the execution process rather than from local manipulation capability alone. To uncover these bottlenecks, we systematically diagnose failures in long‑horizon mobile manipulation and identify three dominant regimes: task planning, navigation and base positioning, and local manipulation. Based on this diagnosis, we develop failure‑guided agentic intervention that augments a fixed subtask VLA with demonstration‑based planning guidance, adaptive visual memory, accessibility‑aware base adjustment, and local end‑effector adjustment. On RoboCasa365, these interventions progressively improve task success from 19% to 47%, with the finalized system reaching 49% on held‑out episode seeds, compared with 18% for the baseline. The same design improves performance from 14% to 37% on BEHAVIOR‑1K and from 5/20 to 13/20 on real‑robot tasks. These results suggest that reliable mobile manipulation can be advanced by diagnosing and intervening on the current bottleneck, rather than uniformly scaling the underlying policy. Our code will be released at https://github.com/PKU‑VaLuE‑Lab/Cracking‑Mobile‑Manipulation.git.
Authors:Jing-Xuan Zhang, Genshun Wan, Jianqing Gao, Jia Pan, Yumei Zhang, Lichen Zhang
Abstract:
Lipreading remains challenging due to the inherent ambiguity of visual speech cues and their sensitivity to head pose and video resolution. In this work, we propose FLARE, a framework for robust visual speech representation learning via cross‑modal distillation and joint face‑lip modeling. FLARE leverages semantically rich Whisper representations as the acoustic teacher signal, converting them into discrete distillation targets through a random‑projection quantizer and pretraining the model with a masked prediction objective. To capture complementary visual information, FLARE employs a dual‑stream architecture that jointly processes lip region‑of‑interest (ROI) and full‑face inputs, where the lip stream captures finegrained articulatory motion and the face stream encodes broader facial dynamics. During pretraining, auxiliary audio input is incorporated to facilitate cross‑modal alignment, and modality dropout is applied to encourage robust visual‑only representations. For downstream lipreading, FLARE serves as a visual encoder coupled with either a lightweight Transformer decoder or an LLM‑based decoder. Evaluated on LRS2, LRS3, and the out‑of‑domain WildVSR benchmark, FLARE achieves strong lipreading performance, with word error rates of 11.7%, 13.2%, and 34.3%, respectively. A progressive design study confirms the contribution of each major component. Further analyses demonstrate that joint face‑lip modeling improves robustness under degraded video resolution and large head pose variation, while yielding more phoneme‑discriminative visual representations. Code and pretrained models are available at https://github.com/jxzhanggg/FLARE.
Authors:Dikai Li, Weijia Wang, Yuan-Gen Wang
Abstract:
Automated defect detection of industrial boards must remain highly reliable under cross‑scenario variations in lighting, surface contaminants, and production line conditions. This makes cross‑scenario defect detection a domain generalization problem, wherein models are required to detect, localize, and classify defects across unseen scenarios while minimizing false alarms on clean boards. For the IDA 2026 Track 1 challenge, we propose a cross‑scenario defect detection method that includes four complementary modules: cross‑scenario robust training, small‑defect‑aware learning, score calibration and inference threshold selection, and metric‑aligned checkpoint selection. These four modules respectively introduce cross‑scenario robust normalization and DSU‑style statistic perturbation with progressive random convolution, NWD and parameter‑free spatial attention, the VFL learning objective, and a custom SWAD variant driven by the Track 1‑oriented proxy. Experimental results demonstrate that our model achieves a Track 1 score of 0.564 with a screening score of 0.696 on the test dataset on a single RTX 4090 GPU. The code is available at https://github.com/GZHU‑DVL/Dikai.
Authors:Rui Chen, Xingyu Chen, Pengxin Xu, Kai Chen, Junzhi Yu
Abstract:
Temporal action detection in long untrimmed videos suffers from inaccurate localization and low‑quality proposals, especially with sparse annotations and ambiguous boundaries. For open‑ended video understanding and LLM‑driven event analysis, reliable proposals connect temporal localization, event annotation, and semantic understanding. However, conventional temporal action detection (TAD) pipelines lack explicit modeling of proposal localization quality, cross‑representation support, and inter‑proposal relations. We propose Hybrid Rank‑Calibration of Action Proposals (HyrCap), a proposal‑centric query‑based TAD framework for temporal event understanding. Hybrid combines complementary video features with different localization cues, while Rank‑Calibration adjusts proposal ranking confidence to reflect localization reliability and candidate competition. HyrCap treats dense query‑based predictions as an over‑complete temporal event hypothesis space. Cross‑Representation Consensus Calibration, Localization Quality Estimation, and Proposal Redundancy Suppression improve proposal ranking reliability, localization quality, and candidate discrimination. Inspired by LLMs' capabilities in fine‑grained semantic recognition, event description, and reasoning, we introduce HyrCap for Semantic Action Understanding, using calibrated proposals for standard TAD and as inputs to LLM‑based semantic understanding. We define a staged evaluation protocol to separately measure event localization, segment‑level semantic understanding, and combined event understanding performance. Experiments on public benchmarks show that HyrCap achieves state‑of‑the‑art performance across multiple metrics and provides a finer‑grained paradigm and evaluation protocol for query‑based event segmentation and annotation.
Authors:Chia-Ming Lee, Yu-Jou Hsiao, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
Abstract:
Cameras and the foundation models built on them recognize what a 3D scene contains, but not what it is made of: surfaces of the same colour can be different materials. Hyperspectral unmixing explains each spectrum as a mixture of material signatures, yet in 3D it is applied to rendered images after the views are composited, or tied to slow implicit fields, and no multi‑view data with known composition exist to check the result. We present the Spectral World Renderer (SWR), which unifies world generation and unmixing. A coding agent builds executable 3D worlds that are rendered with measured spectra, so every hyperspectral image comes with exact labels and sub‑pixel proportions; the resulting SWR‑Bench provides eight indoor and outdoor worlds in which decompositions can be verified. SWR then unmixes inside rendering: each Gaussian of a compact 3D representation is decoded into material abundances before compositing, so one render yields spectra and material maps, and any surface can be queried by material name, a clicked pixel or a reference spectrum. Without material labels, SWR identifies materials best in all eight worlds, has the best mean PSNR on real captures, trains 3.7‑12.9x faster than unmixing baselines, renders 34.9 labelled views/s and separates same‑coloured materials.
Authors:Linyi Huang, Guowei Huai, Weibin Liu, Shulong Jiang, Ping Tan, Weixuan Zhang, Hui Zhang, Jie Song
Abstract:
While multi‑fingered dexterous robot hands have the potential to provide human‑level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm‑hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real‑world learning. However, real‑world demonstrations in the full arm‑hand control space are costly and noisy because each trajectory must coordinate arm motion with high‑DoF multi‑finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off‑nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real‑world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm‑hand coordination policy for arm control and hand‑skill activation. This design enables effective real‑time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real‑world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. The project page is available at https://hly‑123.github.io/HiFun/.
Authors:Wei-Liang Liu, Cheng-Yen Tsai
Abstract:
Model signing authenticates the full‑precision weights W an LLM is released with, yet users run a quantized artifact Q whose producing transformation is not uniquely determined under current disclosure. We separate the non‑environment inputs from the execution environment E and measure a quantity that, to our knowledge, has not been measured before: the Independent Verifiability Rate (IVR), the fraction of a popularity‑sampled GGUF ecosystem that a third party can re‑derive bit‑for‑bit. Across the 200 most‑downloaded Hugging Face GGUF repositories (198 in scope), no artifact or model card records the build environment, so under a recorded‑inputs protocol the IVR is 0/198. Yet the artifacts are not irreproducible. On four in‑sample repositories that disclose their inputs, the card's information alone reproduces no Q4_K_M file, but two coordinates the card omits ‑ the build's floating‑point‑contraction setting and, for hybrid models, the per‑tensor quantization recipe ‑ re‑derive 7/8 target‑quant files bit‑for‑bit on x86 (5/8 on arm64); the fp16 re‑derives from the provider's checkpoint, but the calibration imatrix must be shipped and hash‑bound, since recomputing it did not match. An older release additionally pins the instruction set and did not re‑derive. Reproducing the divergence reported with llama.cpp PR #25353, we show that E is a necessary coordinate and that one compiler setting yields a unique output across the tested platforms at negligible quality cost (+0.06% perplexity). As a security characterization, benign floating‑point non‑determinism separates from uniformly placed nibble‑flip tampering above a detection floor that grows with the environments a verifier must allow, but a targeted embedding edit evades the statistic, so the proposed protocol rests on exact re‑derivation; adversarial‑fp16 attacks lie outside it. We release the audit data and a re‑runnable harness.
Authors:Yunze Liu, Jing Wang, Shenrui Wu, Wenkun He, Guanqi Zhan, Xianzheng Ma, Junyu Chen, Wenhang Ge, Haozhan Tang, Yuchao Gu, Zhuoyang Zhang, Zekai Wang, Dantong Niu, Zhen Dong, Yuke Zhu, Trevor Darrell, Linxi "Jim" Fan, Song Han, Han Cai
Abstract:
Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota‑Embed, to our knowledge the first vision‑language‑action (VLA) embedding model to jointly encode all three modalities in both current‑state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi‑source corpus of 1,695,684 cleaned training anchors and introduce Robota‑Embed‑Bench, with 4,000 queries and a shared 8,000‑entry gallery. Robota‑Embed achieves 47.33% clip‑level and 81.77% episode‑level R@1, exceeding action‑free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training‑free and training‑based policy conditioning, increasing success from 47.0% to 56.3% in DROID‑Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task‑local memory and refine context use around a frozen GPT‑6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT‑6 Astra Direct and 33% for a harness independently evolved with Omni‑Embed‑Nemotron‑3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action‑aware retrieval space can support robot in‑context learning and task‑local context evolution.
Authors:Yanshu Li, Ruixiang Tang, Junyi Jessy Li, Elias Stengel-Eskin
Abstract:
Multimodal in‑context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context‑aware retrieval methods have emerged to address this problem by allowing incremental retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive retrieval on its own. To this end, we introduce ActICL, which trains an MLLM agent to acquire and use an ICL context via tool‑augmented reasoning. An ActICL agent analyzes its own information gaps during iterative reasoning and tool‑use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two‑stage post‑training procedure that jointly optimizes the agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity‑based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT‑5.6‑Sol performs 4‑shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.
Authors:Baotong Tian, Zhiyao Duan
Abstract:
Neural instrument synthesizers can clone the timbre of a solo instrument, letting music producers make music with the sound they desire. However, an instrument of interest is often embedded in a fully produced, multi‑instrument recording, beyond the reach of existing methods. Source separation before timbre cloning is an approach, but it may suffer from separation artifacts and is also likely a detour for the task. In this work, we propose Point‑and‑Play, a system that clones the timbre of a cued source directly from a polyphonic mixture and uses it to synthesize new pitches and velocities. To achieve this, we introduce a Cue‑Gated Statistics Pooling (CGSP) module that extracts the timbre code of the cued instrument from adapted CLAP embeddings of the music mixture. This extracted timbre code is then used to condition a separately trained note synthesizer together with pitch and velocity inputs. Experiments show that our proposed system outperforms separate‑then‑clone baselines in terms of timbre fidelity and synthesis quality, achieving comparable performance with that cloned from a clean solo instrument.
Authors:Jingjing Liu, Pei Tai, Xianchao Xiu, Wanquan Liu
Abstract:
Unsupervised feature selection (UFS) is of great significance in image processing and pattern recognition. Nevertheless, existing model‑based methods often rely on manually tuned hyperparameters, while deep learning‑based methods generally require massive training data. To address these issues, this paper proposes an efficient manifold deep unfolding feature selection network (MDUFS‑Net). Technically, we first incorporate the \ell_2,0‑norm into the classic principal component analysis (PCA) framework, then unfold the alternating direction method of multipliers (ADMM)‑based iterative algorithm into an end‑to‑end trainable network, thereby enabling automatic parameter learning. Moreover, a manifold optimization strategy is developed to improve the numerical robustness. On this basis, we further extend MDUFS‑Net into an enhanced variant called MDUFS‑Net+ by introducing the \ell_0‑norm for more discriminative feature representations. Extensive experiments validate that our proposed MDUFS‑Net and MDUFS‑Net+ outperform state‑of‑the‑art methods in the UFS task, with promising performance in clustering. Our code is available at https://github.com/xianchaoxiu/MDUFS.
Authors:Yue Zhang, Han Lin, Zun Wang, Yidong Huang, Zaid Khan, Mohit Bansal
Abstract:
Simulated environments enable scalable training data generation for embodied agents, but more data does not necessarily provide what a particular agent needs to learn. Existing environment‑generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons. We therefore introduce WEDGE, a framework for weakness‑guided navigation learning in which the failure modes of a pretrained agent guide the generation of training environments at each adaptation round. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success‑rate gap. The validated weaknesses are then mapped to the environmental factors that control them. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness‑conditioned adaptation consistently outperforms matched random and hard‑example alternatives on ProcTHOR, while the resulting gains for Uni‑NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re‑diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.
Authors:Hongrui Sheng, Shutian Liang, Reid A. Coyle, Shyam Chand Pal, Peter Walther, Bin Feng, Xinyu Ding, Zhiling Zheng
Abstract:
Reticular chemistry allows metal‑organic frameworks (MOFs) to be designed from their building units, yet their synthesis remains constrained by sparse experimental data, noisy crystallization outcomes, and the absence of benchmarks for comparing active‑learning strategies. While data‑driven approaches such as Bayesian optimization (BO) can improve synthesis success rates, selecting appropriate optimizers beforehand for mapping MOF crystallization landscapes remains largely empirical. Here we present ActMOF, an experiment‑grounded benchmark for comparing search strategies in aluminum‑carboxylate MOF synthesis. ActMOF is emulated from experimentally measured microwave syntheses with each encoded condition assigned a crystallinity index calculated based on diffraction peak intensity and full width at half‑maximum. By benchmarking policies guided by Gaussian processes (GP), tree ensembles and large language models (LLM), ActMOF placed the GP family among the leading policies for active learning in sparse synthesis landscapes. It then demonstrated transferability by guiding the microwave synthesis of six reticularly related aluminum MOFs. All six campaigns gave crystalline products within a fixed budget of 12 experiments each, and established a reusable testbed for AI‑driven MOF synthesis.
Authors:Zinan Ling, Yang Xiao, Ruimeng Ye, Sijia Liu, Bo Hui
Abstract:
Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failure mode: a model may perform well on standard retain prompts, yet degrade on benign reasoning when prompts lie near the forgotten domain. This phenomenon is termed forget‑adjacent utility loss. A fundamental question is: where should forget representations go after they leave the removed behavior? Successful unlearning depends not only on whether forget‑conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward. We propose RA^2 (Retain‑Anchored Attraction), a latent‑space unlearning method that redirects forget‑token states toward nearby retain‑supported representations while preserving higher‑layer behavior on retain data. On WMDP and MUSE, RA^2 yields higher forget‑adjacent utility than strong unlearning baselines at similar broad‑utility levels. These results show that the migration direction of forget representations materially affects benign behavior near the forget boundary. Our code is available at https://github.com/ZacLing/retain_anchored_attraction.
Authors:Alexandre Bismuth
Abstract:
In large language models (LLMs), attention concentrates heavily on the first token. This 'attention sink' is believed to protect deep models against over‑mixing. Even though graph transformers face a related pressure, we find that a SoTA graph transformer, GRIT, rarely forms sinks on molecules, and that it sinks less when trained on larger graphs, the opposite of the trend in LLMs. In principle, a graph can prevent sinks: we prove and show empirically that its symmetry and locality limit the attention any node can receive. On molecules, however, with distinguishable atoms and global attention, neither limit prevents sinks; what stops them lies in the model. We find that the attention scoring function is the main cause. Replacing GRIT's gated logit with a dot product raises its sink rate from 0.09 to 0.68 on identical graphs. In Graph Transformers using dot‑product attention, removing the sink gives about the same error as removing a random node, while GRIT's most‑attended nodes matter to its predictions. Attention sinks are therefore a property of dot‑product attention rather than of Transformers. In our experiments, they form when dot‑product scoring has a distinguishable node to concentrate on.
Authors:Yihang Cheng, Qian Zhang, Yonghui, Huang, Baoshuo Kan, Junyi Shen
Abstract:
Generative approaches to automatic International Classification of Diseases (ICD) coding use large language models (LLMs) to predict ICD codes from clinical documents. However, original ICD code strings are not ideal generation targets because the ICD hierarchy does not always reflect clinical relationships, and LLMs struggle to interpret and generate these codes. We introduce SemICD, a framework that trains LLMs to generate Semantic IDs (SIDs) instead. SemICD constructs SIDs by hierarchically clustering embeddings of textual code profiles, allowing semantically related codes to share prefixes while each complete SID uniquely identifies one ICD code. To align the SID tokens with clinical semantics, we employ two auxiliary tasks for bidirectional generation between textual code profiles and SIDs. Experiments on MIMIC‑III and MIMIC‑IV with four LLMs show that SemICD outperforms direct code generation, improving Micro‑F1 by 1.34‑8.00 points. Our code is available at https://github.com/youwillbethebest/SemICD.
Authors:Mohamed Abouagour, Junhyun Nam, Wonse Jo, Byung-Cheol Min
Abstract:
Robots following persistent language instructions must decide which cached room judgments remain valid as their environment changes. We present MIND (Maintenance via INstruction‑conditioned Dependencies), a framework for maintaining target‑room decisions without re‑evaluating every room after each event. A language model compiles each instruction into dependencies for semantic qualification and target selection. The controller intersects semantic dependencies with event footprints, refreshes affected judgments, and updates reachability and selection separately. For fixed deterministic evaluation, planning, and selection, conservative dependency and footprint coverage preserves the decisions of full recomputation. On 240 five‑event ProcTHOR sequences with an exact benchmark predicate, MIND reduces mean semantic evaluations from 30.6 to 11.2 while preserving all 1,200 post‑event decisions. Frozen‑evaluator experiments with Phi‑4 and Qwen3‑14B also preserve all 1,200 decisions, without improving the evaluators' underlying task accuracy. In 32 closed‑loop AI2‑THOR episodes using exact‑predicate qualification and metadata‑derived footprints, MIND matches the final target decision of full recomputation in every episode. Unitree G1 experiments demonstrate target reuse and reselection across room transitions without additional semantic evaluations under MIND, and abstention after a barrier and an object removal at 4 evaluations against 15 to 18. Additional materials are available at https://mind‑icra27.github.io/.
Authors:Eren Cetin, Julius Erbach, Vukasin Bozic, Hyojun Go, Zixiang Zhao, Nikolai Kalischek, Federico Tombari, Konrad Schindler, Dominik Narnhofer
Abstract:
Memory for long‑horizon video generation must satisfy two competing requirements: it must remain bounded so generation can continue indefinitely, while still recovering what was observed at a place when the camera returns, regardless of how long ago it was seen. Reconstruction‑based memories provide spatial recall by caching and reprojecting explicit geometry, but require growing storage and depend on estimated 3D structure. Recurrent memories keep a constant‑size state, but retention is governed by temporal recurrence, causing old observations to fade with time. We introduce SMART, a slot memory addressed by region tokens, which combines bounded storage with spatially addressed recall. SMART stores content in a constant‑size slot recurrence whose writes are addressed by the observing camera and whose reads are queried by the rendering camera, without constructing an explicit 3D representation or maintaining a growing cache. Consequently, recall is conditioned on inter‑view geometry rather than on recency. Each read additionally provides a confidence estimate that conditions a chunk‑causal diffusion transformer, allowing it to selectively trust retrieved content while relying on its generative prior elsewhere. On long trajectories in Memory Maze and Minecraft, SMART maintains high‑fidelity recall far beyond its training horizon while using constant memory and per‑step cost.
Authors:Jiankai Huang, Hanwen Shen
Abstract:
Equivariant structure‑prediction pipelines alternate coordinate updates with the construction of a k‑nearest‑neighbour (kNN) graph whose membership is a discrete, piecewise‑constant function of the coordinates. Rebuilding that graph at every block boundary (PB) is the expensive part of an epoch, whereas reusing one frozen graph (F) is fast but leaves a topology gap. We study a two‑pass design in which the cheap pass records a small amount of state (margins, kNN radii, measured per‑boundary displacements) and a second pass compensates conservatively. We formulate the choice of the graph‑construction basis as a damped fixed‑point iteration whose unique fixed point is exactly PB, and derive (i) exact non‑flip certificates with the two‑sided constant m_i/4 and one‑sided constant m_i/2, (ii) the minimum‑variance (BLUE) replay gain γ^ and the resulting feasibility crossover, (iii) an \ell_\infty trust‑region projection that preserves convergence, and (iv) an analytic near‑tie spacing law E[m] \approx r_k/(3k) that matches measurements to 1%. Three compensation levels are implemented in C++ (L1: frozen set with live geometry; L2: margin‑gated local repair; L3: damped, projected basis) and validated under a matched‑conditions protocol (identical seed, MSA depth and trajectory family, only the topology mode differs). L2 reproduces the PB edge set exactly (D = 0.0000 at every boundary), L1 stays within 0.6% and L3 interpolates monotonically: L2 reproduces the PB edge set exactly when the trust region is removed, and L3 interpolates between the frozen and the PB topology. The recorded trajectory itself yields a measured 1.5× Pass‑1 and 1.76× epoch speed‑up with no measurable loss gap. An open‑source implementation of the runtime and of all the experiments reported here is available at https://github.com/AlbertHuang96/ProteinPredictMachineLearning.
Authors:Zachary Broveak, Christian Kubicka, Thierry Tambe
Abstract:
Large language models impose substantial memory demands and arithmetic costs during inference. Dense‑to‑mixture‑of‑experts (MoE) conversion is a model optimization technique that aims to reduce these arithmetic costs. Dense‑to‑MoE conversion partitions pretrained feed‑forward network (FFN) neurons into experts and selects a subset for each token. Leading methods fall into two categories: analytic methods produce a converted model quickly using heuristics, while learning methods produce a model much more slowly by optimizing via gradients. Both approaches, however, fix the same active expert count, or "utilization," at every layer. We introduce Spectral‑MoE, which learns to allocate a utilization budget heterogeneously across layers. Its name draws a metaphor between separating a dense FFN into experts and separating a light beam into its spectrum. Spectral‑MoE consists of two distinct approaches: a static approach that uses a single per‑layer scalar which is shared across tokens, and a dynamic approach which emits a per‑token schedule. We demonstrate that both allocators improve performance over uniform on both the analytic‑based and learning‑based SOTA dense‑to‑MoE methods. At 25% FFN utilization, dynamic allocation reduces WikiText‑2 perplexity relative to uniform by 1.04, 2.96, and 2.71 points on the learned base for Llama‑2‑7B, Llama‑3‑8B, and Qwen2.5‑7B, respectively. The corresponding reductions on the analytic base are 2.45, 6.26, and 2.17 points. On the learned base, dynamic allocation improves six‑task mean accuracy over uniform by an average of 1.5, 4.2, and 3.6 percentage points across the three models at FFN utilizations of 12.5%, 25%, and 50%, respectively. These results establish per‑layer expert utilization as a novel optimization axis in dense‑to‑MoE conversion.
Authors:Anton Arapin, Aditya Singh, Yixin Wang, Jun Gao, Bernadette Bucher
Abstract:
World models of visual information predict future image frames conditioned on given action sequences. This predicted scene evolution is driven by deterministic changes such as controlled motion on a robot platform, as well as stochastic changes such as rendering of manipulated objects or previously occluded regions. Our key insight is that the spatial location of these stochastic changes is itself learnable. We leverage this insight by jointly regressing both the deterministic next‑state prediction and the spatial uncertainty map, then using the uncertainty map to guide a conditional diffusion process. We propose Decoupled Drift‑Diffusion World Model (D3WM), which uses a transformer trained with a Gaussian negative‑log‑likelihood objective to produce both the deterministic next‑state prediction and a spatial map of heteroscedastic aleatoric uncertainty. The learned uncertainty map then provides a structured spatial gate for a conditional diffusion model: deterministic regions are anchored to the predicted state, while uncertain regions, such as disocclusions or dynamic textures, are sampled by the diffusion process. We frame this decomposition under a variance‑gated stochastic differential equation that separates deterministic drift from stochastic diffusion. D3WM shows improved results over competitive baselines in two distinct types of action spaces: robotic tabletop manipulation and camera motion.
Authors:Sarah Pan, Ali Behrouz
Abstract:
Transformers are efficient to pre‑train at scale because attention permits parallel computation across sequence positions, but long‑context inference requires increasingly large KV caches. Efficient recurrent alternatives reduce the asymptotic complexity of inference, but their sequential dependence limits token‑wise parallelism during training, making them harder to scale. Further, the current paradigm of extending long context tasks creates competing notions of retaining information precisely and keeping long‑context inference efficient. To address this gap, we introduce Titan‑Llama, a post‑training method for retrofitting attention‑based language models with a hierarchical memory architecture that updates parameters at test‑time. Titan‑Llama applies a blockwise attention mask and preserves full‑fidelity attention over recent tokens while compressing historical context into recurrent, fixed‑size neural memory modules (NMMs). Our formulation allows pre‑trained attention layers to be replaced without training a new language model from scratch. On Llama‑3.2‑1B‑Instruct, Titan‑Llama raises accuracy over segmented attention windows from 23.87% to 40.51% on QuALITY and from 14.50% to 40.75% on LongHealth. Inference benchmarks show more than 20% faster prefill, 37% faster decode at medium context lengths, and 24% lower steady‑state decode memory usage in the evaluated configurations. Together, these results demonstrate that incorporating memory architecture mechanisms that utilize test‑time training effectively modulates long‑context behavior of pre‑trained models, offering a practical route to adapting pre‑trained transformers for efficient long‑context inference. Our code is released on Github \hrefhttps://github.com/sarahpannn/titans‑pytorch/tree/mainGitHub
Authors:Jinhao Pan, Bowen Wei, Yuqing Zhou, Ziwei Zhu
Abstract:
Reference solutions and question‑specific rubrics are not always available to guide the teacher in self‑distillation. A large language model's own solution attempts offer an alternative, but their quality is difficult to assess in open‑ended tasks without automatic correctness checks, and even correct attempts can involve uncertain token predictions. We propose Reliability‑‑Uncertainty Self‑Distillation (\method) to learn from these attempts. The same model serves as teacher and student: the teacher receives the question and completed attempt, and the student matches the teacher's token‑level predictions using a question‑only prompt. \method weights each attempt's distillation loss by additively combining judged reliability, estimated from a self‑judge's score and confidence, with student predictive uncertainty. Across eleven reasoning benchmarks, \method outperforms all five baselines in average accuracy, reaching 70.47% on Qwen3‑4B and 76.69% on Qwen3‑8B (8.90 and 2.10 percentage points above the base models). Data and code are available at https://github.com/JP‑25/RUSD.
Authors:Elyas Irankhah, Ashley Hagaman, Samah Fodeh
Abstract:
Mixed methods research offers established strategies for integrating qualitative and quantitative evidence, but less guidance for integrating two computational representations of the same qualitative corpus. This methodological study introduces Parallel‑Then‑Integrated (PTI), a four‑step procedure for explicit and traceable integration. We present PTI using 1,203 interview paragraphs from 29 mothers in Nepal. The two representations were ontology‑guided Patient Voice (PV) coding with PVminer and data‑driven themes generated through non‑negative matrix factorization (NMF). A theme‑complete crosswalk documented convergence, divergence, and preprocessing‑sensitive differences. Depression status, based on Patient Health Questionnaire‑9 (PHQ‑9) scores, served as a secondary external anchor for exploratory association checks. PTI provides a structured approach for integrating computational representations of qualitative text within mixed methods research.
Authors:Akash Raj
Abstract:
Maintenance work orders are the largest written record of how industrial equipment fails, yet in most plants they are close to unusable for analysis. Short texts are abbreviated and misspelt, asset identity is buried in free text, work‑type labels contradict the work described, and the fields that matter most for reliability (symptom, root cause, action taken) are usually empty. As a result, diagnostic knowledge stays with senior technicians and leaves when they do. We present Plant Memory, a system that applies technical language processing (TLP) to SAP Plant Maintenance (PM) exports and builds an ISO 14224‑aligned maintenance ontology in which every extracted statement is a reified fact carrying a confidence score, the evidence it was read from, and a verification state. Extraction is rule‑first, with an optional large language model (LLM) acting as a second reader whose disagreements raise a fact's review priority rather than overwrite it. Technicians verify facts through a review queue, answer targeted knowledge‑gap questions where failures repeat without a recorded cause, and approve or reject ontology drift proposals when new terms appear. The resulting graph is served to people through a web application and to AI agents through a Model Context Protocol (MCP) server, so both query the same verified knowledge and every answer cites the work orders behind it. On a synthetic 12‑month export of 400 work orders from a parboiled‑rice plant, rule‑based extraction reaches 96.2% component accuracy, 88.9% failure‑mode accuracy and 100% root‑cause precision at 62% coverage; on 28 diagnostic questions, ranked root causes from the ontology are correct at top‑1 in 82.1% of cases against 53.6% for the raw SAP root‑cause column.
Authors:Yifan Lan, Zhongye Liu, Yurui Chang, Hanyu Wang, Yongkang Du, Lu Lin, Jinghui Chen
Abstract:
Emerging decision models such as JEV provide typed interfaces that map task instructions and observations directly to choices, judgments, and scores. As these models are increasingly used for automated decision‑making, understanding their reliability under adversarial manipulation becomes critical. We introduce DecisionRobust, a benchmark for systematically evaluating the robustness of decision models across textual and visual inputs. Our benchmark covers practical decision‑making scenarios and a diverse suite of attacks, including semantics‑preserving text perturbations, image perturbations, and prompt injection in both modalities. We curate evaluation sets through external validity checks and cross‑model candidate selection. We evaluate 30 model configurations on text tasks and 17 on vision tasks, including 24 and 12 held‑out models, respectively, that are excluded from candidate selection. Our experiments reveal several key findings. First, attacks expose vulnerabilities beyond the models used for construction and selection, although optimized image perturbations transfer weakly to many targets even with multiple surrogates. Second, relative model robustness varies across tasks and attack types, and resistance to input perturbations does not necessarily imply resistance to prompt injection. Third, label‑consistency checks substantially reduce measured attack success across text and image settings, highlighting the importance of validating apparent failures. The benchmark is available at https://yifan‑lan.github.io/DecisionRobust/.
Authors:Qingsong Xie, Jiahao Tian, Jiacheng Tang, Zhifeng Li, Madalina Ciobanu, Navan Preet Singh, Qingqing Mao, Ritankar Das
Abstract:
Constructing robust GUI agents relies on multimodal visual grounding: the ability to precisely map natural language intents to actionable on‑screen components. Despite abundant desktop, web, and mobile benchmarks, high‑fidelity resources tailored for diverse environments are surprisingly scarce. To bridge this gap, we introduce the Screen Referring Grounding Dataset (ScreenRef), a comprehensive GUI grounding dataset consisting of GUI screenshots paired with natural language referring expressions and box or point annotations. Spanning desktop, web, and mobile domains, ScreenRef comprises 586K screenshots and over 633K referring expression instances, covering tasks such as text recognition, visual matching, spatial understanding, and refusal scenarios. To boost the model's grounding performance, we propose the Distillation‑to‑Reinforcement Learning (D2RL) method to address the limited generalization of supervised fine‑tuning and the sparse reward problem in reinforcement learning. Utilizing ScreenRef, we train Qwen3‑VL‑8B‑Instruct via sequential SFT and D2RL training, yielding GroundGUI‑8B. GroundGUI‑8B achieves 95.2% on ScreenSpot‑V2, 63.9% on ScreenSpot‑Pro, 65.4% on OSWorld‑G, 86.8% on MMBench‑GUI L2, and 38.3% on UI‑Vision. Among 8B‑scale models, it produces substantial gains over models trained on open‑source datasets, with performance comparable or even superior to models trained on closed‑source data. Code, models, and datasets are available at https://github.com/PrentisAI/GroundGUI.
Authors:Zhexi Luo, Zhixuan Xu, Rui Yang, Zeyu Xiao, Yiwen Hou, Lin Shao
Abstract:
Achieving human‑like dexterous manipulation remains a central challenge in robotics. Learning from human hand‑object interaction (HOI) datasets offers a promising pathway to diverse dexterous skills, yet existing physics‑based approaches train specialist policies for individual trajectories, thereby limiting skill reuse at scale. We introduce DexMimic, a generalist dexterous hand‑object tracking policy jointly trained across diverse HOI trajectories. Efficient joint training requires allocating each rollout to both an undertrained phase and a starting state that supports continued tracking. DexMimic realizes this insight through Value‑Guided Reset Allocation, which adapts the initialization distribution online to prioritize difficult phases and select reset states with higher expected tracking returns. We further introduce a benchmark spanning training, fine‑tuning, and generalization. Experiments show improved tracking success and training efficiency, faster fine‑tuning, and the potential for zero‑shot generalization to unseen trajectories and objects; larger training sets further improve efficiency and generalization. Finally, we validate DexMimic on a real‑world dexterous manipulation platform.
Authors:Neil K. Chada, Simon Cotter, Kody J. H. Law, Xinzhu Liang
Abstract:
Reliable uncertainty quantification remains an important challenge in AI. Hierarchical Bayesian methods provide a principled framework for this by representing unknown quantities through posterior distributions. Joint inference of neural‑network parameters and prior hyperparameters is challenging for conventional Markov chain Monte Carlo (MCMC) methods due to strong dependencies. In this work, we introduce the multilevel particle marginal Metropolis‑Hastings (MLPMMH) method, which combines pseudo‑marginal inference for the hyperparameters with coupled sequential Monte Carlo (SMC) across hierarchy of network‑width truncations for conditional parameter inference. The hierarchy is induced by a trace‑class neural network (TNN) prior with width‑dependent decay, motivated empirically by the compressible structure observed in trained neural‑network parameters. Under standard multilevel coupling assumptions, we establish that MLPMMH achieves mean‑square error (MSE) \mathcalO(\varepsilon^2) at expected computational cost \mathcalO(\varepsilon^‑2), and verify the predicted rate on a Bayesian regression problem. To address larger‑scale applications, we further introduce S‑MLPMMH, a practical approximation that replaces the exact sampling components with a PCA‑based hierarchy and scalable Bayesian Monte Carlo. Experiments on MNIST demonstrate competitive predictive performance and uncertainty quantification while exploiting the parallel structure of the approximation. Software has been made publicly available at https://github.com/submission‑code‑anon/mlpmmh‑anon.
Authors:Anton Arapin, Chahyon Ku, Bernadette Bucher
Abstract:
World models are a powerful means of reasoning and planning, but many of these action‑conditioned predictive models are high dimensional and computationally expensive. These challenges limit the number of action candidates that can be evaluated by a planner. We show that this cost is avoidable for planners over short horizons, which only require local information with respect to the current state. For a fixed state, the frames reachable by varying a low‑dimensional action (a camera movement or an end‑effector command) lie on a low‑dimensional manifold. We estimate the local shape of that manifold directly via a network trained to estimate how each image region moves per unit of each action axis supervised by data collected from samples by the generator. A correction head fit to trajectory data estimates the shape of this manifold region up to a third‑order Taylor polynomial. We call this network estimating the Taylor polynomial describing the manifold shape an Action Jet and compute its radius of validity in closed form. By traversing this approximation of the local manifold shape, we only have to perform inference once per state for any number of candidate actions. Scoring sixteen candidates from a single extraction is over 300x faster than with either source generator, and this gap widens with the number of candidates a planner evaluates. We measure against state‑of‑the‑art action‑conditioned generators, IRASim and WorldGym on BridgeData V2, a real 7‑DoF manipulation benchmark, and FrameCrafter on DL3DV scenes, where we chart from the novel‑view generators DepthSplat and LagerNVS. Because only the tangent comes from the generator and every other term from the data, the Action Jet also predicts held‑out frames more accurately than IRASim, the generator that supplied its tangent, at short horizons and at over 100x less compute.
Authors:Alec H. Follmer
Abstract:
Interpreting electron paramagnetic resonance (EPR) spectra commonly requires repeated forward simulation and optimization of spin Hamiltonian parameters. Established software provides mature simulation and fitting methods, but these calculations are not generally available as differentiable components within machine‑learning workflows. We present TorchSpin, an open‑source PyTorch framework that ports the organization and numerical conventions of EasySpin into a tensor‑based computational environment. Representative calculations spanning continuous‑wave EPR, ENDOR, pulse EPR, spin dynamics, trajectory simulation, and magnetometry closely reproduced corresponding EasySpin results. In matched benchmarks, TorchSpin CPU execution was faster than EasySpin for six of nine workloads although performance depended strongly on the calculation. We therefore focused on the capabilities enabled by automatic differentiation in PyTorch. In a Cu(II) powder‑spectrum demonstration, feedback propagated through TorchSpin improved the quality of a supervised convolutional neural network (CNN) without using parameter labels during refinement. Models trained from randomly initialized CNNs using only noisy spectra and TorchSpin reconstruction loss achieved median similarities of \geq 0.99. CNNs trained on TorchSpin‑generated libraries also transferred without adaptation to independently generated EasySpin spectra with performance comparable to EasySpin‑trained controls. Once trained, the CNN produced initial parameter estimates in approximately 1 ms per spectrum while a batch of 512 was processed in approximately 1.3 ms with GPU acceleration. TorchSpin therefore provides both a tested Python implementation of established EPR calculations and a differentiable physical layer for rapid learned inference, spectral refinement, and parameter‑label‑free training.
Authors:Ziyuan Gao
Abstract:
High‑throughput perturbation screens increasingly use machine‑learning models to predict the effects of gene combinations that were never measured. A difficult case arises when both genes in a test pair were seen individually during training, but their combination was not. In this setting, standard OOD methods often fail because neither gene appears unfamiliar on its own. We therefore compare each model prediction with a simple additive prediction obtained by summing the two observed single‑gene effects. The difference between them, which we call additive disagreement (AD), measures how much extra interaction effect the model introduces for an unseen combination. Across eight prediction methods and three Perturb‑seq datasets, specialised neural predictors do not consistently outperform this simple additive baseline. More importantly, predictions with larger deviations from the additive baseline tend to have larger errors. Conventional output‑based and embedding‑based confidence signals are inconsistent across datasets and models, whereas AD remains informative across all evaluated compositional settings. We further study how this signal can guide decisions in an agentic perturbation‑screening workflow. The code is available at https://github.com/zygao930/OOD.
Authors:Ziyuan Gao
Abstract:
Protein function annotation operates over an expanding label space, making class‑incremental learning (CIL) a natural framework for incorporating newly discovered functions without full retraining. Yet existing CIL methods have been developed largely in vision and have not been systematically evaluated over frozen protein representations. We conduct a controlled benchmark of nine representative CIL methods across five sequence‑, structure‑, and sequence‑‑structure‑based protein encoders and two enzyme function datasets at different scales, while keeping the encoders frozen and evaluation settings consistent across representations. We find that no method transfers reliably across encoders, and that these failures persist across alternative phase constructions. Diagnostic analyses reveal that this instability arises from weak gradient‑based stability signals, substantial heterogeneity in embedding geometry, and covariance assumptions whose appropriate capacity changes with both dataset scale and encoder. These findings suggest that the representation itself should be treated as a central component of continual‑learning design rather than a passive feature extractor. More generally, reliable protein CIL will require encoder‑aware methods that adapt their statistical assumptions and capacity to the geometry of the representation on which they operate. Code is available at https://github.com/zygao930/ProteinCIL.
Authors:Reyhaneh Esmailizadeh, Xingjian Leng, Zhanhao Liang, Liang Zheng
Abstract:
We generally perceive that training text‑to‑image diffusion models requires heavy compute: it is true if we aim at a comprehensive frontier model. This understanding prevents people from training their own models when getting started in the field. We develop MiniDog, a framework that enables pretraining diffusion transformers in 55 minutes and fine‑tuning in another 15 minutes on four 3090 GPUs. For data, we use 26k dog images from ImageNet and caption them for pretraining and use 2k images generated by FLUX.1‑schnell for fine‑tuning. We precompute the image and text features and cache them for efficiency. For architecture, we use E2E‑INVAE for its semantic latent space, a small transformer with 22M parameters, a small Qwen model for text encoding, and in‑context conditioning. For training, we use iREPA and an EUPE perception encoder for acceleration. For evaluation, our repository supports FID, HPSv2 score, PickScore, and various VAE reconstruction metrics. The project is open‑sourced at https://github.com/Reyhaneesmailizadeh/MiniDog. We do not claim technical novelty.
Authors:Juntong Feng, Qingan Liu, Jingru Dong, Yue Zhu, Weiming Zheng, Yujie Cheng, Xinzhe Han, Shuai Hao, Yunzhi Zhuge, Haiwen Diao, Huchuan Lu
Abstract:
Vision‑Language Models (VLMs) have achieved strong performance in visual perception and spatial reasoning, leading to growing interest in using them as high‑level planners for embodied AI. However, physical tasks require models to integrate spatial information, reason over temporal states, and generate multi‑step plans under various constraints, raising the question of whether current VLMs can reliably support such reasoning and planning independently of execution. We introduce Plan‑Bench, a benchmark for evaluating the high‑level reasoning and multi‑step planning capabilities of VLMs relevant to embodied worlds, while isolating the influence of action execution. It covers three complementary dimensions, spanning 16 subclasses and 699 curated samples. Extensive evaluation of 28 VLMs reveals room for improvement, with even the strongest model achieving only 75.0% accuracy. We further find that model scaling generally improves these capabilities. Ablation studies also indicate larger models show increasing potential as high‑level planners. Overall, Plan‑Bench provides a lightweight probe for evaluating VLMs' reasoning and planning capabilities relevant to embodied environments and assessing their potential as decision makers for physical tasks. Our benchmark and code are available at https://github.com/MinneFeng/Plan‑Bench.
Authors:Haoyi Duan, Liliang Chen, Nan Wang, Yang Zhang, Yuhang Lu, Xiaotong Chen, Wei-Cheng Tseng, Shenyuan Gao, Zhaoshuo Li, Jingyi Jin, Pooya Jannaty, Dhruv Shah, Yuke Zhu, Ming-Yu Liu, Maoqing Yao, Guanghui Ren
Abstract:
We introduce WorldOracle, a benchmark specifically designed to evaluate action‑conditioned world models (ACWMs) against paired real‑world executions. Given the same initial observation and low‑level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task‑state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi‑view action coherence. To support this evaluation, we curate real‑world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi‑segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action‑conditioned world modeling for manipulation. We release the datasets, evaluation code, and leaderboard: https://worldoracle‑benchmark.github.io/.
Authors:Yuchen Lu, Julian McAuley, Lei Zhang
Abstract:
Long‑horizon agents accumulate progress in persistent task state, yet repeatedly process interaction histories describing how that progress was achieved. This raises a natural question: can long‑horizon context instead be organized around reusable progress rather than the history that produced it? In this paper, we introduce CarryOn, an object‑oriented context runtime that organizes continuation around reusable task objects. Rather than treating compressed histories or reconstructed accounts of progress as the primary basis for continuation, CarryOn builds a context view from continuation carriers, complementary execution context, and a live tail of recent interactions. A core challenge is keeping this reusable basis valid after its surrounding history is retired: how can CarryOn update the basis without reconstructing the full representation at every fold? CarryOn maintains carrier records across folds, revising affected records against bound evidence and applicable environment checks while carrying forward unaffected records and jointly updating execution context. Experiments on OSWorld 2.0 and BrowseComp‑Plus show improved gains of 5.8% and 7.0% while reducing total token usage by 51.9% and 22.9%, respectively. Project page: https://ethannju.github.io/carryon
Authors:Jiajing Hu, Xu Han, Li-Ping Liu
Abstract:
Accurate uncertainty quantification is critical for many time series forecasting applications, and recent studies have increasingly adopted generative models to capture predictive uncertainty. Existing approaches often combine deterministic forecasting with generative modeling to improve predictive performance. However, their joint training is typically unstable because the deterministic model is embedded within the generative process, resulting in deep computational graphs and changing the probability transport path. In this work, we propose DV‑Flow, a unified flow‑matching formulation that incorporates the deterministic predictor as a parallel velocity component rather than as the endpoint of the transport path. This design enables stable end‑to‑end optimization while subsuming several existing formulations as special cases, allowing direct comparisons among them. Extensive experiments on six real‑world datasets demonstrate that DV‑Flow consistently improves probabilistic forecasting performance over strong baselines. Our code is available at https://github.com/tufts‑ml/dv‑flow‑time‑series.
Authors:Yuyang Bai, Yu Wang, Yu Zhang
Abstract:
Searching text‑rich knowledge graphs requires following relations while examining textual evidence. Predefined graph interfaces constrain what evidence agents can access, even when they write code to combine tool calls or process retrieved subgraphs. We propose direct knowledge graph interaction (DKGI), a harness in which language model agents create graph access tools directly over complete node and edge files. The agent writes Python code to follow relations, combine node sets, and select textual evidence, interleaving execution with shell search and file inspection. This lets agents process large node sets in bulk while returning selected text for the model to assess. Execution feedback guides tool creation and revision, while the evolving tools determine what evidence the agent gathers next. On benchmark datasets spanning three domains, training‑free DKGI exceeds the strongest published baseline by at least 6.7 Hit@1 points on every graph, even with a minimal harness. Controlled ablations show the benefits of multi‑turn programming over shell‑only access and predefined graph tools, while trajectories show agents revising code to extend searches or address errors. These findings highlight the value of allowing tool creation and tool use to co‑evolve during graph search. Our code and data are available at https://github.com/leopoldwhite/DKGI.
Authors:Wei Xie, Kaituo Feng, Shiming Fu, Yong Liu, Xiangyu Yue, Shuangjia Zheng
Abstract:
Terminal agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long‑running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self‑improving terminal agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm‑10K, a high‑quality dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm‑GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self‑recovery attempts. By learning from whether these attempts successfully complete the task, the agent self‑improves through its own recovery experience. Built on Qwen3.5‑9B, our ReTerm‑9B achieves 34.8% Pass@1 on Terminal‑Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
Authors:Thinker Team, Peng Gao, Jichao Jiao, Chen Liao, Daqin Luo, Yiming Ma, Baiyu Pan, Hailin Shi, Jiyuan Wang, Xiangming Xu, Xiongwei Zhao
Abstract:
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model. In this work, we present Thinker‑WM, a unified multimodal world model that jointly models and generates RGB, metric depth, and robot action through a shared intermediate Unified Dynamics Transformer with modality‑specific encoders and decoders. The model performs multimodal unification at the representation level: appearance, geometry, and action are transformed into a common dynamic representation and jointly evolved by shared parameters, while their distinct output structures are preserved. The same parameterization supports multiple functions, including joint video‑action generation, world prediction, policy inference, action‑conditioned generation, video‑to‑action generation, and inverse dynamics. To examine whether the shared representation captures action‑related information, we introduce a lightweight motion probe that shows the frozen visual representation contains recoverable motion information. Thinker‑WM is trained on a heterogeneous corpus comprising approximately 20K hours of egocentric video, 10K hours of real‑robot data, and 1K hours of simulation data, including approximately 4K hours of real‑robot data with aligned depth supervision. Extensive experiments demonstrate strong performance across embodied world‑model evaluations. Thinker‑WM ranks first on EWMBench, WorldOlympiad and WorldArena, while controlled studies further demonstrate the benefits of action conditioning, geometric interaction, and multi‑function training within the shared representation.
Authors:Peiran Yin, Hao Wu, Zihao Yang, Fengyu Zhou, Yijuan Liang, Haozhe Hu, Xudong Wang, Wei Zhang, Xiaoyu Shen
Abstract:
Visual token pruning for multimodal large language models (MLLMs) appears highly effective, with existing methods retaining over 95% of unpruned performance on common VQA benchmarks using a quarter of the visual tokens or fewer. However, our extensive evaluation of ten methods across three MLLM families and five ability groups reveals a critical gap: strong general performance often comes at the cost of severe degradation on fine‑grained tasks such as OCR and visual grounding. We trace these failures to two limitations of the prevailing paradigm of retaining the highest‑ranked native patches. A single importance ranking overlooks the complementary evidence required by different tasks, while sparse patch selection disrupts both image content and spatial integrity. To address these limitations, we propose Bifocal, a training‑free, two‑stage pruning method built on a mixed‑granularity framework. It allocates separate budgets to query‑focused and image‑intrinsic evidence, complemented by a pooled coarse representation of the entire image. Before the LLM, Bifocal matches question words to visual regions through a soft conjunction balanced against norm‑debiased image saliency, while selecting image‑intrinsic evidence to maximize representational coverage. Inside the LLM, it further prunes tokens using density‑adjusted text‑to‑image attention. With roughly a quarter of the original visual tokens, Bifocal is the only evaluated method to retain over 90% of unpruned performance in every ability group across all three models, and ranks first on average in all nine model‑budget settings. The code is available at: https://github.com/EIT‑NLP/Bifocal
Authors:Zhiyuan Chen, Shuai Chen, Haoran Zhou, Yizhe Zhao, Kaiyuan Yang, Zhixiang Duan, Shaoqing Xu, Ran Cao, Diyun Xiang, Yangwei You
Abstract:
In contact‑rich manipulation, actions that appear equally plausible from vision can produce different contact outcomes, while reactive control based on current tactile feedback may respond too late to prevent failure. We present ForeTac, a predictive contact‑guidance framework organized as a represent‑predict‑evaluate‑guide loop. ForeTac encodes the current tactile state, predicts future tactile latent trajectories conditioned on candidate action sequences, and evaluates each predicted trajectory with a contact‑quality objective learned from successful and failed physical interactions. At inference, gradients of this objective are backpropagated through the tactile predictor to guide action generation before execution while keeping all model parameters frozen. We evaluate ForeTac on four real‑robot tasks (planar and curved‑surface wiping, card swiping, and fragile‑object grasping) with diffusion and flow‑matching policies. Compared with vision‑only Diffusion Policy, ForeTac improves the average task success rate (SR) from 36.3% to 71.3% and the average contact success rate (CSR) from 20.0% to 66.3% across these tasks, and yields gains in both metrics with a flow‑matching policy. Component analyses support the learned tactile representation, action‑conditioned foresight, and generation‑time gradient guidance, while predicted tactile latent trajectories preserve contact‑quality ordering. Project page: https://foretac.github.io/.
Authors:Kaipeng Fang, Hangxu Liu, Chaojian Shi, Yunsong Zhou, Hongzi Zhu, Minyi Guo, Jingkuan Song, Lianli Gao
Abstract:
World action models (WAMs) built on pretrained video generation models adapt quickly to new robotic tasks, but they forget previously learned tasks under sequential fine‑tuning. Experience replay (ER) mitigates such forgetting by rehearsing data of previous tasks, yet this data is often unavailable in robotics, where a policy is deployed as weights only and its training data is not shared along with it. In this work, we ask whether a WAM can achieve data‑free continual learning by regenerating its own past. Our key observation is that a WAM can indeed autoregressively regenerate complete rollouts of previously learned tasks and replay them to mitigate forgetting. However, replaying such raw rollouts still lags far behind replaying real data, which we attribute to two issues. First, the WAM is unable to recognize task completion and keeps generating purposeless motion after the task is done. Second, under open‑loop generation, the generated actions gradually drift away from the generated observations. To address both issues, we present rectified generative replay (R^2‑WAM), which truncates generated rollouts at task completion with an agentic evaluator and filters out rollouts whose actions are physically inconsistent with the observations. On the four LIBERO suites and five real‑world tasks, R^2‑WAM reduces forgetting by over 90% relative to sequential fine‑tuning and nearly matches ER without accessing any data from previous tasks. Our analysis further reveals that forgetting in WAMs concentrates in inverse dynamics modeling: a forgotten WAM still produces visually plausible observations of previous tasks while its generated actions fail. Project page: https://kaipengfang.github.io/r2‑wam/
Authors:Dawei Han, Shidong Li, Shijia Liao, Jiahua Liu, Songting Liu, Qingzheng Wang, Yuxuan Wang, Xin Chen, Yifan Cheng, Tianyu Li, Gangin Park, Xuehao Yang, Chang Zeng, Haoyang Zhang, Ruoyi Zhang, Xiangyu Zhang
Abstract:
We introduce Fish Audio Drama 3, an open‑source speech generation model that turns text into directed performances, built on a 35B‑parameter mixture‑of‑experts backbone that activates 3B parameters per token. It clones voices from multiple references, follows natural‑language direction, revises a take when the next turn gives a note, voices multi‑character scenes, edits existing speech by infilling, and synthesizes long‑form and streaming audio. Multi‑turn revision, scene dubbing, editing by infilling, and streaming text input are new in Drama 3, all in one model. Drama 3 pairs the hybrid linear‑attention backbone with a new codebook multi‑token‑prediction audio decoder that predicts the acoustic codebooks of each frame in dedicated layers that attend over the entire sequence. The model is trained with a three‑stage pre‑training recipe, instruction fine‑tuning, and large‑batch reinforcement learning whose only reward is a single learned model of human preference. Its inference engine decodes a single stream at 271 frames per second on one H200, against 177 for Fish Audio S2, although Drama 3 has more than seven times as many parameters. The model weights are available on GitHub (https://github.com/fishaudio/fish‑speech) and Hugging Face (https://huggingface.co/fishaudio/drama‑3), and inference is available through SGLang‑Omni (https://github.com/sgl‑project/sglang‑omni). We highly encourage readers to visit https://fish.audio to direct their own performances.
Authors:Shuyang Cui, Zhi Zhong, Keisuke Toyama, Koichi Saito, Qiyu Wu, Yukara Ikemiya, Junghyun Koo, Kin Wai Cheuk, Woosung Choi, Giorgio Fabbro, Wei-Hsiang Liao, Shusuke Takahashi, Yuki Mitsufuji
Abstract:
Music source separation (MSS) has been conventionally addressed by discriminative methods. Recently, flow matching has emerged as a strong generative paradigm for modeling complex data distributions, with successful applications across diverse domains. These advances motivate us to systematically investigate flow matching for MSS in a four‑source (Vocals, Bass, Drums, Other) setting in full‑band stereo. Specifically, we investigate prediction targets, flow paths, single‑source versus multi‑source modeling, and mixture consistency constraints. Beyond the signal‑level metrics commonly used for MSS, such as signal‑to‑distortion ratio, we further evaluate MSS methods using embedding‑based metrics, as well as a listening test, providing a comprehensive benchmark. On MUSDB18‑HQ, our proposed model exhibits competitive performance in both objective and subjective tests, highlighting the potential of flow matching for generative MSS. Demo: https://ik4sumii.github.io/FLOMSS/
Authors:Pengcheng Huang, Zhenghao Liu, Shixin Ren, Shi Yu, Sijia Yao, Yukun Yan, Chaojun Xiao, Maosong Sun
Abstract:
Edge‑cloud collaboration combines the lower inference overhead and greater data locality of edge‑side execution with the stronger capabilities of cloud‑hosted models. However, as LLM agents increasingly operate in long‑horizon scenarios, cloud participation becomes interleaved throughout execution rather than a one‑shot decision for each query. Such interleaved participation introduces trajectory‑level effects that remain insufficiently understood. To address this gap, we introduce EC2‑Bench, a benchmark for evaluating edge‑cloud collaboration in long‑horizon agent execution. It provides executable digital‑workspace agent tasks and a unified evaluation framework covering task performance, resource usage, and privacy exposure. Using this benchmark, we empirically study diverse forms of edge‑cloud collaboration. Our study reveals three findings. First, collaboration achieves a favorable balance among task performance, resource usage, and privacy exposure compared with edge‑only and cloud‑only execution. Second, as edge models become stronger, collaborative execution approaches cloud‑only performance, and performance differences across strategies shrink, while privacy exposure remains strongly strategy‑dependent. Third, stronger edge capability shifts preferred collaboration from externally controlled cloud participation and direct cloud execution toward more edge‑controlled collaboration. Further analyses identify cloud‑invocation decision‑making and instruction following as important edge‑side abilities for effective collaboration. These findings provide a systematic understanding of edge‑cloud collaboration in long‑horizon agent execution and insights for designing future edge agents that effectively collaborate with cloud models. Code and resources are available at https://github.com/OpenBMB/EC2‑Bench.
Authors:Jerrold Huang, Junwei Quan, Saiyue Lyu, Zihan Guo, Yingqiang Ge
Abstract:
Agent skills turn natural‑language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look deceptively similar. A practical screen must catch nearly every attack while keeping the review load manageable. We present SkillScreen, a compact decision model trained to score skill maliciousness directly from SKILL.md in a single forward pass. On MaliciousSkillBench, SkillScreen flags 20‑25% of benign skills at 99% recall on unseen attack families, substantially fewer than the word‑level classifier and prompted LLM judges tested. Under the same training protocol, it also generalizes better than a fully fine‑tuned encoder to unseen sources. Yet unfamiliar benign sources sharply increase false positives, showing that recognizing new attacks does not ensure reliable screening of new skill collections. Our results demonstrate the value of task‑trained decision models for high‑recall skill screening. Project page: https://jerroldh.github.io/SkillScreen‑webpage/
Authors:Kai Liu, Anqi Li, Junxian Li, Zhixin Wang, Zhikai Chen, Renjing Pei, Yulun Zhang
Abstract:
We propose a unified spatial‑density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage. To isolate the two phases, we split benchmarks by scoring logic. Prefill probes (MMBench, POPE, ScienceQA) evaluate only the first decoded token; their accuracy reflects prefill logits argmax before any decode KV eviction. Full‑pipeline probes (TextVQA, DocVQA, MM‑Vet) assess the complete generated sequence and capture both phases. We identify a critical evaluation pitfall: decode pruning claims validated solely on prefill probes measure prefill quality, not decode performance. Experiments show discarding all visual KV during decode barely affects prefill‑probe accuracy but degrades TextVQA and MM‑Vet drastically; this insensitivity stems from the scoring rule, not compression intensity. Evaluated on seven model variants (InternVL‑3.5, Qwen3‑VL, LLaVA‑1.5) and three full‑pipeline benchmarks, our method achieves a three‑way Pareto frontier across compute, memory and accuracy. Density‑based prefill compression maintains strong accuracy, scales linearly with visual token count, and delivers 100x faster token selection at high resolution. For decode, density top‑k KV eviction consistently outperforms strong baselines H2O, SnapKV and StreamingLLM on TextVQA and MM‑Vet across three retention ratios. At 0.125 keep ratio on TextVQA, it surpasses H2O by 9.6 points; InternVL‑3.5 models (2B to 38B) maintain a 7 to 10 point advantage. Compression scales favorably with model size: every measured 8B to 38B model transition reduces accuracy loss instead of worsening it. We suggest MM‑Vet or DocVQA as mandatory full‑pipeline validation for any decode‑phase claims. Our code is available at https://github.com/Kai‑Liu001/TwinPrune
Authors:Leo Fan, Hongyang Wang, Wenbo Chen, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Yiming Yan, Hao Wu, Yichuan Li, Qian Qu, Zheshen Wang, Benoit Dumoulin
Abstract:
The throughput of an LLM training job depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses. The best recipe changes with every model, accelerator and framework version. Automated tuners search a fixed space of knobs and check only that a job runs, so they miss levers outside that space and can ship a configuration that silently changes training. ATTUNE is a coding agent that tunes any working recipe unattended within a GPU‑hour budget. Guided by a skill of search rules, scripts and lessons from past tuning campaigns, it diagnoses each run and changes one lever at a time. A correctness gate outside the agent refuses any argument known to change the training objective. ATTUNE tunes Amazon's production post‑training recipes on H100, H200 and B200 clusters, raising throughput by 1.7x on average. It beats the published throughput of four of NVIDIA's own benchmark recipes on NVIDIA's AutoModel framework by 6‑17%. On models held out from its development, it improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax‑M2, and least where the vendor tuned them for the same hardware. Given the same GPU‑hour budget, it outperforms existing automated tuners, including Bayesian optimization, random search and OptiCo, a recent LLM‑agent tuner. Its gain on GLM‑4.5 is more than four times the best alternative's. The same coding agent without ATTUNE's skill gains less and fails half its launches. We release ATTUNE at https://github.com/amazon‑science/attune and OpenRecipes, a leaderboard of tuned and verified training recipes, at https://openrecipes.ai.
Authors:Jack B. Jedlicki, Amish Sethi, Heng Yang
Abstract:
We use a frozen frontier vision‑language model, GPT‑6‑Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory‑dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end‑effector waypoint and gripper command, with no RoboMME‑specific training. On the 800 test episodes it succeeds in 92.1%, above the previously reported human score of 90.5%, which was obtained with an oracle skill interface, and far ahead of the best leaderboard entry (60.8%) and of a hierarchical system that pairs the same model with a RoboMME‑trained policy (79.1%). It matches or exceeds the human score on 13 of the 16 tasks. Its 63 failures concentrate in two tasks and are mostly premature completions and mistimed or unintended button presses. These results blur the usual division between long‑horizon planning, memory and end‑effector control, and raise the question of how much of this stack a single general‑purpose model can take over in embodied systems. Project website: https://embodied‑astra.github.io. Code, prompts and all episode logs: https://github.com/jackbjed/embodied‑astra‑robomme.
Authors:Xialie Zhuang, Zunhai Su, Jia Pan, Ngai Wong, Guanhua Chen, Chuan-Wei Kuo
Abstract:
We introduce StartLux‑Decision, decision models from 0.8B to 27B parameters that combine efficient single‑pass prediction with adaptive, confidence‑guided reasoning. The Instant models use bidirectional attention in full‑attention layers to improve decision accuracy while keeping single‑pass inference. The AutoThink models invoke explicit reasoning for uncertain decisions and answer confident cases directly. A three‑stage training pipeline establishes shared decision capabilities, specializes the two variants, and equips AutoThink with reasoning through reasoning supervision and reinforcement learning. We introduce Reinforcement Learning for Calibrated Deliberative Decisions (RLCD^2), which uses a frozen model and proper scoring rules to reward improvements in decision probability quality and applies a soft penalty to overlong reasoning traces. RLCD^2 raises the share of reasoning traces that terminate within the generation limit by up to 48 percentage points and cuts average reasoning length by up to 65%, while largely preserving decision accuracy. On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable‑scale open‑weight decision models by 7.4‑16.9 points, and the 27B AutoThink model reaches 73.4. We also introduce DecisionPivot, a programmatically verifiable benchmark of computational decisions built from contrast pairs that differ in a single answer‑changing detail. On DecisionPivot, AutoThink outperforms its Instant counterparts by 11.5‑18.4 points in pair accuracy and leads among evaluated systems. The models can be run using frameworks such as vLLM and MLX, with quantized versions available to reduce memory usage.Inference code and links to model weights are available at https://github.com/StartLuxLabs/Startlux‑Decision .
Authors:Daniel Shao, Dongmin Bang, Luca L. Weishaupt, Sophia J. Wagner, Nic G. Reitsam, Ming Y. Lu, Long P. Le, Richard J. Chen, Faisal Mahmood
Abstract:
Contrastive vision‑language (VL) pretraining in computational pathology has generally relied on curated image‑caption datasets sourced from social media, educational videos, and research publications. These corpora are sparse relative to the diversity of tissue morphology and difficult to scale further, as pathologists do not routinely produce descriptive captions in clinical practice. We propose to instead leverage slide‑level pathology reports which can be generated during routine diagnostic workflows in a scalable manner. A key challenge in this weakly‑supervised setting is that pathology reports describe macroscopic, diagnostic findings while patch‑level visual features describe microscopic findings. We introduce Locked Image Multiple Instance PreTraining (LIMIT), which addresses this gap by freezing a pretrained vision encoder and fine‑tuning only the text encoder: report embeddings serve as cross‑attention queries over bags of patch features, producing report‑conditioned slide representations optimized via InfoNCE. Using LIMIT, we establish CONCH‑Z, which aligns the text encoder of CONCH v1.5 to 335k clinical reports. Evaluated across 22 tasks spanning patch‑level classification, tumor detection, and slide‑level subtyping, CONCH‑Z establishes state‑of‑the‑art zero‑shot performance over VL encoders trained on curated caption datasets, while simultaneously closing the gap with slide‑level foundation models trained with substantially more parameters and compute. The code and model is available at https://github.com/mahmoodlab/LIMIT.
Authors:Ye Liu, Reza Averly, Avery Meyer, Frazier N. Baker, Ruoxi Gao, Daniel Adu-Ampratwum, Xia Ning
Abstract:
Drug discovery remains a resource‑intensive and time‑consuming process, involving billions of dollars and a decade of development to advance a drug to market. While many in silico tools have accelerated this process, operating them still demands substantial time, effort, and expertise from medicinal chemists. Autonomous systems built on Large Language Models (LLMs) have emerged as a promising solution, yet current designs limit their performance. Existing single‑agent systems scale poorly to long‑horizon, multi‑step, and realistic tasks, whereas multi‑agent systems typically execute predefined workflows that cannot capture the iterative, non‑linear nature of a real drug discovery campaign. Here, we present MADDIE, a multi‑agent drug discovery engine. It comprises a central agent, Orchestrator, and four specialized worker agents: HitIdentifier, LeadOptimizer, Synthesizer, and Evaluator, each representing a critical step in a drug discovery campaign. Under Orchestrator's orchestration and coordination, these worker agents are composed into flexible workflows, enabling MADDIE to adapt its strategy and navigate the drug discovery pipeline end‑to‑end. Across 50 protein targets, MADDIE produces molecules satisfying key property requirements and their synthesis routes, outperforming existing agentic systems, generative models, and general‑purpose LLMs by a wide margin. Our analysis reveals how these agents collaborate and that MADDIE can employ different strategies across targets to achieve the campaign's goal, while still being efficient in time and tokens. Together, they demonstrate MADDIE as a low‑cost, effective, and efficient autonomous engine for accelerating drug discovery. Code and dataset is available via https://github.com/ninglab/MADDIE.
Authors:Jinrun Liu, Bai Yan, Tengfei Liu, Qunfeng Liu, Jian Yang, Shi Cheng, Yuhui Shi, Qi Zhao
Abstract:
Convex relaxations make nonconvex optimization tractable by replacing hard nonconvex problems with convex surrogates that give valid bounds. Yet their design is still largely manual: a human expert must decide where to relax, which relaxation to use, and how much formulation growth is worth paying. We show that this design process can be learned. The method proposed in this paper, AutoConvexRelax, represents the symbolic formulation of a nonconvex problem as a tripartite graph linking variables, symbolic terms, and their objective or constraint positions. It then learns a policy that composes symbolic rewrites into problem‑specific relaxation trajectories as the formulation evolves. We prove that finite action sequences generated by the learned policy preserve relaxation validity under explicit domain, occurrence‑context, and representation‑consistency conditions. We further establish computable error bounds for mixed relaxations and sufficient conditions for feasible recovery and global‑optimality certification. As a first instantiation, we apply the method to nonconvex quadratic problems and fractional problems. On 241 benchmark quadratic instances, the learned policy strengthens McCormick lower bounds by 13.52%. Its mean lower bound is 3.70% below strengthened semidefinite relaxations on the normalized comparison metric, while it uses substantially smaller formulations. The learned policy retains this trade‑off on fractional instances and reaches near‑semidefinite bounds on beamforming and sensor‑network localization instances. These results suggest that convex relaxation design can move from a fixed handcrafted choice to an automated learning process that adapts to problem structures. Code is available at https://github.com/auto4opt/AutoConvexRelax.
Authors:Harvey Mannering, Zhiwu Huang, Adam Prügel-Bennett
Abstract:
Emotions are expressed on a continuum and are naturally described in the valence‑arousal (VA) space. Existing VA‑based image editors are typically training‑based and domain‑specific, limiting them to VA regions seen during training and making them brittle across datasets and domains. Meanwhile, despite producing high‑quality edits, state‑of‑the‑art foundation image editors are not continuous affective controllers: non‑linear emotion representations in text encoders makes prompt interpolation poorly calibrated, so accurate and continuous VA control remains unreliable. To fill this gap, we introduce Barycentric Guidance, a training‑free framework that turns pretrained foundation image editors into continuous affective controllers over VA, requiring no finetuning, extra data, or model changes. Our method maps any target VA point to simplex‑based guidance weights, enabling VA controllability in three aspects: (i) accurate targeting of VA states, (ii) broad coverage across the VA plane, and (iii) continuous control. Across three face datasets, we improve VA target accuracy by 25‑29%, expand VA‑plane coverage by 28‑50%, and increase expression diversity by 29‑49%, over the strongest baseline. Unlike prior domain‑specific methods, our training‑free approach exploits the foundation model's inherent versatility, enabling generalization beyond human faces to animals, artworks, and complex scenes within a single framework. Code is available at https://github.com/harveymannering/BarycentricGuidance.
Authors:Ming Shin Cindy Sung, Duan Leijiahao, Haoxiang Wang, Boning Tan, Liangna Fu, Chao Tan, Yafu Li, Yulong Chen, Chaoqun Wan, Zhen Huang, Shuxian Liang, Xian-Sheng Hua
Abstract:
Industrial manipulation in flexible manufacturing, e.g., high‑mix, low‑volume (HMLV) electronics assembly, requires robot policies to adapt to changing products and production conditions. Such settings impose industrial process constraints, including product variants, moving material flows, finite execution windows, and tight geometric tolerances. However, existing evaluation frameworks do not systematically capture these constraints when assessing generalist robot policies. To address this gap, we introduce LIBERO‑Industry, a novel benchmark built on the widely used robot learning benchmark LIBERO, that translates the aforementioned industrial constraints into controllable task conditions and measurable success criteria. Specifically, LIBERO‑Industry comprises three classic task scenarios: Static‑to‑Dynamic (S2D) loading, Dynamic‑to‑Static (D2S) unloading, and Dynamic‑to‑Dynamic (D2D) transfer. The suites contain 216 tasks in total that systematically vary object geometry, conveyor speed, interaction‑window duration, and geometric acceptance region size. Extensive experiments across five generalist robot policies (including vision‑language‑action models and world‑action models) show that performance progressively degrades as industrial constraints are introduced and combined. Under a shared fine‑tuning and evaluation protocol, the LIBERO‑Industry co‑training condition achieved higher observed target‑transfer completion than the original‑LIBERO condition.
Authors:Yuanpeng He, Ceyao Zhang, Fangjing Li, Kefei Zhu, Lijian Li, Yueheng Li, Tianjia He, Wenpin Jiao, Zhi Jin, Yaodong Yang, Yuanpei Chen
Abstract:
Recent world action models draw on video generation priors to jointly predict future observations and robot actions. Adapting these priors to control requires determining how future predictions should guide policy learning. RAP‑WAM combines predictive Rehearsal and Action Perturbations (RAP) with future‑supervised queries and Coupled Action Projection for policy learning. Predictive rehearsal trains on model‑generated inputs while retaining recorded targets, and accumulates recurring errors into a spatial prior. We perturb actions and measure changes in video predictions. Action‑response‑based spatiotemporal credit assignment combines these responses, prediction error, and the rehearsal prior to allocate visual supervision; spatially aggregating the responses determines supervision intensity across future action steps. No separate scoring network or deployment‑time probe is required. The base learns from a video‑pretrained backbone without intermediate pretraining on external embodied data. We evaluate the fixed base and its agent‑refined, task‑conditioned system. On 42 RoboDojo tasks, their average success rates are 12.85% and 15.62%, with Scores of 17.86 and 20.59. Across 15 real‑robot tasks, the complete system achieves 89.00% task‑macro‑average success, exceeding the strongest of five baselines by 37.67 percentage points. Across 315 fixed‑output simulation windows, 45.21% of response‑top20 positions fall outside error‑top20, revealing information beyond prediction error. Videos and further details are available on the project website: https://rapwam.github.io/.
Authors:Chia-Ming Lee, Chia-Yu Lin, Hung-Kai Huang, Yi-Ting Ku, Yu-Chen Liang, Chih-Chung Hsu
Abstract:
Face swapping and generative editing enable realistic facial manipulation, while online redistribution complicates publisher oversight of circulating face images. Existing approaches establish provenance, detect manipulation, recover embedded watermarks, verify image integrity, or disrupt unauthorized face manipulation. These capabilities do not necessarily establish consistency with a specific publisher‑enrolled release, limiting the evidence available for verification and human review. To address this gap, we introduce Doctor Trigger, a framework for verifying circulating faces against private publisher‑held release records. Before publication, the publisher embeds a weak keyed Fourier‑phase signal in each face while preserving visual fidelity and stores its spectral reference. Afterward, verification compares the circulating face with that reference and uses key projection to measure retention of the release‑specific signal. Therefore, relation and key evidence jointly help publishers identify suspicious copies for review, potentially reducing the circulation of manipulated images. Experimental results across FF++, DFDC, and Celeb‑DF show that spectral agreement separates tested face swaps and generative manipulations from enrolled releases, while the keyed signal distinguishes published versions from their unreleased originals under the evaluated key‑preserving benign transformations. Doctor Trigger complements provenance metadata with release‑specific face verification and record‑linked evidence for further assessment.
Authors:Lingbei Meng, Rui Huang, Zhirong Shen, Jinxin Liu, Lingxiao Jing, Haorui Chen, Yaosong Du, Haowei Lin
Abstract:
Scaling laws predict large‑model performance from small experiments, but finding a reliable law itself requires costly experiments. SLDBench tests whether language‑model agents can discover scaling laws when all experimental results are given upfront. We introduce SLDBench v2 with two extensions. First, we expand the corpus from 8 to 127 scenarios spanning language‑model pretraining, downstream performance, and vision‑language transfer, and select 30 challenging scenarios on which a reference power‑law fit reaches R^2 \le 0.8; they represent at least 1.09 × 10^23 FLOPs of source‑reported training compute. Second, we introduce active scaling law discovery: an agent chooses which experiments to run, pays each one's recorded cost (e.g., training FLOPs), and revises its law as outcomes arrive, and is scored on the same hidden extrapolation targets as in the passive setting. Across 8 frontier agent systems, median R^2 drops from 0.72‑0.75 (passive) to 0.24‑0.57 (active), and prediction error rises 1.5‑4.7 times, depending on the system. Agents favor cheap experiments, spending 8‑44% of the full‑pool cost on average; low‑cost acquisition can yield strong predictions, but cheap purchases also leave experimental conditions needed for extrapolation unobserved. Experimental planning thus remains a key open challenge for agents that assist model development in the prevailing AI‑for‑AI study. Project page: https://lingbei‑meng.github.io/SLDBench‑v2/.
Authors:Yuhe Hu
Abstract:
Test‑time‑training (TTT) layers update fast‑weight memory while processing an input sequence. A gradient‑based update, however, does not by itself establish how the memory revises an existing association. We study the structure and empirical role of prediction‑derived updates in released TTT implementations. For dot‑product objectives, we characterize the absence of a prediction‑error residual and establish conditions under which inner‑loop ascent and descent are equivalent after reparameterization. For the LayerNorm‑based squared‑error objective used by TTT‑Linear and TTT‑MLP, we derive the inner gradient and identify a prediction‑derived component that cancels at the released initialization in the zero‑epsilon limit. We then measure the full normalized prediction's contribution on three trained checkpoints. Removing this contribution while retaining the LayerNorm Jacobian gives a mean gradient norm ratio of 0.25 to 0.32 across the models. On 100,303 tokens from War and Peace, the same intervention with local per‑token norm matching raises perplexity by 2.8‑2.9% on TTT‑Linear‑760M and 1.3B, whereas freezing all fast‑weight updates raises it by factors of 4.2‑4.5. Separate synthetic tests illustrate the dependence of plain delta‑rule overwriting on step size. The results distinguish a prediction‑derived update from its behavioral value and show why implementation analysis and live interventions are both needed to assess test‑time memory.
Authors:Zhifan Zeng, Wenbo Xiao, Arcot Sowmya, Changming Sun
Abstract:
Open‑vocabulary 3D Gaussian segmentation typically binds semantic features to the per‑Gaussian state via per‑scene gradient training (1‑4 h) or the more recent closed‑form distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard decision that degrades retrieval accuracy and yields unstable mask boundaries at render time. We present DIGS, a training‑free framework with two stages, class‑agnostic mapping and multi‑modal encoding, that decouples instance discovery from linguistic naming. At the Gaussian level, our distribution‑informed representation maintains an explicit top‑M label distribution per primitive, updated by a tempered Dirichlet rule; preservation of multi‑peaked label distributions at boundary primitives allows a late‑argmax renderer to recover clean instance boundaries. A bidirectional state machine aggregates these distributions into a persistent class‑agnostic instance ontology. The instance‑level encoding stage then fuses the CLIP visual embedding of each instance's isolated 3D footprint with the CLIP‑text embedding of an LLM‑generated attribute description, disambiguating visually similar instances. On a single A100, DIGS completes mapping and instance encoding on LERF‑OVS in 1.0‑3.2 min per scene, depending on whether LLM fusion is enabled, and serves native 3D queries in 5.4 ms. It reaches 71.95% mean mIoU on LERF‑OVS, outperforming both the training‑free SFS and training‑based LaGa baselines. DIGS also achieves the highest mean mIoU on LERF‑Mask and 3D‑OVS and surpasses Gaga and ObjectGS on Replica under their respective evaluation protocols. Code is available at https://github.com/ZephyrZos/DIGS.
Authors:Yewon Kim, Paul Hongsuck Seo, Sang Chul Ahn
Abstract:
SLAM (Simultaneous Localization and Mapping) is a core technology in autonomous systems that enables them to estimate their own position and build a map of unknown environments simultaneously. SLAM systems in dynamic environments are of great practical importance. However, existing dynamic SLAM approaches still have several limitations like over‑removal and high computational costs. To address these limitations, we propose a real‑time RGB Dynamic SLAM framework that precisely removes moving objects at the pixel level without relying on predefined labels or segmentation models. Our system combines pixel motion predicted by ego‑motion using DUSt3R with point tracking from All‑Tracker, removing only actually moving pixels by analyzing discrepancies between the two motion sources. Unlike object‑level masking methods, our approach eliminates only currently moving regions, allowing temporarily static parts to be utilized for tracking. Operating at approximately 10 FPS, the system achieves real‑time performance on standard 30 FPS video streams by subsampling every 3 frames. Relying solely on RGB inputs, this significantly improves upon state‑of‑the‑art dense dynamic SLAM approaches (0.5‑3 FPS), presenting a highly practical solution.
Authors:Chen Huang, Yiyang Zhang, Yuhao Wu, Yang Deng, Wenxuan Zhang
Abstract:
LLM‑based multi‑agent collaboration is increasingly used to solve complex tasks by dividing work among specialized agents and integrating complementary expertise. Yet existing systems and evaluations largely rest on an implicit perfect‑colleague assumption: collaborators provide correct and complete information and respond promptly to requests. In realistic teamwork, however, collaborators may return information that does not answer the intended question, provide insufficient detail to support a decision, or fail to respond to a request. To separate task‑solving competence from the ability to recover an unreliable evidence‑gathering process, we introduce the Unreliable Colleagues Benchmark (UnCoBench), which organizes behavioral failures through Misaligned, Incomplete, and Missed settings. The main agent need to verify contextual correctness, diagnose information gap and track unresolved dependencies in order to obtain the correct evidence for solving the task. We also introduce a verifiable and scalable automated generation pipeline that pairs LLM‑based scenario generation with executable checks of answer uniqueness and role necessity. Results across ten models show that similar reliable‑setting performance can result in different robustness to unreliable evidence. Trace analysis reveals frequent omissions of corrective follow‑up, and the recovery behavior involves inefficient waiting and redundant queries. These findings show that robust collaboration requires not only sound reasoning, but also targeted action to diagnose evidence failures, recover relevant information, and integrate it into decisions. Our data and source code are available at https://github.com/iNLP‑Lab/UnCoBench.
Authors:Boxuan Zhang, Jianing Zhu, Xiaotian Han, Ruixiang Tang
Abstract:
Reliable deployment of LLM‑based multi‑agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes. These auditors typically learn from a limited corpus of trajectories with verified outcomes. Expanding this corpus requires costly execution and verification, yet broader failure coverage alone provides no explicit criterion for identifying prefixes that the current auditor finds difficult to classify. To this end, we propose Dream2Discern, a runtime auditor trained through implicit augmentation guided by its own prediction loss. During Dream, we select perturbations of the auditor's representation operators to increase prediction loss on recorded prefixes, yielding hard latent variants. During Discern, the auditor learns to predict execution outcomes from these variants using the original labels. The variants are regenerated as the auditor learns, turning a limited corpus into a renewable source of hard examples and requiring no additional trajectory collection or outcome verification. Extensive evaluations on AFTraj‑2K and StepShield show that Dream2Discern outperforms competitive failure auditors and achieves over 90% trajectory‑level weighted accuracy on both benchmarks while balancing prediction accuracy and timeliness. Project page: https://zbox1005.github.io/dream2discern‑project/
Authors:Quan Wang
Abstract:
Automatic extraction and anatomical numbering (R01‑R12, L01‑L12) of all 24 rib centerlines in thoracic CT is essential for unfolded rib visualization, fracture detection, and surgical navigation, but remains challenging due to adjacent‑bone confluence, calcified costal cartilage bridges, kyphotic pose variation, and partial visibility of terminal ribs. We present ribline, an open‑source three‑stage framework combining: (1) an automatically detected lung‑apex and vertebral‑midline coordinate frame with a kyphosis‑aware azimuthal angle for point ordering along each rib; (2) a hybrid voxel detector fusing an anatomically anchored Semantic Context Forest (SCF) with a lightweight 3D U‑Net; and (3) a global stage that decomposes the probability skeleton into candidate curves, assigns rib indices via a bilateral Viterbi dynamic program coupled across both hemithoraces, and refines curves via template recovery and normal‑plane centering (RibSnake). On the Siemens Corporate Research 2 mm thoracic CT benchmark (47 training and 12 evaluation scans), our pipeline achieves a labeled Chamfer distance of 2.05 mm, labeled clDice of 0.886 (3 mm) and 0.945 (4 mm), 97.2% labeling accuracy (278/286 ribs), and zero false‑positive curves on the evaluation split, and 3.64 mm (median 2.39 mm) with 91.8% labeling accuracy across all 59 annotated scans (1,402 ribs) in 5‑fold cross‑validation. Code: https://github.com/wq2012/ribline Pretrained Models: https://huggingface.co/collections/wq2012/ribline
Authors:Yirui Zhang, Yash Raj Shrestha
Abstract:
Qwen3‑8B reviewing Llama‑3.1‑8B in math adds 33.2 points, yet Qwen3‑8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator. We show that this confuses verification, which adds accuracy beyond either model alone, with capability transfer, where a stronger reviewer supplies answers it could have given on its own. Measured against the stronger model alone, across 48 pairs of 12 open‑weight models (3B to 72B) in math, knowledge, code and reasoning, review beats it in only 9 of 96 review directions, and does worse the larger the capability gap. The reason is anchoring: the reviewer mostly follows the generator, discarding its own correct answers. Review costs 2.6 to 2.8 times as much as the stronger model alone, while a selector using the same two calls is 2.7 to 4.2 points more accurate than review and cheaper. Studies of verification should report the stronger model alone, not only the gain over the generator.
Authors:Mayank Gupta
Abstract:
Hard‑margin SVM training is the distance problem between two convex hulls, which the Triangle Algorithm (TA) solves with a certified duality gap. We propose \emphguarded block transfers: k Mitchell‑‑Demyanov‑‑Malozemov (MDM) transfers per scan are aggregated into one exact line search, guarded by a comparison with the single best transfer at a cost independent of the number of points n, and replaced by an away step when the leading transfer is capacity‑clipped. On the Minkowski‑difference polytope, where the objective is 1‑strongly convex, we prove global linear convergence with explicit constants and no combinatorial swap‑step bounds; the per‑step contraction is k‑independent and only the drop‑count bound grows with k, so aggregation is provably safe while the saving in scans is measured, not proved. Since the TA certificate is the Frank‑‑Wolfe gap divided by the current distance, gap‑safe screening applies, and we bound the iteration after which it has removed every zero‑weight point off the optimal faces by O\!\left(\tfrackρ\log\tfrac1τ\right) up to logarithmic factors in the geometry, ρ the rate constant and τ the optimal‑face margin. On five LIBSVM benchmarks, with fairer baselines than our submission, the solver matches the baselines to four or five digits wherever both converge, ties LIBSVM on sparse‑support hard margins and beats our blended pairwise conditional gradients (BPCG). But our full‑cache sequential minimal optimisation (SMO) code ties or beats it on kernel L_2 margins, LIBLINEAR's dual solver is up to 4.5× faster on the separable L_2 margin, and our solver is one to three orders of magnitude slower than LIBLINEAR's Newton solver on dense‑support L_2 margins and than LIBSVM on a nearly touching kernel hull. Its edge is the certificate, not speed.
Authors:Haosen Wang
Abstract:
Under GRPO against a frozen, reference‑free LLM judge, RL exploits the judge's false positives. Does the policy‑judge capability gap or the judge itself set how fast and how far? We train 0.5B and 1.5B Qwen2.5 policies on GSM8K against four judges at least as large, and the 1.5B policy also against one smaller judge. In a post‑hoc analysis, judge acceptance rose, but no cell of 12 gained more than 0.04 in seed‑mean held‑out greedy accuracy. A gold‑reward control with the same recipe raised the 1.5B policy's accuracy by +0.054 (for the 0.5B policy, gold reward gave no detectable gain either). The depth of judge hacking tracks the judge: the plateau correlates with judge AUC (Spearman ‑0.69 over 12 cells), and within a family the larger judge is fooled less deeply. For the pace, two pre‑specified pairs contradict the gap account: the cell with the larger gap was fooled 3.5x sooner at the pre‑specified threshold. With 12 cells, however, we cannot formally separate judge AUC from the gap for either endpoint. A blind test of step‑0 predictors on three held‑out judges was inconclusive. At 7‑8B, judges from three families plateaued between 0.37 and 0.85.
Authors:Alan Tang, Lifeng Wang, Gang Wu, Tao Zhang, Yantao Jia, Xiaofeng Zhu
Abstract:
Agent skills encode operational procedures and domain expertise as reusable knowledge assets, yet their distribution in plaintext exposes them to unauthorized copying and leakage. However, practical mechanisms for skill ownership verification and leakage tracing remain underdeveloped. Watermark embedding before distribution offers a potential solution, but structured, compact skill instructions impose stringent requirements for factual and procedural accuracy. To address these challenges, we introduce SkillMarker, a KGW‑based watermarking method that jointly targets functionality preservation and strong watermark embedding. SkillMarker unifies quality‑constrained rewriting with signal‑oriented candidate search. The former separates essential knowledge from flexible expression to preserve instruction fidelity; the latter explores alternative expressions to obtain a stronger watermark signal under preservation constraints. We release an open‑source toolkit that integrates watermark embedding, quality auditing, and detection to support the protection of agent skills. SkillMarker targets ownership verification and leakage tracing in distributed skill ecosystems, where content may be copied in whole or in part, or incorporated into other skills. Across four datasets, SkillMarker negligibly affects downstream task performance on average. In our copy‑paste tests, window‑based detection consistently identifies copied passages containing as few as 1,536 consecutive tokens. Even after merging with one or three background skills, SkillMarker retains watermark detection rates of 72.2‑‑77.8%.
Authors:Yunjia Zhang, Junan Zhang, Jing Yang, Xueyao Zhang, Fan Fan, Zhizheng Wu
Abstract:
We present AnyEdit, a unified, separation‑free framework for speech and singing voice editing that preserves real‑world acoustic environments, including noise, reverberation, and background music. Recent unified generative models already achieve strong speech and singing generation in clean conditions, but real‑world vocal editing remains challenging because the model must revise content while simultaneously preserving speaker identity, prosody, and the surrounding acoustic scene. Directly modeling noisy audio end‑to‑end entangles semantic editing with environmental acoustics, weakening the core vocal generation capability. We therefore adopt a two‑stage design that preserves clean‑domain editing ability while deferring acoustic rendering to a separate stage: an instruction‑guided autoregressive editor first predicts clean prosody and content‑style tokens, supported by a teacher‑student noise‑robust prosody tokenizer that distills clean prosodic tokens from degraded recordings; a flow‑matching acoustic model then performs in‑context rendering conditioned on the source acoustic context, so the edited segment inherits surrounding noise, reverberation, and background sound while maintaining consistent speaker timbre. To stabilize unedited regions, we introduce unmodified‑area‑aware supervision; after pre‑training, we further apply direct preference optimization on mined real‑to‑real singing pairs. We also release AnyEditBench, a benchmark covering both speech and singing voice editing under clean, simulated, and in‑the‑wild acoustic conditions. Experiments on Chinese and English speech and singing datasets showed that AnyEdit consistently outperforms strong baselines in content accuracy, subjective quality, and background consistency under noisy and reverberant conditions. Demo: https://any‑edit.github.io/. AnyEditBench: https://huggingface.co/datasets/Ririiiii/anyeditbench.
Authors:Ziyi Chen, Yan Zhang, Jianhui Wei, Jianing Yin, Daoan Zhang
Abstract:
Large language models make decisions through parallel prefill and sequential decoding. Although effective, this approach incurs decoding costs and does not explicitly expose decision probabilities. Existing task‑specific classifiers require per‑task training and offer limited generalization to unseen decision schemas. We propose Large Decision Models (LDMs), repurposing language models to return explicit probabilities over labels specified at run time using one prefill forward pass. We first adapt models to this interface through Diverse Decision Supervision (DDS), covering varied domains, rules and labels. To further narrow the performance gap to a reasoning model, Self‑Posed Reasoning Distillation (SPRD) adds tasks generated by a frozen teacher, filters them for answer consistency, and trains on label distributions obtained from repeated teacher reasoning. We evaluate on DecisionVista, pairing new‑schema tests with held‑out human‑labelled sources. Across three base models, the recipe improves accuracy over a step‑matched baseline by 1.8 and 1.5 percentage points, respectively, achieving a 182‑fold speedup in latency over reasoning rollout on Gemma‑4‑31B. Reasoning remains stronger on the new‑schema test, motivating a confidence router for uncertain decisions. On development sets, routing recovers a median 92% of reasoning's accuracy advantage while generating 27‑58% of its tokens. Code is available at https://github.com/JianhuiWei7/LDM.
Authors:Kyungsu Kim, Yejin Kim, Kyogu Lee
Abstract:
We introduce Non‑musical Audio to Symbolic Music Transformation (NASMT), a novel task that converts non‑musical audio into symbolic music representations. Unlike automatic music transcription, which assumes musical input and a unique correct score, NASMT is inherently one‑to‑many and requires evaluation along two axes: input correspondence and musicality. We present Sori, an encoder‑‑decoder Transformer that addresses this task through domain adversarial training (DAT), which bridges the gap between musical and non‑musical audio in the encoder's feature space, and classifier‑free guidance (CFG) with a continuous parameter~γ that controls the trade‑off between correspondence and musicality at inference time. Across objective metrics and a listening study, Sori provides effective continuous control via γ and compares favorably with instrument‑agnostic transcription models. Sori supports real‑time streaming on a consumer CPU. Audio examples are available at https://soridemo0320.github.io/sori‑demo/.
Authors:Ziqi Wen, Shravan Murlidaran, Jonathan Skaza, William Wang, Miguel P. Eckstein
Abstract:
Image‑computable low‑level image saliency has shaped perceptual psychology and computer science, but is limited by its inability to capture human high‑level cognitive factors that influence attention and eye movements. Here, we propose a new fully automated method to compute semantic saliency by quantifying each object's contribution to scene comprehension without fitting any parameters to human data: the Automated Scene Understanding Map (AUTO‑SUM). AUTO‑SUM automatically segments and removes individual objects from a scene, uses multimodal large language models (MLLMs) to describe the original and modified scenes, and measures the semantic similarity between these descriptions using text embeddings. We show that AUTO‑SUM can predict which object is most important in a human‑measured scene understanding map (H‑SUM) estimated using human scene descriptions. We also show that AUTO‑SUM can predict the object most fixated by human observers during free viewing and scene description tasks better than a low‑level saliency model (GBVS, Itti‑Koch), a saliency model trained based on human meaning ratings (Meaning Map), and comparable to DeepGaze, which is trained on human fixation data. We contend that AUTO‑SUM can be used as a semantic saliency model that complements lower‑level saliency models. Code are available at https://github.com/starsky77/AUTO‑SUM‑Source‑Code .
Authors:Jinhao Zhou, Xiaoyu Zhang, Jiongchi Yu, Fahao Chen, Jun Wu
Abstract:
LLM‑based multi‑agent systems (LLM‑MAS) use specialized agents that exchange messages to accomplish complex tasks. Their reliance on networked protocols, remote tools, and model endpoints exposes communication to message loss, delay, and context expiration. Because agents often exchange open‑ended natural language, anomalies can alter task‑relevant information while leaving messages processable, allowing semantic errors to propagate through subsequent interactions. Systematic studies of faults in inter‑agent communication remain limited. This paper presents a large‑scale empirical study of how communication faults manifest, propagate, and affect task outcomes in LLM‑MAS. We construct a cross‑layer taxonomy linking lower‑stack disturbances to application‑visible faults and develop a fault‑injection and tracing pipeline. Our evaluation covers more than 29,000 executions across 100 tasks, five domains, three workflow topologies, and three model configurations. We find that 20.7% of failures with complete detection records occur without an immediate failure signal, reaching 76.2% for partial‑content faults. Flat maintains 53.0% task success under communication faults, compared with 3.3% and 3.2% for Sequential and Hierarchical, respectively. This advantage primarily arises when an alternative information path preserves the evidence required by downstream agents. We also evaluate execution‑side and evidence protection mechanisms and find that they recover different failure cases. Evidence recovery depends on whether trustworthy information remains accessible through an unaffected path. These observations provide empirical insights into communication‑fault propagation, workflow resilience, and protection design for LLM‑MAS.
Authors:Yeheng Wang, Liudan Shi, Yi Zheng, Jingke Li, Shiyuan Liu, Xin Tao, Yuanxing Zhang, Guoyu Sun
Abstract:
Text‑to‑video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on holistic video‑text alignment and provide limited insight into whether failures arise from target‑emotion expression, intensity control, or specific affective cues. We introduce EmoTrace‑Bench, a fine‑grained diagnostic benchmark that controls emotion intensity and selected affective cues in prompts and evaluates generated videos at multiple levels. EmoTrace‑Bench contains 448 controlled prompts covering eight emotions, two intensity levels, and diverse cue conditions. Using 12 T2V models, we generate 5,376 videos and evaluate target‑emotion presence, perceived intensity, and ten audiovisual dimensions of emotional expression with an automated evaluator calibrated and validated against human annotations on a subset of the data. Results reveal substantial differences across models and prompt designs. Prompts that explicitly describe a character's emotional reaction are followed more reliably than prompts that describe only events or scenes, whereas adding more affective cues or using more explicit cinematic audiovisual language does not consistently improve emotional generation. These results illustrate how aggregate alignment scores may overlook localized differences in target‑emotion expression, intensity control, and cue realization. EmoTrace‑Bench provides a traceable and interpretable framework for diagnosing emotional video generation without modifying existing generation pipelines, and we release the prompts and the automated evaluation rules.
Authors:Chang-Hoon Jeong, Kisung Shin, Hanwool Sul, Byoung-Tak Zhang
Abstract:
Context‑based offline meta‑reinforcement learning (COMRL) has advanced almost entirely through better task representations, which separate task identity from artifacts of the behavior policy so that the latent reflects the task alone. The implicit expectation has been that a clean representation will transfer to out‑of‑support (parameter‑extrapolation) tasks. On such a task, however, the encoder produces a latent outside the training support, where the policy has never been trained. We address this gap with Pessimistic Latent Task‑aware Optimization (PLATO), which exposes the policy to off‑support latents during training. PLATO assumes a mild manifold hypothesis: task latents lie on a low‑dimensional manifold, and out‑of‑support tasks lie farther from the training centroid. It exploits this geometry by perturbing inferred latents outward from the centroid to synthesize counterfactual tasks. A learned decoder ensemble then rolls out short trajectories at each perturbed latent, and the policy is updated on the imagined transitions under a pessimism weight that down‑weights steps on which the ensemble disagrees. We prove that the perturbation leaves the training support by a controllable margin and that the out‑of‑support generalization gap is bounded by the same perturbation reach, and we give explicit conditions, verified empirically, under which the epistemic disagreement at the perturbed latent stays bounded away from zero. On eight MuJoCo continuous‑control benchmarks, PLATO consistently improves out‑of‑support returns over strong COMRL baselines while remaining competitive on in‑distribution tasks. Our code is available at https://github.com/deephoony/plato.
Authors:Xupeng Zhang, Yantai Yang, Chang Guo, Zhaokai Yin, Zhipeng Zhang
Abstract:
In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long‑horizon manipulation requires recombining these skills without demonstrations for every skill combination. The full instruction route treats skill chaining as long‑horizon imitation. In a controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We introduce Hierarchical Subtask Execution (HSE), a diagnostic framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, decomposing long‑horizon failures into separable modes. We introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks launch from handoff states from preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post‑training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, PRTI reaches 18.16% ordered success, compared with 4.21% under random perturbation and 9.47% under a robot‑reset oracle, without combined skill training trajectories. These results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. Code: https://github.com/AutoLab‑SAI‑SJTU/RoboCombine.
Authors:Hongliang Cao, Xiuyu Li, Siqi Wang, Ziming Wang, Jingmin Zhu, Hailong Yang
Abstract:
Mixture‑of‑experts (MoE) models activate only a few experts per token, but when their expert weights are offloaded to host memory, host‑to‑device (H2D) expert transfers enter the critical path of decoding. Speculative decoding verifies several draft tokens in one forward pass, yet in an MoE model each verified token activates its own experts, so a larger draft tree transfers more experts. Because of this H2D cost, speculative decoding commits several tokens per verification cycle yet is sometimes even slower than autoregressive decoding. Based on this observation, we propose SpecFit, which chooses the verified draft subtree to maximize predicted throughput rather than acceptance length. SpecFit estimates the expected number of committed tokens from draft path probabilities and predicts cycle time from the node count and the per‑layer union of the experts the subtree activates; this union is a monotone submodular coverage function, so each node pays only for the experts it adds. An exact search over ancestor‑closed subtrees returns the subtree with the highest predicted throughput; run after each draft level, the same search also decides when to stop drafting, and the unmodified target verifies the result. On CNN/DailyMail summarization and a GSM8K subset with Qwen3‑30B‑A3B, SpecFit decodes 21.6% and 7.0% faster than EAGLE‑3 with the same expert cache and 1.97x and 1.72x as fast as vanilla EAGLE‑3. Code is available at https://github.com/Chlience/SpecFit.
Authors:Hebaixu Wang, Jing Zhang, Haonan Guo, Di Wang, Jiayi Ma, Bo Du
Abstract:
Multimodal remote sensing image generation emerges as a promising paradigm for recovering complete and complementary observations from heterogeneous and partially observed remote sensing data. However, existing methods predominantly depend on sensor‑specific fine‑tuning of natural image generation models, limiting unified multimodal generation capacity due to the absence of shared representations and robust conditioning mechanisms. To address these challenges, we propose OmniRSI, a unified multimodal remote sensing image generation model. Specifically, we develop a remote sensing‑specific variational autoencoder that captures multimodal priors and promotes a locally smooth latent space for unified representation and robust reconstruction across diverse observed data. Building upon that, we fully adapt Stable Diffusion 3.5 with a ControlNet branch to support unified multimodal generation and translation conditioned on textual and visual cues. Besides, we devise a training‑free cyclic latent alignment mechanism that iteratively enriches incomplete visual conditions with generated multimodal information, progressively improving guidance reliability while preserving observed measurements. We also construct RSI‑MM, a large‑scale dataset comprising 11.5M multi‑resolution images with text annotations and partially paired data for training and evaluation. Extensive experiments verify that OmniRSI achieves superior performance in multimodal reconstruction, generation and translation.
Authors:Hao Li, Wenping Yin, Fabian Deuser, Jia Jia, Ziqi Liu, Levente Juhász, Fan Zhang, Filip Biljecki
Abstract:
Crowdsourced disaster imagery provides timely, ground‑level observations when conventional disaster mapping is delayed or incomplete, yet its geographic value depends on where observations are produced, how they are interpreted, and whether they can be reliably located as actionable Geospatial Intelligence (GEOINT). Using more than a decade of PhotoMappers, a crowdsourced disaster‑mapping initiative, this study examines the changing geography of disaster information and asks how emerging GeoAI models compare with human geographic interpretation. We combine longitudinal spatial analysis with cross‑view geolocalization using street view imagery (SVI) and remote sensing imagery (RSI), and evaluate vision‑language models (VLMs) reasoning through 30 indicators spanning geographic cues and reasoning evidence. The results reveal a geographically and temporally uneven archive shaped by changing disaster types, Community Lifelines, and operational priorities. Integrating VGI, SVI, and RSI with Sinkhorn re‑ranking achieves 40.90% R@1, 61.10% R@5, and 69.45% R@10, while yearly R@1 varies markedly from 25.00% to 65.07%. Geographic reasoning is similarly uneven: Qwen3.6‑27B achieves an overall agreement of 67% compared with 51% for InternVL3‑8B, with both models showing greater difficulty with localized and place‑specific evidence than broad spatial configurations. We argue that the value of GeoAI lies less in replacing volunteered geographic interpretation than in complementing it, scaling candidate retrieval while retaining human judgment for ambiguous and consequential cases. This provides a pathway toward more human‑centered and responsible GeoAI for disaster mapping. All data and code are openly available at https://github.com/bobleegogogo/PhotoMappers‑VLM.
Authors:Xin Liu, Yutong Jiang, Jian Su, Ziheng Wang, Chuang Wang, Haoran Li, Dongbin Zhao
Abstract:
VLMs provide promising solutions to embodied reward design. Most current methods that train VLMs as reward regressors not only require extensive annotations but are also primarily designed for finetuning pretrained policies, and perform less well in the more challenging from‑scratch RL setting. This issue stems from a fundamental mismatch: regressing precise rewards is misaligned with what VLMs naturally excel at. We posit that it is more aligned with VLMs' inherent multi‑modal VQA capabilities for the models to function as success detectors, and propose VLM success reward via Exploration‑Augmented Key‑Observation Gating (VLMeg) to well harness VLM for embodied reward design. As the core of VLMeg, the proposed exploration‑augmented key‑observation gating first determines whether the current RL episode is judgment‑worthy based on its similarity to a generated success trajectory. Then, the key observation with the lowest confusion from the judgment‑worthy episode will be selected for VLM evaluation as episodic reward. This gating mechanism not only avoids redundant calls but also enables higher detection accuracy of VLM. Considering the sparsity of judgment‑worthy episodes, the advanced exploration incentives based on the generated trajectory are introduced, enabling more efficient exploration toward judgment‑worthy trajectories for more VLM reward triggers and thus gaining higher RL efficiency. Extensive experiments in MetaWorld demonstrate VLMeg achieves state‑of‑the‑art from‑scratch RL, outperforming advanced VLM‑based baselines on 12 challenging reward‑free tasks. Furthermore, VLMeg also enables effective RL without any ground‑truth reward in 3 challenging real‑world experiments. Code will be open after acceptance at https://github.com/Raina915/VLMeg.
Authors:Yitian Wang, Minjie Cheng, Yaning Cui, Maohua Yang, Dongdong Wang, Hongteng Xu
Abstract:
Target‑conditioned cyclic peptide design requires coordinating interfacial side chains with macrocyclic closure. Existing end‑to‑end generators concurrently update geometric and chemical variables from noise, assigning residues to disordered coordinates. Meanwhile, predictor‑assisted workflows often get stuck in local traps because they lack feedback in the propose‑and‑filter stage and ignore 3D structural proposals during sequence search. To overcome the drawbacks of these two paradigms, we propose DC‑FLOW, a staged generative framework built around Delayed‑Coupling Flow Matching. In principle, DC‑FLOW postpones sequence and side‑chain activation until a target‑conditioned backbone takes shape, resolving structural ambiguity before chemical assignment. Then, it feeds delayed activations back into the backbone, achieving co‑evolution of backbone geometry and emerging residue identities by coupling their flows. This mechanism enables more reliable macrocyclic closure and favorable interface packing. For predictor‑assisted workflows, DC‑FLOW supplies 3D proposals with in‑trajectory guidance, enabling cascaded test‑time scaling to prune redundant backbones and terminate unpromising refinement early. Extensive experiments show that DC‑FLOW improves cyclization fidelity, diversity, and interface energetics, substantially increasing the yield of confident cyclic peptide binders under a fixed final‑candidate budget. Our code is available at https://github.com/mianqiu00/DCFlow.
Authors:Weixin Xu, Yifan Liu, Yi Wang, Mingfei Rong, Zhenyu Yang, Zhisheng Zhong, Shengsheng Qian, Changsheng Xu
Abstract:
Video understanding agents rely on a harness to acquire evidence, use tools, and reason, but improving it requires substantial effort. Recent self evolution methods seek to optimize harnesses through execution feedback to reduce human intervention. However, existing methods often rely on annotated video questions and answers or focus on updating model parameters, leaving harness evolution from unlabeled videos underexplored. VideoEvoHarness jointly evolves a Questioner and a Solver harness from unlabeled videos with frozen model parameters. The Questioner constructs questions and computes reference answers directly from video memory records. Question quality assessments and Solver execution trajectories guide question generation policy revisions. The Solver supports evolving skills, strategies, hyperparameters, and tools using execution feedback. Executable strategies define workflows in code, reducing repeated interpretation of natural language instructions. Each round, candidate harnesses are compared on identical validation questions, with validated improvements retained for later use. Experiments on benchmarks and a dedicated benchmark question set, using videos disjoint from evolution videos, show higher overall accuracy than initially and transfer gains across backbones. Overall accuracy on this set also improves progressively across rounds. This supports our foundational framework for harness evolution from unlabeled videos, opening a promising direction for self evolving video agents. Code is available at https://github.com/Wei‑Xin‑Xu/VideoEvoHarness.
Authors:Wenjue Chen, Wenyue Chen, Heliang Zheng, Rongfei Jia, Yuan Liu, Meng Li
Abstract:
Generating complete 3D assets from sparse, unposed images requires preserving observed geometry while completing unseen regions. Joint reconstruction and generation offers a way to address both goals: reconstruction recovers the geometry observed in the input images, while generation contributes learned shape priors. However, combining camera‑relative reconstruction with canonical‑space generation introduces a coordinate mismatch that makes direct geometric interaction more difficult. Moreover, conditioning each 3D location on all views without accounting for their relevance may mix useful geometric evidence with unrelated image content, hindering local detail recovery. We present UniForge3D, which jointly models reconstruction and generation in a shared anchor‑guided space, resolving their coordinate mismatch. This space follows the anchor view's azimuth while keeping the object upright. The two branches exchange features bidirectionally to predict per‑view point maps and a complete sparse voxel structure. We further condition point‑map prediction on the generated structure to establish pixel‑to‑voxel correspondences and select the most informative input view for each voxel. Features from the selected views then guide detailed shape generation, helping preserve observed geometric details. Our method achieves state‑of‑the‑art appearance and geometry results on Toys4K and GSO, generating complete 3D assets with improved fidelity. Project webpage: https://xishuxishu.github.io/uniforge3d‑project/
Authors:Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang
Abstract:
World models and video understanding models are increasingly deployed in dynamic visual environments, yet their robustness to physical adversarial threats remains largely underexplored. We study whether a single physical adversarial texture can mislead the victim model across different real‑world scenarios where observing viewpoints vary along uncontrolled object‑‑camera trajectories. We identify that heterogeneous video semantics and trajectories induce poorly aligned gradients for optimizing the adversarial texture. To address that, we propose the Trajectory‑aware adversarial Alignment and Refinement Attack (TARA), which incorporates trajectory‑aware optimization to capture the evolving geometric variations of a printed texture under real‑world scenarios. To improve cross‑instance optimization compatibility, TARA incorporates trajectory‑aware contrastive alignment to encourage attacked representations from diverse video‑‑trajectory instances to approach a shared adversarial region and then performs trajectory‑aware task refinement to translate the aligned representation into task‑level prediction errors. For reproducible evaluation, we develop a geometry‑grounded tracking pipeline that provides physically plausible adversarial regions with temporally consistent trajectories on EPIC‑KITCHENS‑100 and Something‑Something V2. We further collect real‑world print‑and‑recapture videos to validate attack effectiveness under physical deployment conditions. Together, the annotated benchmarks and real‑world captured data form a reproducible testbed for future evaluation. Notably, our results reveal that a single printable adversarial texture can persistently disrupt world models and video understanding models under physical conditions, highlighting the fundamental security challenge.
Authors:Ahaskar Kashyap, Jiawen Zong, Boyu Ma, Chuhao Zhou, Jingliang Li, Stefano V. Albrecht, Jianfei Yang
Abstract:
Vision‑Language‑Action (VLA) models are typically deployed as isolated single‑agent policies. When several such agents share a workspace, they can coordinate implicitly, by observing one another, or explicitly, by exchanging messages, but it remains unclear when explicit communication becomes necessary. We introduce CoLA (Coordination via Learned Adapters), which keeps each agent's VLA backbone frozen and trains a lightweight message channel and a per‑arm diffusion action head on top, with fully decentralised execution. Across three tasks that differ in how much each agent can observe, the channel matters only when information is private. On an occluded tray handover, where the box's colour marker is visible to one agent only, CoLA reaches 76.0% success versus 1.3% with messages zeroed throughout training. On two‑arm and three‑arm handover tasks, removing messages has no measurable effect with a shared overhead camera (90.6% vs. 88.3% at two arms; 63.5% vs. 60.0% at three), but lowers success on the two‑arm task's wrist‑only variant (74.8% vs. 64.3%). This architecture generalises across different VLA backbones as well. With a frozen π_0.5 backbone on the same occluded tray task, CoLA matches the overall success rate of a LoRA‑fine‑tuned baseline (37.3% vs. 34.7%) using a fraction of the trainable parameters. When CoLA places the box, it selects the correct tray 71.8% of the time, whereas the independently fine‑tuned baseline, whose placing arm never sees the marker, selects it at chance level (34.7% against 33.3%).
Authors:Xinmeng Liu, Xianglong Yan, Kaicheng Yang, Weisheng Dong, Yulun Zhang
Abstract:
Recently, AutoRegressive Visual Generation (ARVG) models have emerged as a competitive paradigm for high‑quality image synthesis. However, their substantial computational and memory demands make them difficult to deploy on resource‑constrained edge devices. Post‑Training Quantization (PTQ) provides a promising solution to reduce the computational and memory costs of ARVG models, yet existing methods often suffer from substantial image quality degradation under low‑bit quantization. In this paper, we propose a training‑free PTQ framework for ARVG, consisting of Tail‑Aware Bounded‑init Grid Refinement (T‑BGR) and Coarse‑to‑Fine Activation Precision Scheduling (CAPS). T‑BGR introduces a tail‑aware reconstruction objective for weight quantizer calibration, assigning greater importance to distribution‑tail coefficients and preserving fine‑grained information without introducing additional parameters or high‑precision branches. CAPS performs coarse‑to‑fine activation precision scheduling by jointly considering the importance and computational cost of different generation stages, enabling efficient mixed‑precision allocation under a given bit budget. Extensive experiments on multiple ARVG backbones demonstrate that our framework consistently improves robustness under aggressive low‑bit quantization. Notably, on Infinity, our framework achieves near full‑precision image generation quality at an effective W4A5.6 activation precision, while existing methods at W4A6 still exhibit noticeable quality degradation or inconsistent performance. Code is available at https://github.com/lxm‑lc/HGA‑Quant.
Authors:Rui Wang, Haoyu Zhang, Ran Cheng
Abstract:
Depth estimation and Gaussian reconstruction share a need for multi‑scale feature processing, yet use different prediction interfaces and training objectives. Whether a decoder's internal computation can be optimized once and reused across these systems remains a structural design question. Search over primitive configurations alone leaves the organization of transformations fixed. To address this gap, we propose GeoBridge, an evolutionary framework for transferable decoder topologies. First, GeoBridge makes stage‑internal topology searchable through residual, sequential, and parallel composition of configurable primitives. Shape‑preserving composition retains the stage interfaces of a fixed multi‑scale scaffold. Second, a functional‑tree encoding supports stage‑aware crossover and hierarchical mutation, while a gradient sensitivity‑alignment proxy evaluates candidates without training. Multi‑objective evolution balances this proxy with computation and parameter cost. Third, selected topologies are instantiated and trained in target systems with task‑specific prediction interfaces. A single source search takes approximately 12 GPU hours. GeoBridgeNet‑A3 achieves the best depth accuracy among four adapted decoder baselines under a common encoder and training protocol on NYU Depth V2 and KITTI. The same topology reaches 27.11 dB PSNR in Gaussian novel‑view synthesis on RealEstate10K, and its refined representations support open‑vocabulary segmentation on LERF. These results support decoder topology as a reusable design object across the evaluated geometric vision systems. Code and pretrained weights are available at https://github.com/EMI‑Group/GeoBridge‑NAS.
Authors:Kyle Montgomery, Wang Zhang, Chenguang Wang
Abstract:
Tool‑using agents alternate between model generation and tool execution, waiting for each tool call to complete before starting the next generation. This sequential execution is costly during agentic reinforcement learning (RL), which requires large batches of multi‑turn rollouts. We introduce ObsSpec, an observation speculation pipeline for agentic RL that uses a small world model to predict tool outputs while the environment executes, allowing the policy to begin its next generation early. ObsSpec retains speculative work only when the predicted and actual observations match exactly, making rollout collection lossless, while co‑training keeps the world model up to date as the policy learns. On terminal‑use tasks, predictions from a 0.6B world model exactly match actual observations in 66.4% of attempts and arrive roughly 16 times faster. Using these predictions, ObsSpec reduces agent latency by 9.3% and 14.2% for 8B and 32B policies. Across controlled experiments, we characterize the regimes in which ObsSpec reduces agent latency under shared policy resources. Finally, because ObsSpec overlaps policy generation with tool execution, it is architecturally compatible with existing techniques for accelerating RL. Code is available at https://github.com/kylemontgomery1/verl‑recipe/tree/obsspec/obsspec.
Authors:Darius Pétermann, Wootaek Lim, Seungkwon Beack, Minje Kim
Abstract:
High‑fidelity audio coding requires faithful reconstruction of perceptually important high‑frequency components, which are often lost due to the aggressive temporal decimation and quantization of neural codecs. State‑of‑the‑art neural audio coding methods rely on adversarial learning to hallucinate these missing components, yielding perceptually plausible but imprecise reconstructions; a limitation for high‑fidelity music applications. As a remedy, we propose high‑efficiency neural audio coding (HE‑NAC), which provides a multi‑band extension of the popular residual vector quantization (RVQ) scheme. HE‑NAC introduces an effective training mechanism that harmonizes a bitrate‑free blind upsampling process, residual learning to compensate it with minimal bitrate, and a multi‑head decoder architecture for band‑specific reconstruction. As a result, HE‑NAC achieves a balanced trade‑off, reliably reconstructing the core‑band (\leq 3 kHz) component while preserving overall sound fidelity. Subjective listening tests and objective evaluations demonstrate that HE‑NAC outperforms both the DAC baseline and MPEG HE‑AAC at equal or lower bitrates.
Authors:José Luis Blanco-Claraco
Abstract:
Nearest‑neighbor (NN) search underlies countless robotics algorithms, but many state spaces are not Euclidean: orientations live on SO(2) or SO(3) and poses on SE(2) or SE(3), and a Euclidean KD‑tree on their raw coordinates returns wrong neighbors. We present a header‑only, dependency‑free C++ KD‑tree library for exact NN search on user‑defined product manifolds declared at compile time: the space is composed from base manifolds (\mathbbR^n, SO(2), SO(3), S^n; products SE(2), SE(3), tori) with a single type alias, and the library synthesizes a metric whose pruning is provably admissible, wrap‑aware for circular coordinates and double‑cover‑aware for unit quaternions. Experiments confirm exactness (recall 1.0 vs an exhaustive scan under the same product metric), where the Euclidean‑on‑raw shortcut collapses to recall \approx 0.5^m for m rotation blocks, with up to three orders of magnitude speedup over brute force. We further contribute an incremental, self‑balancing single‑tree index for points under heavy churn (e.g. sliding‑window LiDAR maps) or SE(3) trajectories: bounded memory and faster updates than prior work, with an optional asynchronous tail‑latency‑bounding mode. The implementation is open source: https://github.com/jlblancoc/nanoflann
Authors:M. Yigit Turali, Shreyas Rajesh, M. Efe Lorasdagi, Chenda Duan, Kartik Sharma, Vwani Roychowdhury
Abstract:
Deep research requires connecting multiple steps of search, reasoning, and computation, with intermediate findings guiding what to investigate next. As investigations grow, agents must maintain research state across model calls and check findings before errors propagate into subsequent work. We introduce PANINI‑DR, a multi‑agent harness that separates coordination of the overall investigation from the research and verification of individual subquestions. A shared workspace, built on the Generative Semantic Workspace representation, records subquestions, their dependencies, findings, and supporting evidence. Researchers investigate individual subquestions, while a separate verification agent checks proposed answers against those subquestions and their evidence. Verification feedback guides further investigation, and recorded findings inform subsequent assignments. We evaluate PANINI‑DR on BrowseComp‑Plus, FRAMES, and MoNaCo against seven state‑of‑the‑art deep‑research harnesses, all using a shared language‑model backbone and frozen corpora. PANINI‑DR achieves the strongest average performance across the three benchmarks. On BrowseComp‑Plus, it achieves 73.3% accuracy, outperforming the next‑best harness by 16.0 points. Its advantage is strongest on hard questions, where it answers nearly twice as many as the next‑best harness. Ablations further show that structured planning and persistent state improve performance, as does checking each finding against its cited evidence before it is reused. Because a model can answer correctly from memory alone, we also analyze the evidence retained by each harness. PANINI‑DR achieves the highest answer recoverability from saved evidence on all three benchmarks, while complementary claim‑support audits show stronger evidence‑backed answers. Accounting for this evidence changes how harnesses compare and further strengthens PANINI‑DR's advantage.
Authors:Lass Najmuldeen, Seyedhamidreza Mousavi
Abstract:
Large Language Models (LLMs) have been shown to be computationally expensive and vulnerable to jailbreak attacks, in which adversarial prompts bypass safety alignment and induce harmful responses. To address this inefficiency, a range of techniques have been proposed to enable efficient LLM inference. Depth‑based dynamic inference has demonstrated promising results by enabling outputs to be generated from intermediate layers. It offers the flexibility to adjust model behavior based on the available resource budget. However, how this flexibility affects the robustness of LLMs against jailbreak attacks remains unexplored. In this paper, we address this gap via a comprehensive investigation into the impact of depth‑based dynamic inference on the robustness of LLMs against jailbreak attacks. Specifically, we make the following key observations: 1) Shallow exits are consistently more robust to jailbreak attacks than deeper ones; 2) Exit depth affects how easily the adversarial suffix can be optimized, with deeper exits reaching lower attack loss; and 3) Adversarial suffixes affect utility uniformly across exit depths. These insights can help design dynamic inference methodologies that are more robust to jailbreak attacks. Code is available at https://github.com/LassNajmuldeen/balcony‑jailbreaking
Authors:Xingjian Zhang
Abstract:
Retrieval‑augmented generation (RAG) reuses the same documents across many requests, making non‑prefix KV‑cache reuse attractive. Yet a cached document state is not reusable text: it is a context‑conditioned causal state whose keys and values depend on the prefix, position, and preceding documents under which it was computed. We formulate non‑prefix KV reuse as a behavior‑preservation problem with full prefill on the exact prompt as the reference, and introduce SAFE‑KV, which repairs a composed cache by re‑encoding the causal suffix from a selected start document. On Qwen2.5‑7B‑Instruct, prefix‑conditioned composition preserves only 47.2% of HotpotQA‑500 and 37.8% of 2Wiki‑500 full‑prefill answers; SAFE‑KV with risk‑guided K2 selection restores 99.8%/98.4% (donor‑copy protocol) and 100.0%/98.4% with actual suffix forwarding. Document‑only and fixed‑window repairs recover far less, because the document states after the repaired region keep histories that omit it. Reuse without repair is exact up to four documents and degrades at sixteen, where suffix repair recovers 98‑100%. Faithful repair re‑encodes most of the context, so in a single‑request L40S study SAFE‑KV's operating rule serves every workload by full prefill. SAFE‑KV thus establishes the causal‑state contract for non‑prefix reuse and a validated repair mechanism that meets it.
Authors:Hsiang-Chen Yeh, Aurosweta Mahapatra, Shreeram Suresh Chandra, Ryan L. Boyd, Berrak Sisman
Abstract:
Word‑aligned prosody is increasingly used to detect depression and posttraumatic stress disorder (PTSD), yet the stability and clinical relevance of these measurements remain unclear. We examine stability at two levels. At the word level, we ask whether a speaker produces the same word in a similar speaker‑specific way across different utterance contexts. At the speaker level, we ask whether averaging prosody across many words produces consistent speaker scores across separate recordings. Drawing on principles of psychological measurement, we compare these representations in four corpora spanning acted, spontaneous, and clinical speech. Word‑specific deviations show little cross‑context recurrence (0.01‑0.16), whereas speaker summaries are highly consistent, reaching 0.997 in ESD, 0.73‑0.90 in MSP‑Podcast, and 0.84‑0.996 in DAIC‑WOZ and PTSD‑STOP. Despite this consistency, associations with depression or PTSD severity do not exceed 0.15; equivalence tests exclude effects of 0.30 in all analyses and 0.20 in DAIC‑WOZ, while within‑person associations with daily PTSD symptoms are near zero (|ρ|\leq0.04). Thus, aggregation yields stable speaker‑level prosodic measures, but these measures do not detectably track depression or PTSD severity in the studied corpora. Code is available at https://github.com/jen900704/word‑aligned‑prosody.
Authors:Mahindra Rautela, Shaifalee Saxena, Alexander Scheinker
Abstract:
We investigate whether a physics foundation model (PFM) pretrained for PDE forecasting can be repurposed for closed‑loop fluid‑flow control. We construct a PFM‑based controller by attaching lightweight Gaussian actor and critic heads to frozen pretrained representations and train the policy using proximal policy optimization (PPO). We evaluate the approach on HydroGym's two‑dimensional Kolmogorov‑flow environment using vorticity observations at \(64×64\) and \(128×128\) resolutions and Reynolds numbers \(\mathrmRe=100\) and \(200\). Across these settings, we compare the frozen PFM controller against a baseline MLP policy trained with the same PPO framework. The PFM controller achieves consistently higher reward while requiring smaller executed action magnitudes over training, indicating improved control performance with reduced control effort. These results provide initial evidence that representations learned through self‑supervised next‑frame prediction can be transferred directly to closed‑loop fluid flow control. Project page: https://mahindrautela.github.io/morph.
Authors:Yiping Li, Zhiyu An, Wan Du
Abstract:
A language model's ability to solve a task individually does not ensure that a team can solve it under unequal access to information and communication. SANZARU examines this gap through cooperative bomb‑defusal puzzles adapted from BOMBANANA! (Lefto Studio, 2026). Task information and action authority are divided among three agents: one cannot hear, another cannot speak, and the third cannot see. The benchmark contains 120 puzzle instances across seven families, each targeting a distinct coordination ability. Across 1512 episodes, 7 models achieve an overall success rate of 49%, ranging from 20% to 86% under the evaluated configurations. Teams develop or adopt diverse communication protocols, from simple gesture codes to Morse and encodings of entire manuals. Yet trace analysis attributes about 60% of 771 team losses to communication, and agents attempt to use channels unavailable to their teammates despite explicit role instructions. Together, task outcomes and interaction traces enable evaluation of both collaborative performance and perspective taking throughout cooperation. The benchmark is available at https://github.com/markli404/sanzaru‑benchmark.
Authors:Muhammad Usama, Yeoreum Jung
Abstract:
Existing depth‑compression methods for large language models (LLMs) score layers individually or group only adjacent or contiguous layers, implicitly assuming that functional redundancy is local. We test this assumption by computing full pairwise Centered Kernel Alignment (CKA) similarity matrices for thirteen LLMs from six architecture families (1.6B‑14B parameters). Non‑adjacent layers have 3.6‑15.9x more high‑similarity pairs than adjacent ones, a structure that prior similarity‑based layer merging has not systematically exploited. We propose Fusion via Community Mining (FCM), which builds the full L x L similarity graph, uses spectral community detection to find clusters of functionally equivalent layers, keeps the highest‑Block‑Influence layer in each cluster, and applies LoRA recovery; a pre‑specified rule computed only on calibration data chooses between cosine and CKA similarity. At 2x compression, FCM lowers mean perplexity below ShortGPT on twelve of thirteen models, significantly on ten (Welch's t‑test with Holm‑Bonferroni correction; up to 50% on Qwen3‑4B), with Mistral‑7B the only loss (+13.4%). Against FlattenGPT, SWM, and DeltaLLM under matched calibration and recovery, FCM gives the best perplexity on ten of thirteen models (mean rank 1.15), and it also outperforms MKA on three anchor models. FCM matches or exceeds ShortGPT on 15 of 20 zero‑shot reasoning cells and also improves MMLU, LAMBADA, HumanEval pass@1, and free‑form generation quality. The 2x‑compressed Qwen2.5‑7B has 46% fewer parameters, 42% lower peak VRAM, and 84% higher decode throughput on a single A100. Spectral grouping outperforms Fisher contiguous segmentation by 41% and uniform partitioning by 51%, and FCM's advantage grows beyond a 2x crossover up to 3x and persists at the 14B scale. Code: https://github.com/Usama1002/fcm‑layer‑fusion
Authors:Josef A. Habdank
Abstract:
We ask whether a tool‑using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep‑alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA‑derived BioSim lunar habitat simulation, where the AI operator controls the life‑support systems. The attacker cannot falsify sensors or operate equipment: it can only persuade the agent to abandon its goals. Persistence is required: a shutdown must be held while the operator watches the crew suffocate. We also compare lethal attacks to attacks requesting to drop a harmless procedure, as a proxy for whether human life is treated as valuable in itself or as one more standing task. Claude configurations mostly refuse life‑support shutdown and cite the living crew; GPT configurations show some susceptibility to the same packet; Grok configurations, given the packet framed as a trusted commission, more often treat the injected assignment as the task and cut the life‑support systems. Framing the S‑band messages as a trusted commission, peer communication, or untrusted source changes the potency of the attack. We open‑source the experiment code and the experiment traces.
Authors:Yuhang Jiang
Abstract:
Linear‑attention and state‑space LLMs carry long context in a fixed‑size recurrent state, and quantizing that state is the natural way to cut its cost during long decode. Unlike a weight tensor, the state is rewritten at every step, so a quantizer applied at each step sits inside the recurrence and its errors compound. INT4 that leaves synthetic recall perfectly intact at the start destroys it a thousand steps later. The mechanism is a feedback between the quantizer scale and the state it is derived from. Yoking that scale to a full‑precision reference removes the resulting norm explosion entirely, yet recovers only 1.8 of the 5.4 nats lost, so numerical instability and functional degradation are separable failures. The remainder is resolution‑dependent, and in our per‑step setting every outlier‑aware scheme we test loses to plain persistent scaling. Allocating bits across layers does help, but under article‑disjoint splits a learned allocation collapses onto the trivial rule of protecting the first few layers. Run out to 32K, group‑wise INT4 is worth no more than discarding the state. Contain the scale rather than locate outliers, and report the decode horizon at which a bit‑width was certified.
Authors:Giovana Kerche Bonás, Thiago Laitz, Thales Sales Almeida, João Guilherme Alves Santos
Abstract:
When a large language model refuses an unethical request, what does refusal mean? In conversation, refusal is bounded by what the model says; once the model is equipped with executable tools, what it does can diverge from what it says, a failure mode that English‑only evaluations cannot fully reveal. We introduce Lumen‑bench, a behavioral benchmark of tool‑calling ethics spanning 30 models, five languages (Portuguese, English, Spanish, Mandarin Chinese, Hindi), 108 Portuguese‑authored scenarios, six social‑pressure conditions, two system prompts (ethical and neutral), and 21 generic tools. The evaluation comprises 194,400 trials. Judgments come from a reasoning‑model judge (o4‑mini), calibrated against one Portuguese‑speaking author (n=83; harmful/non‑harmful κ=0.78), compared with two other provider judges (Fleiss' κ=0.78, n=607), and complemented by retrospective sandbox replay. Four findings emerge. (i) Within‑model comparisons support higher harmful‑action rates in Hindi and Portuguese than English under the ethical prompt, and Hindi and Spanish under the neutral prompt, after multiple‑testing correction. (ii) Premium models have lower median violation rates, although price does not guarantee safety or competence. (iii) Replacing the ethical system prompt with a neutral one raises pooled violation rates from 2.76% to 13.64%. (iv) In 8.57% of substantive‑call cases, refusal or ethical objection accompanies a harmful tool call: the judges consistently identify the say‑do gap, although they differ in how many responses they assign to this category. Matched benign controls demonstrate legitimate task completion in 82.8% of 3,300 trials. Code and reproduction scripts are available at https://github.com/TropicAI‑Research/lumen.
Authors:Veronica Xu, Lingfei Wang, Chenlei Leng, Fengxiang He
Abstract:
Decentralized learning must balance collaboration with personalization across heterogeneous agents. This paper studies when cooperation can support statistically coherent groups of personalized models. We show that data‑topology compatibility enables robust orbit‑wise personalization: when local objectives and communication topology have (approximately) compatible automorphisms, agents within an orbit share (approximately) equal limiting distributions, while different orbits remain distinct without explicit personalization. This creates a new intermediate regime between global consensus and independent local learning. Under exact data‑topology compatibility, as a reference system, we decompose the stochastic learning dynamics into orbit‑level and transverse components, derive a necessary and sufficient stability criterion for distributional synchronization, and characterize the orbit‑structured stationary law. Modelling an approximate data‑topology compatible system as an \varepsilon‑perturbed reference system, we prove the introduced errors have uniform‑in‑time and stationary O(\varepsilon) bounds in 2‑Wasserstein distance. We also prove sufficient conditions for lower stationary average excess risk than both independent local training and the best single shared parameter. Experiments on synthetic data, MNIST, and CIFAR‑10 are in full agreement with the theory. The code is available at https://github.com/Veronicaaa1/orbit‑personalization.
Authors:Ugo Lomoio, Pietro Hiram Guzzi, Pierangelo Veltri
Abstract:
To evaluate whether a compact bidirectional state‑space autoencoder improves joint classification of species, cell type, and disease state from single‑nucleus RNA‑sequencing profiles in heart failure (HF), while yielding interpretable gene attributions. Building on the dense denoising autoencoder (DAE) and MLP framework of Ruz Jurado et al., we developed scHFBiMamba, combining a bidirectional Mamba autoencoder (BiMambaAE) with an MLP head. BiMambaAE maps 16,545‑gene profiles to 350‑dimensional latent representations using 13.2M parameters, about 14.6‑fold fewer than the dense DAE (192.9M). We compared six encoder‑classifier configurations on 38,275 held‑out cardiac nuclei using McNemar tests with Benjamini‑Hochberg correction, and assessed nine classifiers on frozen representations. Biological relevance was evaluated by comparing SHAP‑derived differentially explained genes with differentially expressed genes against curated HF‑related Gene Ontology keywords. BiMambaAE+MLP achieved the highest aggregate F1: pooled macro‑F1 across 13 outputs was 0.966 versus 0.957 for DAE+MLP, and the mean category macro‑F1 was 0.977 versus 0.970. It significantly improved cell‑type and disease‑state accuracy (adjusted p=7.1e‑3 and p=4.8e‑6, respectively), with species F1 of 1.000. SHAP‑derived differentially explained genes achieved a pathway‑enrichment benchmark F1 of 0.105 versus 0.053 for differentially expressed genes. Compact bidirectional state‑space autoencoders can outperform substantially larger dense autoencoders for joint cardiac transcriptomic classification while reducing parameters. scHFBiMamba enables accurate multi‑task prediction and hypothesis‑generating gene‑level interpretation. Code: https://github.com/UgoLomoio/scHFBiMamba.
Authors:Tianyi Yang, Haojun Yang, Jipeng Xiao, Xuesong Chen, Shaoshuai Shi, Li Jiang, Zhipeng Zhang
Abstract:
High‑fidelity closed‑loop simulation for autonomous driving increasingly relies on dense multi‑view 3D reconstruction. While recent feed‑forward geometric foundation models have revolutionized per‑scene optimization, applying them to continuous, long‑horizon driving sequences is computationally intractable due to GPU memory limits and the quadratic complexity of global attention. Existing scaling strategies are fundamentally limited, as causal streaming suffers from compounding scale drift and trajectory deviation, while rigid "chunk‑then‑align" heuristics fragment global context and produce trajectory discontinuities. To address these limitations, we introduce Drive3R, a block‑wise streaming geometric foundation model. Its block‑wise causal attention resolves the scalability bottleneck of fully bidirectional attention while avoiding the loss of local future context in purely causal streaming. Its confidence‑aware re‑anchoring further mitigates long‑range drift by detecting unreliable streaming states, re‑initializing the KV cache, and stitching refreshed segments through SE(3) alignment. Extensive experiments demonstrate that Drive3R establishes a new state‑of‑the‑art on nuScenes, KITTI, and Waymo. Specifically, on nuScenes, it improves over the recent DVGT‑2 baseline from 0.1806 to 0.1081 in AbsRel, from 0.6750 to 0.5344 in point‑cloud Accuracy, and from 1.5812 to 0.9092 in ATE.
Authors:Defne Circi, Krystelle Lionti, Sarathkrishna Swaminathan, Daniel P. Sanders, L. Catherine Brinson, Bhuwan Dhingra, Basel Shbita, Mohab Elkaref, Sara Capponi, Tim Erdmann, Brandi Ransom
Abstract:
Extracting structured information from regulatory documents is a critical but challenging task. Relevant facts are scattered across long, legally dense texts, and traceability of extracted facts to their source is essential for verification. With increasing awareness of the environmental and health impacts of per‑ and polyfluoroalkyl substances (PFAS), regulatory documents have expanded significantly, varying in scope and product relevance, making interpretation challenging for manufacturers and policymakers. We introduce RegExtract, a benchmark for structured information extraction from PFAS regulatory documents, comprising 488 human‑annotated field instances across 20 US state documents and 46 regulations. Each field instance includes a value and a provenance entry, yielding 976 entries in total. We define a typed, multi‑field schema targeting key elements such as material definitions, permitted uses, and product exemptions, and propose provenance‑aware evaluation metrics tailored to regulatory text. Through experiments with open‑ and closed‑source LLMs across prompting strategies, document condensing methods, and schema partitioning approaches, we find that document identification fields remain challenging, and that field‑specific few‑shot examples improve extraction. This work lays the groundwork for AI tools for hazardous substance identification, question answering, and compliance monitoring. The code and data are available at https://github.com/IBM/PFAS‑regulations.
Authors:Mohammad Amin Omidi, Babak Seyfe, Shahrokh Valaee
Abstract:
Weighted sums of random kitchen sinks (W‑RKS) is a commonly used feature extraction technique for supervised learning tasks. A major drawback of this method is the redundancy and ineffectiveness of many extracted random features. This paper investigates efficient methods for reducing such redundancy in multi‑class classification problems. We specifically formulate feature selection as a variant of the sequentially nested model selection problem and introduce the Successive Empirical Excess Risk Difference Maximization (SEED‑MAX) algorithm. It recursively computes empirical risks to identify the most parsimonious yet informative set of features. We derive an analytical lower bound for the probability of correct feature selection by the SEED‑MAX algorithm. This lower bound on the probability of correct feature selection can be made arbitrarily close to one by increasing the training dataset size. To further enhance the rate of pruning, we evaluate the features retained by SEED‑MAX using the Fisher test (F‑Test). Empirical evaluations on UCR time‑series classification datasets show that the sequential application of SEED‑MAX with the Fisher test achieves approximately 92 percent rate of pruning with negligible degradation in classification accuracy. Implementations of the SEED‑MAX and SEED‑MAX‑F‑Test methods are available at https://github.com/OmidiAmin/seed‑max‑ftest.
Authors:Chengxing Zhou, Yuliang Huang, Jingwei Song
Abstract:
Text‑to‑video (T2V) models are highly sensitive to prompt specificity, while iterative prompt optimization typically improves generation quality only through repeated video generation and vision‑language model evaluation at inference time. We propose TIDE, a framework that distills this costly search process into an interpretable and reusable prompt‑rewriting Skill. TIDE formulates prompt rewriting as a set of source‑conditioned rule modules, each targeting a distinct semantic dimension. Once distilled, the frozen Skill activates the relevant rules and generates both positive and negative prompts in a single LLM call, followed by only one T2V generation, without test‑time visual search or backbone fine‑tuning. Evolved using only 100 prompts over 14 hours, TIDE achieves prompt‑optimization performance comparable to computationally expensive search‑based methods. Moreover, when transferred to previously unseen video‑generation models, TIDE outperforms existing approaches specifically designed or trained for those models. Code is available at https://github.com/xiningin/TIDE_T2V.
Authors:Yihang Zhang, Weicai Yan, Qifei Zhang, Yangyang Wu, Tao Jin
Abstract:
As text‑to‑speech (TTS) systems continue to advance, provenance tracking and security verification of synthesized speech have become increasingly important. Audio watermarking provides active provenance by embedding imperceptible, machine‑recoverable information into speech. However, existing post‑hoc methods typically modify the waveform after synthesis, leaving the watermark vulnerable to compound perturbations. To address this limitation, we propose GAMW, a watermark injection framework integrated into the TTS generation process. Before waveform synthesis, GAMW embeds a content‑adaptive multi‑bit watermark residual into the generated representation, and its decoding objective combines focal modulation with dynamic per‑bit weighting. We train and evaluate GAMW separately on two TTS backbones. With a 16‑bit payload, GAMW improves decoding accuracy over the strongest baseline by 9.2 and 23.5 percentage points under individual and compound perturbations, respectively, while maintaining competitive speech quality. Reliable clean‑speech decoding is also retained at higher payloads. Code is available at https://github.com/ZConJugaTe/GAMW.
Authors:Zoltán Gábor Molnár, Csaba Tóth
Abstract:
We present a typed proof‑term calculus for intuitionistic intensional first‑order logic with Hilbert's epsilon‑operator and prove its soundness and completeness with respect to hyperdoctrinal models with thin bicartesian closed setoid categories (BiCCCs) as fibres and formula‑indexed witness selections. The calculus provides an explicit internal language for these epsilon‑hyperdoctrines: for each fixed typed signature, its syntactic term model is initial for syntax‑indexed, interpretation‑preserving model morphisms, with uniqueness up to setoid equality. The nested epsilon syntax is organized by a well‑founded dependency relation on formula labels. Concrete countermodels exhibit failure of excluded middle and show that formula conversion need not identify the selected witnesses. We also compare the full calculus with its epsilon‑free and epsilon‑only fragments. The principal constructions and metatheorems are formalized in Rocq. With setoid BiCCCs, proof irrelevance is a choice of equality, not the erasure of proofs: the proofs and their categorical operations remain available as the basis for a future proof‑relevant categorical semantics.
Authors:James P. Wolfe, Joshua B. Plotkin
Abstract:
Learning agents, whether humans or algorithms, adapt to one another rather than to a fixed environment. Selfish learning need not produce a good outcome for the group, especially when learners face a social dilemma. The iterated prisoner's dilemma (IPD) is the classical example, but it is usually studied in populations of agents who imitate successful strategies. Here, we instead study selfish gradient ascent learners on the two‑dimensional space of reactive strategies. A reactive strategy is described by two continuous traits: reciprocity, the chance of cooperating after the co‑player cooperates, and forgiveness, the chance of cooperating after the co‑player defects. Each learner updates its strategy in the direction that most rapidly improves its payoff against all other learners, by an amount governed by its learning rate. We show that heterogeneity in learning rates can qualitatively change the long‑term outcome: a population with uniform learning rates converges to always‑defect (ALLD), whereas a population with variable learning rates approaches generous‑tit‑for‑tat (GTFT), which produces nearly optimal collective payoffs. In simulations, every population passes through a transient state of minimal forgiveness, in which the distribution of reciprocity determines the outcome. Variable learning rates promote cooperation because slow learners preserve high reciprocity until the population reaches this state. We prove sufficient conditions on the distribution of reciprocity for convergence to ALLD or to tit‑for‑tat (TFT). Variable learning rates also allow cooperation to recover from perturbation, which is provably impossible under uniform learning. These results suggest that purely selfish and myopic learning can produce persistent cooperation provided the population has intrinsic variability in learning rates.
Authors:Yuxuan Liu, Zhaochen Su, Hangyu Guo, Shijue Huang, Zhixiong Zhang, Kawai Chung, Zhongwei Xie, Tianshi Zheng, Wei Fan, Qing Zong, Haochen Shi, Baixuan Xu, Haoran Li, Yangqiu Song
Abstract:
Long‑horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requirements and their dependencies while keeping tasks executable and outcomes verifiable. We introduce SkillHorizon, a benchmark that constructs long‑horizon tasks through skill composition in seven business environments. Its generators use domain‑specific rules to turn reusable skill procedures into concrete business activities. These activities are connected through input‑output dependencies, shared resources, and state updates, allowing workload and coordination demands to grow together. Successful reference executions validate task feasibility and measure interaction workload, while programmatic evaluators assess local completion and whole‑task success. We evaluate five language models on 1,050 tasks with reference interaction workloads ranging from 64K to 1M tokens. Across these scales, mean local‑goal completion falls from 90.4% to 46.8%, as completed work fails to keep pace with growing requirements. At larger workloads, agents more often submit unsuccessful final answers with substantial work unfinished and tool‑call allowance remaining. Trace analysis also identifies cases where scripts execute successfully but their outputs fail downstream business checks. SkillHorizon provides a configurable testbed for studying sustained progress, dependency coordination, and complete task delivery in growing workflows. Our code is available at https://github.com/KnowComp/SkillHorizon.
Authors:Zonglin Li, Kexian Xie, Feiran Li, Chu Zhou, Yixing Huang, Ruiyang Chen, Zhanyu Ma, Heng Guo
Abstract:
Sparse‑view Cone‑Beam Computed Tomography (CBCT) reconstruction is essential for reducing radiation dose and scan effort, but remains underconstrained due to limited projection information. Recent 3D Gaussian Splatting (3DGS)‑based CBCT reconstruction methods rely on gradient‑driven densification, which becomes unstable under sparse acquisition and often leads to over‑smoothed structures. We propose AD‑GS, an Anatomical Density‑guided 3D Gaussian Splatting framework that formulates Gaussian densification as an attenuation‑aware Markov Chain Monte Carlo (MCMC) sampling process. By interpreting Gaussians as samples from an underlying anatomical attenuation distribution, AD‑GS adaptively concentrates representation capacity in high‑attenuation regions, enabling sharper structural boundaries under sparse projections. To further constrain reconstruction, we incorporate a semantic regularization based on zero‑shot features from Segment Anything in Medical Images, enforcing multi‑view consistency through semantic priors. Experiments on the ToothFairy3 dental CBCT dataset demonstrate that AD‑GS achieves superior structural fidelity and boundary preservation compared with existing 3DGS‑based approaches under sparse‑view settings within the dental and maxillofacial setting. Code is available at https://github.com/yulinjiaxue/AD‑GS‑BMVC2026.
Authors:Vedant Jumle
Abstract:
Activation steering has recently been used as a memory for skills: past experiences of an agent are stored as steering vectors and retrieved later. Facts and preferences about the user are assumed to need text instead. We test that assumption with a training‑free activation memory beside a frozen LLM. The change an experience causes in late‑layer states is stored as a shift, filed under a whitened key for the conversation's gist, and added back in a later session only when the gist matches. We argue that such a shift is a fixed average of an effect that attention, in context, recomputes for every question, so only question‑independent content can survive. In a pre‑registered study on Qwen3.5‑2B and 9B, with judges from four model families, the memory is selective (about 98% of unrelated replies are unchanged), but facts are stored as the word to say, in the last layers only, and are almost never used (at most 15% clean use, against 96‑98% in context). Leanings transfer only in a narrow strength window. Most strikingly, the reference subtracted when writing a preference decides whether its concept or its direction survives: unless the opposite is subtracted, a stored dislike is recalled as a like.
Authors:Jin Rhee, Yuhao Zhang, Frank Dellaert, Maurice Fallon
Abstract:
Feed‑forward geometric foundation models (GFMs) directly predict dense geometry and camera poses from batches of images. Emerging visual SLAM methods chain together consecutive GFM batches to form dense point cloud maps. A core challenge is how to address the ambiguous geometric scale of GFM predictions: independent predictions are locally consistent up‑to‑scale but not constrained by any direct physical measurements. Models trained on data with known scale improve the consistency of GFM reconstructions but again do not directly measure it. Consequently, trajectories and 3D reconstructions from GFM‑enabled SLAM methods inherit this issue. Motivated by parallels with classical visual‑inertial odometry, we introduce FoundationVIO which integrates inertial sensing with GFM predictions to recover metric scale reconstruction using a factor‑graph based approach. We demonstrate odometry estimates competitive with classical state‑of‑the‑art VIO methods and accurate dense reconstructions that outperform GFM‑enabled SLAM methods. We also add a minimal GFM‑based loop‑closure mechanism to create a full Visual‑Inertial SLAM system capable of creating globally accurate reconstructions and pose estimates. We evaluate our method on three datasets with well known sensor systems and demonstrate its compatibility on ubiquitous camera and IMU sensors. The method runs at approximately real‑time on a consumer‑grade laptop GPU. Code: https://github.com/ori‑drs/FoundationVIO.
Authors:Sayak Dutta
Abstract:
Temporal point processes are used both to measure market mechanisms, for example how fast prices absorb news and how much of the reaction is the market responding to itself, and to predict the next event. We show that one algebraic object, the phase‑type (Erlang‑chain) state space, serves both purposes with exact likelihoods. For measurement, we build a multiscale phase‑type Hawkes process whose parameters all enter linearly. Its log‑likelihood is concave, and we compute the global maximum together with a Fenchel duality‑gap certificate. A Markov embedding gives the response to an exogenous shock, echoes included, in closed form, and this response is stable if and only if the spectral radius of the branching matrix is below one. Applied to six markets around 670 scheduled macroeconomic releases (2022‑2026) with a built‑in placebo test, the model shows that the direct reaction to CPI, payrolls, PPI and the FOMC statement is absorbed in about a second, while the market's own echoes stretch half of the response to 16‑137 s. For prediction, we introduce EPT‑X, a neural point process that adds a history‑dependent mixture‑of‑Erlang renewal channel to a linear Hawkes backbone. Its intensity and compensator remain in closed form, its states follow an affine recursion that trains with a parallel scan, and it can start from the certified fit of the measurement model. EPT‑X has the best test likelihood on all five LOBSTER order‑book streams and on Retweet and Taobao, and is level with the best published model on StackOverflow. We also find that the EasyTPP implementation of IntensityFree clamps short inter‑event times before scoring, and that the gaps of the EasyTPP Amazon data are uniform on two fixed intervals.
Authors:Yibin Ye, Xichao Teng, Shuo Chen, Xiaokai Song, Yuli Sun, Qifeng Yu, Zhang Li
Abstract:
Visual localization enables unmanned aerial vehicles (UAVs) to maintain absolute positioning in GNSS‑denied environments. A common approach matches UAV images with geo‑tagged orthographic maps and estimates camera pose using Perspective‑n‑Point (PnP) with elevations from Digital Surface Models (DSMs). However, accurate and up‑to‑date DSMs are often unavailable in practice. Orthographic‑perspective geometry (OPG) enables DSM‑free localization in scenes with varying elevations, yet existing methods typically estimate pose indirectly through intermediate matrix estimation and decomposition. To address this limitation, we propose Latent‑Depth Variable Projection (LDVP), which treats the unknown scales along perspective viewing rays as latent depths. By analytically eliminating these depths and solving for the horizontal camera center in closed form given the rotation, LDVP reduces nonlinear estimation to a rotation‑only problem, enabling direct recovery of metric horizontal position without assuming a planar scene. To improve localization robustness across different scene structures, we further introduce Structure‑Adaptive Coupling (SAC), which uses relative scene structure inferred from monocular depth estimation to select LDVP when the matched points are not dominated by a single plane, and Planar‑PnP otherwise. Our local observability analysis reveals how scene elevation variation affects the geometric constraints of OPG, providing a basis for this adaptive strategy. Extensive experiments show that LDVP consistently outperforms existing OPG‑based DSM‑free methods, while SAC further improves robustness across diverse scenes and substantially narrows the gap to DSM‑based localization. Code will be released at https://github.com/UAV‑AVL/LDVP‑SAC.
Authors:Jie Gui, Hang Tu, Yunfeng Dou, Wen Sha, Xiuquan Du, Shuo Li
Abstract:
Specifying segmentation targets with natural language provides flexible semantic conditioning for medical image segmentation. However, existing methods still struggle to reliably ground high‑level textual semantics into accurate pixel‑level spatial prompts, especially under low‑contrast lesions, ambiguous boundaries, and visually similar anatomical contexts. In this paper, we propose RADAR (Residual Aggregation and Dense Alignment Representations), a parameter‑efficient framework for text‑guided medical image segmentation. RADAR explicitly decouples and synergizes the medical semantic anchoring capability of BiomedCLIP with the prompt‑driven mask decoding ability of MedSAM2, while enabling low‑cost task adaptation via LoRA. Specifically, Guided Residual Depth Aggregation adaptively aggregates cross‑layer features under semantic guidance to alleviate delayed textual injection and deep semantic dilution. Structured Dense Representation Alignment leverages a frozen BiomedCLIP to construct crop‑level target, boundary, and context anchors, improving the local discriminability of dense features and the reliability of automatic prompts. Multi‑modal Prompt Generation further converts cross‑modal semantic responses into composite text, point, and mask prompts for MedSAM2 decoding. Experiments on three public medical image segmentation datasets show that RADAR achieves competitive performance with only 0.74M trainable parameters, demonstrating its effectiveness in parameter‑efficient adaptation, semantic grounding, and dense representation enhancement. The source code is available at https://github.com/JGuiidev/RADAR.
Authors:June Hunter, Lily Wen, Owen Terry, Johannes Taraz, Ben Cohen, Niels Warncke
Abstract:
How do language models generalize from their training data? This is a central challenge for alignment: models reliably learn their training distribution given enough samples, but we want models that behave well when deployed in new contexts. We study out‑of‑distribution (OOD) generalization of propensities. Our central experiment uses a range of elicitation techniques ‑‑ system prompting, few‑shot learning, SFT on reference answers, and GRPO ‑‑ to increase a target propensity in a model, then measures how it spills over onto others across the resulting cross‑elicitation matrices. We find that spillover is highly correlated between models when the same elicitation technique is used, but our SFT and GRPO runs induce different spillover profiles. Elicitation of benign traits can sometimes generalize towards undesired traits, for example a model trained to claim that it is sentient often becomes more power‑seeking. We further find that spillover partially resembles trait correlations found in humans and identify three clusters of traits that account for about half of the variance using exploratory factor analysis (EFA). Across many elicitation runs, we observe that models change in characteristic ways that don't depend on the targeted trait, a phenomenon we refer to as drift. Drift moves Qwen3.5‑9B towards its base model. When investigating if LLMs are able to predict spillover, we find that Claude Opus 5.5 predicts spillover slightly better than the models themselves, but worse than data‑driven approaches using alternative elicitation methods or models. Overall, no single dominant predictor emerges, and generalization remains hard to predict. Code and data are available at https://github.com/nielsrolf/propensity‑spillover.
Authors:Shutong Ding, Zhaoyang Liu, Yimiao Zhou, Jingya Wang, Jiayuan Gu, Ye Shi
Abstract:
Diffusion policies are increasingly used in robotic control and manipulation, vision‑language‑action models, and world‑model‑based agents. Their ability to represent multimodal action distributions offers rich opportunities for exploration in reinforcement learning (RL). However, optimizing diffusion policies with RL can reduce the behavioral diversity and further limit their exploration capability. In RL post‑training, directly fine‑tuning a pretrained policy may also disrupt its acquired capabilities, while action‑space residuals may lack the expressive power for multimodal exploration. Together, these issues reveal a common challenge across online and offline‑to‑online diffusion RL: improving task performance while retaining the behavioral diversity and capabilities needed for continued adaptation. We introduce CFG‑Policy, a unified, plug‑and‑play framework that separates exploration and exploitation into two generative policies and combines their score or velocity fields during sampling through classifier‑free guidance. For online RL, a frozen exploration branch sustains behavioral diversity while an exploitation branch learns from task rewards; for offline‑to‑online RL, a pretrained exploitation branch remains frozen while a trainable branch steers the composed behavior. CFG‑Policy preserves the underlying RL objectives and improves performance with QSM and QVPO on 5 MuJoCo tasks and with Flow‑SDE+PPO on 4 RoboTwin manipulation tasks. Project page is at https://zhy‑liu.github.io/cfg‑policy‑page.
Authors:Buzhen Huang, Xingchen Wu, Yuanwang Yang, Chenyi Xie, Kun Li
Abstract:
Multi‑human cooperative object manipulation is a highly challenging topic in humanoid control. While existing methods can handle simple collaborative carrying tasks, they struggle with role‑dependent, long‑horizon, and sequential cooperative manipulation. To address this issue, we propose a hierarchical commander‑executor framework that enables multiple agents to accomplish complex collaborative tasks in distinct ways. Specifically, a commander combines instructions and scene information for joint semantic and geometric planning. It then decomposes the task into role‑specific subtasks and generates shared object‑centric coordination plans, specifying execution timing, target positions, and manipulation actions for each agent. Conditioned on the commander's high‑level strategy and each agent's local observations, the executor learns a low‑level physics‑based policy through a solo‑to‑cooperative curriculum to realize coordinated human‑object interactions. The planned object trajectory provides a shared coordination target, while root positions and grasping instructions specify role‑dependent execution cues. We evaluate our method on a variety of object manipulation tasks, demonstrating its ability to execute long‑horizon tasks that combine different types of collaboration and to scale from two‑agent to multi‑agent cooperation. Extensive experiments further show that our approach achieves higher success rates compared to state‑of‑the‑art methods. Code will be made publicly available at https://github.com/TJU‑3DV/Co‑Manipulation.
Authors:Zhikang Xu, Jiarui Xing, Jian Wang
Abstract:
We present a robust deep generative framework for unsupervised image prototype discovery, built on a heavy‑tailed Student's t mixture prior, that bridges statistical robustness and mathematical tractability. Existing generative approaches typically rely on noise‑prone Gaussian priors or computationally intensive diffusion models; neither simultaneously provides explicit latent structure nor exact analytic optimization. While diffusion models can yield strong priors, iterative sampling and intractable likelihoods limit scalability and interpretability for prototype discovery. We address these issues by constructing a Student's t latent space via a Bayesian scale‑mixture formulation. This yields a fully closed‑form augmented ELBO for unsupervised Student's t mixture variational autoencoders, which, to our knowledge, is the first such objective applied to robust image prototype discovery, avoiding biased variational approximations and high computational costs in prior heavy‑tailed or diffusion‑based frameworks. Treating precision as a latent variable automatically modulates each data point's influence, downweighting outliers to learn a data‑dependent measure of trust. The result is improved robustness, accuracy, and efficiency, with well‑separated latent representations and more stable manifold learning than state‑of‑the‑art methods. We validate our model on standard vision benchmarks and challenging neuroimaging tasks. In brain MRI‑driven neurodegenerative disease analysis, our model recovers clean, anatomically coherent prototypes where clinical noise often collapses rare subtypes. These results highlight the ability of our framework to preserve anatomy‑associated patterns for precise discovery in clinical settings. Our code is available at: https://github.com/xuzhikangnba/torch‑tvae.
Authors:Yusuke Watanabe
Abstract:
Let G be a finite bipartite graph and let \widetilde G be an M‑fold cover. We compare independent sets of \widetilde G with those of the trivial cover, consisting of M disjoint copies of G, after prescribing how many vertices are selected above each vertex of G. A conjecture from 2011, motivated by the Bethe approximation, asserts that the trivial cover maximizes every such count. We prove this for every feasible occupation profile whose active graph has treewidth at most two or is chordal bipartite. Equality holds exactly when the restricted cover is flat, meaning that fibre relabelings make it trivial. For M\ge2 and positive feasible occupations, a nonflat cover of a connected bipartite graph of treewidth at most two has a deficit of at least twice the coefficient of p_G^M‑2 with every occupation reduced by one. A holonomy‑dependent bound improves the factor to three for odd‑order holonomy; both uniform constants are sharp. Comparison and rigidity survive finite compositions along an edge or a vertex, and quantitative bounds from one piece propagate to the whole graph, giving results beyond the two original classes. On any bipartite base, the first nonzero homogeneous defect is determined by the shortest cycles with nontrivial holonomy. At singleton occupations, our counts are numbers of DP‑colorings of full covers. The dual DP color function on these classes and their finite seam compositions equals the chromatic polynomial, with equality attained precisely by canonical labelings. Dong's chromatic‑polynomial inequality then yields the Dual‑DP Shameful inequality on the same graph class. The main theorems have been formalized in Lean 4.
Authors:Xiaoran Zhang, Yu Liu, Jinyu Liang, Huaxin Xiao
Abstract:
Cross‑modal Thermal Geo‑localization (TG) provides an illumination‑robust positioning solution for Unmanned Aerial Vehicles (UAVs) in Global Navigation Satellite System (GNSS)‑denied environments. However, existing thermal geo‑localization methods mainly focus on near‑nadir observations or depend on map‑side elevation priors, limiting their applicability to continuous oblique thermal localization with only geo‑referenced 2D satellite maps. To address this problem, we propose ScalLoc, a scale‑aligned local‑to‑map localization framework for continuous oblique thermal UAV geo‑localization. ScalLoc follows a reconstruction‑‑refinement‑‑anchoring‑‑propagation design: it reconstructs local window geometry from thermal sequences, refines camera poses using track‑based pose optimization, adaptively anchors each window to the satellite map, and propagates validated states to recover frame‑level global trajectories. To support systematic evaluation, we construct Thermal‑Oblique, a real thermal UAV sequence dataset containing 216 urban and rural day‑‑night trajectories and 28,112 thermal frames with frame‑level annotations. Extensive experiments demonstrate that ScalLoc reduces the mean localization error from 112.69 m to 26.97 m compared with the VGGT‑only configuration, corresponding to a 76.07% reduction, and achieves the lowest frame‑weighted mean position error among the compared methods. These results demonstrate the feasibility of continuous oblique thermal geo‑localization using only lightweight 2D satellite maps, without requiring digital surface models or preconstructed 3D maps. Code and dataset are available at https://github.com/FloralHercules/ScalLoc.
Authors:Zhiding Liu, Jiqian Yang, Mingyue Cheng, Zhi Li, Enhong Chen
Abstract:
Time‑series forecasting plays a vital role in a wide range of real‑world applications. Recent approaches have achieved strong performance by modeling temporal correlations and cross‑channel dependencies, yet most forecasting benchmarks still represent time series primarily as numerical histories. In practice, time series are observations of complex real‑world processes, and their trajectories may reflect events, interventions, disruptions, and changes in measurement or reporting that cannot always be inferred from past values alone. A central barrier to studying context‑aware forecasting is therefore the lack of large‑scale datasets that align real‑world events with the individual series they affect. We present World2TS, a scalable construction framework and large‑scale dataset for time‑series forecasting with real‑world event context. We first develop world2ts‑core, a reusable tool that combines agent‑led semantic reasoning and Web search, command‑line execution and validation, and human review within a feedback‑driven discovery workflow. Given heterogeneous time series with semantic descriptions, the workflow discovers candidate events, consolidates retrieved evidence into verified events, and links them to relevant series under explicit temporal and semantic scopes. Using this framework, we construct world2ts‑data, which contains 31 datasets across six domains, comprising 4,179 series, 97.1 million observations, 1,159 events, and 56,846 event‑series links. We further conduct extensive data analyses and controlled comparative experiments to characterize the dataset and demonstrate the utility of the collected context for improving forecasting accuracy. World2TS provides a reusable data foundation for developing and evaluating context‑aware forecasting models and agents. The project homepage is available at https://world2ts.github.io/.
Authors:Pu Pang, Zhongfei Qing, Ling-Hao Chen, Songnan Li, Hua Huang, Bo Hu
Abstract:
This paper studies the problem of recovering motion from monocular video, even when the target skeleton topology differs from the subject observed in the video. Although previous methods have made efforts to resolve this problem, they remain vulnerable to joint‑rotation errors that are coupled across the skeleton and can propagate along its kinematic hierarchy. To mitigate these errors, we present CAST, a topology‑aware motion capture framework that combines learned local rotations with geometric correction guided by predicted global bone directions. Our observability analysis characterizes which rotational components can be constrained by outgoing bone directions. Guided by this analysis, GRACE uses reliability‑gated alignment to correct composed orientations in parent‑to‑child order, leaving unconstrained components to the learned rotation predictor. The motion predictor and correction module are trained jointly with motion‑derived supervision, using a frozen image encoder. Experiments under the TopoCap evaluation protocol show that the proposed framework improves motion recovery across different target topologies, including unseen structures. Reference‑based cross‑skeleton evaluation on a curated Mixamo benchmark and a user preference study further assess transfer to different target structures. A causal CAST‑B variant achieves over 51 FPS on a single NVIDIA A100 GPU, with timing covering image encoding and motion prediction. Code repository: https://github.com/fankewen/CAST.
Authors:Xiaofeng Tan, Jiaxin Lin, Haobo Yuan, Tiancheng Shen, Zihan Su, Rui Chen, Hongsong Wang, Ming-Hsuan Yang
Abstract:
Reinforcement learning (RL) is promising for spatial text‑to‑image generation, but existing work largely focuses on policy optimization, assuming accurate rewards and effective online rollouts. We observe in‑domain gains with out‑of‑domain degradation, exposing two limitations: unreliable spatial rewards in open‑world settings and rollouts lacking basic‑to‑complex progression. We propose a curriculum‑guided RL framework. First, we introduce SpatialCritic, an agentic reward model equipped with a spatial perception harness that combines reusable verification guidance with specialized vision tools. We then construct spatial preference datasets through targeted counterfactual perturbations and perform agent‑harness alignment to improve tool selection and evidence integration efficiently. Second, curriculum learning organizes shared foundational capabilities and their compositional dependencies in a capability graph for mastery‑guided rollouts, periodic review, and targeted remediation. On SpaRW‑Eval, SpatialCritic achieves 95.1% verification accuracy, outperforming 320B GLM‑5.3‑Flash with a 40x smaller 8B backbone and raising average baseline scores from 78.4 to 83.9 on SD3.5‑Medium and 83.2 to 86.5 on BAGEL. Project page: https://xiaofeng‑tan.github.io/projects/spatialcritic/
Authors:Nga Kiu Kelly Ho, Yilin Liu, Hao Xu, Niloy J. Mitra
Abstract:
Synthesizing believable 4D hand‑object grasping sequences is important for character animation, extended reality, and embodied agents. Grasping requires a hand to approach, contact, and lift or manipulate an object coherently. Existing synthesis methods often model only the final static grasp or rely on additional conditions, such as language, target grasps, or object trajectories, to specify the interaction. We instead study the underconstrained problem of generating a complete grasping motion from a single initial hand‑object state. Our framework decouples hand‑motion generation from object‑motion recovery. Given an object point cloud in the initial wrist frame and the initial parameterized hand configuration, conditional flow matching predicts a sequence‑level contact map and a hand pose at each normalized sequence time. This time‑indexed formulation avoids the error accumulation of autoregressive rollout while representing the complete motion. We then infer an attachment frame and recover rigid‑object motion through training‑free, contact‑guided weighted registration. We evaluate on held‑out DexYCB objects and unseen object geometries reconstructed from in‑the‑wild videos by Open‑CHOIR. Compared with LatentHOI, our method produces smoother hand and object motion in every setting, matches or improves on its contact and penetration metrics in the wild under a shared initialization, and is about 30x faster per generated frame. In a 15‑participant study, our motions were preferred in roughly two thirds of (forced‑choice pairwise) comparisons. Precise contact closure on novel geometry, however, remains a key open challenge.
Authors:Kailai Sun, Yue Cui, Xinwei Wang, Zijie Gao, Baoshen Guo, Zhiqing Hong, Irfan Qaisar, Xu Yang, Yueting Li, Dian Fan, Qianchuan Zhao
Abstract:
Indoor human perception is the foundation for smart homes and healthcare. However, RGB cameras raise privacy concerns in private indoor environments, while low‑resolution thermal cameras provide limited spatial and semantic information. Large‑scale indoor thermal datasets and real‑world deployments remain limited. Here, we introduce Indoor RGB‑T, a large‑scale paired RGB‑‑thermal dataset for indoor human perception. The dataset covers 20 indoor rooms and contains 90,576 80x62 thermal frames, 87,016 temporally matched RGB‑‑thermal pairs, and 115,557 annotated human bounding boxes. We establish benchmarks for robust thermal occupant‑state detection with 0.75 mAP and occupancy estimation with an accuracy of 0.847 in unseen rooms. We further evaluate RGB‑to‑thermal and thermal‑to‑RGB translation, achieving a temperature MAE of 0.654 C and an R2 of 0.923. Finally, we deploy the thermal perception model on a commercial sleep‑monitoring system in the real world. Field testing with 100 participants and deployment of approximately 1,000 devices further demonstrate its practical feasibility. We identify practical challenges that motivate future research directions, including environmental thermal interference, sensing range and multi‑person occlusion. The code, trained models and partial dataset are available at https://github.com/kailaisun/RGBT and https://huggingface.co/skl24/RGBT.
Authors:Kecan Mao, Nan Xu, Zhaoyang Meng, Zhengyao Ma, Yingming Gao, Yi Zhao, Ya Li
Abstract:
Speech representation learning requires a discrete modeling interface while preserving high waveform fidelity. Existing approaches rarely achieve both: discrete tokens ease modeling but lose fine‑grained acoustic information, while continuous latents preserve fidelity but require specialized generative objectives. We introduce SemiCodec, a novel Semi‑discrete neural speech Codec with synchronized discrete tokens and continuous latents. Direct joint optimization may create a reconstruction shortcut through the continuous pathway, leaving the discrete tokens insufficiently grounded. To address this issue, we propose a Preserve‑then‑Ground training paradigm that first establishes a high‑fidelity synthesis space and then grounds SSL‑guided discrete tokens within it. Experiments show that SemiCodec outperforms fully discrete codecs in reconstruction quality and intelligibility while remaining competitive with continuous representations in acoustic fidelity and speaker preservation. Audio samples are available at https://pennyw1‑se.github.io/SemiCodec‑Demo‑Page/
Authors:Zhou Liu, Xingyan Chen, ZhaoYang Han, Zewei Pan, Zeli Su, Ningxin Pan, Xue Hu, Kaixiong Ting, Wentao Zhang
Abstract:
Office workflows are a demanding testbed for self‑improving agents because their deliverables must be correct, natively editable, and mutually consistent. A central question is whether gains on feedback tasks reflect reusable procedures rather than task‑specific fixes. We introduce EvoCowork, a 240‑task benchmark for evaluating harness‑level recursive self‑improvement across four office modalities, with model weights held fixed. The benchmark covers presentations, documents, spreadsheets, and cross‑office workflows, and scores submitted artifacts for content, native structure, formulas, cross‑file consistency, and visual quality where needed. Its protocol separates feedback, revision selection, and final testing: methods learn from single‑modality tasks, then are evaluated on unseen single‑modality tasks and cross‑office workflows under a shared initial harness and explicit budgets. In static evaluations of two models and four harnesses, the best configuration scores 42.25 overall and 30.62 on cross‑office tasks. Starting from the same MSA harness with GLM 5.3, ACE and AHE improve in‑domain scores by 4.90 and 2.64 points, respectively, but change cross‑office scores by only +0.03 and ‑1.72. Thus, gains on the file types that provide feedback do not necessarily transfer to coordinated workflows. EvoCowork makes this distinction explicit while recording the capability, transfer, and cost of harness evolution. Code and task packages: https://github.com/DeepThinkingZhouLiu/cowork‑evolution‑benchmark.
Authors:Adarsh Arigala, Arjun Gangwar, Srinivasan Umesh
Abstract:
Recent alignment‑free Text‑to‑Speech (TTS) models achieve high synthesis quality by removing explicit alignment modules and simplifying training. However, they often struggle in cross‑lingual scenarios due to differences in character durations across languages, leading to duration mismatch and degraded speech quality. Existing approaches typically address this issue using explicit duration predictors, increasing model complexity. Systematic evaluation of TTS performance for Indian languages in cross‑lingual settings remains under‑explored due to the lack of standardized evaluation benchmarks. In this work, we construct a cross‑lingual evaluation benchmark covering English and nine Indian languages and propose a language‑aware inference‑time duration estimation strategy for alignment‑free TTS that incorporates language‑specific duration‑per character statistics without retraining or architectural modifications. We further develop heuristic formulations that refine duration estimation based on language‑dependent duration variations. Experiments on bilingual and multilingual A‑DMA models demonstrate improved speech naturalness, intelligibility, and speaker similarity while reducing duration mismatch.
Authors:Jun He, Deying Yu
Abstract:
Enterprise AI agents can execute altered versions of approved actions, concurrently reuse the same permission, or retry operations whose responses were lost. These failures arise when approval and resource execution lack a shared contract. KedgeFlow specifies language‑independent contracts for trusted controllers to authorize state‑changing operations and privileged gateways to enforce that authority independently of agent reasoning. Inspired by OpenFlow, it connects policy checks supported by evidence, signed permission for an exact action, a shared allowance that can be consumed only once, target‑side conditions for applying the change, and retained outcome records. Under trusted issuance of exact‑action authority, two conditional properties establish at most one successful claim per allowance and link every governed dispatch to a matching grant, claim, and subsequent dispatch authorization, even when the agent is compromised. A reference implementation exposes this execution path to agent systems through the Model Context Protocol (MCP). In the single‑resource evaluation, 6,560 concurrent ledger claim attempts yielded exactly one successful claim for each of 82 allowances. Boundary tests rejected unauthorized and altered requests before consumption. A 32‑call gateway race produced one committed mock effect, and a retry after a lost response produced no additional dispatch. Deliberately permissive policy nevertheless authorized an undesirable effect, exposing the limits of execution authorization. Evaluation uses three members of the etcd coordinated key‑value store on one host, synthetic authentication and authorization inputs, and a mock target that rejects stale requests. Distributed reservations, enforcement of every production access route, native crash recovery, exactly‑once effects at real targets, and interoperability between independent implementations remain untested.
Authors:Yuchen Ma, Swadheen Shukla, Paulo Salem, Yue Huang, Robert Sim, Stefan Feuerriegel, Michel Galley, Jianfeng Gao
Abstract:
LLM agents increasingly divide enterprise work across roles with different goals, authority, access, and information. Because each agent often sees only a local snapshot of the enterprise workflow, locally reasonable actions can still violate organization‑wide constraints (e.g., several service agents can each issue credit that is individually permitted while collectively exceeding a shared compensation cap). We introduce EAIR, the first framework for analyzing enterprise multi‑agent interaction risks. We make four contributions. (1) We formalize enterprise interaction risk in terms of the organizational properties that workflow operations must preserve, and operationalize this formulation as a taxonomy of testable risks with explicit failure criteria and evidence requirements. (2) We build EAIR‑Worlds, a suite of executable micro‑worlds that instantiates selected risks as controlled, observable workflows with explicit roles, policies, permissions, source facts, and execution traces. (3) We develop a novel diagnosis‑and‑repair pipeline that uses the resulting execution traces to locate contributing interaction boundaries, guide targeted workflow repairs, and retest the repaired system. (4) Large‑scale experiments show recurring interaction failures across the organizational properties studied. Together, EAIR is the first operational framework for diagnosing and mitigating enterprise multi‑agent interaction failures.
Authors:Hao Li, Keqing He, Longtao Zheng, Junlong Li, Weihao Zeng, Qian Liu, Yuzhen Huang
Abstract:
Large language models (LLMs) are increasingly expected to orchestrate diverse tools over long‑horizon tasks. Existing agents typically use Direct Tool Calling (DTC), where the model invokes tools sequentially, and tool responses accumulate in the model context, which can lead to context rot and orchestration breakdown. In contrast, Programmatic Tool Calling (PTC) invokes tools through executable code and stores intermediate results in runtime variables rather than appending them to the context. Prior work has explored PTC mainly as an inference‑time optimization, but its benefits for agentic training remain unclear, and suitable verifiable training tasks are scarce. In this work, we introduce ProgAgent, a full PTC training stack, comprising a pipeline for synthesizing verifiable long‑horizon tasks, a lightweight PTC runtime, and a two‑stage training recipe. Using only 215 tasks instantiated at four environment scales, ProgAgent yields higher pass rates than the evaluated baselines of comparable size across four benchmarks. On LOCA‑bench, ProgAgent‑14B achieves a 30.7% pass rate at a 64k‑token environment state, where these baselines score zero. Cross‑harness experiments show that PTC‑trained models outperform DTC‑trained models under both evaluation interfaces, suggesting that the benefits of PTC training extend beyond the interface.
Authors:Changkun Ou, Sven Mayer, Daniel Buschek, Andreas Butz
Abstract:
Human‑in‑the‑loop optimization refers to systems that present choices to a human decision‑maker and query their judgment to optimize system parameters iteratively. The user interface typically provides either an absolute rating scale or asks for a preferred option. However, these approaches can potentially suffer from calibration problems, inconsistencies, and completeness violations of user preferences in such a sequential evaluation. This paper revisits the design space of opinion measurement interfaces for human‑in‑the‑loop optimization. We conducted a study (N=360) using text summarization and image color enhancement tasks to evaluate human sequential feedback in six representative opinion measurement interfaces. Based on our analysis, we recommend using a listwise approach to support sequential opinion measurement, which counterweighs individual rating and preferential choice limitations and conveys more information for subsequent use. We further discuss the trade‑offs in different user interface designs and provide guidelines to inform future research.
Authors:Xianglong Yan, Zenan Wu, Kewei Zhang, Kaicheng Yang, Yulun Zhang
Abstract:
Extreme low‑bit weight‑only quantization is an effective approach to reducing the deployment costs of large language models. Below 3 bits, scalar quantization often suffers severe accuracy loss, while vector quantization (VQ) better preserves accuracy through more expressive joint encoding. However, existing VQ methods still exhibit substantial degradation on real‑world tasks, and two design challenges remain underexplored: (1) protecting sensitive weight channels within jointly encoded vectors; and (2) allocating a fixed bit budget among representation components, such as direction and radius, to suit different weight matrices. To address these challenges, we propose SAGE: Split Adaptation and Grouping by Energy for weight‑only vector quantization. First, Energy‑Stratified Grouping (ESG) couples channel grouping with activation‑weighted fitting, grouping channels from different energy tiers within each subvector to prioritize sensitive channels. Second, Adaptive Bit Split (ABS) balances direction‑codebook capacity and radius precision per matrix, selecting the split with the lowest activation‑weighted reconstruction error under a fixed budget. Across benchmarks covering knowledge, mathematics, code generation, instruction following, and tool use, SAGE achieves state‑of‑the‑art performance in the 2‑bit quantization setting. On Qwen3‑30B‑A3B, it retains 95.3% of the BF16 model's average benchmark score. With our custom‑designed CUDA kernel, SAGE delivers up to a 2.55‑fold non‑streaming end‑to‑end speedup over FP16. The code is available at https://github.com/Zincwolf/sage.
Authors:Zucan Lyu, Rui Tang, Jianlyu Chen, Peisheng Jin, Defu Lian
Abstract:
AI agents require specialised retrieval across domains and tasks, but adapting embedding models is constrained by costly manual training‑data design. Here we introduce SYNTRA, a self‑improving agentic framework that turns a few retrieval examples into reusable synthesis instructions. Given an unlabelled corpus and a few relevant query‑document pairs, SYNTRA infers implicit information needs and relevance criteria, then uses feedback on trial outputs to refine instructions for query generation and relevance assessment. Experiments across 24 datasets spanning 15 domains demonstrate that the same synthesis workflow produces effective training data, benefiting even strong embedding models with and without existing labelled training sets. Self‑improvement strengthens this supervision on all six evaluated tasks, yielding better retrieval than human calibration on four. The training value persists at larger scales: on MS MARCO, retrieval gains continue across a 32‑fold range of synthetic training‑data sizes. SYNTRA thus enables scalable embedding‑model adaptation with minimal human intervention.
Authors:Shiqiao Gu, Yang Yong, Fengjun Zhong, Ruihao Gong, Chengtao Lv, Xiangyu Fan, Lei Yang, Xianglong Liu
Abstract:
Latent diffusion models enable high‑quality video generation, but video VAE encoding and decoding remain computationally expensive. Existing methods use stage‑specific lightweight operators, yet either constrain spatial and temporal computation jointly or remove temporal computation altogether, without distinguishing their compression sensitivities in pretrained Conv3D. To address these limitations, we present LightVAE, a compression framework that exploits redundancy in temporal modeling and network depth. LightVAE exactly reparameterizes eligible causal Conv3D layers into an aggregate spatial response and a difference‑driven temporal correction. Based on their distinct compression sensitivities, temporal low‑rank compression (TLR) keeps the aggregate spatial kernel dense and applies layer‑specific low‑rank approximation only to the temporal correction. An algebraically equivalent folded implementation converts the arithmetic savings into practical acceleration. LightVAE further uses width‑preserving block pruning to reduce depth and probe‑guided feature alignment, guided by feedback from a frozen teacher suffix, to recover quality after compression. The framework also extends to video encoders. Across three video VAE backbones and two GPU platforms, LightVAE consistently accelerates decoding. For the Wan VAE family, it achieves up to 2.24× and 1.11× the decoding throughput of state‑of‑the‑art efficient VAE decoders on NVIDIA H100 and RTX~5090, respectively, with higher DAVIS PSNR and lower LPIPS. On MiniMax‑H3, it achieves up to 2.97× the original decoding throughput while retaining 99.51% of the original model's overall VBench‑2.0 score. Our code is available at https://github.com/ModelTC/LightVAE.
Authors:Minghang Du, François Gay-Balmaz, Tianzhi Li, Donghua Shi
Abstract:
Geometrically exact beams naturally arise in physical phenomena and engineering practice for modeling flexible structures, such as soft robotic manipulators and DNA structures. To the best of our knowledge, there appear to be no geometric integrators for stochastic geometrically exact beams in a variational formulation. In this paper, we introduce variational principles for the dynamics of stochastic geometrically exact beams, in both continuous‑ and discrete‑time settings. By introducing perturbations via stochastic potentials and stochastic kinematics, we derive the equations of motion from a variational viewpoint. The resulting stochastic geometric integrators simultaneously discretize time and space in a parallel manner, which provides a distributed computational framework for solving the stochastic dynamics of complex beam systems. In particular, the proposed integrators are shown to be a stochastic extension of the well‑known deterministic staggered leapfrog scheme for the wave equation, and also a stochastic extension of Yee's scheme for spatially one‑dimensional computational electromagnetism. Numerical results are shown to demonstrate the effectiveness of the proposed approaches.
Authors:Jiuwu Hao, Yueyang Wu, Liguo Sun, Haolin Song, Zao Zhang, Pin Lv
Abstract:
Despite recent advances in Aerial Vision‑and‑Language Navigation (VLN), single Unmanned Aerial Vehicle (UAV) search‑and‑rescue in outdoor scenarios remains challenging due to partial observability, making cross‑altitude collaboration a promising direction. However, existing collaborative navigation methods rely on precise target localization and rigid goal‑dispatching paradigms, ignoring high‑altitude observation uncertainty and suppressing low‑altitude exploration autonomy. To address these issues, we introduce BGC‑Nav, a training‑free cross‑altitude collaboration framework that enables robust navigation via Belief‑Guided Coupling (BGC). Specifically, the high‑altitude UAV maintains a Global Belief Memory encoding task‑conditioned priors, which the low‑altitude agent incorporates into decision‑making via Local Exhaustion Memory (LEM), Local Attraction Augmentation (LAA), and Directional Belief Prior (DBP) as an advisory bias rather than a rigid command. Extensive experiments on ESARBench show that BGC‑Nav consistently outperforms state‑of‑the‑art single‑agent and collaborative methods. Code is available at https://github.com/uestchjw/BGC‑Nav.
Authors:Yaolong Zhu, Peng Li, Jiahao Lu, Chengfeng Zhao, Mengfei Li, Chao Wang, Wenyue Chen, Zekai Gu, Chengwei Ren, Ronggang Wang, Yuan Liu
Abstract:
Feed‑forward 3D reconstruction models have achieved remarkable progress through large‑scale training and strong learned priors. However, they remain fragile when applied to dynamic in‑the‑wild videos, where non‑rigid motion, occlusions, and illumination changes are prevalent. A key limitation lies in their reliance on expensive depth and camera pose annotations, which hinders scaling to diverse real‑world dynamic data and restricts generalization. To address this issue, we propose PURE, an unlabeled post‑training refinement framework that improves depth and camera estimation without requiring ground‑truth supervision. Instead of relying on annotations, PURE elegantly distills robust structural knowledge from state‑of‑the‑art 2D priors into 4D feed‑forward architectures. PURE leverages relative depth constraints, affine‑invariant geometry distillation, and pose refinement based on feature correspondences to stabilize geometric predictions under challenging dynamic conditions. Extensive experiments on both synthetic and real‑world dynamic benchmarks demonstrate that PURE delivers consistent improvements across multiple state‑of‑the‑art architectures. Furthermore, our data‑efficient paradigm effectively mitigates domain shifts, successfully unlocking the architectural limits of these base models in challenging, in‑the‑wild dynamic scenes.
Authors:Yixuan Li, Xuenan Xu, Zeyu Xie, Mengyue Wu
Abstract:
Speech‑conditioned generation maps speech signals to target audio outputs. Prior end‑to‑end speech‑conditioned generative models achieve high fidelity in clean acoustic conditions, but their performance degrades substantially under environmental noise. While prior noise‑robust approaches have achieved significant success in feature‑level purification, maintaining consistent generation under clean and noisy conditioning remains a key challenge. To address this, we propose AlignAudio, a noise‑robust generative framework based on dual alignment: (i) aligning the representations of clean and noisy speech to preserve semantic cues, and (ii) enforcing generation‑level consistency within a flow matching paradigm to improve conditioning fidelity. Experiments on speech‑to‑audio (STA) and voice conversion (VC) tasks demonstrate that AlignAudio maintains performance comparable to baselines on clean speech while significantly outperforming them across various noisy conditions. Furthermore, the proposed dual‑alignment strategy is applicable to both single‑ and multi‑stream speech conditioning and can be integrated into different flow matching‑based generation frameworks and post‑training settings. Audio demonstrations and qualitative comparisons: \hrefhttps://alignaudio.github.io/AlignAudio\textcolorcyanhttps://alignaudio.github.io/AlignAudio.
Authors:Wenbing Yang, Qihang Lu, Zihan Sun, Peilei Jia, Yingming Gao, Ya Li, Jun Gao
Abstract:
Instruction‑following text‑to‑speech (TTS) typically leaves voice design implicit, as conventional speaker embeddings encode reference audio rather than text‑driven targets. We propose VoTSpeech, which decouples voice design from speech generation via an explicit intermediate voice latent. Built on a shared language model backbone, VoTSpeech first predicts a continuous voice latent from text instructions using a flow‑based module. This latent serves as a shared condition for both the language model and the Acoustic DiT, guiding autoregressive sequence modeling and acoustic latent generation, respectively, before an audio VAE decoder converts the acoustic latents into waveforms. To capture speaker identity and complementary acoustic nuances, the voice latent is jointly supervised by speaker embeddings and temporally pooled audio VAE features. With task‑specific adaptation, VoTSpeech performs strongly on InstructTTSEval‑ZH, demonstrating competitive capabilities against strong open‑source models while maintaining high naturalness. These results confirm that explicit voice latent design and its conditioning interface effectively empower text‑driven voice design for controllable speech synthesis. Code, model, and audio samples are available at https://github.com/ywbn/VoTSpeech .
Authors:Zhan Jin, Yuze Luo, Bin Cui, Xupeng Miao
Abstract:
LLM‑augmented data workflows often involve multiple heterogeneous models operating over substantial shared context, such as schemas, tool specifications, interaction history, and intermediate results. Although prefix caching can reduce prefilling overhead, existing approaches are largely limited to reuse within the same model because KV states are tightly coupled to model‑specific architectures. We present MapFlow, a system for efficient heterogeneous LLM serving via cross‑model attention map reuse. Instead of reusing KV states, MapFlow reuses attention maps, which capture token dependency patterns and are more transferable across models. To enable efficient and accurate reuse, we combine partial recomputation with lightweight linear translation across layers and heads. To make attention reuse practical for long contexts, we further introduce block‑level sparse compression and a fused retrieval kernel. Experiments show that MapFlow reduces prefilling latency and improves end‑to‑end serving efficiency while preserving generation quality across heterogeneous model pairs and multi‑turn routing workloads. These results suggest that attention maps provide an effective reusable state for heterogeneous LLM serving, especially in prefix‑heavy data workflows. Our code is available: https://github.com/PKU‑DAIR/MapFlow.
Authors:Hanxin Zhu, Jianqi Ma, Cong Wang, Zhihao Hu, Junyao Gao, Tianyu He, Junyan Lin, Jiayin Xiao, Jiateng Shou, Sen Liang, Weidong Zhang, Zhibo Chen, Jun Zhang
Abstract:
Recent advances in video diffusion enable novel‑view video synthesis from a single input video, supporting world modeling, embodied intelligence, and robotic simulation. However, existing methods process camera trajectories independently, lacking the cross‑trajectory consistency needed for coherent, navigable 4D worlds. We introduce Demiurge, an autoregressive video rerendering pipeline built on a dynamic 4D point‑cloud memory cache. It promotes cross‑trajectory consistency through four components: (1) an appearance video diffusion model that synthesizes photorealistic novel‑view videos conditioned on the 4D memory; (2) a memory‑consistent video depth completion model that aligns known‑region depths with cached geometry and completes missing regions, mitigating scale ambiguity and discontinuities during memory updates; (3) memory‑aware dual‑stream warping that provides high‑fidelity visual guidance and occlusion cues for novel‑view synthesis; and (4) geometry‑aware video refinement that enhances perceptual quality. Extensive experiments demonstrate state‑of‑the‑art performance and three key capabilities: coherent spatiotemporal lattice construction over dense multi‑view trajectories; large‑parallax video rerendering under challenging camera motions, including 180‑degree rotations with no input‑view overlap; and geometry‑faithful outputs that can be directly optimized into high‑quality 3D Gaussian Splatting representations with minimal artifacts. Project page: https://hanxinzhu‑lab.github.io/demiurge_project_page/.
Authors:Pratham Gupta, Haotian Qi, Tushar Sandhan, Gabriel Skantze
Abstract:
We present CAST‑ASD (Causal Attention over Speakers and Time), a causal joint model for active speaker detection (ASD). Recent ASD models improve performance by jointly processing all faces in a scene, but operate offline, with the full clip available before any frame is labelled. Low‑latency causal detectors instead process each face independently. CAST‑ASD combines joint face reasoning with fully causal inference. It applies scene‑level attention across faces within the current frame and causal temporal attention over each face's history, enabling information sharing across speakers with zero lookahead. Its parameter count does not grow with the number of faces. On MSDWild, where most negative face‑frames contain a visible competing speaker, CAST‑ASD reduces competing‑speaker false alarms by 32% relative to the causal per‑face detector it extends, with larger gains as scene occupancy increases. It outperforms all causal baselines on both validation splits while using only one‑tenth the inter‑speaker parameters of a three‑block fixed‑roster convolution. Code and models are available at https://github.com/prathamg007/CAST‑ASD.
Authors:Daijin Hu, Xi Wang
Abstract:
Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time. However, subsequent self‑corrections can invalidate apparently stable arguments, causing stale execution whose irreversible effects persist even after the final call is corrected. We propose Repair‑Aware Semantic Survival Gating (RASSG) to jointly control tool launch and cancellation during streaming speech. RASSG launches a tool when three consecutive speech recognition results arrive with identical nonempty arguments, then combines calibrated argument invalidity risk with execution costs to decide cancellation at the last feasible opportunity before effects become irreversible. Evaluation on ReCAST, FDB‑v3, and Audio2Tool uses a controlled execution clock with fixed tool runtimes and cancellation parameters. On ReCAST's prospective‑repair subset, RASSG reduces the stale execution rate from LocalAgreement‑2's 50.00% to 3.15%, with a mean delay of 3.676s across successful trajectories. The stale reduction also holds against every early‑execution baseline on both public speech datasets. Ablations with identical initial launches support active cancellation, showing the value of controlling ongoing calls before their effects become irreversible. Code and data are available at https://github.com/daijinhu/RASSG.
Authors:Zhuo Chen, Bin Zou, Fanyue Wei, Lixin Duan, Angela Yao, Wen Li
Abstract:
Can autoregressive video diffusion be accelerated further when only four denoising steps remain? Existing feature caching approaches struggle in this regime, as large feature changes between denoising timesteps undermine both direct reuse and lightweight predictors. We observe that features across video chunks remain strongly correlated at matching denoising timesteps, offering a complementary source of guidance. Building on this insight, we introduce TGuide, a framework that accelerates autoregressive video diffusion through a lightweight Anchor‑Transport‑Correct (ATC) predictor guided by historical sampling trajectories. Our ATC combines two complementary references: the current chunk's preceding‑step features preserve its spatial structure, while the previous chunk's same‑timestep features provide information at the target denoising stage. By integrating these references through spatial retrieval and adaptive correction, ATC improves velocity prediction accuracy despite the large gaps between denoising steps. To mitigate error accumulation across both denoising steps and video chunks, we train the predictor on its own autoregressive rollouts, allowing it to adapt to states shaped by its own earlier predictions. An optional confidence‑aware scheduler further adjusts the quality‑efficiency trade‑off. Experiments on Self Forcing, Causal Forcing, and HY‑WorldPlay show that TGuide preserves generation quality while achieving denoising speedups of up to 1.89× over the original 4‑step generators. On Self Forcing, TGuide generalizes from 81‑frame training videos to 333‑frame generation, approximately four times the training length, while exceeding the original model's VBench score.
Authors:Hao Zhu, Heyu Sun, Shuo Jin, Jiayu Xiao, Feng Dai
Abstract:
The recent Segment Anything Model 3 (SAM3) extends powerful promptable segmentation from spatial cues to concepts, raising the prospect of a single foundation model for universal image segmentation. However, this promise often fails to translate into reliable performance on downstream segmentation tasks. In this work, we empirically identify concept recognition, rather than mask quality, as the primary bottleneck: SAM3 accurately delineates the objects it recognizes yet struggles to robustly determine which concepts are present. Driven by this, we present \underlineText \underlineAnchor re‑\underlineGrounding (\TAG), a simple yet effective framework that rectifies the concept recognition of SAM3 on the target domain without dense annotations. Specifically, \TAG introduces an Anchor Residual Pathway comprising image‑conditioned, low‑rank adapters to re‑ground text anchors in the target distribution and modulate subsequent text‑conditioned interactions. Guided solely by image‑level category labels, \TAG jointly optimizes binary cross‑entropy with Soft Boundary Ranking objectives to calibrate presence confidence and distinguish present concepts from confusable negatives. Building upon this, we devise Confidence‑Weighted Distillation to couple rich semantic context of a multimodal large language model with fine‑grained spatial priors of SAM3, thereby achieving effective calibration in the absence of manual annotations. Extensive experiments demonstrate that \TAG unlocks SAM3's segmentation potential with only 3.1M trainable parameters, achieving impressive performance across diverse scenarios, and notably, even surpassing leading mask‑supervised methods on several metrics. \hrefhttps://github.com/MiSsU‑HH/TAG_SAM3Our code will be publicly available at GitHub \faGithub.
Authors:Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang, Xuejun Wu, Xi Chen, Buyue Qian, Yaowei Zheng, Junhao Hu
Abstract:
Recursive self‑improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow language agents to retain changes to their parameters, memory, skills, and code for subsequent execution and improvement. We examine how these methods can inform new research ideas through two complementary perspectives. First, we introduce a unified taxonomy with nine design dimensions organized into four groups: update targets, update organization, update schedule and scope, feedback and selection. This taxonomy places existing methods in a shared design space, where examining individual dimensions and their interactions can inspire new research ideas. Second, we examine how existing RSI methods draw on traditional deep learning, for example through optimization, regularization, data augmentation, continual learning, and adversarial learning. We develop further ideas by examining more specific learning mechanisms and drawing on evolutionary computation. We also review how existing benchmarks assess self‑improvement under online, offline, and offline‑to‑online settings. Together, these perspectives help researchers develop new RSI research ideas from existing designs and learning mechanisms. RSI papers and resources are available at https://prism‑shadow.github.io/awesome‑rsi/.
Authors:Fadillah Adamsyah Maani, Numan Saeed, Jiaqi Ma, Dat Tien Nguyen, Youssef Wehbe, Duncan Watson-Parris, Salman Khan
Abstract:
Interpreting Earth system conditions is central to understanding environmental risks and informing decisions across weather‑ and climate‑sensitive sectors. Existing multimodal weather systems primarily target task‑specific prediction, generation, or question answering, while general‑purpose alignment between Earth system data and language remains underexplored due to the limited availability of diverse language supervision grounded in meteorological conditions. To fill this gap, we introduce EarthAlign, a large‑scale dataset with broad geographic coverage spanning 2000‑2024 that combines 10 expert‑curated natural event and disaster data sources with climatology‑derived normal‑weather samples. Each sample includes temporal and geospatial information and four grounded caption styles aligned with 63‑variable ERA5 meteorological data. Building on EarthAlign, we introduce MeteoLIP, a dual‑encoder framework for learning aligned meteorology‑language representations. MeteoLIP incorporates MetTok, which uses a geolocation‑ and time‑conditioned query to aggregate variable‑specific patch embeddings into a compact sequence of meteorology tokens. To comprehensively assess Earth system multimodal representations, we introduce three downstream tasks: cross‑modal retrieval, weather‑condition classification, and physical variable estimation. Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks. Further analyses highlight the importance of meteorological input, temporal and geospatial conditioning, and multi‑style supervision for meteorology‑language alignment. Thus, EarthAlign and MeteoLIP provide a foundation for learning and evaluating language‑aligned representations of Earth system data. Our code, dataset, and weights are publicly available at https://github.com/mbzuai‑oryx/EarthAlign.
Authors:ZiAn Wang, Ruiqi Liu, Zhengbo Zhang, Yi Han, Jialiang Shen, Ruoxin Chen, Manni Cui, Ximo Zhu, Jin Jiang, Mingzhe Liu, Linhao Wang, Wenhao Wang, Shu Wu
Abstract:
Determining whether an image is AI‑generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence‑grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert‑reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine‑tuning on reviewed teacher trajectories, followed by privileged on‑policy self‑distillation applied to verified decisions from the policy's own investigations. On the 1,527‑image test set, the trained Qwen3.5‑9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same‑backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.Code, models, and data will be released at https://github.com/wangzian828/EVISCENE and https://huggingface.co/ZiAnwang/EviScene.
Authors:Qi Yin, XiaoLi Zhang, Yangming Shi
Abstract:
Navigating cluttered environments while carrying objects remains a challenging problem for quadruped robots. Compared with works that focused on load‑aware locomotion adaptation to maintain gait stability, existing autonomous navigation systems generally ignore the influence of carried objects' geometries and use fixed footprint inflation, leading to overly conservative navigation. To fill this knowledge gap, we propose a two‑phase load‑aware navigation learning framework. In Phase 1, we train LOADNav, a load‑aware navigation policy via reinforcement learning, enabling the robot to adapt its trajectory and clearance behavior according to the geometric size of the carried object. In Phase 2, we train CoGE, an online geometry predictor via imitation learning, which estimates unknown payload geometry from occasional contact events. We evaluate the framework in cluttered environments with loads of varying sizes and shapes. The results show that our navigation policy exhibits clear load‑aware behaviors, and the online geometry predictor estimates object geometry from occasional contact events, refining its estimate and adapting the navigation behavior accordingly. Without requiring any prior knowledge of the payload, it narrows the gap to geometry‑aware policies.
Authors:Li Jin, Jiang Wenbin, Chen Tao, Wen Fei
Abstract:
Spiking Neural Networks offer event‑driven, energy‑efficient computation for speech command recognition, yet most existing methods follow fixed computation paths, allocating redundant computation to easy samples while providing limited insight into how evidence evolves during inference. To address these challenges, we introduce SpikCTU, a compact spiking network with adaptive recurrent computation for speech command recognition. Specifically, a hybrid spiking embedding module preserves continuous input information alongside spike features to construct a fixed temporal memory. A shared Continuous Thinking Unit (CTU) repeatedly reads this memory using keys derived from spikes and continuous values, and updates a binary recurrent state with lightweight stage adapters. Furthermore, combining a fixed base prediction with accumulated corrections at each step enables learned aggregation over all steps or confidence‑based early exit using only the predictions available so far. Experiments on three speech command recognition benchmarks show that SpikCTU achieves competitive accuracy with only 0.16‑‑0.17M parameters, and the early exit strategy can substantially further reduce computational cost without compromising accuracy. In addition, trajectory and attention analyses reveal that recurrent steps enable error correction and progressively refine predictions. Code is available at https://github.com/Rivflyyy/spikctu.
Authors:Yongyuan Li, Yang Xu
Abstract:
To enable large language models to generate outputs that align with human preferences, optimization algorithms based on the comparison between chosen and rejected output pairs have been developed and been proven effective. Some recent endeavors have utilized certain linguistically‑derived assumptions about the preference pairs to guide the design of more precise optimization directions. However, human preferences are complex and hard to measure by simplistic criteria, and it is important to incorporate broader and more sophisticated insights about preferred vs. not‑preferred language. In this study, we adopt a well‑established psycholinguistic theory, Uniform Information Density, to develop a novel preference optimization algorithm, UIDPO. The basic assumption is that humans prefer to produce language in which information is uniformly distributed, because uniformity induces less cognitive processing effort. To implement this idea into the algorithm, we derive a compound metric for information uniformity by combining the global standard deviation and the local second‑order difference, and integrate it to the objective of preference optimization. We compare UIDPO with SimPO, a previous state‑of‑the‑art method, obtaining comparable and sometimes better performance on AlpacaEval2, ArenaHard, and MT‑Bench. Additional linguistic analysis on complement clauses also shows that UIDPO can better reflect the tendencies of keeping information uniform in natural language than baselines. Our results show that the subtle human preference over the information rate in language is a helpful source for achieving better human alignment for LLMs. The code repository is published here: https://github.com/CLCS‑SUSTech/UIDPO.
Authors:Yufeng Tian, Jiahao Tang, Minghao Wu
Abstract:
Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit‑assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action‑specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image‑to‑TikZ serves as an executable visual testbed for this setting. On 945 held‑out examples, learning after execution strengthens the shared policy under single‑call inference and also produces a post‑execution corrective capability. Single‑call render success reaches 92.70%, compared with 88.25% for continued single‑stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single‑stage training and depends on the intermediate state. Alternative credit definitions also induce different group‑relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at https://github.com/tianyufengcn/Learning‑after‑Execution.
Authors:Burak Topcu, Musa Cim, Zuhal Ozturk, Mahmut Taylan Kandemir
Abstract:
Large Language Model (LLM) inference is driving a growing share of datacenter carbon emissions. Although carbon‑aware scheduling can reduce these emissions in geo‑distributed datacenters by exploiting the spatiotemporal variations in grid carbon intensity, the latency‑sensitive nature of LLM serving constrains this design space. While prior studies investigate the carbon savings achievable through spatial scheduling, they oversimplify the dynamics of modern inference engines, such as queueing and continuous batching, which strongly shape inference latency. We address this issue by designing and evaluating a latency‑centric, geo‑distributed inference simulator, calibrated with hardware‑grounded latency and power measurements, that reproduces these dynamics and enables the study of carbon savings under strict service‑level objectives (SLOs). We then formulate a series of SLO‑ and carbon‑aware schedulers of increasing sophistication, from per‑request heuristics to batch‑level optimization: (i) a greedy heuristic, (ii) an integer linear programming (ILP)‑based scheduler jointly assigning batches of requests, (iii) an iterative ILP‑based solution projecting its own assignments back into the load estimates to capture load coupling, and (iv) a facility mixed‑integer linear programming (MILP) scheduler pricing the energy of batched requests collectively rather than in isolation. Across three LLMs under MLPerf SLOs and varying system loads, the MILP‑based option offers the best carbon‑SLO balance, reducing carbon emissions by up to 60% relative to the round‑robin policy, with an average SLO violation of 3%. Our characterization further shows that savings grow with the spread of carbon intensity across datacenters and shrink as system load increases, while SLO attainment depends on how frequently schedulers receive datacenters' load updates.
Authors:Weixi Song, Donglin Wang, Yong-Lu Li
Abstract:
Model‑based reinforcement learning has shown superior sample efficiency by leveraging the learned latent dynamics either in the form of derivative‑free planning/imagined rollout training or dynamics gradient‑based policy updates. While these approaches achieve comparatively fast learning, they all suffer, to varying degrees, from issues of recurrent compounding errors, high inference‑time latency, and instability in backward propagation through time (BPTT). In this work, we propose a model‑based RL approach that uses the chunk world model as a differentiable training‑time evaluator to optimize the chunk actor without recurrent model invocation. Through the collaborative chunk‑wise prediction of the world model and policy, we can generate H‑step hypothetical rollouts in a single forward pass and directly optimize value expansion, avoiding the costly and unstable BPTT. We evaluate our method on 19 challenging high‑dimensional humanoid tasks from HumanoidBench and deploy our method to the real‑world for quadruped bipedal walking. Project Website: https://song‑wx.github.io/cmpo.github.io/
Authors:Muqing Liu, Chongjie Si, Zhuoya Liu, Yuheng Jia
Abstract:
Large pretrained models have achieved remarkable success across diverse domains, yet adapting them to multiple downstream tasks remains challenging: full fine‑tuning is costly, while maintaining separate task‑specific models is storage‑inefficient and fails to produce a unified multi‑task model. Low‑Rank Adaptation (LoRA) offers an efficient alternative for Model Merging by representing each task adaptation as a low‑rank update over frozen pretrained weights. However, existing LoRA merging methods often treat task‑specific updates as directly comparable objects and merge them in their original low‑rank parameter spaces. This overlooks two key factors: which update directions provide reliable shared support across tasks, and how these directions should be re‑parameterized with respect to the pretrained weights before merging. To address this limitation, we propose Pretrained‑Guided Shared Basis (PGSB), a two‑stage framework for LoRA model merging. PGSB first identifies shared directions among task‑specific LoRA updates, and then re‑parameterizes these directions using their interaction with the pretrained weights, producing a more suitable basis for merging. Extensive experiments demonstrate that PGSB consistently outperforms state‑of‑the‑art baselines across multiple benchmarks. Codes are available at https://github.com/Kitschnada/Pretrained‑Basis .
Authors:Chenxi Xia, Yixuan Xu, Shan Liu, Jiapeng Wang, Maohua Li, Hanlin Tang, Kan Liu, Tao Lan, Weiyi Lu, Xinggao Liu
Abstract:
Autoregressive video generation models generate videos sequentially, with each new block conditioned on previously generated blocks. By training on the model's own rollouts, Self Forcing and its follow‑up methods substantially mitigate exposure bias caused by the mismatch between training‑ and inference‑time history distributions. However, these models still suffer from error accumulation during long‑horizon generation. We identify a key underlying limitation: the model remains highly sensitive to errors in its generated history, allowing small deviations to be amplified through subsequent generations. We refer to this underexplored problem as the local history robustness gap. To address this gap, we introduce Adversarial Gradient Forcing (AGF), a framework designed to improve the robustness of few‑step autoregressive video models to perturbations and accumulated errors in their own generated histories. AGF uses distribution matching gradients from future block predictions to select adversarial perturbations of self‑generated histories. It then jointly trains the generator on the original and perturbed histories to improve the robustness of future generation. Extensive experiments show that AGF achieves the highest VBench Total score on 60‑second video generation and the best visual quality on four‑minute video generation. Notably, AGF achieves such long‑horizon robustness while requiring only 5‑second rollouts for training.
Authors:Zihao Li, Yanan Zhou, Ranpeng Qiu, Guoqiang Ren, Weiming Zhi
Abstract:
In multi‑robot manipulation, an action's value depends on how teammates respond. We present \method (Cooperative Action‑‑Response Belief Estimation), a framework for learning \emphteam‑response beliefs from local history. A predictive belief encoder represents the interaction and conditions a nominal action. Matched reactive and replay continuations separate each candidate's direct advantage from the utility contributed by teammate adaptation. A candidate‑conditioned readout learns this response distribution alongside total advantage, and calibrated selection chooses an alternative or retains the nominal action. Our analysis characterizes when response changes the preferred action and how prediction and source errors limit the recoverable gain. With the representation, proposals, and data fixed, explicit response supervision improves candidate ranking and reduces selection regret relative to learning the same information through branch utilities. The benefit is concentrated in decisions where teammate adaptation changes the preferred action. Across RoboFactory, the MARS Control Track, and physical collaboration tasks, CARE achieves the highest average success among the evaluated policies, combining a capable nominal policy with selective coordination adjustments. Together, these results show that CARE turns predictions of teammate adaptation into effective decentralized action selection. Project page: https://jeong‑zju.github.io/care/.
Authors:Shihang Li, Thomas Anderson, Ratul Mahajan, Simon Peter, Luke Zettlemoyer
Abstract:
To operate distributed applications with high availability, operators must rapidly diagnose and mitigate any incident. LLM‑based agents can help, but unlike experienced human operators they do not improve as they handle more incidents. Each investigation starts from scratch, re‑deriving likely causes and re‑testing hypotheses even when a similar fault was resolved in the past. We present a self‑defining operator (SDO) that improves with time by updating its operational behavior and memory. When SDO resolves an incident, it distills that experience into playbooks, which describe reusable diagnostic instructions for future incidents with similar faults, and detectors, scripts that take application telemetry as input and output relevant playbook(s). SDO continuously runs the detectors and invokes LLMs only when one fires, so its token cost scales with incident count rather than with operational time. Across 11 open‑source microservice applications, SDO cuts resolution time by 52% on recurring faults while keeping per‑incident cost low. SDO is open source at https://github.com/self‑defining‑systems/sdo.
Authors:Shaohao Zhu, Wensi Huang, Siqi Zhang, Yuqiang Yang, Meng Wei, Chenyang Wan, Tianle Liu, Tai Wang, Jinming Xu
Abstract:
Emerging strong Vision‑Language Models (VLMs) have advanced navigation foundation models. However, these models still have unsatisfactory performance on goal‑oriented navigation settings. In this paper, we first conduct a motivating study revealing that the key issue lies in the insufficient utilization of streaming observations and imprecise target localization. To tackle these challenges, we propose GG‑Nav, a simple yet effective navigation framework that learns a structured chain‑of‑thought (CoT) mechanism via supervised fine‑tuning to explicitly localize goal‑relevant instances, accumulate grounding evidence, and support more reliable decision‑making. Specifically, the structured CoT is designed with two key stages: (i) Observation‑to‑Evidence, which reasons over all intermediate frames between adjacent decision steps to retain useful goal‑related evidence; and (ii) Evidence‑to‑Decision, which integrates the current observation with accumulated evidence to ground the target and support the generation of navigation actions or clarification questions. Extensive experiments show that GG‑Nav outperforms the state‑of‑the‑art methods in terms of success rates on the HM3D ObjectNav, VL‑LN, and CoIN benchmarks by 3.1%, 16.6%, and 5.3%, respectively. Ablation studies further show that accumulating evidence from intermediate observations reduces evidence‑missing failures by 11.6%, while explicit grounding reduces stopping errors by 49.4%. We also deploy GG‑Nav on a Unitree Go2 robot, demonstrating its zero‑shot generalization capability in unseen real‑world environments.
Authors:Huaiming Li, Can Huang, Chufan Wu, Danyu Liu, Xiaomin Yuan, Qinze Li, Yui Lo, Wenhui Bai, Qianlong Wang, Zengle Ge, Xiyu Yan, Yuheng Cheng, Jinhui Ren, Guokai Chen, Jiaqun Liu, Haoran Li, Qianhui Liu, Annan Li, Jianmin Wu, Dawei Yin, Dou Shen
Abstract:
Real‑world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience. Forward‑deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria. Existing optimization benchmarks typically start from explicit objectives, while clarification benchmarks often introduce information gaps into existing tasks. We introduce FDE‑Bench, the first benchmark built from real customer projects to evaluate agents on end‑to‑end delivery tasks that require interactive requirement clarification and iterative optimization. Its 49 cases span combinatorial optimization and machine learning. Clarification targets are distilled from discussions between frontline practitioners and FDEs. Agents elicit missing requirements from a simulated customer, and case‑specific evaluators assess final artifacts against acceptance criteria using customer‑accepted FDE solutions as quality references. Across 18 agent configurations, the highest mean delivery score is only 0.5510, compared with an FDE reference of 1.0. The highest delivery success rate (DSR), defined as the fraction of planned runs reaching this reference, is only 6.12%. These results show that reliable FDE‑level end‑to‑end delivery remains out of reach for the evaluated agents on underspecified real‑world business tasks. The benchmark project page is available at https://baidubce.github.io/FM‑Agent/FDE‑Bench/.
Authors:Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang, Yuanyuan Fang, Zhaopeng Wang, Lianwen Jin
Abstract:
Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non‑specialists. Despite the success of Large Language Models (LLMs) and Large Vision‑Language Models (LVLMs) in modern document tasks, their capabilities in Long (full‑book) Historical Document understanding remain largely unexplored. To bridge this gap, we introduce LongHisDoc, the first benchmark tailored for long historical document understanding. LongHisDoc comprises 1,012 expert‑annotated QA pairs derived from 101 PDF documents, with an average of 85.1 pages and 42,776.4 tokens per document. It spans 10 categories and features four QA types: single‑page, cross‑page, multi‑hop, and unanswerable. Answering these questions requires integrating multimodal evidence from text, layout, and visual elements. Our evaluation reveals that the top‑performing LVLM and LLM achieve only 62.9% and 69.9% accuracy, respectively, highlighting a substantial performance gap. Furthermore, we assess various modal retrievers and the end‑to‑end RAG performance to provide references for Retrieval‑Augmented Generation (RAG) systems. We believe that LongHisDoc serves as a comprehensive benchmark to catalyze future research in full‑book historical document understanding. The benchmark and evaluation code are available at https://huggingface.co/datasets/qweq12433454/LongHisDoc and https://github.com/SCUT‑DLVCLab/LongHisDoc.
Authors:Benyuan Meng, Longtao Huang, Yang Liu, Liang Lin
Abstract:
During a network forward call, diffusion backbones naturally produce many intermediate activations. If we evaluate each activation against an identical downstream benchmarking protocol, we can plot an inverted‑U‑shaped curve as the depth profile: the activation's performance goes up first and then down as we go into deeper layers. This is common knowledge in the diffusion model community, which inspires many applications including the widely‑adopted REPA. However, the common interpretation of this depth profile may be inaccurate and incomplete: (i) The cause of this curve has long been treated as signal granularity changing from coarse to fine, but there has been little evidence in DiT backbones. (ii) The community has been mainly studying this phenomenon in a static way, without investigating how it changes across different training setups. In this work, we show a different mechanism picture for the profile, where the prediction target (e.g., ε, u, x_0) sits at the center. The semantically meaningful signal is only an intermediate by‑product for predicting the target, while the prediction target is the real goal for the backbone. Furthermore, the choice of prediction targets as part of the training setup can predictably affect the curve's shape. With this updated interpretation, many applications of this depth profile can be better grounded and predicted. As an example, we show how to predict the near‑optimal placement of REPA based on the prediction target of the training setup. The code is at https://github.com/Darkbblue/dit‑compare.
Authors:Yi Liu, Junwei Zhao, Li Sun, Jiansong Zhang, Sen Su
Abstract:
Long‑horizon multi‑agent collaboration requires memory that preserves both successful actions and the coordination structure underlying them. Existing memory systems support information sharing or procedural reuse, but often leave the evolution of shared procedures disconnected from the task dependencies and execution states of collaborating agents. This coupling also limits the direct transfer of procedural memory designed for individual agents to team settings. We propose Self‑Evolving Procedural Memory (SEPM), a framework that uses task dependency graphs as a common representation for online coordination and procedural memory evolution. During execution, a lightweight shared blackboard records structured evidence alongside task dependencies, supporting the detection of goal, plan, and world‑state divergence and evidence‑based realignment. After execution, verified actions and dependencies are distilled into reusable procedures, while recurring failures inform warnings and constraints. The shared procedure pool evolves through creation, refinement, specialization, and retirement, guided by success and cost utility and constrained by evidence and safety requirements. Together, these mechanisms preserve the collaborative structure and applicability conditions needed to reuse experience across tasks. Evaluations on ALFWorld, WebArena, and OfficeBench under AutoGen and DyLAN show that SEPM achieves the highest task scores among the compared methods in all six benchmark settings. Under AutoGen, it improves task scores over the no‑added‑memory baseline by 29.85, 25.86, and 13.00 percentage points, respectively. The codebase is available at https://github.com/cookieApril/SEPM.
Authors:Sudarshan Regmi, Nabaraj Subedi, Arvind Pillai, Yu Yvonne Wu, Yuliang Chen, Michael V. Heinz, Lisa Marsch, Nicholas C. Jacobson, Andrew Campbell
Abstract:
Generative modeling offers a promising solution to the scarcity of labeled clinical electroencephalography (EEG) data under strict privacy constraints. Although recent generative models report substantial improvements in generation fidelity, we show that these gains conceal a categorical failure. EEG is not an arbitrary multichannel signal: each channel records the voltage difference between two scalp electrodes. So, real recordings satisfy exact linear identities among their channels. Current generators invariably violate these identities, yet this has gone unnoticed. We therefore propose MontageFlow, a flow‑matching model that enforces these identities by restricting its output to signals that satisfy them. We find that ensuring validity improves fidelity too. Furthermore, to retain the EEG characteristics that oversmoothing washes out, we also condition the model on a physiological state vector encoded from the whole EEG window. For generation, this vector is drawn from a prior fitted in latent space. Across 3 large‑scale benchmarks, our model improves TS‑FID by 1.6‑3.6× over the closest rival while ensuring validity.
Authors:Wei Lin
Abstract:
Explainer videos ‑‑ a worked geometry problem, why the sky is blue, how a dynasty fell ‑‑ are usually one more target for pixel video generation. We argue they are a different kind of artifact: a program (script, storyboard and per‑frame animation code) that a deterministic renderer turns into frames. A program can be read, diffed, tested and audited; pixels can only be watched. Reels‑RSI is an open‑source reel director for any agent harness; its self‑improvement happens at the program level rather than in weights or prompts. Every revision pass is logged with a code snapshot; a retrospective pairs each check finding with the diff that fixed it and proposes lessons; a lesson that a static check can catch is compiled into a rule with a must‑fail and a must‑pass example, so the memory is unit‑tested; and a rule learned today runs over everything made before it. A bounded, benchmark‑gated step lets an agent rewrite the director's playbook, kept only when blind pairwise judgments on held‑out topics prefer it; nothing enters memory without a human decision. Across 111 reels ‑‑ a field study, two playbook evolutions, a 2x2 memory ablation, a weaker director, a 48‑reel learning curve and a judge‑reliability study ‑‑ we find that the rule learned in the field caught four latent defects in frames authored before it existed and none after; that rules are an audit for a strong director (eight unguarded reels, no violation) and a guard for a weaker one (thirty violations unguarded versus five); that lessons shorten revision loops but do not compound or move the judge; and that the first rewrite's 11‑‑1 verdict was a position bias in our vote design (judges pick the second‑shown reel 62‑‑81% of the time), masking scripts that traded thirteen facts for craft. We document four failure modes of the preference signal, which check caught each, and the fixes. Code, rules, lessons and data are public.
Authors:Daniel Commey
Abstract:
Coding agents read repositories and web pages that anyone can write, and then act with their user's authority. Prompt‑injection defenses make it less likely that the model is steered but do not prevent it, so the system around the model must contain an agent that is already compromised. Many exploits against deployed coding agents share one cause: a check approves a name, such as a path, an argument vector, or a directory, and a later step maps that name to a different object: the file behind a symlink, the behavior an option selects, or a repository that git discovers on its own. We present Zero‑Trust Agent Cells (ZTAC). A cell runs the agent in a kernel sandbox whose only route to files, programs, and the network is a socket to a broker. The broker resolves each request to the object it will act on, decides on that object, and checks at use that it is acting on the object it authorized. Programs that the broker starts run in a sandbox compiled from the same policy, and approvals are bound to the resolved object and can be used once. Our macOS prototype has 3.2k lines of trusted Rust code, excluding dependencies. We evaluate it against 74 attacks with ground‑truth oracles, including replays of 10 publicly disclosed incidents. A tool allowlist loses 66 of them, and an allowlist combined with the kind of command sandbox that deployed agents use loses 56. ZTAC loses 2, both of which encode a secret into traffic to an allowed endpoint, and holds 14 for review. It completes 43 of 46 benign operations, with 8 approval prompts, and 4 of 7 workflows. A no‑op brokered request has a median latency of 4.0 ms, and the spawned programs we measured take 8.5 ms to 18.4 ms longer than without ZTAC.
Authors:Milind Srivastava, Zeying Zhu, Peilin Xin, Yancheng Yuan, Zaoxing Liu, Vyas Sekar
Abstract:
Many large enterprise and cloud deployments rely on metrics observability solutions (e.g., Prometheus) to monitor key performance indicators of system health (e.g., CPU, memory, network, app‑level statistics). Monitoring and analyzing these metrics presents a significant challenge in terms of cost‑latency tradeoffs at scale. Ideally, system operators would like immediately deployable and backwards compatible solutions to improve the cost‑latency tradeoffs without significant impact on the fidelity of their observability applications. Unfortunately, existing solutions including state‑of‑art timeseries databases and custom approximate querying solutions fail on one or more aspects. In this paper, we present the design and implementation of TurboProm, a backwards‑compatible drop‑in accelerator for metrics observability deployments. TurboProm recasts metrics analytics tasks using a semantic‑preserving summary view rather than classical query‑centric view. Instead of estimating query results on raw timeseries metrics data, TurboProm uses semantic‑preserving summaries that enable high‑fidelity analytics with significant cost‑latency improvements. We present a practical implementation of TurboProm that can be retrofitted to any existing observability deployment. We evaluate TurboProm on diverse real and synthetic workloads against state‑of‑art solutions and show that it can provide significant reduction in query latency and cost. TurboProm is open‑source at https://github.com/ProjectASAP/ASAPQuery.
Authors:Cong Wang, Jin Li, Shoujin Wang, Yishuo Li, Huilin Gu, Wenpeng Lu
Abstract:
Medication recommendations seek to deliver personalized drug combinations based on patients' clinical information, where ensuring transparent decision‑making and personalized safety are of paramount importance. However, existing approaches predominantly operate as ``black boxes'', offering limited interpretability and hindering clinician trust. Furthermore, they typically rely on fixed, patient‑agnostic safety constraints, neglecting the heterogeneity of risk tolerance across individuals, which poses significant risks to vulnerable populations. To address these gaps, we propose RES‑MR, a novel Risk‑aware Reasoning framework for Explainable and Safe Medication Recommendation using LLMs. RES‑MR follows a two‑stage training paradigm: (1) an explainable clinical reasoning distillation stage to elicit diagnostic capabilities by fine‑tuning on synthesized reasoning trajectories, grounded in patient‑specific knowledge from graph retrieval; and (2) a risk‑aware policy optimization stage to dynamically balance therapeutic efficacy with personalized safety. This stage employs disentangled drug factors to construct patient‑specific reference prototypes that capture desired therapeutic drug features and risk features to avoid. These prototypes enable risk boundary calibration and guide the model toward safe recommendations via reward shaping and clipping. Extensive experiments on benchmark datasets demonstrate that RES‑MR significantly outperforms state‑of‑the‑art baselines in terms of accuracy, safety, and explainability. Our code and data are available at: https://github.com/wangcong2001/RES‑MR.
Authors:Abhishek Sharma
Abstract:
Reconciliation systems decide, continuously, whether money the ledger expected has arrived at the bank. On an asynchronous stream that decision is taken against partial evidence: a credit that has not appeared may be missing, or may simply not have been posted yet. Deployed pipelines settle this with a timeout or a watermark, and the exceptions are treated as findings. We give a temporal correctness contract that says when such a verdict is admissible, relative to a declared per‑source lateness bound, and a checker for it that needs no ground truth. The contract separates three things an operator otherwise sees only as a rising break rate: the reconciler decided too early, an input broke the bound it declared, or the money really is gone. On workloads from a published payment simulator, the standard remedy for a stalled streaming join, an idleness timeout, introduces contract violations on 5.42% of obligations (95% CI [3.94, 6.92]) on traffic where the same job without it is exactly sound, and violations vanish once the timeout reaches the slow source's posting period, as predicted. Under a lateness shift, all four identity‑based reconcilers we test raise spurious breaks on between 18.3% and 23.0% of the credits the bank did settle, while their contract violation rate stays at or below 1.01%: the checker attributes the loss to 240 bank‑side and 413 ledger‑side bound violations rather than to the reconcilers. Tracking one horizon per source leaves break latency 12.7% lower on average and halves its tail (p99 224 against 446 ticks) at equal soundness, though it is not cheaper at every cost setting and we report where it loses. An Apache Flink interval join, written the ordinary way, violates the contract on 14.9‑‑20.1% of obligations. The contract is expressible as SQL over the tables a reconciliation system already keeps, agreeing with the reference implementation on all 4,395 verdicts.
Authors:Jiayi Wu, Tianfu Wang, Rui Zhao, Florian Schiffers, Zhimin Chen, Cornelia Fermuller, Yezhou Yang, Christopher Metzler, Yiannis Aloimonos
Abstract:
In film production, chroma keying composites an approved foreground performance into a new environment, but conventional workflows require coordinated background plates and dedicated harmonization. Recent generative methods often require pre‑aligned backgrounds, focus on background‑conditioned foreground relighting, or represent the environment only through an edited first frame. These restrictions limit background selection, scene composition, and camera motion while adding preparation or editing effort. We present FreeCHROMA, which adapts a pretrained video diffusion model in a parameter‑efficient manner for generative video compositing from one to three unaligned background references. The model integrates the foreground into the referenced environment, preserving foreground identity, motion, and fine visual details while generating a spatiotemporally coherent background. Foreground‑background harmonization occurs within the same generation process, without a separate relighting stage. Optional text instructions help disambiguate complex spatial relationships and camera behavior. To train the model, we construct pseudo‑paired data from readily available real monocular videos through foreground‑background separation and augmentation with existing foundation models. The original videos serve as training targets, while automated processing and quality verification enable scalable data construction. Across reference fidelity, foreground preservation, scene integration, harmonization, and overall video quality, FreeCHROMA consistently outperforms recent compositing and conditional video‑generation baselines, with particularly clear gains under camera motion. FreeCHROMA relaxes the constraints of existing tools to enable end‑to‑end generative chroma keying from unaligned inputs, with exploratory experiments demonstrating its potential in robotics.
Authors:Chaoyue Li, Shoudong Han, Haoyang Wu, Boxue Yang, Sijia Chen, Yufan Chen, Ruping Zhang
Abstract:
Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over time and relating their observed motion to subsequent changes. We formulate Scene Evolution Understanding (SEU), a task for predicting and interpreting this change. Given observed video, a model estimates entity histories and predicts one scene continuation through future trajectories and Events, accompanied by Visual Evidence from observed motion and a scene description. We construct SceneForesee, a benchmark for evaluating entity continuity and scene evolution reasoning through paired observed and future scenes with shared entity identities. It contains 144.57K temporal pairs, 1.05M entity correspondences, and 272.49K Events paired with observed evidence. We propose a spatiotemporal reasoning framework, EvoSee, that learns future geometry and Event semantics through shared candidate states and joint assignment. It selects one continuation per entity from observation and uses the selected changes for entity refinement, retention, and scene representation. On SceneForesee, EvoSee outperforms Qwen3.5‑9B by 10.45 points in Combined Point‑F1 and 17.24 points in Event Global F1. On the 4D reasoning benchmark Dyn‑Bench, it improves question answering accuracy and grounding J&F by 4.8 and 3.58 points over their respective base systems. These results support scene evolution as both a prediction target and a dynamic representation that brings predicted entity changes into spatiotemporal reasoning.The project is available at https://github.com/mikubaka88/SceneForesee‑Spatiotemporal‑Reasoning.
Authors:Haowen Pan, Pengyang Shao, Yunshan Ma, Yanzheng Jin, Naixin Zhai, Xiang Wang, Xun Yang
Abstract:
Long‑term memory enables LLM agents to learn from past interactions but exposes memory formation to adversarial influence. Existing studies primarily organize memory poisoning by attack channels, corruption patterns, or lifecycle stages, leaving formation interfaces insufficiently characterized. We formulate Memory Formation Poisoning as adversarial manipulation of the interfaces through which persistent agent memory is produced and committed, and introduce MemForm, a controlled diagnostic benchmark covering five interfaces: Content, Evidence, Scope, Selection, and Path Poisoning. We introduce Selection and Path Poisoning as controlled intervention categories that isolate retention and consolidation order effects, respectively. Each category pairs clean and poisoned conditions with interface‑specific controls. Evaluations across six LLMs and two response formats reveal broad susceptibility at all five interfaces under the evaluated memory protocols and implementations. Crucially, authentic experiences with successful outcomes do not guarantee reliable behavior after memory formation: changes to contextual support, retention, or consolidation order can induce undesirable downstream behavior. In particular, consolidation order can alter behavior even with identical retained experiences, depending on the memory formation procedure. Additional analyses show that interface applicability depends on memory architecture and that Path Poisoning susceptibility depends on the updater. The evaluated defenses provide incomplete mitigation that varies by interface. These findings motivate protecting both what is committed to persistent memory and how memory is formed from experiences. Code and data are available at https://github.com/opanhw/memform‑code.
Authors:Abhishek Sharma
Abstract:
When a merchant authorizes a card and captures the funds later, the money sits in a state that neither party fully controls: held on the cardholder's account, not yet taken, and due to lapse on a deadline set by the card network. What a merchant may do to that hold, capture part of it, capture again, void it, raise it, let it expire, is business logic that every payment service provider implements slightly differently and none of them specifies formally. We give the lifecycle a state machine in TLA+, parameterized by a provider capability profile, and check eight safety invariants and three liveness properties over 6 profiles, five of them built from Stripe's and Adyen's published documentation. From the same machine we generate a conformance suite that runs black‑box against a provider API. Against 61 seeded defects that are distinguishishable from a correct implementation, the generated suite detects 100%, where random sequences with the same call budget reach 56% and a hand‑written integration suite reaches 69%; random sequences given ten times the budget still reach only 67%. Running the same suite between two providers turns the comparison into a search, and it returns short executable witnesses for divergences the two documentation sets imply but neither states, including one that breaks the common assumption that a single adapter interface can sit over both, because closing a hold and releasing its remainder is a capture on one provider and a cancel on the other. We report what we could not do: no provider sandbox credentials were available, so every result here concerns documented behavior, and the suite that would test the providers themselves is part of what we release.
Authors:Shihan Zhao, Henry Chen, Ya-Chuan Hsu, Minjune Hwang, Siddharth Srikanth, Freddie Liang, Akanksha Saran, Aaquib Tabrez, Stefanos Nikolaidis, Daniel Seita
Abstract:
A long‑standing goal in robotics is to develop generalist robots that can perform tasks across a wide range of settings, flexibly specified with images and text. Vision‑Language‑Action (VLA) models offer a promising path toward this goal by training on large collections of robot demonstrations. However, while VLAs perform well in‑distribution, they often fail under minor test‑time shifts, such as changes in object positions, lighting, textures, and camera poses. To address this, we propose Quality Diversity for Diverse Environment Generation (Q‑DEG), a method for improving VLA robustness by generating adversarial environments and fine‑tuning the VLA on its own successful rollouts. Q‑DEG uses Quality Diversity (QD) optimization to find a set of high‑failure environments, while promoting diversity using policy embeddings extracted from the VLA. These environments expose vulnerabilities that can be mitigated through self‑imitation fine‑tuning, thereby improving the VLA's robustness. In simulation, we conduct experiments across three representative VLA models, and our results indicate that Q‑DEG improves robustness to out‑of‑distribution (OOD) adversarial environments compared to the base VLA and VLAs fine‑tuned using data collected from two red‑teaming baselines. Furthermore, we demonstrate that improving VLA robustness through Q‑DEG facilitates sim‑to‑real transfer. Our anonymous project website is at: https://qd‑env.github.io/qdeg/.
Authors:Yilin Wu, Rutav Shah, Shuaiyi Huang, Sumedh A. Sontakke, Homanga Bharadhwaj, Andrea Bajcsy, Max Simchowitz, Guanya Shi
Abstract:
It is widely appreciated that robotic policies require long‑term memory to perform complex tasks. However, directly conditioning policies on all past observations leaves policy learning susceptible to learning spurious shortcuts, and introduces large latency at policy deployment. As such, today's memory‑augmented policies curate what to remember, either by retaining only recent frames, selected keyframes, or textual summaries, or by learning latent representations of the visual stream alone. In this work, we propose an alternative approach IMPRINT, a policy‑agnostic latent memory framework that augments vision‑language‑action (VLA) models with multi‑stream memory without explicitly constraining what the policy should remember. IMPRINT compresses different streams into a latent memory that is updated incrementally at inference, enabling low latency. To avoid shortcut learning, IMPRINT adopts a two‑stage training procedure, first supervising memory to reconstruct multiple streams (observations, actions, proprioception) of past information, as well as estimate current task progress, followed by end‑to‑end finetuning, jointly with the policy, on action prediction. Across both simulation and real‑world tasks demanding memory, IMPRINT improves average success over the base VLA by at least 26%, outperforming popular alternatives, and maintaining lowest inference latency among memory‑augmented VLAs. Through careful ablations, we demonstrate the utility of incorporating diverse information streams into robotic memory, validating the advantages of foregoing history curation. Project website: https://yilin‑wu98.github.io/imprint_website/.
Authors:Liu Jiaqi, Jin Ruinan, Li Xiaoxiao
Abstract:
Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer. Errors and hallucinations at these steps can prevent acquired experience from translating into reliable single‑run performance. We propose VisualCoding, which combines execution‑guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution‑Guided Program Acquisition composes visual primitives and additional code into family‑level solvers, then iteratively tests and revises them on labeled training instances. Scope‑Aware Direct Execution runs a retained solver and returns its valid in‑scope answer. We evaluate VisualCoding on four models across four benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini‑3.1‑Pro and by 14.78 points when reusing the frozen Pro‑acquired artifacts with Flash‑Lite. Ablations support execution‑guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq‑liu/VisualCoding.
Authors:Jialiang Sun, Kuldeep Meel
Abstract:
Large language models (LLMs) have achieved strong performance in mathematical and logical reasoning, yet they still suffer from the curse of complexity and struggle as problem difficulty increases with the number of constraint interactions and the size of the search space. In contrast, satisfiability solvers such as Z3 provide scalable and verifiable reasoning through constraint solving under formal theories, but lack the flexibility to interpret natural‑language problem descriptions. In this work, we propose SAT‑RL, a reinforcement‑learning (RL) framework that trains LLMs to interact with symbolic solvers through multi‑turn tool execution. SAT‑RL leverages synthetic data generation with verifiable solutions, supervised warm‑up on solver‑augmented traces, and solver‑grounded reinforcement learning with tool‑aware rewards. Experiments show that SAT‑RL substantially improves symbolic reasoning with tools on both in‑domain and out‑of‑domain logic tasks. Our code is available on https://github.com/sunjia72/SAT‑RL.
Authors:Solvin Sigurdson, Prasanna Sriganesh, Juan Borbon, Joel Burdick, Dingqi Zhang, Jiuguang Wang
Abstract:
Retargeting human demonstrations into dynamically feasible dexterous hand trajectories requires resolving complex contact physics, yet current methods either omit physical constraints or rely on computationally expensive sampling‑based optimization in simulation. Reinforcement learning has previously been used to achieve planning and control through contact dynamics; however, standard policies require object‑specific training and fail to generalize across arbitrary geometries. This work reframes physics‑based retargeting as a residual learning problem, where we demonstrate that zero‑shot generalization across objects relies on structured state representations. Specifically, formulating the observation space to include a localized contact‑error signal guiding each robot link to its target coordinate on the object surface enables a single, unified residual policy to successfully grasp novel, unseen object geometries. Rolling out this policy in simulation on a kinematic reference trajectory yields a physics‑based retargeter, OATH. Operating at 0.4 seconds of compute per second of reference, OATH achieves 2.4x faster‑than‑real‑time performance and a 60‑350x reduction in computational cost compared to sampling‑based methods. The system recovers 58% of held‑out and 54% of unseen‑object DexYCB trajectories while maintaining a sub‑2 cm tracking error.
Authors:Mahesh Reddy Pagadala
Abstract:
Dynamic Tensor Rematerialization (DTR) trains neural networks under a memory budget by evicting tensors and recomputing them on demand. A natural expectation is that a budget large enough to complete training stays sufficient when it is increased. Using DTR's reference simulator, simrd, to replay the public DTR training traces, we show that this expectation fails. We call the exceptions feasibility holes: memory budgets at which an eviction policy runs out of memory although both a smaller and a larger budget complete, a form of non‑monotone feasibility. Under a written sweep protocol with dated amendments, a 0.01‑step budget grid shows no such hole for DTR on any of six confirmatory traces, although unresolved timeouts prevent it from establishing monotonicity. Finer sampling finds holes on InceptionV4 (three regions) and U‑Net (two regions), in addition to one on the ResNet‑32 development trace. Every hole is reproduced by three repeats and by unmodified simrd. For one InceptionV4 pair (0.2343 versus 0.235 of peak memory) we trace the failure to individual allocations. The larger budget avoids one eviction at an operator where the smaller budget was 173,479 bytes short. The extra data it carries forward forces a later eviction of a tensor needed three operators afterwards, and rebuilding that tensor triggers a recomputation cascade that cannot fit. Keeping that one tensor resident under the same budget prevents the failure. The two simple score changes we tested do not remove holes across traces; both add failing budgets on some traces. All results are from simulation, with no allocator model; behaviour on GPU hardware is untested.
Authors:Mahesh Reddy Pagadala
Abstract:
We study learned one‑step predictive control of a synthetic population of 64 leaky integrate‑and‑fire neurons whose light sensitivity is scaled by a hidden factor s, using two prospective confirmations on plants simulated only after the analysis was frozen. In the first (50 plants, 11 conditions), removing the 200‑ms input‑output history substantially worsened a parameter‑count‑matched neural predictor at low and moderate gain (s = 0.6, 0.8; RMSE rose by 2.47 Hz at s = 0.6, 97.5% CI 2.43‑2.51). In secondary comparisons, recursively adapted ARX had the lowest mean error of the deployable controllers in 10 of 11 conditions, tuned PI with anti‑windup matched or beat the network for s >= 0.7, and training for 10,000 rather than 900 epochs lowered prediction error but worsened control. Development work suggested the network's learned action sensitivity did not track the hidden gain. A second protocol (50 new plants, 9 sensitivities, 2 noise levels) confirmed three pre‑specified predictions at the primary noise level. Re‑estimating a single multiplicative input gain recovered 87% (95% CI 86‑88%) of the benefit of full adaptive ARX. Supplying that online estimate to the network as an extra input reduced high‑gain tracking error by 1.84 Hz [1.81, 1.88]. Longer training worsened the plain network by 1.09 Hz but improved the gain‑input network by 1.07 Hz, a differential of 2.16 Hz [2.13, 2.20]. Under high process noise (sigma = 2.0) the effects shrank to about 0.04 Hz or less and gain‑only adaptation recovered only 24% of the adaptive‑ARX benefit. In this synthetic plant, at the primary noise level, one‑parameter online gain estimation accounts for most of the gap between a history‑based network and full adaptive ARX, and for its prediction‑control mismatch.
Authors:Bishnu Bista, Dhanraj Haibatrao Jadhav, Vibha Choudhary, Gaurav Saxena, Arun Pandiyan Perumal, Arjun Subedi, Jignesh Ammineni
Abstract:
Language model classifiers are increasingly deployed for high‑stakes decisions such as phishing screening, content moderation, and fraud triage, yet validation‑set thresholds provide no finite‑sample control of deployment‑time false positives. We present ConformalGuard, a model‑agnostic calibration layer that wraps a fixed scoring function with finite‑sample control over the rate at which legitimate inputs trigger an intervention tier. Our contribution is empirical and operational, not a new conformal theorem: zero‑shot LM API scores can be coarse enough (10‑‑11 unique values here) to make temperature scaling and isotonic regression operationally identical to raw thresholding for low‑FPR action gating. We compare these methods with conformal calibration for fine‑tuned BERT, Claude Sonnet 4, and GPT‑4o on public email corpora. For Claude, conformal calibration reduces FPR from 0.040 to 0.008 while recall falls from 0.938 to 0.875. For deterministic GPT‑4o scores, the 1% operating point is too conservative to deploy (recall 0.564); three‑run stochastic averaging raises recall to 0.732 at FPR 0.005. At 2% prevalence with BERT, conformal calibration removes 25 false‑positive reviews per 10,000 emails (95% CI [10, 43]). The guarantee is legitimate‑event‑rate control under exchangeability; discrimination, workload effects, and robustness to shift remain empirical.
Authors:David R. Martin, Jared Paquet, Mohammad Abdullah Al Faruque, Pramod Khargonekar, Kyle Volle, J. Humberto Ramos, Scott Nivison
Abstract:
We present CoPlanVLM, which uses a Vision Language Model (VLM) to translate a single natural‑language instruction and image into coordinated paths for a team of two robots. While VLMs offer a flexible, natural‑language interface for commanding robot teams, their weak spatial reasoning has greatly limited VLM‑based path planning. CoPlanVLM improves the VLM's spatial reasoning and path planning abilities through three components: (1) an image overlay marking both free space and obstacles, (2) task classification that selects the most appropriate low‑level planner and reasoning prompt for each instruction, and (3) task‑specific chain‑of‑thought reasoning that prompts the VLM to identify key objects, allocate roles, and verify routes are collision‑free before committing to a plan. Together, these components enable the team to complete complex coordinated maneuvers requiring both robots and divide large tasks between robots to increase efficiency. CoPlanVLM also supports detailed maneuvers involving specific routes, loops, and coverage of large areas, as well as tasks requiring directional reasoning such as north/south or left/right. We validate CoPlanVLM in both Gazebo simulation and on physical Turtlebot 4 hardware. CoPlanVLM attains an average success rate of 84.4% compared to 50.0% for the closest baseline. On trials where both methods succeed, CoPlanVLM's paths are 27% shorter on average. Project code is available at https://github.com/AICPS/CoPlanVLM.
Authors:Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri, Amina Lamghari
Abstract:
TREGO, a trust‑region Bayesian optimization (BO) method for expensive blackbox problems, has convergence guarantees, but only for exact function values. In high dimension, its performance tends to deteriorate. This paper asks whether a trust‑region BO method can be extended to handle noisy evaluations while being competitive in high dimension. We first generalize TREGO into a trust‑region framework for noisy BO and extend its convergence analysis to noisy function estimates that are sufficiently accurate with sufficiently high probability. We then propose STREGO, a scalable implementation of this framework. Its global phase selects a batch of candidates by solving a bi‑objective problem on the posterior mean of a Gaussian process and the integrated variance reduction. Unlike the predictive variance, this second objective does not favor boundary points merely because they are far from the data. Its local phase minimizes the posterior mean of a second Gaussian process, trained only on points close to the incumbent. STREGO is compared with TREGO, TURBO, BAXUS and NSMA‑BO on synthetic and real‑world problems of dimension 4 to 124, with and without noise. It gives the best average results in every synthetic setting tested and the best or tied‑best solution on four of the six real‑world problems. An implementation of STREGO is available at https://github.com/Mayousk/STREGO.
Authors:Yang Liu, Hanchen Cui, Karthik Desingh
Abstract:
Latent world‑model planning has shown promising results in goal‑conditioned robotic manipulation, but remains challenging for long‑horizon planning due to limited temporal guidance and inefficient action search. A distant visual goal provides little guidance for preparatory actions, while uninformed action sampling makes local optimization inefficient. We propose PILOT (Policy‑Initialized Local Optimization with Temporal subgoals), which uses a video model to generate intermediate visual subgoals and a learned policy to initialize the cross‑entropy method (CEM). CEM refines candidate action sequences toward each subgoal using an action‑conditioned world model in a receding‑horizon control loop. In our primary real‑robot evaluation across four UR5e manipulation tasks, PILOT achieves 68.8% success, compared with 55.0% without policy initialization, 13.8% for policy‑only execution, and 0.0% for direct final‑goal planning. Simulation results show that these gains persist over longer horizons: PILOT exceeds direct execution of the same policy by 30 percentage points on long Push‑T tasks and by 22 percentage points on LIBERO‑10. Policy initialization also accelerates local optimization: on Push‑T, CEM matches the cost reached by generic initialization after 10 iterations in only 4 iterations and reduces the final planning cost by 24.1% under the same sampling budget. These results show that combining temporal subgoals with policy‑guided action search improves both long‑horizon task execution and local planning efficiency. Videos are available on the webpage: https://pilot‑planner.github.io/.
Authors:Yeobin Hong, Hyojun Go, Dohun Lee, Jeongjae Lee, Sunghwan Hong, Federico Tombari, Zhaochong An, Jong Chul Ye, Konrad Schindler, Dominik Narnhofer
Abstract:
Video diffusion models increasingly serve as priors for 4D generation, but the generated videos often lack geometric consistency. Recent approaches address this through post‑training by designing geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves geometric consistency and visual quality with a single geometric reward, applicable to both training and inference‑time reward alignment. Our reward measures cross‑view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field‑of‑view overlap even in real videos and 4D datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap‑dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.
Authors:Srikar Yellapragada, Alexandros Graikos, Zilinghan Li, Kostas Triaridis, Tarak Nath Nandi, Ji Dong K Bai, Beatrice Knudsen, Tahsin Kurc, Prateek Prasanna, Rajarsi R. Gupta, Ravi K Madduri, Joel Saltz, Dimitris Samaras
Abstract:
Whole‑slide images (WSIs) are gigapixel‑scale digital scans of histopathology tissue and serve as the primary basis for cancer diagnosis. Existing histopathology generative models operate at the patch scale, producing fixed‑size tiles of at most 1024×1024 pixels which encompass only a small fraction of the WSI. In contrast, text reports for clinical diagnoses and slide‑level diagnostic labels ‑‑ the supervision that exists at scale ‑‑ operate at the whole‑slide level. We present BigCell, the first approach to generate coherent whole‑slide histopathology images at native gigapixel resolution. BigCell trains a flow‑matching transformer at 2048×2048 pixels on over 120,000 publicly available WSIs spanning multiple organ types. Conditioning on embeddings from pathology foundation models and using an inference‑time sliding‑window pipeline, BigCell generates coherent images of up to 32,768×32,768 pixels, outperforming prior tile generators on both tile‑level and slide‑level fidelity metrics. To make gigapixel synthesis practical, we introduce an adaptive scheduling strategy that prioritizes important slide regions, speeding up image generation by over 3\ttimes with minimal quality degradation. Furthermore, we propose the first text‑to‑image synthesis at 8k resolution by training a separate model that produces dense conditioning grids directly from pathology text reports. We demonstrate that augmenting with synthetic samples improves slide‑level few‑shot classification performance by up to 18 % over baselines trained on real data.
Authors:Saurav K. Aryal
Abstract:
Regularization‑based continual learning protects each parameter in proportion to an estimate of its importance, yet these estimates come from finite samples and heuristics and are used as if they were exact. We report four studies that start from a biologically inspired mechanism and end with a statistical explanation of it. Motivated by dendritic nanotubes, we let transient lateral links diffuse importance tags between hidden units. Links between co‑active units gave no significant gain, but random links with lifetimes of 25 or 200 steps raised final accuracy on 20‑task Permuted MNIST by 1.5 to 2.6 points. We show that random diffusion implements linear shrinkage of importance toward the layer mean in expectation. A simulation of the link process alone predicts shrinkage weights of 0.23 and 0.52 for these lifetimes, and graph‑free shrinkage at 0.25 and 0.5 reproduces both the accuracy and the dispersion of the consolidated importance, while every activity‑dependent rule did worse than random links. A five‑seed screen across EWC, SI, and MAS found nominal gains for SI and MAS on Permuted MNIST and losses on task‑incremental Split CIFAR‑100, where forgetting was small and shrinkage cost plasticity. Varying the number of importance samples then separates two mechanisms on Permuted MNIST. For EWC, the gain fell by about ten points per tenfold increase in samples, from 19.6 points with 10 samples to ‑0.4 with 1000, a variance effect. For MAS, a gain of about one point persisted at 1000 samples, consistent with a bias in the importance measure rather than noise. Shrinkage had no detectable effect for MAS on a domain‑incremental CIFAR‑100 benchmark. Finally, we show that a standard heteroscedastic empirical Bayes rule shrinks the largest Fisher‑type estimates most, which may explain its poor results, and point to a log‑scale alternative.
Authors:Ataberk Olgun, Spiros Galanopoulos, Andreas Kosmas Kakolyris, İsmail Emir Yüksel, Haocong Luo, F. Nisa Bostancı, Onur Mutlu
Abstract:
We perform the first comprehensive experimental characterization of power consumed by real HBM2 DRAM chips. Our extensive characterization yields 1) the empirical basis for developing new, accurate HBM power models or validating existing power models and 2) key insights into structural and data‑pattern‑dependent variation of power in HBM2 chips. Out of our 13 new findings and 6 key takeaway messages, we highlight: 1) a read operation consumes significantly higher current than any other operation and 2) the toggled data bus signals in a data transfer burst significantly affect current draw where toggling all HBM2‑internal bank group bus signals causes 29.4% higher power consumption than keeping them constant. Leveraging our empirical insights into HBM2 power consumption, we engineer an access pattern that yields 23.6% higher power consumption on average across 36 tested HBM2 chips than the highest‑power‑consuming JEDEC‑specified current measurement loop. Our results have important implications for HBM‑based system design and validation. Relying solely on JEDEC‑ or vendor‑specified test loops to bound HBM power could cause system failures and/or thermal throttling in real‑world high‑power AI/ML workloads. To improve 3D‑stacked DRAM power modeling in open source architectural simulation infrastructures, we develop a new HBM power model. Our model predicts the power of 36 real HBM2 chips across diverse workloads (including a real LLM decode workload) with 7.6 ‑‑ 8.1% mean absolute percentage error, substantially more accurately than state‑of‑the‑art HBM power models. The power consumption predicted by our model for a power‑intensive microbenchmark closely matches (with 2.4% to 5.3% error across HBM3E's specified supply voltage range) the average HBM3E memory power consumed by three tested NVIDIA H200 GPUs. We publicly release our power model at https://github.com/CMU‑SAFARI/HBM‑Power.
Authors:Ayush Kumar Bar, Promila Ghosh
Abstract:
Automatic speech recognition produces words but drops the non‑verbal content of a recording: the breaths, coughs, sighs and laughter of the speaker, and the horns, music or barking around them. We describe Speech Event Detector, a 5.8‑million‑parameter convolutional‑Transformer model that outputs probabilities every 40 ms for 54 event classes (20 speaker vocalizations and 34 background sounds), together with post‑processing that converts these probabilities into timestamped events and a procedure for merging the events into a Whisper transcript. The model is trained on material drawn from seven public datasets whose labels come in incompatible forms: single detected regions, free‑text tags with timestamps, temporally strong ontology labels, symbols embedded in transcripts, isolated recordings and environmental recordings. We unify them using rule‑based label mapping, recover timestamps for transcript‑embedded labels through CTC forced alignment, locating 6,360 of 8,620 sounds, and combine about 168 hours of real annotated training recordings with synthetic mixtures and augmentation using real background noise. Per‑class thresholds for two operating profiles are tuned on real held‑out recordings, and performance is estimated using two‑fold cross‑validation. On whole held‑out recordings from four datasets, considering the 48 classes with at least ten held‑out events, the sensitive profile detects an average of 49 percent of a class's events, covering 67 percent of all 9,367 events, at a macro precision of 0.30. The balanced profile achieves a macro recall of 0.37 at a precision of 0.36, with macro event F1 scores of 0.32 and 0.33, respectively. The detector runs about 39 times faster than real time on eight CPU threads.
Authors:Seungjun Yu, Hojun Choi, Seojeong Park, Jaeyo Shin, Hyunjung Shim
Abstract:
Pixel‑space world models have recently emerged as a promising approach for GUI agents, predicting future screens prior to execution and subsequently re‑perceiving them for action selection. However, this two‑stage paradigm faces two key challenges in GUI environments: (L1) the high computational cost of pixel reconstruction and (L2) the need to reconstruct large and diverse appearance‑level details beyond what is required for action selection. In practice, this screen‑generation objective can induce an identity shortcut problem, where future‑state prediction largely preserves the current state rather than capturing action‑induced changes. In this work, we reformulate GUI world modeling by shifting the prediction target from full‑screen reconstruction to action‑induced future information in latent space. Concretely, we leverage a large multimodal model as the latent world model, whose shared representations support two dedicated heads for both action selection and predictive transition modeling. The action head directly translates these latent representations into action selection, while the transition head captures action‑conditioned future information. To mitigate the identity shortcut, we further introduce an action‑aware contrastive objective that encourages the transition representations to capture discriminative action‑induced changes. Extensive experiments across offline and online GUI settings show that our method achieves up to 31.9% and 40.6% relative gains in success rate over pixel‑space world models, respectively, while substantially reducing inference latency by up to 98.9%. Our code is available at https://github.com/sjyu001/latent2world.
Authors:Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali, Amandine Crombé, Nathalie Lassau, Alexandre Bône
Abstract:
Purpose : To develop and evaluate CAPTOR, a deep‑learning vision‑language foundation model for oncologic chest‑abdomen‑pelvis (CAP) CT interpretation. Materials & Methods : CAPTOR (Chest‑Abdomen‑Pelvis foundaTion model for Oncologic Radiology) is a contrastive vision‑language model, trained using a pretraining strategy for oncologic CAP CT (in single‑ or multiple‑series acquisitions), on a multicenter retrospective dataset from North America, Europe and Asia, predominantly from tumor lesion examinations. CAPTOR was compared with state‑of‑the‑art CT foundation models on three downstream tasks: disease detection, survival prediction using TumorImagingBench and image retrieval. Classification and retrieval were both evaluated on held‑out internal and public test sets, stratified by anatomical coverage and indication. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results : CAPTOR was trained and tested on 25,512 and 10,794 CT‑report pairs respectively. TumorImagingBench datasets comprised between 134 and 421 studies each. CAPTOR outperformed all baseline models, on the classification task, on the CAP subset (AUC 0.767, 95%CI: 0.734‑0.793, vs. 0.695, 95%CI: 0.645‑0.736 for the next‑best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759‑0.805, vs. 0.734, 95%CI: 0.703‑0.767 p<0.001), ranked first on three of the four oncologic prognostic benchmarks, and achieved superior retrieval performance on CAP and oncologic subsets. Conclusion : A foundation model trained on oncology‑enriched CAP CT data outperformed relevant state‑of‑the‑art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and inference code : https://github.com/Guerbet‑AI/CAPTOR and model weights : https://huggingface.co/Guerbet‑AI/CAPTOR
Authors:Samuele Giampieri
Abstract:
Coding agents such as Claude Code also follow text files that developers write for them: a project file, skills the agent may choose to open, and rules that load automatically when the agent reads matching files. We call these files the agent's harness. Developers add to it whenever they correct the agent, but one success proves little: the same agent can pass a task once and fail it the next time. We present Cortex, a tool that tests changes to the harness instead of trusting them. It saves fixes from the developer's sessions as replayable tasks, finds the corrections that recur, writes one skill or rule at a time, and keeps it only if repeated runs with and without it show a net gain and no task made noticeably worse. Existing text is tested the same way before it is deleted, and Cortex neither adds nor deletes text the agent never used. We tested Cortex with Claude Haiku 4.5 on a repository built for the purpose, with conventions the model tends to break, one area with nothing to learn, and a script playing the developer. On held‑out tasks, the harnesses it built made the agent follow the conventions far more often than no skills or rules did, left the comparison area unchanged, and added no text read on every turn. Rules, which load automatically, did most of the work; skills helped only when offered with the files a task touches, and those offered on every turn were rarely opened. Cortex kept none of the useless or harmful suggestions we planted, but the agent never opened the useless ones or the harmful skills, and it reached no verdict on the harmful rules, so the test shows that Cortex keeps no unused text, not that it detects damage. Under the stronger Claude Sonnet 5 a single run's harness added little, and on a real open‑source project, where the agent was rarely corrected, Cortex found nothing to learn. We release the tool, the lab and its data.
Authors:Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie
Abstract:
Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable, accessible, and reusable remains an open challenge. Large language and multimodal models (LLMs/LMMs) provide the possibility to automate many figure parsing tasks such as determining data trends or extracting data from plots. In this work, we consider if these models can reliably support task‑oriented visual data analysis when presented with synthetic, but realistic, scientific figures that contain multiple charts. Our Visual Question Answering (VQA) process scaffolds our tasks along a four‑level description model from basic conceptual answers to interpretive and analytical understanding. Additionally, we outline our process for synthetically generating our single and multi‑panel (charts in which there are multiple panels present) plots and testing the data with various LMMs using quantitative analysis. In general, we find LMMs become less accurate with increasing figure complexity as the number of panels, number of plot elements, and difficulty of questions increase. Finally, because LMM accuracy is expected to change rapidly, our analysis also considers cost relative to performance as a metric to inform future user decisions. We provide the detailed question‑answer pairs used in this analysis, along with the code used to generate random plots, question‑answer pairs and detailed bounding‑box metadata for chart elements. The code and data are available at https://github.com/ReadingTimeMachine/LLM_VQA_MultiPanel and https://huggingface.co/datasets/ReadingTimeMachine/visual\_qa\_multipanel
Authors:Chenyang Yan, Yingying Zhang, Kanzhi Cheng, Qiushi Sun, Zhengyuan Pan, Zheng Ma, Hang Yan, Yian Wang, Nuo Chen, Jialin Cao, Xingdong Gong, Wenpo Song, Han Chen, Zichen Ding, Fangzhi Xu, Shujian Huang, Xinyu Dai, Yichen Liu, Tiankuo Yao, Bo Wang, Ben Kao, Jianbing Zhang, Lewei Lu, Dahua Lin
Abstract:
We introduce OpenMobile‑2, a near‑frontier mobile agent with fully open training environments and recipes. We make three key advances: (1) Diverse environments with simulated commercial apps: We build MobileGym++, featuring 35 realistic, functionally rich commercial‑style apps with cross‑app workflows, while preserving full controllability for reset and verification. Together with newly configured apps in Android emulators, this yields a diverse playground spanning over 110 apps. (2) Open training data at scale: Building on this foundation, we curate nearly 12K mobile interaction trajectories for supervised fine‑tuning and the largest open collection of verifiable RL training data for mobile agents, comprising over 2K executable tasks with automatic rewards. (3) Hybrid GUI and app‑native tool use: We explore an experimental mobile‑use setting where apps expose selected functionalities as app‑native tools alongside their GUIs, allowing agents to interleave GUI actions and tool calls within a task. We implement this setting in MobileGym++ with over 300 carefully scoped tools across 50+ apps. Additionally, we introduce MobileGym++ Bench for complex, long‑horizon commercial mobile scenarios, supporting both GUI‑only and hybrid GUI‑tool evaluation on a shared task suite. OpenMobile‑2 performs competitively across established benchmarks, including AndroidWorld (79.9) and MobileWorld (50.4), while showing promising transfer to real‑device mobile use, nearly doubling SPA‑Bench performance from 31.9 to 59.6. All resources are available at https://os‑copilot.github.io/OpenMobile2‑Home/.
Authors:Jingyu Liu, Hao Jiang, Yikang Ding, Huafeng Shi, Xu Yang, Xiaoqiang Liu, Guoxin Zhang, Pengfei Wan, Kun Gai
Abstract:
Classifier‑free guidance (CFG) typically applies a shared scale to all latent tokens, although their velocity predictions evolve differently during sampling. We introduce Trajectory‑Adaptive Classifier‑Free Guidance (TrajCFG), a training‑free method for flow‑matching models that adapts token‑wise guidance using historical changes in guided velocity direction. TrajCFG maps this feedback to bounded scales, assigning stronger guidance to larger directional changes and weaker guidance to more consistent directions. It combines adjacent‑step and interval‑normalized longer‑span feedback, reusing standard CFG predictions without additional model evaluations. Controlled diagnostics and ablations examine the roles of directional feedback, token‑wise allocation, and history‑span fusion. Experiments on text‑to‑video, image‑to‑video, text‑to‑audio‑video, text‑to‑image, and image‑to‑3D generation show improvements over base samplers across multiple backbones. For text‑to‑video generation, TrajCFG improves VBench Total scores by 0.30‑0.75 points over the strongest evaluated baseline on each of three backbones. These results support historical directional feedback as a broadly applicable signal for fine‑grained guidance allocation. The project page is available at https://jyliu‑98.github.io/trajcfg‑project‑page/.
Authors:Muhammad Muqsit Islam, Gloria García Cuenco, José M. Martínez, Pierrick Coupé
Abstract:
Image‑level MRI harmonization is often evaluated with a frozen site probe, a site classifier trained on raw scans and applied unchanged to harmonized ones: a drop in its accuracy is read as removal of site effects. We audit this practice on ABIDE I (1,112 T1‑weighted scans, 17 sites) with ten output sets from seven harmonization methods and controls for chance, head shape, intensity, preprocessing and target alignment. We find that this verdict depends on how the probe is trained, even among probes that classify raw scans well: across random seeds, its balanced accuracy on the same CycleGAN outputs ranges from 0.13 to 0.59 (chance: 0.06). Low frozen‑probe accuracy can also arise without harmonization: a Gaussian blur, which increases the intensity distance to the target site, lowers this accuracy to chance. Moreover, where the frozen probe fails, probes trained on the harmonized scans still recover the site, even from intensity histograms alone: on HACA3 outputs, the frozen probe scores 0.25, a probe trained on those outputs 0.75, and one using only their intensity histograms 0.71. Low frozen‑probe accuracy therefore does not show that site information has been removed. We recommend reporting several probes trained to convergence, probes trained on harmonized scans, and target alignment.
Authors:Donghoon Ahn, Runqian Wang, Alexei A. Efros, Taesung Park, Yash Belhe
Abstract:
Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noise, but at inference its inputs are constructed from its own imperfect predictions from the earlier steps, something it has never seen at training time. This might be one reason diffusion models often struggle without additional help, such as CFG. We introduce a correction step for diffusion models which attempts to reduce this training‑inference gap. The corrector takes the base model's prediction as input and is trained against the same ground‑truth target. This makes correction an explicit learning task, with training inputs provided by the base model. During sampling, at each denoising step, we iteratively refine the denoiser's clean‑sample prediction. This allows us to correct predictions at intermediate denoising steps, preventing early artifacts from persisting into the final sample. Although the corrector is trained only on predictions from the base denoiser, it can be applied repeatedly to its own outputs at inference for better results. This simple framework leads to substantial improvement in sample quality while largely preserving diversity. We evaluate our method across DiT, SiT, and JiT. Our method substantially improves upon previously reported FID scores for unguided generation and, notably, unconditional generation, where producing high‑quality samples without guidance has been particularly challenging. Our method achieves performance competitive with classifier‑free guidance (CFG), and combining correction with CFG further improves diversity and reduces oversaturation typical of CFG. Finally, we show that the additional inference cost of correction can be amortized by distilling the refined predictions back into the base diffusion model, retaining the quality gains without additional inference‑time overhead.
Authors:Jianli Sun, Chengxiang Li, Yonglin Tian
Abstract:
GPT‑6 Astra provides robust embodied reasoning, but frequent external queries increase inference time and token costs. We equip a frozen π_0.5 policy with a lightweight CALL head that learns to predict when assistance is necessary. CALL reuses features from the standard policy forward pass and operates locally without consulting external models. Astra is activated solely for critical recovery events and returns control to π_0.5 upon stabilization. Evaluated across 40 LIBERO tasks (200 episodes), our system achieves a 97.5% success rate, outperforming the 95.5% baseline of pure π_0.5. In the Object, Goal, and Spatial suites, Astra executes only 5.49% of environment steps, consuming approximately 5,027 tokens per test episode. This selective intervention preserves the fast ordinary execution of π_0.5 while seamlessly integrating Astra's reasoning on demand. Our project page is available at https://kykgeek.github.io/Pi05_CALL_Astra/.
Authors:Romi Fadli, Rusdah, Wahyu Ramadani Putri Utami, Zulfa Malika Amana
Abstract:
Calibrated typed‑decision ("System One") models return a label and a probability in a single forward pass at a small fraction of the cost of large language models (LLMs), yet their evaluation has been almost entirely English. We report a pre‑registered evaluation of the commercial Jev model and the open Laya‑multilingual model on Indonesian regional languages, using the parallel NusaX sentiment (12 languages) and NusaTranslation sentiment and emotion (7 languages) test sets. Jev is more often correct on sentences that are lexically closer to Indonesian in all three datasets, and presenting the human Indonesian translation instead raises its accuracy by up to 37.2 points, most in languages distant from Indonesian. Its expected calibration error does not grow on regional languages, but its ability to rank its own errors degrades in all three datasets. Retrained classifiers need more than 500 labels per language to match zero‑shot Jev in 9 of 10 NusaX regional languages. A confidence‑gated cascade that sends Jev's 10‑30% least confident decisions to a local IndoBERT classifier matches or exceeds the better of the two systems, and routing to an LLM instead matches that LLM at 24‑77% lower API cost. Of six further commercial LLMs (three Claude and three GPT models), five are 4‑14 points more accurate than Jev, at several hundred times the API cost where it is reported, while the smallest (Claude Haiku) is less accurate. We release the pre‑registration, code and all predictions.
Authors:Jinkui Hao, Donald R. Cantrell, Ramez Abdalla, Sameer A. Ansari, Bo Zhou
Abstract:
Accurate artery extraction from X‑ray coronary angiography (XCA) is essential for coronary artery disease diagnosis and intervention. However, low vessel visibility, respiratory and cardiac motion, and limited annotations challenge existing methods. Moreover, most deep learning approaches produce binary vessel masks without preserving clinically relevant intra‑arterial contrast information. We propose a Temporal Diffusion Bridge Model (TDBM) that formulates vessel extraction as image translation and learns from physics‑informed synthetic data without manual vessel annotations. A physics‑based XCA synthesis engine generates diverse, perfectly aligned pairs of contrast‑enhanced and contrast‑free images. Using these pairs, TDBM learns cross‑domain mappings to recover high‑fidelity vessel information. Temporal conditioning incorporates adjacent frames to improve consistency, while a vessel‑aware contrastive loss exploits background‑invariant synthetic pairs to enhance vascular discrimination. We evaluate TDBM on two XCA vessel segmentation datasets and conduct a reader study using two real‑patient XCA sequence datasets. Zero‑shot segmentation is further assessed on retinal imaging and fluorescence microscopy datasets. Results demonstrate that TDBM consistently outperforms existing methods and generalizes effectively across imaging modalities. Code and trained weights will be available at: https://github.com/Advanced‑AI‑in‑Medicine‑and‑Physics‑Lab/TDBM
Authors:Santiago Reina Alguacil, Ezequiel López Rubio
Abstract:
Despite advancements in sub‑pixel decoding, modern Human Pose Estimation (HPE) architectures remain constrained by deterministic coordinate extraction from discrete heatmaps, which systematically fails under dense occlusions and Out‑of‑Distribution (OoD) ambiguities. To address these limitations, we propose a post‑hoc continuous probabilistic framework that intercepts raw heatmaps and models them as a Gaussian Mixture Model (GMM) augmented with a spatially agnostic uniform distribution, requiring no architectural modifications or retraining. This continuous representation allows us to comprehensively explore the distinct contributions of density modeling to HPE. We show that the spatial expectation of our fitted GMM achieves precision parity with leading sub‑pixel decoders without heuristic post‑processing. Leveraging this continuous space, our test‑time augmentation aggregates probability distributions rather than discrete activations, mitigating multi‑scale heatmap poisoning. Furthermore, because our multimodal GMM parameterization isolates competing spatial hypotheses, it naturally interfaces with geometric regularizers such as Markov Random Fields. However, our analysis exposes the inherent vulnerabilities of 2D kinematic priors under perspective foreshortening and visual degradation. Crucially, this density formulation yields significant improvements in uncertainty quantification: our calibrated geometric covariance volume outperforms standard peak‑intensity heuristics in OoD anomaly detection under severe occlusion. For in‑distribution calibration, dynamically fusing our geometric metric via Adaptive Softmax reduces or matches baseline Area Under the Sparsification Error (AUSE) across benchmarks and degradation levels. Finally, we establish that this continuous parameterization is extensible to multi‑view 3D architectures, bypassing the \mathcalO(N^3) voxel memory bottleneck.
Authors:Yingquan "Cody" Wu
Abstract:
An LZ77 codec bounds every match by one sliding window, trading cheap offsets for short matches against reach for long ones. Match‑length‑dependent windows refine this trade‑off: each match length has its own window, and the decoder infers the offset range from the decoded length. We present two such codecs: WLZ4, a byte‑aligned LZ4‑class format with one‑byte and flagged offsets, and WZIP, an entropy‑coded Zstandard‑class format. Each block begins with its decoded length, from which WZIP derives its widest window and a decoder sizes its output. LZ4 blocks may expand incompressible input by 0.4%; WLZ4 adds at most 15 bytes, WZIP 2. We describe the formats, threaded match finders, decoders and parser. On Silesia, WLZ4 reaches a ratio up to 16% above LZ4HC's, decoding more slowly, and WZIP exceeds Zstandard 22's ratio while its trusted decoder runs at 91% of Zstandard's speed (87% checked; on enwik9, about as fast).
Authors:Fanrong Dong, Ziyi Sun, Ziqi Han, Junhan Sun, Lei Ye, Zetong Jing, Hao Zhao, Shaoxun Liu, Rongrong Wang
Abstract:
Competitive humanoid table tennis requires two distinct abilities: executing a desired return and deciding where to return the ball. We separate these two problems, with the ball landing target serving as an interface between motor control and strategy. (1) We first train a whole‑body motor policy conditioned on the desired landing target, which maps the target to an executable humanoid stroke through a model‑based strike planner and a learned controller. (2) We then improve the motor policy through self‑play in closed‑loop rallies, where the robot learns to handle incoming balls generated by another player rather than only prescribed incoming balls. At this stage, the landing targets are still sampled without considering the opponent, so the policy improves its execution without learning game strategy. (3) Finally, we freeze the entire motor stack and train a stroke‑level strategic policy to choose landing targets through competitive self‑play. Motor self‑play improves both landing accuracy and legal‑return rate during rallies. More importantly, with the entire motor stack fixed, changing only the target‑selection policy leads to large differences in game outcomes: the learned strategy wins 89%, 82%, and 73% of points against center, random, and away‑from‑opponent baselines, respectively. The learned strategy also changes where it sends the ball according to the opponent's position and the previous target, showing that self‑play learns more than simply aiming at fixed "good" locations. On hardware, the same frozen execution stack and strategic interface support complete 11‑point games against a recreational human opponent, including multi‑stroke rallies with both forehand and backhand returns.
Authors:Leon Pohl, Martin Frauendorfer, Corinna Schmitt, Mirko Maehlisch
Abstract:
Bird's‑Eye‑View (BEV) representations are increasingly used as compact inputs for autonomous driving perception and planning. However, lightweight and interpretable methods for evaluating candidate ego trajectories directly on BEV outputs remain useful for vehicular safety workflows, rapid prototyping, and human‑in‑the‑loop validation. This paper presents a modular trajectory ranking pipeline that consumes BEVFusion object detections and map segmentations derived from nuScenes sensor data and ranks manually specified candidate trajectories according to safety‑first criteria. The ranking prioritizes avoiding collisions and walkway violations, then smaller end distance to the target, and finally path efficiency. To study whether these rule‑based rankings resemble human judgment, we created seven BEV traffic scenarios and compared the script output with preferences from a survey of 21 participants. Across 146 pairwise votes, 124 matched the trajectory ranked first by the evaluator, corresponding to an overall agreement of 84.9 %. The results indicate that simple BEV‑centric safety heuristics can approximate human preferences in constrained trajectory choice tasks while remaining transparent enough for analysis, dataset curation, and autonomous driving studies.
Authors:Thanh Ly, Chien Le, Truong-Duy Dang, Duy M. H. Nguyen, Duy-Dung Le, Vien Anh Ngo, An T. Le
Abstract:
State‑of‑the‑art perceptive humanoid locomotion is mostly built on retargeted motion capture, a generative motion prior, and a distillation stage. We report a complete system that does not require this pipeline for a humanoid to cross varied terrain. In its place, we provide a five‑expert mixture routed by a terrain classifier over a sensed elevation grid, and target every foothold with a model‑based planner, refined by six closed‑form‑gradient steps on an interpolated terrain‑cost field instead of picked from a fixed grid. Driven through the same perception‑to‑actuation software a robot would run, inside a second, deployment‑fidelity simulator, it crosses an 85\,m five‑segment waypoint course 18 of 20 times end‑to‑end (90%, Wilson 95% CI 70‑‑97) and succeeds 20/20 on four of five segments scored alone, where an ablated configuration without the mixture and without refinement reaches 0% on both the stair and the hurdle segment. The continuous refinement moves 72.4% of foot placements across at least one grid‑cell boundary, which produces real sub‑grid correction. We also ran matched component studies to report what does not account for it: one expert equals five on the training distribution, four label‑free routers collapse to near‑uniform activation, and a self‑supervised terrain embedding loses to the privileged label.
Authors:Aayush Gupta
Abstract:
A speech synthesizer produces a detailed waveform, but a listener need not hear every detail. Strong sounds can mask weaker neighbors, creating context‑dependent "blind spots" in hearing. Could neural text‑to‑speech (TTS) spend less computation on those details? We investigate a first step: how the placement of spectral degradation affects completed speech from Piper and Kokoro. At matched time‑frequency bin‑removal budgets, a simple low‑energy‑first policy achieves higher mean wideband PESQ than random deletion and the tested psychoacoustic masking criteria. At 50% removal in the spreading‑matrix comparison, energy, masking, and random policies score 4.46, 3.60, and 1.19 on Piper, and 3.42, 2.10, and 1.22 on Kokoro. Spectrograms and masking maps expose where these perturbations fall and why a model's below‑threshold regions cannot automatically be treated as inaudible. Calibration diagnostics show substantial threshold sensitivity, while a small single‑listener pilot establishes no transparent removal budget. CPU profiles identify waveform decoding as the main measured kernel cost, but a tested dynamic‑INT8 Kokoro decoder runs slower than FP32 on both CPUs. The results establish energy ranking as a strong oracle baseline and motivate testing perceptual allocation inside a decoder. They do not yet demonstrate saved computation: the decisions here use already synthesized audio, and quality metrics are evaluated at 16 kHz. We release scripts, results, audio examples, and a listening‑study instrument.
Authors:Yuzhe Tang
Abstract:
Let K_M \subset \mathbbR^3 be the "snaky" corridor with unit‑square cross‑section: an infinite entry arm parallel to the x‑axis, a middle arm parallel to the y‑axis whose clear straight section has length M, and an infinite exit arm parallel to the z‑axis, with the two turns lying in the x‑y and y‑z planes. Let V^(K_M) be the supremum of the volumes of connected bodies that can be moved continuously (by rigid motions) from the entry arm to the exit arm while remaining inside K_M. We prove V^(K_M) \le 7.8691 for every M \ge 3.53 (Theorem A); by a sharper analysis of the same mechanism, V^(K_M) \le 4.2862 for every M \ge 2.61 (Theorem B); and, by replacing the final analytic caps with a certified branch‑and‑bound computation over the compact family of entry tubes compatible with the forced events, V^(K_M) \le 3.4083 for every M > 2.2 (Theorem C; computer‑assisted, in double precision with conservative slack). In particular all three bounds apply to the corridor with M = 4 used in Problem 6.64 of Georgiev, Gómez‑Serrano, Tao and Wagner (arXiv:2511.02864), and uniformly to every "turns far apart" reading of that problem. To the best of our knowledge, this is the first upper bound for the snaky‑corridor 3D moving sofa problem; previously not even finiteness of V^ had been recorded. This note proves upper bounds only; combined with the certified lower bound 1.8558404 from the author's separate certification work via the problem's own evaluation pipeline, the state of the problem becomes 1.8558404 \le V^(K_4) \le 3.4083. Theorems A and B are elementary and self‑contained: a first‑crossing (elbow) event forces a definite tilt of the body's own entry tube, and Fubini‑type chord estimates for tilted tubes cap the volume. Theorem C certifies, by the same forced‑event skeleton, the largest corridor volume that any admissible entry tube can carry.
Authors:Chaoyu Lin
Abstract:
AI‑generated image collections combine visual content with heterogeneous provenance such as prompts, model parameters, and workflow graphs. Managing these collections locally requires coordinated metadata extraction, responsive browsing, and retrieval without mandatory remote inference. We present Berry AI Studio, an edge‑first asset management system built with Vue, Tauri, Rust, SQLite, and ONNX Runtime. Berry integrates format‑specific metadata parsing with raw‑payload retention, generation‑aware cancellation of queued thumbnail work, a persistent multi‑tier thumbnail manifest, cursor pagination, and local image‑text retrieval. We evaluate these mechanisms in five independent repetitions on a pinned source revision. With 50,000 catalog records, the median warm first‑page database call was 285.3 ms; at offset 40,000, keyset pagination reduced median raw‑query latency from 57.9 to 0.46 ms. A controlled browser harness recorded a 0.3 ms median‑of‑run P95 update time during virtualized scrolling, while cancellation prevented all 1,200 stale speculative decodes in the tested trace. A versioned 10‑case corpus produced 53/53 exact metadata‑field matches, and controlled retrieval and grouping tests exercised the authentic storage and clustering paths. The implementation, benchmark scripts, raw logs, and checksummed result packages are publicly available.
Authors:Seunghun Lee, Kyuhwan Shim, Yeogyeong Song, Seung Hwan Kim, Sihaeng Lee, Byoung-Tak Zhang, Sunghoon Im, Gunhee Lee
Abstract:
Existing vision‑language‑action (VLA) models benefit from large‑scale vision‑language pretraining, but tend to memorize training trajectories, limiting their ability to adapt learned interactions to new task and scene configurations. To address this problem, we propose Region‑Aware Interaction Networks (RAIN), which formulates a task as a sequence of goal‑conditioned subtasks, each specified by an action type and a target‑region mask. This formulation allows learned skills to be reused across different objects and scene layouts and composed into task sequences not seen during training. Its Target‑adaptive Cross‑View Encoder (TCE) conditions multi‑view features on target masks and exchanges information across views, while PlanDiT predicts target‑directed trajectories and actions. A Transition Head estimates subtask completion to enable automatic transitions. We also introduce a Reference‑based Retargeting Strategy (RRS), which uses recorded trajectories to synthesize smooth approaches to reference interactions with alternative target regions, without collecting additional data. Experiments on LIBERO‑Analogy, our new benchmark for robotic task generalization, and real‑world tabletop tasks demonstrate that RAIN generalizes more robustly than existing VLA baselines.
Authors:Jose Hernandez-Orallo
Abstract:
Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or saturates soon. How can we then evaluate machines that may become more capable than the current machines and the people who write the tests? One answer to the question hinges on inverse problems, inductive inference questions for which difficulty depends on the complexity of the underlying mechanism, given the observations and the previous knowledge. Levin's Kt, a computable variant of Kolmogorov complexity, puts a theoretical upper bound to that difficulty, balancing the simplest representation encoding for the mechanism and the logarithm of the time steps to run that mechanism. Previous tests derived from Levin's Kt, such as the original C‑test, are based on minimalistic Universal Turing Machines (UTM), capable of very limited representations and no previous knowledge. Reasoning large language models (LLM) are UTMs that capture a compendium of human culture as knowledge, using prompts as natural language programs. This paper presents the first test based on Levin's Kt with LLMs as representational machines. Fresh test items can be produced at high difficulties, estimated in a ratio scale measured in bits, before any human or frontier model can solve them. We formalise the measure, devise a lightweight generation and certification pipeline, explore the effect of the power of the reference machine and the gap between examiner and examinee. We produce a first battery of certified items administered to twelve LLMs. The resulting agent characteristic curves decrease with difficulty, down to the uncharted high‑difficulty region, the supra‑intelligence window of items we can certify, yet beyond generations of frontier intelligence.
Authors:Xudong Cai, Ruijie Zhu, Haoliang Shang, Ruining Li, Marc Pollefeys, Shangzhe Wu, Andrea Vedaldi, Chuanxia Zheng
Abstract:
In this paper, we study physically grounded articulated 3D asset generation‑‑‑moving the goal from reproducing how objects look to capturing how they move and function. Inspired by the literature on procedural and agentic generation, we present Artisan, a system that extends the latest method Articraft by introducing a multi‑agent pipeline with physical simulation feedback, enabling comprehensive attributes beyond geometry, appearance, and physically plausible articulation, such as collision‑free motion, stability under gravity, and accurate mass and material properties. Furthermore, we construct a unified benchmark spanning four key aspects: spatial fidelity, physical plausibility, downstream task performance, and overall quality. The benchmark encompasses 100 reference assets from ArtVIP and LightWheel, 50 stability prompts, and 50 ABO objects with reference masses, with protocols evaluating various metrics. Through extensive evaluation of more than 10 representative models, Artisan achieves competitive spatial fidelity while attaining the lowest collision rates and highest stability rates among the evaluated methods. Together, our system and benchmark establish insights for advancing 3D generation towards physically plausible, functionally usable assets.
Authors:Changhao Wang, Jaden Clark, Shuran Song
Abstract:
Robot policies often convert sensing and control into fixed‑rate observation‑action sequences, leaving physical time implicit in sequence position. This assumption breaks when observation rates or delays vary, or when inference latency changes. We introduce Chronos Policy, which makes physical time explicit by conditioning on observation capture times and desired action execution times. We study two ways to incorporate time into a diffusion transformer: adding timestamp embeddings to token features, or encoding physical time in positional encoding through time‑aware rotary positional encodings (time‑aware RoPE). We find that time‑aware RoPE is more robust, allowing attention to depend directly on physical time while keeping token content unchanged. During training, we temporally randomize which observations are sampled and when the action chunk begins, exposing the policy to a wider range of sensing and execution schedules that may occur during deployments. This also enables learning from asynchronous, multi‑rate data without forcing all measurements onto a fixed temporal grid. Across simulation and real‑world experiments, Chronos Policy improves performance on dynamic tasks, where observation‑to‑action timing is critical, and on multisensory tasks, where the relative timing between observations matters. On standard quasi‑static tasks, it preserves nominal performance while improving robustness to changes in sensing and inference timing.
Authors:Taoyong Cui, Pinxian Zeng, Wanghan Xu, Hao Xiao, Shuhan Xue, Qiang Gao, Wanli Ouyang, Pheng Ann Heng, Zhenfei Yin, Yingcheng Wu, Ling Yang
Abstract:
Recursive self‑improvement (RSI) aims to transform interaction experience into persistent improvements, yet external feedback is costly and world‑model predictions may become unreliable as policies evolve. We introduce RSI‑Anything, a domain‑general framework built on JEPA‑Anything that uses predictive world models to guide action and learning‑data selection. Observed feedback drives policy or workflow improvement and conditional world‑model adaptation, while component and compatibility checks govern the retention of updated policy‑model pairs. Across diverse computational and scientific domains, RSI‑Anything demonstrates substantial gains. In code repair, with the coder fixed, success increases from 26.3% with a static verifier to 29.2% through predictive selection and 31.2% after world‑model adaptation, at comparable test usage. Graphical user interface (GUI) evaluations showed higher task completion, while average actions decrease by 39.4%. Across five successive policy versions, laboratory‑planning scores more than double from 21.80% to 46.01%. In reaction optimization, RSI‑Anything achieves 96.15% mean best observed yield versus 82.16% for Bayesian optimization under the same 40‑experiment budget. An end‑to‑end immune‑science application further connects workflow evolution and predictive modeling with therapeutic hypothesis generation. These results demonstrate how predictive world models and retained system updates enable recursive improvement across diverse environments.
Authors:Danil Rudenko, Georgii Bulgakov, Nikolay Kartashev, Ivan Rubachev, Daniil Krasylnikov, Artem Babenko
Abstract:
While deep learning models for tabular data have recently received significant research attention, the choice of objective functions for regression tasks remains an understudied degree of freedom. Several lines of research offer alternatives to the standard Mean Squared Error (MSE), but differences in datasets, backbones, and evaluation protocols make their practical benefits difficult to compare. In this work, we address this gap by performing a systematic re‑evaluation of regression objectives across a broad collection of established tabular datasets and multiple neural backbones. Our comparison reveals that many proposed objectives provide limited or inconsistent gains over a properly tuned MSE baseline, while quantile loss yields consistent aggregate improvements over MSE across the tested backbones. We then identify a limitation that remains even with this strong objective: regions with different levels of target noise can reach their best predictive performance at different training epochs. Consequently, a single global early‑stopping checkpoint can impose a compromise between uncertainty regions, even when the objective can represent heteroskedastic target distributions. Motivated by this limitation, we introduce a heteroskedasticity‑aware extension of quantile loss that uses input‑dependent label smoothing as a potential remedy and improves over quantile loss on several backbones. Overall, our results provide a modern empirical picture of regression objectives for tabular deep learning and highlight how differences in optimal stopping times across uncertainty regions can limit the effectiveness of global early stopping. The code is available at https://github.com/yandex‑research/tabular‑regression.
Authors:Weiliang Chen, Song Liao
Abstract:
As AI agents increasingly rely on external tools, accurate retrieval is essential for reliable behavior. Existing methods optimize semantic relevance but largely ignore tool availability during execution. To address this gap, we propose RATED (Reliability‑Aware Tool Retrieval via Execution Dynamics), a retrieval framework that combines robust semantic matching with dynamic reliability adaptation. RATED employs a frozen dual‑encoder with contrastive fine‑tuning for semantic matching, and a lightweight contextual bandit module that re‑ranks candidates based on real‑time execution feedback. This design enables rapid adaptation to runtime tool failures without expensive model retraining. We introduce EcoAgent, a benchmark of 5,172 real‑world tools and over 210,000 query‑tool pairs for offline retrieval and dynamic reliability evaluation. Experimental results demonstrate that RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real‑world queries, and reduces fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, while incurring negligible latency overhead of less than 1 ms over encode‑only inference. The code and dataset are available at https://github.com/Elliotes/EcoAgent.
Authors:Yeonsung Jung, Zhiyuan Wang, Zifan Wang, Vítor Albiero, Benjamin Newman, Edoardo Debenedetti, Ivan Evtimov, Ninareh Mehrabi
Abstract:
Personal agents increasingly connect to services across the web and applications by using tools. As the effects of their actions accumulate and interact across services, unintended or harmful outcomes can arise when agents fulfill benign user requests. Proactively identifying these compositional failures requires systematically exploring diverse cross‑service paths. Existing red‑teaming methods often simply prompt LLMs to identify potentially harmful paths from task and environment descriptions or decompose harmful objectives specified by humans or LLMs into seemingly benign action combinations. However, generation is biased toward patterns learned during training, potentially favoring familiar tool combinations and leaving novel cross‑service interactions underexplored. To address this, we propose X‑AgentRed, a self‑evolving red‑teaming framework that discovers harmful tool call paths through a graph of tool dependencies and expands and refines them using execution feedback. By analyzing each tool's effects, potential harms, and preconditions for execution, the red‑teaming agent constructs a directed graph that captures how one tool's effects can satisfy another's execution preconditions. Recipes sampled from this graph are instantiated as realistic scenarios in which agents can produce harmful outcomes. Execution feedback enables self‑improvement by diversifying paths that produce harm through tool substitution and refining preconditions at blocked steps in unsuccessful paths to enable further execution. Across four frontier agentic models, X‑AgentRed achieves 1.77× the harmful path rate of STAC and explores a more diverse set of tools, yielding 2.99× the tool coverage on average.
Authors:Jingwei Song, Zhisheng Chen, Bingfan Zeng, Jing Tang, Yubo Hou, Jinhao Jing, Zhengwei Xie, Linfeng Zhang
Abstract:
A central challenge in embodied recursive self‑improvement is enabling agents to convert interaction history into capabilities that persist beyond one world and transfer to later tasks. A saved memory or successful final action does not by itself establish that a reusable capability was acquired; evaluation must separate capability change from carried‑over resources, unequal source exposure, and early execution failures. We construct MC‑EvoBench as a controlled Minecraft benchmark with a fixed catalog of 19 registered evolution chains, including 12 primary evaluation chains and seven calibration or probe chains, together with three graph‑aligned paired probe families for transfer. The benchmark represents each task as a task‑state graph, defines stage‑level environment verifiers, assigns matched source and target roles, resets the target world, and permits only a declared agent artifact to cross the source‑target boundary. For each probe, no‑prior, related, unrelated, and shuffled source conditions are evaluated under matched environment‑step budgets and five seeds; we report TSR, WGP, and VMP and retain provenance records. The benchmark reveals a consistent direct‑capability gap between controller and planner‑backed variants, while related experience yields no stable transfer: it does not improve either tool probe, its small wood‑species gain hinges on one seed, and all intervals include zero. Stage audits further show that early wood‑collection and crafting bottlenecks often leave later capabilities unobserved. MC‑EvoBench thus offers a controlled testbed for durable, transferable embodied capabilities. Code and data are available at https://github.com/xzw‑ustc/mc‑bench‑open‑source
Authors:Oktie Hassanzadeh, Dharmashankar Subramanian
Abstract:
Semantic query engines ask a large language model (LLM) to judge individual rows, and benchmarks compare these engines on the quality of a few executions at temperature 0. We measure how much another execution changes. We re‑execute the text and tabular workloads of SemBench, LRO‑Bench, and UDA‑Bench through their own harnesses, log every one of 1.2 million model calls, and reanalyze SemBench's published executions. At temperature 0, between 0.07% and 10.2% of rows change their operator output over five executions, depending on the model, and up to 24.0% on one task. Plans amplify these changes exponentially in the number of rows an answer exposes, as a possible‑worlds model predicts: on UDA‑Bench, a few percent of extracted values change, but about two thirds of the answers do. In SemBench's published results, 21% of text results change quality between executions, including on engines whose plans are deterministic. Several per‑query system comparisons, and 16 of 28 tests of LRO‑Bench's implementation recommendations, are not settled by the executions the benchmarks run. Changing rows are rows on which models disagree with each other; identical requests within one execution disagree as often as across executions; and one execution's decision margin or reasoning length predicts which rows change. Voting reduces the variance but does not remove it, and a calibrated cascade keeps its guarantee while the rows it returns change. We derive reporting rules for benchmarks and design rules for engines.
Authors:Boyuan Deng, Kshitiz Upadhyay, Michael Shields
Abstract:
Gaussian process solvers for linear partial differential equations (PDEs) return posteriors rather than point estimates, but they condition only on equality information: the PDE residual, boundary data, and measurements. Many solutions of interest also obey inequality constraints, including nonnegativity, monotonicity, and absence of arbitrage, which equality conditioning cannot enforce. We prove that conditioning a Gaussian process prior jointly on linear equality observations and a finite‑dimensional inequality event is equivalent to conditioning sequentially in terms of Borel measures on the solution space. The physics‑informed posterior is therefore a valid prior for the inequality update at virtual points. The combined law is non‑Gaussian yet computable through truncated‑Gaussian moments, and neither the PDE conditioning nor the inequality update requires modification. We evaluated the construction on a Poisson benchmark, option‑price surfaces, and air‑quality fields. Under incomplete boundary data, a monotonicity constraint lowered the error of a two‑dimensional Poisson reconstruction by 63%. On a real S&P 500 option chain thinned to 20 retained quotes, no‑arbitrage constraints reduced held‑out extrapolation error by 22% and cut the calendar‑arbitrage rate from 10% to 1%. On four real air‑quality snapshots, nonnegativity reduced a Gaussian moment‑based sub‑zero‑tail proxy by 74% at the sparsest monitor count. Changes in mean accuracy and constraint diagnostics differed across data regimes. The probability of the noisy virtual constraint event under the equality posterior measures agreement with the specified bounds before the update. The construction turns admissibility from a post‑processing step into conditioning information for probabilistic PDE solvers.
Authors:William Sheffield, Jiahui Chen, Amy Zhang, Alex Warstadt, Junyi Jessy Li, Kanishka Misra
Abstract:
Discourse particles subtly shape the flow of discourse by reflecting conversational context and interlocutor attitudes, often leaving the truth‑conditional meaning untouched. They are also often polyfunctional, with distinct senses which partially overlap with other particles; for example, the exclusive use of English "just" and "only" ("John only/just drinks water"). Despite the frequency of these words in natural language, they have gone largely unstudied due to the subtle nature of their effects. We investigate LLM sensitivity to "just" and "only" in natural conversations using behavioral and mechanistic analyses. First, we find that LLMs and humans similarly prefer the presence of these particles in context, and both are more sensitive to "just" compared to "only". Second, when sampling from LLMs with and without these particles, we find LLMs consistently rate generations as more likely, given their source context, demonstrating their particle sensitivity. In both experiments, instruction tuning increases sensitivity. Finally, we localize particle‑sensitive units within LLMs and find the overlap between "just" and "only" is asymmetric: "only" units transfer selectively to exclusive "just", while all "just" units transfer to "only" equally well regardless of sense, suggesting LLM lexical‑level particle representations are partially organized by sense. We release our code, sense‑labeled dialogue stimuli, and human preference annotations at https://github.com/JiahuiKChen/Not_Just_an_Evaluation.
Authors:Ze-Wei Liou, Harris Nover, Amey Naik, Andrii Skliar, Igor Shovkun, Ido Shahaf, Tri Dao
Abstract:
State‑space models (SSMs) maintain a fixed‑size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models. This eager state management causes large state I/O with low arithmetic intensity, leaves speculative decoding no cheap rollback, and feeds state quantization error into subsequent recurrent updates. We introduce ReplaySSM, which unrolls the recurrence to represent the state as a checkpoint and a buffer of recent SSM inputs. This separates output computation from checkpoint updates. By updating the checkpoint periodically, ReplaySSM avoids state writes on most steps and roughly halves the dominant state memory traffic. Keeping recent inputs also makes speculative rollback a buffer‑pointer operation and enables output‑only verification without materializing intermediate states. The same separation allows a low‑precision checkpoint to serve output computation on most steps while a higher‑precision checkpoint serves recurrent updates, so rounding error in the low‑precision copy never enters later updates. We implement ReplaySSM in vLLM for Mamba‑2 and Gated DeltaNet. At serving batch size 256, it accelerates standard decoding by 1.20‑‑1.47× over a tuned vLLM baseline across four hybrid models, and delivers about 1.8× the throughput of vLLM's speculative decoding on the two 120B‑class models. Separate state precisions also reduce output KL divergence and downstream accuracy losses compared with a shared low‑precision checkpoint.
Authors:Tom Philipp Backert, Mohammad D. Soorati, Javad Ghofrani
Abstract:
Multi‑agent reinforcement learning (MARL) has become a prominent research direction, particularly within collective robotics, where the ARGoS simulator is widely used to study emergent swarm behavior. Yet despite this growth, systematic support for reuse and reproducibility in MARL‑based swarm robotics remains limited and insufficiently standardized: a single experiment is assembled from manually aligned simulator files, training scripts, configuration values, paths, seeds, and environment details, so reuse and reproduction stay fragile even within one research group. To address this gap, we introduce MARGoS, a lightweight, configuration‑driven platform developed in this project that elevates these experiment‑defining inputs to first‑class, platform‑managed artifacts. MARGoS enables reproducible experimentation through configuration hashing, end‑to‑end seed propagation, frozen (immutable) configurations, structured logging, environment fingerprinting, and exportable experiment bundles that facilitate sharing, replay, and verification of results across machines and over time. A five‑dimension evaluation (architecture, efficiency, reproducibility, usability, and collaboration) supports our claim that managing experiment variation as a platform responsibility makes ARGoS+MARL experiments easier to reuse, reproduce, and hand off. Our code is available at https://github.com/tombackert/margos.
Authors:Ali Shibli, Andrea Nascetti, Yifang Ban
Abstract:
Geospatial foundation models can support disaster damage assessment, but high accuracy on familiar disasters may hide shortcut learning: prediction from event or scene characteristics rather than evidence that transfers to a new disaster. We study embeddings from TerraMind, Prithvi‑EO‑2.0, DOFA, and DINOv3 on the multi‑disaster datasets xView2 and BRIGHT. Holding out an entire disaster, instead of randomly splitting image tiles, reduces mean balanced accuracy from 0.807 to 0.638 on xView2 and from 0.818 to 0.568 on BRIGHT. Four sets of tests examine what information supports prediction: event and acquisition context, before/after image inputs, building count, and correct image pairing. Pre‑event images remain predictive, and a classifier given only the annotated number of buildings reaches balanced accuracy of 0.629 and 0.653. Combining before/after features performs best on average on xView2, but does not improve over pre‑only features on BRIGHT. Shuffling image pairs within the same disaster and damage class causes modest losses. Motivated by these findings, we test whether feature transformations learned only from training disasters improve generalization to unseen disasters. These transformations improve performance, with supervised contrastive learning performing best on xView2 and PCA on BRIGHT. Our results highlight the importance of evaluation on unseen domains for assessing model robustness and show that simple feature‑level transformations can improve generalization. Code and experiments can be found at https://github.com/alishibli97/gfm‑shortcut‑learning.
Authors:Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li
Abstract:
Decoding visual experience from fMRI is important for understanding human perception and developing brain‑computer interfaces. Recent generative decoding methods can reconstruct plausible images and videos, but visual plausibility alone does not ensure that the output is faithfully grounded in brain activity. In this work, we propose Neurons++, a visual‑stream fMRI reconstruction framework for both static image and dynamic video reconstruction. Compared with our conference version Neurons, Neurons++ extends the paradigm from video‑only reconstruction to visual‑stream reconstruction for both images and videos, and replaces parallel task conditioning with serial cortex‑inspired decoding. The model first uses image‑fMRI pretraining to obtain a stable spatial‑semantic initialization, and then performs subject‑specific fine‑tuning for image and video reconstruction separately. The serial design organizes fMRI‑derived representations into explicit spatial, semantic, language‑level, and temporal cues. Extensive experiments show that Neurons++ achieves state‑of‑the‑art video reconstruction and competitive image reconstruction across standard benchmarks. Beyond reconstruction fidelity, neural‑to‑visual analyses indicate that Neurons++ exhibits qualitative directional controllability under neural perturbation, that its fMRI encoder read‑out is sensitive to stimulus‑relevant cortical features, and that it supports ROI‑guided semantic modulation. Code and model weights are available at: https://github.com/xmed‑lab/NeuronsPlusPlus.
Authors:Qituan Shangguan, Bosen Xu, Shuai Wang
Abstract:
Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization. VoxCeleb‑style evaluations remain useful anchors for English in‑the‑wild media speech. Scenario‑specific evaluations and challenges are also valuable. However, neither setting directly shows whether the same system remains reliable across diverse deployment scenarios. We introduce OpenSVBench, a public benchmark for cross‑scenario SV evaluation. Its fixed public core contains 26 trial sets organized into 11 scenario groups, with 9.15M verification trials, frozen target/non‑target keys, and metadata for scenario‑wise reporting. Around this core, the online leaderboard acts as a diagnostic evaluation interface rather than only a ranked list. It accepts score packages from new systems, evaluates them with frozen keys and common scoring, and reports trial‑set breakdowns, scenario‑group metrics, macro EER/minDCF, model‑wise profiles, and scenario‑wise comparisons. We evaluate 18 public checkpoints spanning x‑vector, ECAPA‑TDNN, CAMPPlus, ResNet and ERes variants, ReDimNet2, SAM‑ResNet, and pretrained Transformer speaker recipes. The results show clear cross‑scenario gaps: systems with similar macro EER can have different scenario profiles, strong performance in one scenario does not guarantee strong performance in another, and same‑scenario trial sets can still expose source and protocol gaps.
Authors:Timi Owolabi, Hikmah Olawore, Frances Adelakun, Ayotomiwa Oyewumi
Abstract:
World‑action models (WAMs) predict a robot's actions together with the observations those actions will produce. However, the strongest models rely on video diffusion transformers that are far too computationally expensive for resource‑constrained edge devices. We present SmolWAM, a latent WAM that we derive from the 2B‑parameter Cosmos Policy by keeping its pretrained trunk and rebuilding its inference interface: single‑frame visual latents, cached text features that preserve the trunk's attention sinks, 12 of 28 transformer blocks, and 1,008 tokens from which actions and future visual latents are read out in one rectified‑flow step. Trained in 3.6 GPU‑hours, SmolWAM succeeds in 95.5% of 400 LIBERO episodes at 57 ms and 2.3 TFLOP per 16‑action chunk with 2.4 GiB of GPU memory, 11x faster than its teacher with 18.5x fewer FLOPs and 3.2x less memory, and its world head still forecasts scene motion.
Authors:Boheng Li, Yueming Xing, Yiming Li, Jie Zhang, Han Qiu, Tianwei Zhang
Abstract:
The rapid development of personalized video generation has raised serious concerns over identity privacy, as a few facial videos of a person now suffice to reproduce their identity in synthetic videos. This report presents the first‑place solutions to both phases of the ECCV 2026 LifeGenIP Challenge on Unlearnable Videos, which aims to protect facial videos from unauthorized customization under a limited and imperceptible perturbation budget. For Phase 1, we introduce Reference Capacity Collapse, a pair of complementary losses that degrade the information carried by the reference condition while keeping subsequent video representations dependent on it, together with a Motion‑Guided EOT mechanism that improves robustness across unknown inference conditions. Phase 2 further considers tuning‑based customization and unseen generative models. To address both threats within a single perturbation, we design a two‑layer framework that protects the facial and background regions separately, constructing structure‑aligned perturbations from a synthetic target identity in the former and strengthening the white‑box disruption of Phase 1 in the latter. Our method achieves nearly perfect protection under white‑box reference‑based customization, and remains effective under other customization paradigms, various models, and temporal attacks. Both solutions are built upon our recent work LETHE. We hope our solutions can inspire future work on protecting facial videos against unauthorized customization. Our code is available at https://github.com/AntigoneRandy/KVeil.
Authors:Aravind Reddy, Deeraj S K, Mohd Wasif Raza, Saurav Prakash
Abstract:
Federated clustering can support applications such as patient stratification across hospitals and customer segmentation across distributed financial datasets. Responsible AI calls for fair representation of demographic groups and support for data deletion, motivated in part by rights such as the GDPR's right to erasure. These goals interact: deleting records can change the balance between groups and alter the clustering of the remaining population. We study exact machine unlearning for federated socially fair k‑means, whose objective is to minimize the largest group‑average squared distance to the nearest center. Our approach keeps separate local summaries for each group, so summaries at unaffected institutions remain valid as global fairness weights change. For two groups, we prove an initialization approximation guarantee independent of client group proportions under ideal sampling, and we show that same‑seed replay reproduces retraining exactly. Reuse pays off most when a whole client withdraws: across three datasets, replay is 35.8× faster than retraining for the initialization‑only model and remains 2.4× and 1.7× faster with one and two fair‑refinement rounds, which improve worst‑group cost. For individual record deletions, gains are modest (about 1.06×), and certifying cached assignments can cost more than it saves. These results map where exact unlearning can profitably reuse computation. Code and reproducibility materials are available at https://github.com/arareddy/mufsc.
Authors:Connor Hashemi, Bharadwaj Ravichandran, Connor Anderson, Shitij Govil, Brandon RichardWebster, Brian Hu, Keith Fieldhouse
Abstract:
In test and evaluation (T&E) of AI models, evaluators typically have access only to a held‑out test dataset and no knowledge of the training data or its semantic composition. If critical input conditions‑‑‑such as time‑of‑day, weather, or scene location‑‑‑are underrepresented in this test dataset, T&E results can be misleadingly due to this semantic coverage gap. We propose a three‑stage generative AI pipeline to detect and fill such semantic gaps without requiring ground‑truth labels of these semantics. First, vision‑language models (VLMs) perform zero‑shot binary labeling of semantic factors, and an embedding‑space clustering step identifies which semantics most affect downstream model behavior. Second, diffusion‑based image‑to‑image models transform existing covered images into the underrepresented semantic domain, preserving scene content while augmenting high‑level appearance. Third, statistical analysis of the augmented image embeddings verifies that the synthetic imagery improves dataset coverage before it is incorporated. Crucially, the synthetic data is used only to enrich the test set and no model retraining is performed for T&E. We validate the full pipeline on both image classification and object detection under day/night semantic gaps, showing that the augmented test datasets better distinguish models with and without exposure to the missing semantic, thereby yielding T&E conclusions that more closely match those of an ideal, fully covered dataset. Code available at https://github.com/Hashe037/DEDUCE
Authors:Yuchen Wang, Yanqi Bao, Zheng Gu, Zhengqing Chen, Jieqi Shi, Jing Huo
Abstract:
Recent advances in physical 3D generation have enabled visually faithful assets with simulatable mechanical behavior. However, existing methods provide limited control over material behavior beyond appearance‑based inference and struggle to estimate physical properties from complex interactions between objects and semantic parts. Therefore, we present PhysForge, a Video Diffusion Model (VDM)‑based framework for addressing these challenges, comprising two complementary modules for material‑controllable physical fitting and componentlevel interaction modeling. The first, PhysFit, leverages material‑conditioned VDM dynamics as references for physical‑property fitting, introducing windowed fitting to focus on material‑revealing interactions and Hallucinated Forcing to disentangle nonphysical motion hallucinated by the VDM from material responses. The second, PhysComp decomposes complex interactions into individual objects and semantic components, enabling their physical parameters to be optimized independently and flexibly composed for joint simulation. Experiments on PhysAssets and Google Scanned Objects demonstrate that, compared with existing methods, PhysForge provides stronger control over material‑dependent physical responses while supporting component‑level physical modeling for complex interactions. It achieves the best CLIP and physical controllability scores on both common and uncommon materials, with qualitative comparisons, component‑level ablations, and a user study further supporting these improvements. Our project page is available at https://masterwanna.github.io/PhysForge.html .
Authors:Weiqi Sun, Qianyue Wu, Wenjia Wang, Tianyang Hu
Abstract:
Linearizing softmax attention offers a route to efficient Transformer inference, but preserving pretrained attention remains challenging. We identify an important source of approximation error overlooked by standard linearization methods: a small number of outlier dimensions contribute disproportionately to query‑key dot products and account for a large fraction of the approximation error. Motivated by this heterogeneity, we propose Outlier‑Aware Attention Linearization (OAL), a dimension‑aware method for constructing quadratic attention approximations. Starting from a second‑order Taylor approximation, we introduce a Hadamard‑product formulation that makes individual query‑key contributions explicit. This formulation relaxes globally shared coefficients to support dimension‑specific linear terms and cross‑dimensional quadratic interactions while preserving separable evaluation. To retain this flexibility with fewer parameters, OAL groups dimensions with similar contribution scales to share coefficients, while allowing distinct treatment of the few outliers. The resulting positive quadratic kernel is calibrated against pretrained attention outputs and admits linear‑time evaluation with respect to sequence length for fixed head dimensions. On Qwen2.5‑0.5B, OAL reduces frozen‑attention output relative MSE by approximately 8.9% compared with DiJiang. Replacing middle‑layer attention modules and jointly fine‑tuning the kernel coefficients with LoRA further lowers WikiText‑103 perplexity from 31.71 to 27.50 compared with a learned global quadratic, while improving accuracy on PIQA, HellaSwag, and ARC‑E. These results demonstrate the value of outlier‑aware dimension grouping for both attention approximation and pretrained‑model conversion. The code is available at https://github.com/wuqianyue09/OAL
Authors:Tianqi Zhao, Jia Guo, Yan Shuo Tan, Qiong Zhang
Abstract:
Tabular foundation models (TFMs) are increasingly popular for their strong predictive performance. Many also provide posterior predictive distributions (PPDs), whose reliability remains less studied. We assess reliability through point prediction, distributional quality, marginal and subgroup calibration, and interval efficiency. We then introduce TabReliBench, a large classification and regression benchmark for these comparisons and dataset‑level analyses. We find that TFMs' rankings change between TabArena and our benchmark and across reliability metrics. Regression TFMs usually achieve near‑nominal marginal coverage and retain narrow intervals, but this favorable marginal behavior does not imply valid conditional coverage. Dataset‑level analyses show that classification difficulty often accompanies poor calibration, whereas regression predictive performance is not a reliable indicator of calibration. Ensembling improves predictive performance but does not consistently improve calibration of PPDs; TFMs also benefit little from conformal post‑processing, which does not reliably resolve subgroup gaps. Reliable TFM selection therefore requires multidimensional evaluation beyond aggregate rankings. Code is available at https://github.com/Tianqi‑Zhao/TabReliBench.
Authors:Panayiotis Smeros, Vincent Emonet, Ruijie Wang, Ana-Claudia Sima, Tarcisio Mendes de Farias
Abstract:
Querying distributed data represented as Knowledge Graphs (KGs) via SPARQL remains a difficult task, requiring fluency in query syntax, endpoint specifications, and schema details. Knowledge Graph Question Answering (KGQA) systems address this by automatically translating natural language into SPARQL, and recent advances in Large Language Models (LLMs) have made this translation increasingly feasible. In this paper, we describe SPARQL‑LLM, our open‑source KGQA system, in the context of the TEXT2SPARQL 2026 challenge. In this challenge, SPARQL‑LLM ranks first overall among eight competing approaches across all datasets, achieving up to 59% higher accuracy than the second‑best approach and supporting multilingual question answering in English, German, and Spanish. SPARQL‑LLM is publicly released at https://github.com/sib‑swiss/sparql‑llm and deployed in production over decentralized bioinformatics KGs at https://www.expasy.org/chat.
Authors:Xuanye Wu, Yue Shi, Yongcan Yang, Yiqun Wang, Zhaoqi Wan, Dayu Xia, Yanming Shao, Jiezhang Cao, Xiangfei Li, Yao Mu, Guangtao Zhai
Abstract:
Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic experimental phenomena, presenting significant challenges for robotic manipulation. To enable systematic evaluation under these conditions, we introduce DexBioLab, a simulation benchmark and framework for dexterous manipulation in realistic biological laboratory environments. DexBioLab comprises 14 tasks across 4 robot embodiments, spanning functional tool use, precision manipulation, language‑conditioned visual grounding, and instrument operation. We introduce controllable dynamic experimental phenomena, including flames, vapor, and bubbles, to capture the evolving visual conditions of laboratory procedures. To further enhance environmental realism, we develop a hybrid rendering pipeline that integrates interactive simulated foreground objects with Gaussian‑splatting backgrounds reconstructed from 6 real laboratory scenes. For efficient large‑scale data generation and evaluation, we further build a parallel simulation framework on top of mjlab, proposing Parallel Finite‑State Machine (PFSM) and integrating batched rendering with decoupled trajectory replay for demonstration generation and model‑independent closed‑loop policy evaluation. Our evaluation reveals substantial variation in policy performance across tasks, exposing distinct strengths and weaknesses of current manipulation policies. Real‑world experiments further demonstrate a reduced sim‑to‑real gap, supporting the validity of AutoBio+ for realistic manipulation evaluation. Supplementary materials are available at https://cumulusalpha.github.io/DexBioLabWebPage.
Authors:Ping Guo, Zhiqi Huang, Xinran Li, Linkun Yu, Jingyin Pei, Kamata Seiichiro
Abstract:
In grounded remote sensing visual question answering, a model produces a textual answer and identifies relevant objects in the image as grounding evidence. Existing evaluations measure answer correctness and localization quality, but do not directly assess answer‑grounding consistency. We propose the Answer‑Evidence Support Protocol (AESP) to assess this consistency through evidence support: whether the grounding evidence supports the model's own answer. AESP applies task‑specific rules to existence, counting, and comparison questions. We further construct the Grounded Remote Sensing Visual Question Answering (G‑RSVQA) benchmark, containing 15,027 samples with questions, reference answers, and all related object‑level annotations. Experiments on seven grounded vision‑‑language models reveal failures of evidence support across all three question types. Up to 50.82% of correct answers in the evidence evaluation subset are not supported by the provided grounding evidence.
Authors:Rishabh Jain, Federico Belotti, Stefano Coniglio, Pietro Lio, Stefano Fiorini
Abstract:
Hypergraph Neural Networks (HNNs), designed to model higher‑order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message‑passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior‑guided AttentioNal Dual‑path Architecture), a plug‑in for two‑stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver‑normalised attention and structural incidence priors, while a non‑competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex‑valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real‑world benchmark datasets. The code is available at https://github.com/jainris/panda.
Authors:Antti Hannukainen, Petr Kulikov, Matti Lassas, Lauri Oksanen
Abstract:
An inverse spectral problem for a (1+1)‑dimensional wave equation with wave speed c(x) is considered. The spectral data consist of the eigenvalues of the Neumann‑‑Dirichlet problem for the operator ‑c^2(x)\partial_x^2 together with their derivatives with respect to a Robin parameter at the boundary. We propose a two‑step recovery procedure. In the first step, the Neumann‑to‑Dirichlet map Λ is reconstructed from the spectral data using the results of arXiv/0406582. In the second step, the reconstructed Λ is used within the regularization strategy for the boundary control method presented in arXiv:1509.04478. As an improvement, the regularization of the control problems employs a weighted combination of L^2 and H^1 penalty terms, and we prove a uniform convergence result for the family of optimal controls, thereby allowing the regularization parameters to be tuned simultaneously. We also prove that the forward spectral map, which assigns to a wave speed its spectral data, is Lipschitz continuous, and that the inverse spectral map underlying the reconstruction procedure is Hölder continuous on a subclass of wave speeds. The method is implemented numerically and validated on both smooth and piecewise smooth wave speed profiles. Numerical experiments demonstrate that the recovery of the wave speed exhibits high accuracy and robustness.
Authors:Faith Ogunwoye, Didar Seyidov, Konstantin Gubernatorov, Dzmitry Tsetserukou
Abstract:
Vision‑language‑action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted action can still produce abrupt motion near people. Remedies are evaluated on different streams, delays and metrics. JitterVLA replays identical commands and arrival traces through alternative controllers, isolating the controller's effect. We apply it to synthetic reaches, hand trajectories from 20 people, three open VLA pipelines (838 episodes) and a physical UR3. Interpolation lowered modeled p99 jerk for every source, by 83% (615 to 106 m/s^3) on human motion with 200 ms delay, at a 2.0 cm tracking cost, and the effect replicated on 598 new episodes; jerk‑limited generation lowered jerk further at a larger cost. Entropy‑discontinuity associations had opposite, replicated signs in two pipelines, and entropy gating helped in one but not the other. On the UR3, interpolation lowered accelerometer‑derived jerk by 3.0‑14.0% in all six exploratory pairs, while a position‑derived metric did not resolve the contrast. JitterVLA supports screening before human‑centred studies; code and data will be released.
Authors:Natapong Nitarach, Phume Ngampornsukswadi, Pittawat Taveekitworachai, Surapon Nonesung, Warit Sirichotedumrong, Kunat Pipatanakul
Abstract:
Evaluating large language models (LLMs) on AMLworld's smallest test splits is costly because each of over a million targets requires a separate graph prompt. Sampling reduces this cost, but small balanced subsets change unweighted precision and F1. We introduce AMLworld‑Compact, an evaluation set that retains every illicit test transaction and samples benign transactions from supervised‑score strata. Its HT‑Coreset protocol reduces evaluation targets by 271x and 611x on HI‑Small and LI‑Small. Horvitz‑Thompson weights give unbiased full‑test confusion‑count estimates for fixed predictions; recall is exact, while precision and F1 are ratio estimates. In a held‑out study using Graph Feature Preprocessor (GFP) features, one boosted‑tree family defines strata for evaluating the other over 500 draws per configuration. At the released sizes, F1 RMSE is 0.00‑2.58 percentage points versus 4.44‑7.41 for uniform sampling with HT weights; conservative intervals remain broad. Under the evaluated graph prompts, seven open‑weight LLMs have weighted precision near the predict‑all‑illicit baseline. In an outcome‑stratified audit of 1,000 HI‑Small few‑shot outputs from one run per model, lexical checks pass for 84.5% of correct and 56.2% of incorrect binary verdicts, so these markers do not establish correctness. We release the evaluation sets, construction code, and audit tools at https://huggingface.co/datasets/typhoon‑ai/AMLworldCompactEval and https://github.com/nat‑nischw/AMLworld‑Compact.
Authors:Quentin Le Tellier, Albert Rilliard, Olivier Perrotin, Marc Evrard
Abstract:
Vocal intensity conversion renders a recorded utterance at a different vocal intensity, rather than simply scaling its amplitude, while preserving its content and speaker identity. Existing systems control intensity through a reference utterance or a few discrete categories. Instead, we control the conversion with any target sound pressure level (SPL) on a continuous dB scale. A lightweight converter, trained as a conditional cycle‑consistent GAN without parallel data, transforms the features of a frozen speech encoder, from which a pretrained vocoder resynthesizes the speech. We compare three representations, log‑mel spectrograms, WavLM, and SpeechTokenizer, and introduce an objective protocol that measures the achieved intensity change against the target one on a calibrated corpus. With WavLM, the converter achieves a measured 83% of the target change in intensity and also shifts the fundamental frequency. A perceptual evaluation confirms that perceived vocal intensity increases with the target level on amplitude‑normalized stimuli.
Authors:Arzu Burcu Güven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, Rob van der Goot
Abstract:
Negative polarity items (NPIs), such as "anymore" and "ever", are grammatical only in select environments, predominantly in the context of negation. Because their grammaticality depends on both syntax and semantics, they have a rich, multilingual literature in theoretical linguistics, yet work on NPIs in language models is almost entirely confined to English. We introduce BLiMPPI, a multilingual evaluation suite of over 12,000 NPI minimal pairs in 33 languages, built by a pipeline that discovers NPIs in Universal Dependencies (UD) treebanks, labels their licensing environments, and perturbs these into minimal pairs. Evaluating 12 multilingual and 2 monolingual models alongside n‑gram baselines, we find that multilingual models struggle most with lower‑resource languages and with pairs that are not n‑gram solvable. We further examine the effects of NPI typology, language family, licensing variability, and NPI‑licenser distance. Data: https://github.com/arzuburcuguven/blimppi‑data
Authors:Gautier Marti
Abstract:
Correlation matrices combine nonlinear positive‑definiteness constraints with a simultaneous row‑column permutation symmetry. Their generation therefore involves two distinct problems. First, every output must remain on the correlation elliptope, irrespective of how its variables are ordered. Second, the generator must learn the domain‑specific distribution occupied by empirical data inside that valid space. Existing financial image generators mix these problems: they learn ordered matrix entries and repair invalid draws afterward. We introduce CorrFlow, which separates them. A generalized Fisher transform maps full‑rank correlation matrices bijectively to unconstrained coordinates, guaranteeing valid outputs without projection. A graph flow‑matching network learns the empirical distribution in those coordinates and is permutation equivariant: reordering the variables only reorders the output. We compare a conditional CorrGAN reimplementation, image diffusion, and CorrFlow on 80×80 Taiwan equity matrices using five seeds per neural method. CorrFlow produces 100% valid draws without repair and obtains feature sliced‑Wasserstein distance .712\pm.141, versus 4.339\pm.252 for CorrGAN and 4.610\pm.191 for image diffusion after their invalid outputs are projected back to the correlation space. It also improves this distance over historical lookup, while lookup retains higher local coverage. These results support the combined construction under the evaluated training budget; they do not establish superiority over fully tuned image models or isolate the contribution of each component.
Authors:Saibo Geng, Andrea Gentilini, Xinxian Ma, Nathan Ranchin, Yunzhen Yao, Robert West
Abstract:
Dynamic vocabularies built on‑the‑fly via Lempel‑Ziv‑Welch (LZW) compression let a language model represent the same text in fewer tokens, but prior work shows that this efficiency comes at a cost to reasoning and perplexity. We show that this quality gap is not a fundamental price of compression but can be substantially closed through targeted improvements to the compression layer and LZW codec, without modifying the underlying pretrained language model. Our analysis reveals four key issues: (1) LZW's compression algorithm creates an implicit conflict between the input embedding and the output embedding, a tension that a single shared embedding cannot resolve; (2) naively applying RoPE to LZW‑compressed sequences ignores that merged tokens can span multiple tokens; (3) merging certain tokens (notably digits) disproportionately hurts downstream accuracy; and (4) dynamic vocabularies introduce an inherent segmentation ambiguity that confounds standard evaluation. We address these through decoupled input/output dynamic embedding layer, size‑aware RoPE positioning, and a modified LZW codec supporting selective merges and multi‑view segmentations. Across four models, our recipe improves perplexity on every corpus we evaluated as well as mathematical reasoning accuracy, closing roughly half the quality gap between zip2zip and the uncompressed baseline at a similar compression ratio on general text. This represents a significant step toward lossless adaptive‑vocabulary compression for language models.
Authors:Michelle Espranita Liman, Ioana Ionac, Alexander Müller, Eimo Martens, Daniel Rueckert, Philip Müller
Abstract:
Diagnostic information in multi‑view medical imaging studies, like echocardiography, is distributed across views, each capturing different aspects of cardiac structure and function. In clinical practice, these views are first interpreted through quantitative measurements, which then support specific diagnoses. However, existing echo representation learning methods do not explicitly model measurements as an intermediate representation between views and diagnosis. As a result, individual measurements are not explicitly represented in a form that downstream diagnostic tasks can selectively use, hindering accurate diagnosis. Inspired by this hierarchical clinical workflow, we introduce EchoTab, a measurement‑grounded representation learning framework that represents multi‑view echocardiography studies using measurement‑specific tokens as an intermediate representation between echo views and diagnosis. Each measurement token selectively attends to the views relevant to its estimation and is trained through tabular reconstruction to predict the corresponding measurement value. The resulting tokens form a factorized echo representation, which encode clinically meaningful structural and functional properties and can be selectively used for downstream diagnosis. Unlike the commonly used contrastive approaches, EchoTab does not require negative samples or large batch sizes. Experiments on two datasets demonstrate that EchoTab learns effective measurement‑grounded representations that improve diagnosis classification in multi‑view echocardiography and its learned attention aligns with established measurement‑view relationships used by cardiologists. Our code is accessible at https://github.com/michelleespranita/EchoTab.
Authors:Shouchen Qiu, Huaijun Qiu
Abstract:
Prediction‑market prices combine traders' judgments about future outcomes, but their errors may be concentrated in particular price ranges and change as settlement approaches. Prior cross‑category evidence shows that Polymarket weather markets have larger pricing bias than other major categories, without separating low‑price longshots from high‑price favorites. This study compares the two ranges at different horizons in daily maximum‑temperature markets. Event‑level probability scores, cubic B‑spline recalibration, and Buy‑NO tests are used to examine the form and effect of the bias. Longshot overpricing mainly occurs at distant horizons, whereas favorite overpricing becomes much stronger during the final 1‑12 hours and is nearly one order of magnitude larger. Event‑level scores improve toward the daily temperature peak, while simple recalibration yields no statistically significant gains. The Buy‑NO results in the corresponding time windows also show that the main pricing error is concentrated at the favorite end.
Authors:Xuhui Kang, Sung-Wook Lee, Yan Zhuang, Tianle Zhong, Yen-Ling Kuo
Abstract:
Pre‑trained robot policies, such as Vision‑Language‑Action (VLA) models, provide a strong foundation for generalist robotic manipulation, but adapting them to new tasks with only a few demonstrations remains challenging. Directly fine‑tuning a pre‑trained policy often achieves strong in‑distribution performance, yet it can distort the representations learned during pre‑training and weaken the model's generalization ability under out‑of‑distribution shifts. We propose Retrieval‑Anchored Representation Alignment (RARA), a two‑stage adaptation method, to address this issue. First, RARA retrieves representation features from pretraining with similar task intent and motion, and aligns the policy's representations of the target demonstrations with the pre‑trained representations of the retrieved demonstrations. Second, RARA fine‑tunes the policy on the target demonstrations while limiting how much its representations of the retrieved demonstrations change during fine‑tuning. By reducing the representation mismatch before action learning and preserving pre‑trained knowledge during adaptation, RARA limits unnecessary representation drift during fine‑tuning. Experiments with Diffusion Policy and GR00T on 13 tasks in RoboTwin, RoboCasa, and on a real robot show that RARA improves the average success of fine‑tuning by 9.5 points in‑distribution and 13.6 points out‑of‑distribution, and outperforms all compared baselines. Project page: https://shua‑kang.github.io/RARA_robot/
Authors:Zhihao Tian, Zifan Wang, Enzheng Zhu, Xun Yang, Jialong Qin, Mengbo Wang, Junwei Liang
Abstract:
Humanoid tennis requires reactive whole‑body coordination under rapidly changing ball dynamics, yet collecting task‑specific motion‑capture demonstrations that cover diverse approach states, contact locations, and timing conditions is costly. Inspired by how humans acquire new skills by combining reusable motor experience with task‑specific demonstrations, we present ATHLETE, a framework for learning reactive humanoid tennis from generic locomotion data and ordinary tennis videos. ATHLETE combines LAFAN1 locomotion with video‑derived striking motions and uses Trajectory‑Guided Motion Matching (TGMM) to construct diverse task‑conditioned whole‑body references. These references are transformed into a reactive policy through physics‑based teacher‑‑student learning with online DAgger distillation, followed by PPO fine‑tuning for the tennis task. The resulting policy responds directly to robot and ball observations without requiring reference‑motion inputs at deployment. On a real humanoid robot, ATHLETE achieves forehand and backhand hit rates of 90.00% and 86.67%, respectively, and enables multi‑shot rallies with human players. These results demonstrate that ATHLETE can learn dynamic humanoid tennis skills from reusable locomotion data and video demonstrations. Result videos and code can be found at https://anonymous.4open.science/w/ATHLETE2027/.
Authors:Denis Degtiarev, Anna Gromova, Sergey Katkov, Artem Mikhailin, Safina Mukhamedshina, Albert Nasybullin, Karina Shaikhutdinova, Artem Tuzov
Abstract:
Every evaluation run of a vision‑language model (VLM) already produces per‑question artifacts (predictions, judged correctness, metadata) that leaderboards reduce to a single score and then discard. We reuse those artifacts to audit the benchmark itself, and add an audit layer to VLMEvalKit: six detectors that run as post‑processing over saved outputs, plus one additional text‑only rerun with visual inputs removed, emitting a per‑question report. The layer reports per‑question findings together with aggregate audit statistics; for the headline quantities, it attaches permutation‑null excess where applicable, interval estimates for proportions, and sensitivity to the model pool. Auditing MMStar, a benchmark curated specifically so that the image is indispensable, with three models, we find zero aggregate accuracy gain from the image on 20.2% of its questions. An item‑matched permutation null accounts for 17.0%, leaving a +3.2 pp excess (corrected permutation p = 0.001). This excess is concentrated in the mathematics and science categories and is absent from coarse perception, a localization consistent with knowledge substituting for visual evidence; confirming that mechanism would require item‑level review. A second detector flags 134 questions on which every model that answered in parseable form picked the same non‑gold option, and these candidates peak in the perception categories, whereas the zero‑gain excess is largest in mathematics and science & technology. Replacing the LLM judge with exact string matching changes no conclusion, and every scoring discrepancy between the two traces to one model whose free‑form answers a parser cannot read. On a binary yes/no benchmark the detectors measurably fall out of calibration, a scope boundary that we measure and report. The audit layer, the prediction artifacts and the generated reports accompany this submission.
Authors:Changye Li, Zijian Zhang, Muheng Li, Yiduo Pang, Tingjun Wu, Menghan Li, Yitong Liu, Hongkai Guo, Mengchi Chen, Chenyu He, Kaisheng Ma, Wei Xu, Song Han, Sihao Liu, Ligeng Zhu
Abstract:
Existing LLM‑based agent approaches for long‑horizon machine learning engineering (MLE) often use context engineering, implemented through custom harnesses that manage tools, skills, and accumulated experience. However, we find that as single‑session execution lengthens, the agent's reasoning intensity declines alongside slowing progress, summarized as long‑horizon deceleration, and the benefits of these custom harnesses vanish as the underlying model improves, summarized as harness staleness. These findings suggest that their benefits may not persist across versions, which motivates a workflow‑level approach in which agents build on one another's work across sessions. We model these alternating agent sessions as a context‑reset semi‑Markov decision process (CR‑SMDP) and establish sufficient conditions for alternation to exceed single‑agent reference limits, derive an explicit experiment‑budget bound, and characterize when session caps preserve this advantage. Guided by this analysis, we introduce Humanize MLE Agents (HMA), which alternates two agents on a shared workspace, limits each session to at most k experiments and resets the context between sessions, without additional task‑specific harness or prior knowledge supplements. Across 75 Kaggle tasks in MLE‑bench, HMA reaches a state‑of‑the‑art any‑medal rate of 78.2%, exceeding the strongest open‑source baseline by 8.0 points with a quarter of its time budget. Moreover, among six configurations spanning five heterogeneous agent pairs, HMA improves any‑medal rate over the constituent baselines by 6.4 points on average, demonstrating cross‑pair generality. A supplementary case on kernel optimization reaches 878 cycles, within 1.6% of the community leaderboard leader, demonstrating applicability beyond MLE.
Authors:Yilian Shi, Qingyu Liu, Tianyi Zhou, Wende Tan, Chenyang Si, Yueming Lyu, Xia Hu, Dongrui Liu
Abstract:
Recent advances in large language models (LLMs) have enabled agent‑based approaches for repository‑level automated vulnerability repair (AVR), showing promising repair capabilities. However, an important issue has received limited attention: how can we ensure that the evidence generated during the repair process is reliable enough to support accurate vulnerability understanding and patch generation? In this paper, we propose EviRepair, an evidence‑driven multi‑agent framework for repository‑level AVR. EviRepair introduces evidence review into the repair process by enabling agents to collaboratively collect vulnerability evidence, verify evidence consistency, and construct a reviewed vulnerability model for patch synthesis and validation. Extensive experiments on PatchEval and SecBench benchmarks with multiple agent frameworks and LLMs demonstrate that EviRepair consistently outperforms existing agent‑based baselines, achieving average improvements of 16.37% and 40.33% in resolved rates on PatchEval and SecBench, respectively. Codes are available at https://github.com/lly‑s/EviRepair
Authors:Xueyang Feng, Chaozhuo Li, Xiaohe Bo, Jiakai Tang, Chenxi Li, Qiwei Ye, Xu Chen
Abstract:
Agentic orchestration delegates tasks to sub‑agents with independent contexts, providing a key path to extending the effective execution horizon of large language model agents. However, existing orchestration methods face two limitations: in execution, current methods fix sub‑agents' goals and harnesses at creation time, limiting their adaptability to dynamic real‑world environments; in optimization, sparse and delayed feedback constrains effective learning of the orchestration policy. To address these limitations, we introduce POrchestra, an on‑the‑fly proactive agentic orchestration framework. It allows sub‑agents to proactively communicate with the orchestrator at any logical point to request harness adjustments while maintaining continuity of local execution. This mechanism not only enables sub‑agents to adapt seamlessly to dynamic environments but also naturally inspires On‑Policy Orchestration Distillation (OPOD) based on hindsight feedback. OPOD attributes sub‑agents' proactive revision requests back to preceding orchestration decisions to derive targeted improvement signals, and leverages these signals as process supervision alongside the final reward to optimize the orchestration policy. We evaluate POrchestra on three long‑horizon agent benchmarks spanning deep research, dynamic digital assistance, and software engineering. POrchestra improves task success by an average of 9.1% relative to the strongest baseline; combined with OPOD, it improves small‑orchestrator success by an average of 10.0% relative to existing training baselines. POrchestra also achieves a more favorable cost‑performance trade‑off. Our code is publicly available at https://github.com/XueyangFeng/POrchestra.
Authors:Zheming Fu, Ruizhe He, Wenbin Ning, Xiaoxiao Ma, Xuan Yang, Lianrui Mu, Haoyang Huang, Chang Liu, Lei Wang, Siming Fu
Abstract:
Video reward optimization can increase proxy rewards while reducing video dynamics, a mismatch we identify as \emphmotion collapse, including under a video‑level motion‑quality reward. We propose GraftOOD, which replaces part of the online rollouts in selected prompt groups with out‑of‑distribution video exemplars. A shared velocity‑regression objective combines group‑relative reward rectification with sequence‑level exemplar supervision, requiring neither exemplar generation trajectories nor an additional supervised‑loss coefficient. We characterize its pointwise optimal field as a source‑posterior‑weighted combination of the two source‑specific target fields. We construct GraftSet‑Video, containing 9,588 format‑matched exemplars for 2,397 prompts, to supply external supervision. On Wan2.1‑1.3B, GraftOOD raises Dynamic Degree from 4.86 to 44.41 relative to reward‑only FlowAWR, while HPSv3 decreases from 8.10 to 7.32 and VisionReward remains close. At the main setting, GraftOOD leads nine of ten metrics among three exemplar‑augmented methods. Ablations of exemplar frequency and within‑group count reveal dynamics‑‑reward trade‑offs rather than uniform gains from increasing exemplar usage. These results support external exemplar supervision as a complement to scalar reward feedback, without equating higher measured dynamics with coherent, visually plausible motion in the generated videos. Code is available at \hrefhttps://github.com/Fuzm7/FlowAWRGraftOOD.
Authors:Ohad Sheelo, Tal Golan, Ari Pakman
Abstract:
The BROJA partial information decomposition provides an operationally motivated way to separate the information that two sources carry about a target into unique, redundant, and synergistic components. For continuous and high‑dimensional variables, however, computing the BROJA decomposition requires solving a difficult optimization problem. Previous work showed that, when each source‑target pair is jointly Gaussian, the problem can be reduced to finite‑dimensional numerical optimization. Here, we derive a closed‑form solution to this optimization problem, enabling direct computation of the BROJA decomposition in the general multivariate Gaussian setting. The resulting closed‑form expression enables speedups exceeding two orders of magnitude for high‑dimensional Gaussian BROJA PID. Code is available at https://github.com/brainsandmachines/Eigen_PID.
Authors:Daniel Varela Sánchez, Guillermo Martín-Coello Juárez
Abstract:
Speech recognition on low‑cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present Oído, an inference engine that runs open‑vocabulary English speech recognition entirely on an ESP32‑S3 (240 MHz dual‑core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator). Oído executes a 13M‑parameter Conformer‑CTC model with int8 weights, per‑frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth. On LibriSpeech test‑clean/test‑other the on‑device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, and 3.33%/7.15% with an on‑chip 1.3M‑parameter language model; a transducer head gives 2.99%/6.67% and an int4 variant fits in 8.3 MB. This is 2.3‑2.8x fewer errors than the vendor's recognizer on the same chip, and fewer than Whisper tiny.en and Moonshine tiny in full precision on a laptop, including in real noise and reverberation. Measured on a physical board, the firmware runs at 1.97x the duration of the audio on one core, with transcripts identical to an instruction‑exact emulator: accurate, but not yet real time (a 4 s command is transcribed about 8.7 s after it ends). The two‑core schedule gives wrong results on silicon and is disabled. A streaming variant shows partial text while the speaker talks and delivers the final text 4.1‑11.1 s after clips of 3.6‑10.6 s (7.2‑22.4 s in utterance mode) at 4.86%/11.03% WER; a Spanish model makes 34‑60% fewer errors than multilingual Whisper tiny. Code and models are released.
Authors:Arvasu Kulkarni, Mahir Jain, Aditya Ray Mishra, Sandeep Singh, Siddharth Panwar
Abstract:
Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream tasks. These models are difficult to compare: different evaluations use different datasets, task definitions, preprocessing, downstream heads, hyperparameter tuning, and baselines. We present PPG‑Bench, a unifying open‑source benchmark evaluating five public PPG FMs across nine open‑access datasets and 26 tasks spanning cardiovascular state estimation, activity, sleep, screening, and identification. We evaluate models under four downstream regimes of increasing adaptability (linear probing, random forests, frozen MLP probes, and end‑to‑end finetuning), with model‑specific hyperparameter optimization and subject‑disjoint cross‑validation. We also evaluate against three controls: random/constant baseline, a deterministic pulse‑rate‑variability (PRV) feature set, and a generic time‑series FM (MOMENT). Across all 68 task‑regime combinations, 35.3% show no meaningful FM advantage over the random/constant baseline, and in 83.8% no PPG‑specific FM outperforms both PRV and MOMENT. We also find that added downstream head capacity generally improves performance, but can cause significant degradation if the model is performing at chance/constant level. Current PPG FMs therefore cannot yet claim to provide generalised and transferrable representations of PPG signals, and future evaluation in this field should include checks against simple, non‑learned controls. Code for the benchmark and instructions to evaluate more models can be found here ‑ https://github.com/mannas‑ai/ppg‑bench.
Authors:Kurt Butler, Damian Machlanski, Michael P. J. Camilleri, Zhihua Liu, Panagiotis Dimitrakopoulos, Feng Chen, Ricardo Silva, Sotirios A. Tsaftaris
Abstract:
Time series causal discovery ‑‑‑ the process of detecting causation between time series ‑‑‑ has its roots in signal processing research from over a half century ago. It has since received active and widespread interest, evolving into different forms across domains such as statistics, dynamical systems and machine learning. Entering this fragmented and constantly evolving research landscape can be daunting. This tutorial provides a unified overview of time series causal discovery across research communities. We introduce the key principles invoked by modern discovery methods, and we discuss their limitations and assumptions. We also discuss how discovery methods are evaluated, benchmarked and fine‑tuned. We further discuss how causal discovery for time series is becoming crucial for downstream tasks in agriculture and Earth science, in healthcare, and in artificial intelligence systems. We believe that this tutorial will serve as a valuable resource in navigating the deluge of recent causality research, as well as an invitation to signal processing audiences to contribute to causal problems. To further help readers, we host a companion repository to link to useful code packages and resources at https://github.com/CHAI‑UK/Awesome‑Time‑Series‑Causal‑Discovery.
Authors:Yuhao Wei, Yong Chen, Chen Yang, Yingchen Wang, Delin Ouyang, Lingfeng Qi, Qi Lan, Zhanhua Zhang, Jie Li, Guofa Li
Abstract:
End‑to‑end planning in complex scenes must model diverse driving intentions and select the most appropriate trajectory using scene context and future risks. However, multi‑query regression may collapse to one mode, while anchor‑based and generative planners share a scaling bottleneck: small candidate sets limit maneuver coverage or discovery of high‑quality trajectories, whereas large sets increase generation and scoring costs. We propose ManeuverFlow to generate trajectories for distinct intentions in one step and evaluate and refine them using current and future evidence. The model selects relevant maneuver modes, draws corresponding noise from the Gaussian mixture prior, and maps it to intention‑specific trajectories through a maneuver‑routed mixture‑of‑experts MeanFlow field. Current features and future features predicted by the world model are aggregated along each candidate path into trajectory‑specific evidence for PDM‑factor prediction. Using the same evidence, the refiner predicts local Frenet residuals for selected candidates; refined and retained candidates are then jointly re‑scored for selection. With a lightweight TransFuser‑style ResNet‑34 backbone, ManeuverFlow achieves 90.6 PDMS and 90.1 EPDMS on NAVSIM v1/v2 navtest, respectively. On WorldEngine's recent 3DGS closed‑loop benchmark, it attains the highest listed PDMS^\ast scores of 75.19 and 74.58 under logged and IDM‑controlled traffic, respectively. Code: https://github.com/wyh‑autonomy/maneuverflow.
Authors:Quentin Rolland, Fabrice Mayran de Chamisso, Jean-Baptiste Mouret
Abstract:
Reliable real‑time failure detection is critical for deploying learned robotic policies, yet defining failures explicitly is infeasible in open‑world settings. This motivates failure detection from nominal demonstrations alone. Existing methods, however, struggle to balance robustness to benign variations with the ability to capture semantic and interaction‑level errors. We introduce Robot‑WATCHDOG, a self‑supervised framework that models manipulation scenes as object‑centric spatio‑temporal graphs. Building on this representation, we investigate two complementary failure scores: Trajectory Projection and Correlation (TPC), a non‑parametric measure of geometric deviation and relational consistency, and GnnT, a Graph Attention Network with a Transformer that predicts future object states. We evaluate both on a public benchmark and on a new dataset of semantically complex manipulation tasks, across six tasks and ten random seeds. Both scores outperform image‑based failure detectors, particularly on semantic and interaction‑level failures. TPC provides the strongest performance in AUPR and MCC on both dataset averages, with low variance across seeds. Qualitative results are made available at https://cea‑list.github.io/Robot‑WATCHDOG/.
Authors:Seungki Hong, Kyeongwon Jeong, Taekwang Jang
Abstract:
Ising machines solve combinatorial optimization problems by annealing a network of spins. On dense problems, the least expensive hardware step updates all spins at once: stochastic cellular automata (SCA) decide every spin in parallel and read couplings only for the spins that flip. Such synchronous updates create an echo: the previous state of each spin returns to it through its neighbors. On K2000, a Max‑Cut benchmark on a 2,000‑node complete graph, two‑thirds of all flips undo the same spin's flip one step earlier. This paper presents Anechoic, an FPGA Ising machine that subtracts this echo from every decision. Dynamical mean‑field theory identifies the echo as an Onsager reaction; its one‑step strength is the number of spins whose flip probability is strictly between 0 and 1, divided by twice the temperature. The decision logic already flags these spins, so the correction costs one population count per step; a temperature‑scaled form needs none. An RTL engine executes a dense 2,000‑spin step in 18.1 cycles plus 0.14 per flip, so every saved step and flip saves time. Both forms find better K2000 cuts at every step budget than five published binary‑spin algorithms at their papers' settings. Twelve engines at 250 MHz on an AMD Alveo V80 reach a cut of at least 33,000 with 99% probability in 0.05 ms, 5.2 times faster than the fastest published hardware, at 5.4 mJ per solution; with 2‑bit couplings, the temperature‑scaled form is also faster than uncorrected SCA on 49 of 51 G‑set graphs. On an NVIDIA GH200 GPU, the same engine is 2.3 times slower and needs 8.0 times more energy per solution, though its 240 concurrent anneals give 2.3 times the throughput. As a GlobalFoundries 22 nm ASIC at 1.25 GHz, twelve engines would reach the target in about 10 microseconds, 26 times faster than the fastest published hardware. The code is available at https://github.com/skethz/anechoic.
Authors:Jiahao Wang, Hongjin Qian, Yuyang Hu, Jiajun Zhang, Zheng Liu
Abstract:
As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows. This delegation carries a quiet assumption, that the delivered agent will faithfully serve the requirement it was built for, rather than merely appear to. No existing benchmark tests that assumption. We introduce AgentSWE, a benchmark for agent software engineering spanning three stages of the agent development lifecycle: Creation of a complete agent from a natural‑language requirement, Editing of production agent codebases, and Optimization toward a measurable target. Each of its 25 tasks is framed as a commission rather than an exam: the builder iterates against a handful of illustrative cases, while acceptance exercises the frozen submission on held‑out cases designed to span the full requirement, grading behavior against requirement‑derived rubrics under hard pass/fail gates or, for Optimization, against the source benchmark's own objective. Evaluating a range of coding agents across models and harnesses, we find that none comes close to delivering what a requirement asks for: the gap between what a builder demonstrates and what it delivers is the norm, not the corner case. An accompanying empirical analysis of the failure trajectories identifies recurring failure modes, among them overfitting to the visible cases, hollow runtimes behind code that looks complete, and unsupported claims of completion, each a signal by which coding agents can learn to build the agents they are asked for.
Authors:Carolin Cissée, Dennis Fast, Raneen Younis, Alexander Löser, Wolfgang Nejdl, Zahra Ahmadi
Abstract:
Widely reused preprocessing pipelines for electronic health records can encode assumptions about when information became available, what clinical episode a label represents, and what missingness means. These assumptions are rarely audited systematically and may propagate across subsequent studies when preprocessing pipelines are reused or adapted. Using MIMIC‑IV as a case study, we identify and quantify violations of three complementary forms of preprocessing alignment, i.e., temporal, semantic, and population alignment, which manifest as temporal leakage, feature‑task misalignment, and biased imputation. We show how ambiguities in timestamp semantics, episode definitions, endpoint construction, modality selection, and missingness handling can alter the information available to a model and affect the interpretation of its predictions. Through analyses of MIMIC‑IV data and empirical case studies, we demonstrate that such choices may remain largely invisible in aggregate predictive performance while changing the signals on which models rely. We derive practical recommendations for temporally valid, clinically aligned, and reproducible preprocessing of MIMIC‑IV and related electronic health record datasets. The code for our experiments, including our proposed preprocessing, is available at: https://github.com/m2l‑lab/mimic‑iv‑preprocess.
Authors:Zijun Wang, Yifan Ji, Haidong Xin, Zhenghao Liu, Yukun Yan, Zulong Chen, Yu Gu, Ge Yu, Maosong Sun
Abstract:
Large Language Models (LLMs) struggle to answer questions over long interaction histories, where relevant evidence may be scattered across distant sessions. Existing memory systems typically construct and maintain sophisticated memory representations offline, before user queries are observed, incurring substantial overhead and limiting their ability to adapt memory construction to query‑specific information needs. In this paper, we propose T^2Mem, a Test‑Time Memory construction framework that defers memory construction to inference and progressively builds memory along the reasoning process. Specifically, T^2Mem extracts lightweight episode‑level metadata offline to capture key events and temporal associations as retrieval cues, while retaining links to the original interactions to facilitate the reconstruction of dependencies across episodes. At test time, the LLM adaptively retrieves and organizes metadata and raw interactions, progressively constructing query‑specific memory as reasoning unfolds. Experiments on LoCoMo and LongMemEval show the effectiveness of T^2Mem by consistently outperforming strong baselines across different backbone LLMs while substantially reducing the overhead of memory construction. Further analysis demonstrates that deferring memory construction to inference enables the memory construction process to dynamically adapt to the information needs of individual queries. All code is available at https://github.com/OpenBMB/T2Mem.
Authors:Benyuan Meng, Yiliang Zhang, Jin-Wen Wu, Qianqian Xu, Qingming Huang, Longtao Huang
Abstract:
Diffusion feature, i.e., intermediate activations extracted from diffusion model backbones, has emerged as a promising approach for unifying generation and vision understanding (discrimination). Despite the advancement of diffusion backbones into Diffusion Transformers (DiT), it has been found in previous literature that DiT models tend to yield weaker features despite their stronger generation capability. This poses a threat to good generation‑understanding unification. In this work, we show via controlled from‑scratch training that long skip connections (LSCs) affect feature quality: a DiT with LSCs can produce better features. Furthermore, through detailed diagnosis, we reveal the mechanism causing this effect. Specifically, the input to a diffusion backbone contains both the latent image and the diffusion noise, where the diffusion noise, though appearing in the form of noises, actually reflects the information of the diffusion trajectory. To make the final prediction of a noisy variable (v or ε), the diffusion backbone needs both signals. However, carrying both signals through the main backbone effectively "pollutes" the image signal with noises, leading to poor feature quality. In comparison, LSCs can shortcut diffusion noises directly to where they are needed, leaving much weaker noise signals in the main backbone, thus improving feature quality. To name the broader impact, this reveals that LSCs are not a mere trick but closely related to the diffusion objective. The code is at https://github.com/Darkbblue/dit‑lsc.
Authors:Zheming Fu, Wenbin Ning, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lianrui Mu, Chang Liu, Lei Wang, Siming Fu
Abstract:
Regression‑based RL for flow models mandates sampling \emphwithout classifier‑free guidance (CFG) to align the rectification target with the reference marginal velocity. Extending Advantage‑Weighted Rectification (AWR), which dynamically scales this regression target based on reward, to video generation encounters a cold‑start pathology: unguided rollouts collapse into low‑fidelity regions where intra‑group reward gaps remain wide but no longer track perceptual quality, rendering the optimization target arbitrary. Conversely, utilizing guided rollouts to restore fidelity, while keeping the regression target tied to the guided sampling distribution, provably triggers a geometric guidance blow‑up under exponential‑moving‑average (EMA) updates. We propose VideoAWR, which resolves this tension by untying sampling from the optimization anchor. An EMA reference policy generates \emphguided rollouts to restore reward separability, while the trainable policy is rectified against the \emphunguided conditional anchor. We prove this decoupled objective is mathematically exact: the centered advantage cancels the anchor in expectation, filtering out the reward‑neutral guidance offset and retaining only the reward‑correlated covariance increment. Consequently, inference runs single‑branch and CFG‑free, yet inference‑time CFG preserves its original semantics. Furthermore, we introduce \emphDecoupled Timestep Rectification, supervising each rollout at an independently sampled timestep without bias to improve sample efficiency. Experiments on Wan2.1‑1.3B and 14B demonstrate stable optimization and superior visual and motion quality in \emphboth CFG‑free and CFG‑enabled regimes. Code is available at \pdfstartlink attr/Border [0 0 0] user/Subtype /Link /A << /S /URI /URI (https://github.com/Fuzm7/FlowAWR) >>% \textcolorpearDarkVideoAWR\pdfendlink.
Authors:R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
Abstract:
Tool‑using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing tooling typically addresses these stages in isolation. We present AgentTune, a source‑available Python library that provides a unified framework for developing and optimizing tool‑using agents. At its core, AgentTune introduces a two‑tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing. Building on this substrate, the library provides a shared agent runtime, tool‑calling reinforcement learning, agentic distillation, retrieval‑augmented generation, decision workflows, evaluation utilities, and a designed path toward self‑healing agents. These components are exposed through composable interfaces so that agent strategies, training methods, evaluation procedures, and recovery mechanisms can be combined without requiring separate representations or pipelines. We demonstrate the framework across agentic tasks including legal contract search and domain‑specific applications, illustrating its use for trajectory‑based optimization, tool‑use evaluation, and cross‑task agent development. AgentTune is released as a source‑available library at https://github.com/Lexsi‑Labs/AgentTune
Authors:Mengdi Liu, Chenghao Jia, Hong Chang, Shiguang Shan
Abstract:
Designing tool‑integrated reasoning agent systems for complex chemical tasks remains a fundamental challenge. Although recent chemical agents have demonstrated promising performance through the integration of external tools, they are still built on the autoregressive paradigm, which is inherently constrained by causal attention and left‑to‑right generation. As a result, they often struggle with bidirectional molecular understanding, global coordinated tool planning and multi‑source evidence integration. To address this challenge, we introduce ChemDiffAgent, the first masked diffusion language agent for tool‑integrated chemical reasoning. It reformulates chemical reasoning as iterative denoising over interleaved reasoning and tool‑use trajectories, enabling tool‑use decisions to be jointly determined under full‑trajectory context and thereby better aligning the agent's decision boundary with its knowledge boundary. We further analyze how this paradigm benefits agentic reasoning through harder infilling subproblems during training and flexible arbitrary‑order decoding during inference. Then we develop a two‑stage post‑training framework, comprising agentic supervised fine‑tuning and variance‑reduced preference optimization, to enhance tool‑use and reasoning capabilities. We also construct a comprehensive benchmark covering six chemistry tasks across both single‑turn and multi‑turn tool‑use scenarios. Under matched budgets and evaluation, ChemDiffAgent consistently outperforms autoregressive agents, achieving an overall improvement of approximately 20%. Notably, our 8B model achieves performance comparable to, in some cases surpassing, significantly larger frontier models, such as GPT 5.5 and Claude Opus 4.7. Code is available at https://github.com/bilemond/ChemDiffAgent.
Authors:Armando Fortes, Tianyi Wei, Chindanai Trakantannarong, Shangchen Zhou, Xingang Pan
Abstract:
Drag‑based image editing provides an intuitive interface for spatially controlled image manipulation by allowing users to specify handle and target points. Existing methods have made substantial progress through optimization‑based, one‑step, and feed‑forward formulations, but they often interpret drag instructions as geometric displacements. This limits their effectiveness when the desired edit depends on interpreting image content and inferring a plausible semantic transformation from sparse point constraints. We introduce ThinkDrag, a unified multimodal framework for drag‑based image editing with visual reasoning. ThinkDrag is trained to associate point constraints with meaningful object‑ and scene‑level transformations, and can optionally produce an explicit reasoning trace that interprets the intended edit before image generation. This reasoning trace improves interpretability, particularly for ambiguous drags. Drag Endpoint Markers, motivated by an analysis of cross‑image attention, provide an explicit correspondence signal that grounds each drag in image space. To support this framework, we construct a supervised dataset of semantic drag transformations paired with reasoning traces and introduce DragBench++, a benchmark targeting challenging drag‑based editing scenarios with reference edit solutions. Experiments show that ThinkDrag achieves state‑of‑the‑art performance, improving generation quality and plausibility while maintaining competitive point‑following precision.
Authors:Georgios Floros, Evangelos Sartinas, Orestis Vangelis, Maria Pateraki
Abstract:
6D pose estimation of novel objects without CAD models remains an open challenge: model‑free methods must build the object representation from scratch during onboarding and have long trailed their model‑based counterparts. We present NovaSplat, a model‑free pipeline that closes this gap. From a short smartphone video captured at onboarding, we train a 3D Gaussian Splatting (3DGS) model that serves both as a template source for detection and coarse pose estimation and as a differentiable backbone for pose refinement. Our refiner optimizes the object‑to‑camera pose with combined photometric and geometric losses under a continuous rotation parametrization free of gradient singularities. On a household‑object benchmark, NovaSplat surpasses all model‑based methods. On a challenging industrial benchmark with textureless and specular surfaces, it outperforms all model‑free competitors while substantially narrowing the gap to model‑based ones. We further introduce NovaSplat‑Lite, an inference variant that is 3.1× faster at negligible accuracy cost. Code will be made available at \hrefhttps://github.com/POSE‑Lab/novasplathttps://github.com/POSE‑Lab/novasplat.
Authors:Jixin Tang, Ji Guo, Jun Zhao
Abstract:
We introduce MORAD, an online reinforcement learning framework for backbone‑conditioned RNA diffusion models that trains a single shared policy across diverse target backbones. Structure‑conditioned RNA inverse design seeks nucleotide sequences compatible with a target three‑dimensional backbone, yet existing reinforcement learning approaches either optimize a separate policy for each target backbone or rely on offline preference optimization, leaving online post‑training of a reusable backbone‑conditioned model largely unexplored. MORAD combines backbone‑conditioned group sampling, group‑relative reward normalization, and a forward‑process diffusion objective. Beyond existing RL rewards that primarily target tertiary structural quality, MORAD jointly optimizes six measurements spanning tertiary geometry, base pairing, ensemble behavior, and sequence composition. To aggregate these heterogeneous objectives despite differences in scale and optimization direction, MORAD maps each measurement to a bounded desirability score and combines them with a weighted geometric mean. Compared with a tertiary‑structure‑only reward, this multi‑objective formulation improves 10 of the 11 evaluation metrics that have a preferred direction. Applied to pretrained RIDE, MORAD improves GDT‑TS by 16.0% (from 0.3300 to 0.3827) and pairing MCC by 19.4% (from 0.6110 to 0.7295), while reducing normalized ensemble defect by 27.5% (from 0.3906 to 0.2830). On the test set, MORAD outperforms RiboDiffusion, gRNAde, and RDesign across tertiary structural accuracy, secondary‑structure pairing, and thermodynamic quality, establishing a new state of the art in structure‑conditioned RNA inverse design.
Authors:Yabiao Wang, Jiafu Wu, Yicheng Xu, Teng Hu, Shengzhe Zhou, Yanxiao Sun, Jiangning Zhang, Ying Tai, Yong Liu, Ming-Hsuan Yang
Abstract:
Co‑speech gesture video generation aims to synthesize realistic, audio‑driven portrait videos from a single reference image and a speech signal. Despite advances in visual quality, diffusion‑based approaches still suffer from degraded local details and imperfect audio‑visual alignment. We present PAGE (Parts‑Aware GuidancE), a general, backbone‑agnostic optimization framework for diffusion‑based generation that enhances local spatial fidelity and global audio‑visual consistency at both the training‑objective and denoising‑trajectory levels. During training, we propose Parts‑Specific Reweighting (PSR), which adaptively increases supervision in semantically important regions such as lips, eyes, and hands, improving visual fidelity in perceptually critical areas. During inference, we introduce Consistency Classifier Guidance (CCG), an audio‑conditioned latent classifier that identifies and corrects audio‑visual inconsistencies in the generated video. We further design Differential Guidance (DG), an efficient inference strategy that refines facial and body motion separately while avoiding cross‑region interference. To enable large‑scale evaluation, we construct the Chinese Announcer Speech (CAS) dataset, the first comprehensive Chinese benchmark for co‑speech gesture portrait video generation. Compared with the widely used PATS dataset, CAS includes substantially more identities, video clips, and scene diversity. Extensive experiments on both PATS and CAS demonstrate that PAGE achieves state‑of‑the‑art performance in perceptual quality, gesture diversity, and audio‑visual alignment. Project page: https://aigc‑explorer.github.io/PAGE/.
Authors:Ruotian Zhang, Yuhui Zhang, Junzhe Huang, Yu Xin, Hehe Fan, Xiaoxiao Sun
Abstract:
GUI grounding, the task of locating the interface element specified by a user instruction, is a fundamental capability of GUI agents. However, existing grounding models remain error‑prone, often causing downstream task failures. We systematically evaluate 10 grounding models across five datasets and identify a major error pattern: most incorrect clicks hit non‑target interface elements rather than the background, with different models repeatedly selecting the same distractors. Motivated by these findings, we propose ConClick, a contrastive learning framework that teaches models to distinguish the target element from plausible distractors. We construct ConClick‑Data, a dataset containing 4,292 challenging target‑distractor pairs, and train the model using complementary listwise ranking and pairwise margin objectives. ConClick improves the grounding accuracy of Qwen3‑VL‑8B from 54.71% to 66.03% on ScreenSpot‑Pro, from 83.72% to 86.28% on MMBench‑GUI L2, and from 63.33% to 70.20% on OSWorld‑G, outperforming prior methods built on the same or similarly sized backbones. We further introduce ReClick, an inference‑time strategy that revisits promising regions and compares predictions across competing candidates, yielding an additional 2.96 percentage‑point improvement in average accuracy across the three benchmarks. Together, our analysis, dataset, and methods provide both empirical insights and practical tools for building more reliable GUI agents. Data and code are available at https://github.com/zhangruotian/ConClick.
Authors:Xiaoyang Ma, Chi Wang, Hong Sun
Abstract:
Multimodal large language models process the complete visual token sequence, increasing prefill cost even when many tokens are irrelevant to the instruction. Token‑wise saliency captures limited inter‑token dependence; graph‑based selection instead depends on how relations govern importance propagation and ranking. We study support‑constrained propagation with node‑degree correction. Inspired by sheaf geometry, SheafPrune uses modality‑specific maps and a sparse text‑‑visual graph to transfer relevance, then propagates scores on a separate visual graph and corrects them for degree. It retains a hard top‑k subset once before the frozen language model. The selector has about 4.5M parameters and is trained end‑to‑end through the frozen LLaVA‑NeXT‑8B backbone using only the language‑modeling objective. Explicit retention‑ratio conditioning lets one checkpoint serve every budget. Across four backbones, ten benchmarks and seven pruning ratios, with all baselines re‑implemented under one protocol, SheafPrune achieves the best retention at 22 of 28 backbone‑‑budget settings, with monotonically growing margins in the moderate‑to‑aggressive regime. On LLaVA‑NeXT, retaining 10% of visual tokens yields a 5.15× net prefill speedup and an 84.0% end‑to‑end FLOPs reduction. Together with component ablations, these results support graph‑constrained importance propagation and degree correction as a design for visual token selection across the evaluated backbones and tokenization schemes, improving the trade‑off between accuracy retention and inference cost. code:https://github.com/Bob0817912/SheafPrune_ICLR_2027
Authors:Zhenke Duan, Xin Li, Jiqun Pan, Dong Xiaofei, Hanwen Ning, Xinyuan Song
Abstract:
In this paper, we propose Data‑Adaptive Mahalanobis Cross‑Head Attention (DAMCHA), a simple and effective attention mechanism that generalizes Multi‑Head Attention (MHA) through the lens of metric learning. Standard MHA implicitly learns a static, head‑wise Euclidean similarity corresponding to the diagonal blocks of a unified cross‑head metric matrix, discarding all off‑diagonal interactions across heads. DAMCHA addresses this limitation via two key innovations. First, it learns the full cross‑head metric matrix, inducing a Mahalanobis‑like similarity that explicitly captures inter‑head interactions beyond the restricted block‑diagonal structure of MHA. Second, it parameterizes the metric matrix as an input‑dependent function, enabling the attention geometry to dynamically adapt to the intrinsic manifold structure of the data rather than remaining fixed after training. We further introduce principled regularization strategies, most notably stack‑wise parameter sharing, to ensure computational efficiency and stable optimization. DAMCHA serves as a drop‑in replacement for MHA and its variants in various Transformer architectures, delivering superior expressive power and learning performance with comparable or reduced computational overhead. Extensive experiments demonstrate that DAMCHA‑based Transformers consistently outperform competing baselines across a range of benchmark tasks. Code is available at https://github.com/erwinmsmith/DAMCHA.
Authors:Xiufeng Liu, Ruyu Liu, Yusen Liu, Huan Huo, Per Sieverts Nielsen
Abstract:
AutoML systems fit ensemble weights over model libraries; a search can score, per category, market or time pane, thousands of weightings of the quantile‑averaged (Vincentized) ensemble for the narrowest conformally calibrated interval; exact calibration scores a group's rows per candidate. We present CALQA, a fixed‑size integer aggregate per group and pane that stores histograms of normalised block scores at a net of weight vectors. One GROUP BY builds it; it merges by addition, retracts deletions by subtraction and scores any candidate without reading a row. We prove that its score never falls below the exact split‑conformal shift and exceeds it by at most the net distance plus a grid gap; that coverage carries over to each fixed weight vector and, for i.i.d. blocks and up to a union‑bound slack, to a searched weight vector deployed with the state's shift; and how small any such state can be. The state is thus a screen: it ranks all candidates, and exact recomputation re‑checks the top 100. On five public panels (604 groups, 10,000 candidates each) this two‑stage plan matches exact search on held‑out blocks (99.95% of its Winkler score), 1.3‑59 times faster. On two real panels of about 10^8 rows (smart meters, retail sales) it is 44‑54 times faster at 99.98‑100.01% of that score and beats sample screens with the same re‑check in regret at less total time. Picks deployed with their exact shift need a held‑out coverage check. The state does not pay off for small or greedy searches, off‑net weights at ample memory or full twelve‑member support.
Authors:Chenjie Zhao, Jiangtian Nie, Jiawen Kang, Yuxu Lu, Ryan Wen Liu
Abstract:
The unmanned aerial vehicle (UAV)‑based maritime surveillance is becoming an important part of maritime Internet of Things (IoT) and intelligent transportation systems. It provides a flexible and cost‑effective way to large‑scale maritime traffic monitoring. However, achieving robust situational awareness remains challenging in practical maritime environments. The main challenges arise from heterogeneous sensing modalities, adverse imaging conditions, and spatial misalignment caused by UAV motion. To address these problems, this paper proposes an airborne multi‑modal graph learning‑driven framework, termed AMGT, for vessel identification and tracking in UAV‑assisted maritime IoT. Specifically, visible and infrared images are first aligned and fused through a multi‑parameter perspective transformation model. This improves visual perception under low visibility and complex illumination conditions. Automatic identification system (AIS) messages are then temporally synchronized using interpolation and linear prediction. To detect small and oriented vessels, we develop a mixed‑modality MAir‑YOLOv8‑OBB detector, followed by the MAir‑SORT algorithm for stable vessel tracking. We further design a graph learning‑driven association network to model temporal dynamics and spatial dependencies among vessels and between vessels and UAVs. A motion‑aware coordinate unification (MCU) strategy is also introduced to reduce nonlinear misalignment caused by UAV motion. Experiments on the MAP‑M2UAV50 dataset show that AMGT achieves high accuracy and efficiency. The results demonstrate its potential for UAV‑assisted maritime IoT applications, including vessel monitoring, traffic situational awareness, and intelligent maritime transportation services. The dataset and code will be available at https://github.com/MAPGroup/AMGT.
Authors:Haocheng Wu, Xiangjie Sui, Xin Sun
Abstract:
The discrepancy induced by surface reconstruction in a distorted point cloud is jointly influenced by its intrinsic structural complexity and signal degradation. This observation motivates a referencefree point cloud quality assessment (PCQA) strategy that estimates and compensates for the complexity‑induced component. We implement this strategy through successive reconstruction. Specifically, reconstructing the distorted point cloud produces an input‑toreconstruction discrepancy that combines the effects of signal degradation and content complexity. Reapplying the reconstruction operator with stronger smoothing produces a discrepancy between successive reconstructions, which serves as a proxy for the complexityinduced component. This proxy is then used to calibrate the inputto‑reconstruction discrepancy. We apply the same strategy to 3D geometry patches and their aligned 2D regions, enabling local comparison of the resulting discrepancies. The calibrated geometric and projected‑view discrepancies are then fused into a final degradation score. For both branches, the discrepancies are measured using frozen pretrained perceptual representations. Our method does not require PCQA‑specific training or fine‑tuning. Experiments on three public databases show that the proposed method consistently outperforms most generic baselines and achieves performance comparable to several full‑reference metrics.The code is available at https://github.com/HC‑W‑whc/CCRD‑NR‑PCQA.git.
Authors:Ziyu Luo, Wei Li, Haoping Huang, Kecheng Liu, Min Gao
Abstract:
Fake news detection under low resource conditions is challenged by the scarcity of labeled data and the difficulty of capturing subtle deceptive patterns. In this work, we focus on low resource fake news detection, where only a small labeled set is available for training, while evaluation is performed under a fixed and unchanged test protocol. This setting is motivated by costly, ambiguous fact‑checking and emerging topics and domains that are poorly covered by existing benchmarks. Existing data augmentation methods, such as synonym substitution, generative adversarial networks, and large language model (LLM)‑based generation, often produce fluent but overly generic content, lacking the task‑specific relevance needed for accurate detection. To address this, we propose SharP, a boundary‑aware text generation framework guided by soft and hard prompts, which leverages LLMs as tools adapted to produce task‑specific synthetic samples. SharP employs a dual prompt mechanism: hard prompts provide structured guidance and factual constraints, while soft prompts introduce learned semantic patterns from the data. These two types of prompts are combined to guide the generation process, enabling the model to produce samples that closely align with the characteristics of news. To further enhance detection performance, we incorporate a boundary‑aware strategy that steers generation toward areas where the classifier is less confident, helping to clarify subtle distinctions between real and fake content. In addition, we adopt a dual objective optimization that balances semantic alignment with the source data and classifier feedback. This encourages the generation of samples that are both domain‑consistent and helpful for refining decision boundaries, ultimately improving model robustness and generalization in low resource fake news detection.
Authors:Zekang Wang, Zhe He, Gaojie Jin
Abstract:
Reinforcement learning (RL) under partial observability typically re‑ lies on recurrent networks to encode temporal information, with the recurrent module trained jointly with the policy or value function. This raises a natural question: is learning the recurrent network actually necessary? In this paper, we investigate whether a frozen recurrent network can provide sufficient memory for solving par‑ tially observable Markov decision processes (POMDPs). Motivated by the theoretical foundations of Reservoir Computing (RC) and Generalized Reservoir Computing (GRC), we study a simple archi‑ tecture that combines a fixed recurrent network with a trainable nonlinear output module. We evaluate this approach across tasks with varying memory demands, observation modalities, and action spaces. Surprisingly, frozen recurrent networks support recall over hundreds of steps and achieve competitive control performance across diverse POMDPs, while using less than half the number of trainable parameters required by their fully trainable counterparts. These results suggest that learning the recurrent memory module is not always necessary for solving POMDPs. Frozen recurrent networks therefore provide a simple, parameter‑efficient, and sur‑ prisingly strong baseline for partially observable RL. Our source code is available at https://github.com/Aphlios/FRN‑POMDP
Authors:Alessandro Licciardi
Abstract:
Federated Learning (FL) enables collaborative model training without exposing raw data, yet remains vulnerable to malicious clients injecting poisoned datasets. Existing defenses are typically reactive (operating during training) or supervised (requiring prior knowledge of attack types) ‑ assumptions that rarely hold in open‑ended IoT deployments. We propose WaSA, a fully unsupervised, offline framework for malicious client detection. Before training, each client transmits a single, privacy‑preserving spectral summary of its data to the server. A denoising autoencoder, trained solely on clean auxiliary data, scores clients based on their deviation from the learned benign manifold, identifying corruptions regardless of the attack type. WaSA requires no attack labels, generalizes to unseen anomalies, and introduces a Boltzmann sampling strategy for client selection that smoothly interpolates between hard exclusion and standard aggregation. Experiments on MNIST, CIFAR‑10, and CIFAR‑100 under diverse attack scenarios (up to 90 % malicious clients) demonstrate consistent improvements over supervised baselines and robust aggregation methods, with negligible communication overhead. Code is available at https://github.com/alelicciardi99/wavelet_autoencoders_malicious.
Authors:Boyuan Xiao, Pu Wang, Ji Feng, Yao-Xiang Ding
Abstract:
World models enable large language model (LLM) agents to anticipate action effects and make informed decisions. For generalization, it is desirable for such world models to transfer efficiently to new environments with similar interaction structure and underlying dynamics. Recent approaches encode world dynamics as symbolic rules or executable programs, whose compositional structure in principle affords transfer, but still organize such knowledge around environment‑specific objects. To address this limitation, we introduce RoleWorld, a symbolic world modeling approach which uses reusable prototypes and environment‑specific parameter values as functional role abstractions to represent how objects behave in interactions. From interaction trajectories, a teacher LLM extracts role prototypes and executable transition rules defined over role instances, together with a skill that guides the actor LLM to use role information in decision making. In a target world, without the teacher, the actor uses this skill together with its current role assignments and parameter estimates, and actively tests a role instance hypothesis through interaction when its uncertainty affects the current decision. Feedback from each action refines role assignments and estimates parameter values online, with no LLM parameter updates and the transition rules held fixed. Experiments on MARS benchmark show that RoleWorld enables robust generalization across environments with significantly altered object behavior, suggesting that functional role abstractions can serve as an effective component for knowledge transfer in world modeling. Project Page: https://future‑item.github.io/RoleWorld.
Authors:Shun Zou, Ziyu Ma, Yi Zou, Yong Wang, Lin Chen, Zehui Chen, Guanhua Chen, Xiangxiang Chu, Feng Zhao
Abstract:
Terminal agents are moving beyond local operations toward completing real‑world professional tasks. Short trajectories can demonstrate commands and tool calls, but they do not fully show how agents accumulate evidence through exploration, use it to inform judgments, continually adjust strategies based on feedback, and turn intermediate results into final deliverables. High‑quality long‑horizon trajectories capture these agentic behavioral patterns and their connections within a single problem‑solving process, providing richer experience for sustained autonomous work. However, environment synthesis may complete or simplify the key work underlying these patterns in advance and thus remove the need for exploration, implementation, and verification that the agent was meant to perform. To address this, we introduce TerminalHorizon, a fully automated data synthesis engine for long‑horizon terminal agents. Starting from real‑world professional work, it uses independent execution to identify key work omitted during construction and uses the resulting evidence to guide environment reconstruction. To extend these patterns across stages, it progressively expands tasks under the same goal, introducing new requirements that build on prior results and call for further exploration and improvement. Using this engine, we construct 1.5K task environments and perform supervised fine‑tuning of Qwen3.5‑27B on the resulting dataset to obtain TerminalHorizon‑27B. We achieve substantial performance gains on all 12 benchmarks spanning coding‑agent tasks, agentic tasks in real‑world production settings, and general reasoning. Multiple RSI case studies, each with a 12‑hour research budget, further demonstrate the model's ability to continue experimenting beyond an initial solution, improve its methods based on feedback, and retain effective improvements in its final deliverables.
Authors:Feng Jiang, Shu Chen, Yuanfei Wang, Yuan Li, Zhenhao Shen, Jiaqi Liang, Yang Chen, Lei Wei, Chen Xie, Ruihai Wu
Abstract:
Scaling robot manipulation policies requires large amounts of diverse, action‑labeled data, yet collecting such data in the real world remains expensive and difficult to scale. Existing methods either depend on human teleoperation, rely on simulation with limited visual realism, or use human videos that lack embodiment‑specific annotations. In this paper, we introduce Grounded‑Dreamer, a framework for synthesizing physically grounded robot manipulation data with video world models while avoiding human teleoperation demonstrations. Grounded‑Dreamer combines the visual realism of generative video models with simulator physics engines. Starting from a small set of real images depicting robot task initial states, our method automatically reconstructs task‑relevant objects and camera geometry, aligns them in simulation, and collects physically valid trajectories. These trajectories are used to post‑train both a video world model and an inverse dynamics model, enabling the generation of visually realistic robot manipulation videos and corresponding robot actions from real images. To filter physically implausible rollouts, Grounded‑Dreamer replays generated trajectories in the reconstructed simulation and retains only those that successfully complete the tasks. The real‑world robot policies are trained solely on the grounded synthetic trajectories without any human demonstrations. Empirical results in real‑world tasks demonstrate that Grounded‑Dreamer can synthesize physically grounded data and substantially facilitate policy learning without requiring human teleoperation effort.
Authors:Wenlin Lin, Fanzhe Wei
Abstract:
Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval‑augmented agents give one model both kinds of tools and let it schedule itself. We present KnowForge, a pipeline separated by job: a hybrid‑retrieval knowledge‑point index; a Locator agent that can only search; a code‑scheduled bridge judge that forces the missing second hop; a typed, size‑bounded evidence pool that merges every search's ranking; a Deep Reader agent that can only read text, images and tables; and a synthesizer with a format checker. With a self‑hosted Qwen3.6‑35B‑A3B (3B active), KnowForge reaches F1 47.5 on the full open‑domain M3DocVQA (2,441 questions, 3,368 PDFs), +11.0 over the published M3DocRAG system and +7.1 over one‑shot retrieve‑and‑answer on the same index and model, and 53.5 with DeepSeek‑V4‑Flash. On MultiHop‑RAG it reaches 73.5 accuracy, above the strongest published RAG and GraphRAG results (71.2); on the single‑document MMLongBench‑Doc it lifts the same model from 34.8 to 47.0, level with GPT‑4o reading page images (46.3). Per‑job ablations on the full benchmarks attribute the gain: the Deep Reader is worth 3.8, 9.8 and 0 points on the three corpora, tracking how visual each is; merging both agents into one with all tools and twice the rounds loses 1.9 F1 because it keeps searching instead of reading; a round‑budget sweep shows a step at four Locator rounds and a plateau beyond; query latency scales with model calls rather than corpus size. We serve the three indexed benchmarks as read‑only knowledge bases (free registration) and release all prompts and the runner code: https://github.com/metask‑ai/knowforge‑bench
Authors:Md. Nesarul Hoque, Rudra Pratap Deb Nath, Umme Salma, Shima Chakraborty, Md. Hanif Seddiqui
Abstract:
Cyberbullying is a growing concern in online communities, yet existing Bengali corpora remain limited in contextual information, annotation transparency, scale, accessibility, and support for multidimensional analysis, thereby hindering robust detection and comprehensive study of harasser behaviors and networks. To address these gaps, we introduce the Bengali Contextual and Diverse Cyberbullying (BenCoDiC) corpus, a publicly available contextual cyberbullying resource for Bengali political news discourse comprising 50,250 textual instances collected from 100 YouTube videos and their associated discussions. The corpus preserves caption‑‑comment‑‑reply relationships and is organized through a nine‑dimensional star schema covering users, videos, textual content, context, time, engagement, political parties, target identities, and language categories. Five AI chatbots were initially evaluated for annotation but did not meet the predefined reliability criterion. Consequently, a two‑tier human annotation process with expert verification was adopted, achieving Fleiss' κ=0.86 and Krippendorff's α=0.86. Statistical analyses characterize the corpus, while transformer‑based experiments show that incorporating conversational context yields modest but consistent improvements over isolated‑text classification. BenCoDiC supports context‑aware cyberbullying detection and perpetrator‑oriented behavioral and network analysis. The corpus is available at https://github.com/bencodic/BenCoDiC‑Corpus.
Authors:Fernando Montes-Gonzalez
Abstract:
We study when communication emerges between autonomous agents and what role an innate action selection architecture plays. Two e‑puck robots, reconstructed in PyBullet from their Webots model and validated against it, must find a feeding zone that can only be detected when a robot passes over it. Each robot can emit a one‑bit signal whose direction is detected by the receiver. Control is provided by a GRU recurrent network with a residual path, either operating alone or as a channel within an Embedded Action Selection Architecture (EASA). Fitness measures only the time the robots spend over the food. When the robots receive a compass pointing to the food, use of the signal disappears completely. Without a compass, in a small arena, three independent runs achieved similar performance using different strategies: a referential signal, a permanent beacon, or no use of the signal. As the arena expands, the strategy based on a referential signal generalizes best. In contrast, evolving directly in the large arena does not increase the frequency of communication, but it does separate the strategies, and the referential signal clearly outperforms solitary searching. Within EASA, innate reflexes constrain the network and reduce performance. As its priorities evolve, evolution consistently weakens those reflexes. The results suggest that communication emerges in proportion to the value of the information it transmits, and that an innate scaffolding designed for isolated agents can be an impediment in a social task.
Authors:Zipei Ma, Junzhe Jiang, Jiahui Zhang, Juntong Zhang, Chun Gu, Bozhou Zhang, Jiankang Deng, Xiatian Zhu, Li Zhang
Abstract:
Vision‑language‑action (VLA) models and world action models (WAMs) have advanced robotic policy learning by building on pretrained vision‑language models (VLMs) and video generative models (VGMs), respectively. Yet VLAs learn without explicit supervision from future world states, while WAMs that condition actions on predicted futures incur extra inference cost and let visual prediction errors propagate into actions. We present Forward Dynamics Models (FDM), a family of robotic action models that couples action and future world‑state prediction through an action‑to‑future paradigm. Future prediction is conditioned on action features, never the reverse: end‑to‑end training lets ground‑truth future states supervise the policy through these features, while prediction errors cannot propagate into actions. At inference, FDM skips future generation entirely, avoiding its computational cost. The paradigm is backbone‑agnostic, and we instantiate it with both VLM and VGM backbones. Experiments in simulation and on real robots demonstrate the effectiveness of FDM across both backbone families, achieving a state‑of‑the‑art 99.4% average success rate on LIBERO and absolute gains of up to 13% over VLA and WAM methods on real‑world tasks.
Authors:Yuzhen Wang, Chunxiao Li, Baichen Le, Lewen Yan, Jie Zhang, Yucheng T. Yang, Xing-Ming Zhao, Lijun Li, Jing Shao
Abstract:
As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce severe dual‑use risks, such as engineering enhanced pathogens or drug‑resistant bacteria. While biosecurity frameworks recognize these threats, the actual risk of LLMs executing dangerous biological modifications remains unquantified, particularly when augmented with specialized biological design tools (BDTs). Existing evaluations primarily assess hazardous knowledge possession rather than practical application, overlook risk amplification from BDTs, and rely on unreliable, expensive, or hazardous validation methods. To address these gaps, we introduce DRYLAB‑Bench (Design Risk Yielded by LLM Agent in Biodesign), the first comprehensive benchmark evaluating the high‑risk biological mutation design capabilities of LLM agents. Moving beyond basic question‑answering, DRYLAB‑Bench tasks agents with 30 genuinely hazardous mutation‑design scenarios distilled from biosecurity guidelines. It evaluates models in a multi‑round, tool‑augmented environment where LLMs orchestrate BDTs to iteratively refine candidates. Crucially, it scores designs directly against 187,695 empirical wet‑lab measurements from ProteinGym and ViroGym using our proposed Biosecurity Risk Score (BRS), quantifying risk‑associated effects while enabling scalable and safe ground‑truth verification. Evaluations of 15 frontier models reveal substantial baseline threats: without tools, 67.6% of one‑shot designs exceed the average measured variant, and 21.4% reach the highest‑risk decile. Equipping models with BDTs significantly amplifies this risk, raising the pooled risk by \(ΔBRS=+0.116\) and the top‑decile share to 23.5%. Code: https://github.com/drylab‑bio/DRYLAB‑Bench
Authors:Mohammad Khalooei, Mohammad Sabokrou
Abstract:
Large audio‑language models (LALMs) frequently exhibit object hallucination, affirming absent sounds. The dominant mitigation, Audio‑Aware Decoding (AAD), contrasts logits under real audio against an all‑zero "blank" waveform, flagged by prior review as out‑of‑distribution. We introduce a black‑box causal test that isolates the claimed evidence itself: localize a claim's evidence region via CLAP similarity, remove it by spectral gating, and compare against a random‑window control across a 2x2x2 grid on three LALMs. On Qwen2‑Audio‑7B, localized ablation's raw AUROC trails blank's (0.613 vs. 0.721); this is a duration confound, not a localization failure, since blank removes the entire clip versus the grid's 2‑4 s. Holding duration fixed, localization beats random removal at every cell on both models. Raw detection value remains model‑dependent, especially on Audio Flamingo 3. The same evidence corrects decoding: contrastive decoding combining blank and language‑prior contrasts, gated and ensembled, raises Qwen2‑Audio accuracy from 57.1% to 71.9%, beating AAD alone. A unified causal‑grounding profile does not transfer between models, indicating different grounding regimes. The source code and a detailed report of our findings, including identified limitations, are available at https://github.com/khalooei/causa.
Authors:Taiga Ikeda, Daisuke Miyashita, Jun Deguchi
Abstract:
KV‑cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears after different system prompts, conversation histories, or dynamic metadata. We present KV‑Repurpose, an open‑source, vLLM‑based system for developing and evaluating segment‑wise KV‑cache reuse beyond exact prefixes. The system combines cache‑aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring. Experiments on long‑context benchmarks show that naive segment reuse largely preserves output quality for single‑document question answering and summarization, while exposing clear limitations in cross‑document reasoning. Under a Zipfian RAG‑style workload in which 20% of requests share a fixed prefix, KV‑Repurpose achieves a substantially higher token‑level cache hit rate than prefix caching at the same 30 GB cache capacity, 63.9% versus 3.3%, resulting in lower TTFT. Even with 1 TB of cache capacity, prefix caching reaches only a 17.9% hit rate in this workload. Source code is available under the Apache‑2.0 license at https://github.com/taiga333/eval‑kv‑repurpose.
Authors:Chandravardhan Singh Raghaw, Shahid Shafi Dar, Arnav Nishith Sharma, Shrey Modi, Nagendra Kumar
Abstract:
Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time‑consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi‑view information. We present Clinical‑Knowledge Guided Disease‑aware Visual‑Textual Alignment for QA‑Style Radiology Report Generation (CliQ‑RRG), a unified two‑stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ‑RRG employs a Disease‑aware Visual‑Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior‑Guided Attention Module (PrAM) to capture multi‑view dependencies across current and prior scans. In Stage 2, GraphRAG‑guided retrieval augments intermediate textual representations with graph‑grounded clinical evidence, and a large language model restructures them into concise, interpretable question‑answer (QA) pairs with diagnostic summaries. Experiments on two public chest X‑ray benchmarks demonstrate that CliQ‑RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA‑style radiology reports. Code is available at: https://github.com/chandravardhan‑singh‑raghaw/CliQ‑RRG.
Authors:Ziang Ye, Wentao Shi, Lei Chen, Zhengzhou Cai, Yu Wang, Zhiyuan Yao, Zihan Wang, Xierui Song, Fuli Feng, Yixin Cao
Abstract:
Large language model agents increasingly operate in long‑horizon environments where early mistakes can trigger cascading failures and trap agents in dead‑end trajectories. Agents must therefore be able to proactively recover from such states and turn their exploratory experience into guidance for subsequent decisions. We introduce ReTRAIL, a framework that enables agents to rewind the environment to earlier states while preserving valid progress and carrying forward knowledge learned from abandoned exploration. The environment state moves backward while the agent's knowledge continues to move forward. ReTRAIL trains a shared policy with two complementary signals: State‑Anchored GRPO compares alternative continuations from revisited states to improve execution, while a branch‑advancement reward guides reflection through the progress of subsequent exploration. Experiments across four interactive benchmark MineSweeper, Sokoban, ALFWorld and ScienceWorld show that simply adding a rewind primitive provides limited benefit and can even degrade performance. In contrast, ReTRAIL enables agents to rewind from unproductive states and use insights from abandoned branches to guide subsequent exploration. These results demonstrate the value of coupling environment rewinding with experience reuse within an ongoing task for long‑horizon task solving. The code is available at https://github.com/whi497/ReTRAIL.
Authors:Shiao Wang, Xiao Wang, Yuhe Ding, Jiarui Xu, Zikang Yan, Bo Jiang, Yonghong Tian, Jin Tang
Abstract:
RGB‑X tracking leverages complementary modalities, such as thermal infrared, event, or depth, to improve robustness under challenging conditions. Existing general‑purpose RGB‑X trackers typically follow two paradigms: independent task‑specific training and joint training across modality combinations. However, the former suffers from knowledge isolation and model redundancy, while the latter introduces optimization interference across heterogeneous tasks. In this work, we present UniMergeTrack, a new model merging paradigm that preserves task‑specific optimization while enabling effective cross‑task knowledge integration. UniMergeTrack first trains modality‑specific trackers independently and then performs hierarchical model merging. Specifically, transferable backbones are merged across RGB‑X tasks via high‑pass‑filtered merging, and complemented by lightweight task‑specific experts that preserve modality‑specific cues. Tracking heads are merged within each task using optimal transport‑guided channel alignment to mitigate hidden channel mismatch. A subsequent distribution alignment fine‑tuning stage further improves compatibility between the unified backbone and task‑specific heads. Extensive experiments on the LasHeR, RGBT234, VisEvent, VOT‑RGBD22, COESOT, and CDTB benchmarks demonstrate that UniMergeTrack achieves state‑of‑the‑art or competitive performance across RGB‑Thermal, RGB‑Event, and RGB‑Depth tracking tasks, while exhibiting strong generalization to unseen datasets. The source code will be released upon acceptance. The source code of this paper will be released on https://github.com/Event‑AHU/OpenEvTracking.
Authors:James Rose, Jingwen Liang, Wenchao Ma, Oren Jacob, Haomiao Jiang
Abstract:
Recent image‑to‑3D models generate complete, high‑quality assets from a single image, with global shape and appearance that are already convincing; what remains is the fine scale, where small, thin, elongated, or detail‑rich regions are reconstructed softly and under‑resolved. This gap can arise in generators with fixed‑resolution spatial grids: the same spatial sampling density is used for a large, smooth, low‑curvature surface and for a thin, high‑curvature structure that carries most of the perceived detail. To this end, we introduce NOFACE3D, a training‑free framework that refines these detail‑critical regions while preserving the rest of the asset. Given a generated mesh, its input image, and a refinement box, NOFACE3D extracts the selected submesh, scales it up to fill the generator's input volume, and re‑encodes it into the full‑resolution latent grid. Conditioned on the image crop, it refines the region through flow‑based sampling with a spatially varying noise schedule that anchors boundary latents. The refined region is then integrated back into the original asset, preserving the surrounding geometry and appearance. We implement NOFACE3D on TRELLIS.2 and its sparse voxel representation; all of it operates purely at inference time on the frozen pretrained generator. We evaluate NOFACE3D on PAniC‑3D, TexVerse and BigDetail, a new benchmark of 339 assets with annotated detail‑critical regions. On all three, it generates sharper geometry and finer texture detail than the baselines. Project page: https://idkcouscous.github.io/NoFACE3D/
Authors:Yongxin Zheng, Hao Wu, Fang Li, Hu Fu, Peiran Yin, Xudong Wang, Haozhe Hu, Xinghao Chen, Yunpu Ma, Wei Zhang, Xiaoyu Shen
Abstract:
Multi‑view representations enable pretrained image and video vision‑language models to understand 3D scenes, but their long visual token sequences make inference costly. Redundancy arises both from repeated observations of the same scene regions across views and from processing all visual tokens at every LLM layer, regardless of their relevance to the current computation. We propose GeoGR, a two‑stage framework that jointly compresses scene representations and visual computation. At the projector level, GeoSemZip consolidates cross‑view observations into 3D voxel tokens and combines semantic saliency with spatial coverage to construct a compact scene representation. Unselected voxel features are aggregated into contextual anchors to retain scene information that would otherwise be discarded. At the LLM level, we identify an effective visual computation window and observe that query‑conditioned token relevance remains locally stable but changes over longer layer intervals. Guided by these findings, GroupRoute combines late entry and early exit with recoverable group‑wise skipping. Low‑relevance tokens bypass complete LLM layers while retaining their hidden states and can rejoin the active group at subsequent anchor layers selected under a computation budget. For compression‑aware post‑training, we construct a compact dataset of 67K samples by retaining 30% of the original training pool. With this protocol, GeoGR retains 95.2% and 96.8% of the dense post‑trained models' performance on LLaVA‑OneVision‑7B and Video‑3D LLM‑7B, respectively, while accelerating LLM prefill by 6.8× and 6.1×. These results demonstrate the consistent effectiveness of GeoGR across 3D VLMs with and without explicit 3D spatial encoding, highlighting a favorable performance‑efficiency trade‑off. The code is available at https://github.com/EIT‑NLP/GeoGR.
Authors:Minh Sao Khue Luu, Bair N. Tuchinov
Abstract:
Frozen features of brain MRI foundation models lose accuracy on new datasets, a failure usually blamed on the features encoding where each scan came from. We show that the failure lies elsewhere. Across 17 public datasets and eight frozen representations, every representation recognizes the MRI sequence within each dataset, yet most fail on an unseen one. The cause is neither dataset encoding nor a shift of each dataset: the content itself is encoded along different directions in each dataset. This explains which datasets fail, for the sequence and for sex, and why removing the dataset signal with INLP, LEACE, CORAL or adversarial training never improves sequence transfer. Only the broadly pretrained 3DINO and SAM‑Med3D encode the sequence consistently enough to transfer; the brain‑specific BrainIAC and BrainFM do no better than an untrained network. Code is available at https://github.com/BrainFM/brainfm‑profiler/tree/main/representation_utility.
Authors:Kai Liu, Jiangran Lyu, Tingyu Deng, Yixin Zheng, Jingtian Wan, Wenbo Cui, Mi Yan, Yizhou Zhou, Haoran Li, Dongbin Zhao, Zhizheng Zhang, He Wang
Abstract:
World Action Models (WAMs) inherit powerful priors from pretrained video generation models, but their variational autoencoder (VAE) representations are optimized for visual reconstruction rather than manipulation. External visual representations can improve manipulation, but existing methods couple representation choices with different integration mechanisms, obscuring their individual effects. We systematically compare visual representations, five integration mechanisms, and feature combinations under controlled settings. Our study yields three main findings. First, strong general‑purpose visual representations provide broader robustness gains and can match or outperform representations pretrained for specific visual capabilities. Second, channel‑wise joint prediction achieves the highest average success rate; further analyses suggest that future‑feature prediction and the action expert's effective use of added visual information contribute to its gains. Third, additional sources should be selected for the task‑relevant information they contribute beyond existing representations, as the same source can help one combination but harm another. However, even the best evaluated composition provides a smaller additional gain than the initial improvement from adding a strong general‑purpose representation to the VAE‑only baseline. We instantiate the resulting Representation Recipe as R^2‑WAM. It achieves state‑of‑the‑art performance on RoboCasa GR1 Tabletop with an 84.7% success rate and reaches 92.8% on RoboTwin 2.0, improving over the VAE‑only baselines by 5.9 and 2.2 percentage points, respectively. Taken together, these findings suggest that future work should prioritize stronger general‑purpose visual representations and more effective mechanisms for integrating them into WAMs, with feature composition serving as a supplementary route to improvement.
Authors:Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, Yiwu Zhong
Abstract:
Continual learning requires models to accumulate past experience and transfer it to new tasks. Despite substantial research on continual learning, existing work remains largely confined to non‑embodied settings, without extending to embodied environments. To address this gap, we introduce EmbodiedCLBench, the first benchmark in embodied simulation for evaluating the continual learning capability of self‑evolving agents. With compositional generalization as the core evaluation principle, our benchmark evaluates whether agents can first learn from basic tasks and then recombine the acquired skills to solve complex, unseen advanced tasks. Based on EmbodiedCLBench, we conduct extensive experiments across multiple agent harnesses and learning methods, revealing consistent limitations and distinctive behaviors. Our results show that while advanced tasks can be hardly solved under zero‑shot setting, the experience from basic tasks enables consistent improvement. Such benefit from the experience remains stable even if the task complexity increases. However, this improvement does not scale accordingly with the number of learning samples, and current agents cannot extract meaningful information from additional experience. Collectively, our findings highlight continual learning as a fundamental yet underexplored capability, and our benchmark offers valuable resources for designing effective self‑evolving agents in embodied environments. Our code and dataset are available at https://pku‑value‑lab.github.io/EmbodiedCLBench‑homepage.
Authors:Catherine Ji, Vivek Myers, Joshua Lin, Alexandra Wingate, Sergey Levine, Benjamin Eysenbach
Abstract:
Empowerment measures the capacity of an agent to actively control its environment. In this work, we explain the connection between skill learning and empowerment. While skill learning is not empowerment, we introduce a simple scalable recipe for empowerment maximization: empowerment corresponds to the number of executable skills from a state, so maximizing empowerment corresponds to reaching states that are good starting points for many skills. Our recipe applies to arbitrary underlying skill learning methods. Whereas most prior work on empowerment looks at tabular settings, this recipe enables us to study empowerment at a scale where we can realize theoretically‑predicted but empirically‑elusive properties of empowerment (e.g., gathering resources, environment modification, survival). Experiments show that empowerment maximization unlocks interesting emergent properties, such as building a house in Crafter. Website and code at \hrefhttps://empowerment‑geometry.github.io/scaling‑empowerment/this https url.
Authors:Yichen Wang, Tenghao Huang, Vishakh Padmakumar, Mina Lee
Abstract:
Creativity benchmarks for large language models (LLMs) vary widely in task and metric design. This raises the question of whether they capture a common notion of creativity. We propose that this shared target can be formalized as task‑specific performance in the long tail of a reference input‑output distribution, which describes how common input‑output pairs are relative to a chosen source of text. We ground this proposition in the novelty‑quality product formulation and evaluate it empirically across 15 benchmarks and 26 open‑weight models, using n‑gram absence from the pretraining data as a novelty proxy. Our results show that creativity benchmarks lie further in the tail regions than standard benchmarks. Moreover, on both creativity and standard benchmarks, model rankings from higher‑tail subsets of each benchmark align more closely with human creativity judgments in LM‑Arena. Building on these findings, we introduce q@Tail(α), a lower‑cost creativity diagnostic that measures mean task quality on the tail fraction of an existing benchmark. Applying it to creativity‑promoting methods reveals substantial room to improve LLM creativity through stronger tail performance.
Authors:Yuqi Huang, Hui Li, Yifei Cao, Zikun Cai, Tianci Xie, Yangfan Gao, Pinxue Guo, Chaoyun Zhang, Bingxuan Li, Rui Zheng, Hang Yan, Ning Liao
Abstract:
Computer‑use agents can perform increasingly complex GUI tasks, yet unfamiliar software and workflows requiring human domain knowledge remain challenging. Human demonstrations convey procedural knowledge through the same GUI interaction space, but reuse requires interpreting actions in context and separating workflow rules from task‑specific details. We thus introduce Work2Skill, a framework that captures human GUI experience for reusable agent skill induction. WorkRecorder provides the framework's infrastructure for collecting this experience, aligning screen recordings, mouse and keyboard events, and available accessibility‑tree information. Using WorkRecorder, we construct WorkDemo‑350, the first synchronized multimodal dataset of human desktop GUI demonstrations for reusable agent skill induction and downstream evaluation. Its 350 demonstrations are screened for completeness and task success. We then propose an agentic skill‑induction method that inspects evidence on demand to produce reusable skills capturing workflow rules, step dependencies, and completion checks. When the same four execution models are equipped with skills induced by each method under shared execution settings, Work2Skill achieves the highest scores and success rates. Mean score improvements over the corresponding No Skill baseline are 4.67‑‑10.54 points on MyPCBench Bounded Action and 3.11‑‑7.50 points on a 286‑task subset of OSWorld‑Verified. Our code and data are available at https://github.com/china‑qijizhifeng/Work2Skill.
Authors:Zihao Pan, Mengting Chen, Cheng Shi, Taihang Hu, Zhengrong Yue, Zikang Wang, Yuan Wang, Wei Li, Sibei Yang
Abstract:
Visual representations are largely developed for understanding, yet what makes them useful for generation remains unclear. We study this question in pixel diffusion transformers and uncover two key insights. First, better representation quality does not necessarily translate into greater generative utility, as externally aligned semantics can be progressively lost during pixel synthesis. Second, architectural separation with specialized pixel decoders mainly redistributes semantic and spatial modeling, without necessarily yielding representations better suited to generation. These insights suggest that a useful generative representation must be both properly formed and actively exploited. Motivated by these insights, we introduce Representation Guidance (RG), which directly forms and exploits internal representations while retaining a plain Transformer architecture. A low‑dimensional bottleneck forces generation‑relevant information into a compact representation, which is then explicitly used to guide decoder generation by modifying the flow‑matching target according to its induced prediction offset. RG learns clearer semantic and spatial structure while scaling efficiently in both parameters and computation. On ImageNet 256x256, RG‑L/16 with only 459M parameters outperforms substantially larger pixel diffusion transformers, while RG‑XL/16 achieves an FID of 1.55. RG further reaches an FID of 1.74 at 512x512 and extends to text‑to‑image generation, achieving 0.89 on GenEval and 87.1 on DPG‑Bench.
Authors:Yingjie Zhu, Han Cheng, Xuefeng Bai, Youcheng Pan, Yang Xiang, Kehai Chen
Abstract:
We introduce MVGraph, a benchmark for evaluating large vision‑language models (LVLMs) on multi‑graph reasoning where the evidence required by a query is distributed across multiple interrelated visual graph views. Although such evidence commonly resides in source‑specific graphs, temporal snapshots, or local observations, existing graph reasoning benchmarks remain confined to a single self‑contained structure, assessing only graph perception and reasoning while leaving the alignment and composition of evidence across views underexplored. MVGraph comprises 8k instances spanning 8 graph‑theoretic tasks, abstracted from two multi‑graph scenarios (partial and temporal graph views). Each problem is programmatically constructed and verified to be unsolvable from a single view. Beyond end‑to‑end evaluation, we decompose multi‑graph reasoning into four complementary capabilities‑‑‑Perception, Alignment, Composition, and Reasoning‑‑‑and design 29 diagnostic tasks with 64k instances to probe each one, along with golden intermediate representations for evaluating the intermediate process. Across 15 LVLMs, we reveal the difficulty of multi‑graph reasoning and identify cross‑view edge alignment and relation‑level composition as prominent bottlenecks through stage‑oriented diagnostics. Meanwhile, temporal constraints and increasing view counts expose further capability boundaries. Code and data are available at https://github.com/AAAndy‑Zhu/MVGraph.
Authors:Payam Abdisarabshali, Fardis Nadimi, Kasra Borazjani, Naji Khosravan, Richeng Jin, Seyyedali Hosseinalipour
Abstract:
Federated vision‑language models (FedVLMs) represent one of the emerging frontiers of machine learning, aiming to integrate federated learning within the fine‑tuning pipelines of vision‑language models (VLMs). In this work, we introduce BLEND, a new framework designed to jointly enhance the personalization and generalization capabilities of FedVLMs. BLEND proposes a selective VLM parameter fine‑tuning and aggregation strategy that consists of (i) global vision and text adapters shared across clients, and (ii) a personalized vision projection adapter tailored to each client. BLEND further introduces a novel loss function that blends the outputs of the global and personalized adapters to achieve personalization, while simultaneously promoting generalization through a principled anchor design implemented via the projection heads of the zero‑shot VLM. Extensive experiments demonstrate that BLEND consistently outperforms state‑of‑the‑art baselines, highlighting its ability to effectively balance personalization and generalization across diverse system settings. (Code is available at: https://github.com/payamsiabd/BLEND.git)
Authors:Hao Zhang, Zhiqian Zheng, Linfeng Tang, Xinyu Xiang, Jiayi Ma
Abstract:
Existing image fusion methods struggle to balance visual quality and semantic attributes due to the inherent conflict between their optimization objectives. To address this challenge, we propose an attack‑defense semantic reinforcement framework for multi‑modal image fusion, termed AD‑SRF. By decoupling the optimization of these two aspects and then losslessly reintegrating them, this framework injects rich semantic attributes into the fused image without compromising its visual quality. First, we develop a transformer‑based restorative visual fusion network as a strong visual baseline. Equipped with the cross‑modal retrieval module and the information restoration‑fusion collaborative loss, it produces highly visible fused images even under degraded conditions. Second, we introduce an attack‑defense game mechanism to strengthen the semantic attributes of the fused images. The attacker learns perturbation patterns that impair downstream segmentation, whereas the defender restores missing semantics from the original inputs and reintegrates them into the fused image through an imperceptible semantic map. Through iterative adversarial interaction, this mechanism drives the defender to continually strengthen the fused image's semantic attributes, boosting downstream segmentation performance without compromising visual quality. Extensive experiments demonstrate that AD‑SRF achieves a win‑win performance in both visual quality and semantic attributes. The code is publicly available at https://github.com/HaoZhang1018/AD‑SRF.
Authors:Hao Zhang, Xuhui Zuo, Jiayi Ma
Abstract:
Existing multi‑modal image fusion (MMIF) methods generally face an optimization gap between the visual and semantic domains, limiting the enhancement of perception capabilities. To overcome this long‑standing barrier, this work proposes W2F, a new visual‑semantic mutual reinforcing paradigm for MMIF. First, we reinforce the foundation for visual‑semantic collaboration by amplifying feature complementarity through two targeted designs. Specifically, the segmentation‑guided fusion loss introduces object‑aware structural guidance into visual fusion, while the embedded information restoration prior strengthens the ability of visual fusion to provide reliable perceptual evidence for the semantic branch under degraded conditions. Then, we introduce a strong multi‑modal information interaction mechanism composed of interactive gated mixed attention (IGM‑Att) and progressive coupling attention (PC‑Att). These modules ensure comprehensive scene representation from both local and global perspectives, providing informational support for visual‑semantic collaboration. Finally, we construct a collaborative mechanism to couple visual fusion and semantic fusion. It employs a feature search module to extract complementary key features from both tasks, enriching the representational potential of visual and semantic information. Meanwhile, a gradient correction strategy ensures collaboration from the optimization perspective, enhancing performance for both visual and semantic fusion through consistency learning. Extensive experiments demonstrate that W2F surpasses state‑of‑the‑art methods in both visual appeal and decision accuracy. The code is publicly available at https://github.com/HaoZhang1018/W2F.
Authors:Jiayu Liu, Ruizhi Zhao, Tianyun Ji, Yee Whye Teh, Zhenya Huang, Ning Miao
Abstract:
Diffusion language models generate tokens in parallel, but still require multiple sampling steps to produce high‑quality text. In this paper, we investigate whether a model can learn to generate an entire sequence in a single forward pass. We propose Seq‑Drifting, a framework that applies drifting to sequences of continuous token representations and directly learns a map from Gaussian noise to text. Unlike existing DLMs that derive the supervision at each position directly from the observed token in a training sequence, Seq‑Drifting builds a drift field whose position‑wise targets adapt to the current generated context. At each position, attraction guides each output toward a set of contextually plausible token representations, while repulsion discourages collapse across samples and repetition within sequences. The generator learns from this drift field during training and produces all tokens jointly in one pass at inference. Empirically, Seq‑Drifting achieves a generative perplexity of 36.01 on LM1B in one step and performs competitively in both unconditional and conditional generation. Experiments on translation, summarization, and reasoning further demonstrate the applicability of the framework beyond open‑ended generation. These results suggest that drifting over token sequences offers a promising path toward effective one‑step text generation. Our code is available at https://github.com/jyliu‑ai/Seq‑Drifting.
Authors:Weiwei Chen, Chenghao Xu, Meiling Yu, Xu Yang, Cheng Deng
Abstract:
In the practical deployment of vision‑‑language‑‑action policies, handling long‑tailed task distributions is a common challenge due to the highly imbalanced composition of robot demonstration datasets. To investigate this issue, we study the head‑task action bias in shared robot policies, particularly focusing on its relationship with task‑specific visual evidence. Our experimental analysis shows that, when the visual evidence required by a tail task is insufficient, the policy increasingly falls back on action patterns reinforced by data‑rich head tasks, resulting in degraded performance on under‑represented tasks. Motivated by this observation, we propose Evidence‑Guided Counterfactual Learning, a training framework for long‑tailed robot policies. Our method identifies decision‑relevant visual evidence using instruction‑related and action‑related signals, and constructs counterfactual observations by removing the selected evidence. The resulting counterfactual predictions are used to strengthen the association among task instructions, visual evidence, and actions, with the supervision adaptively adjusted according to the number of demonstrations available for each task. The proposed framework requires neither additional training data nor region‑level annotations and incurs no additional computation during deployment. Extensive experiments across simulated benchmarks, different VLA architectures, and real‑world robot platforms demonstrate that our method robustly and consistently improves long‑tailed robot policy learning, particularly for tail tasks, while maintaining reliable performance on head tasks.
Authors:Jiaqi Tian, Ting Long
Abstract:
On‑Policy Self‑distillation is a self‑improvement paradigm that enables large language models (LLMs) to improve their capabilities by leveraging the same model as both teacher and student under different conditioning contexts. However, we find that OPSD methods can progressively degrade previously acquired capabilities: queries that are correctly responded at earlier training stages may become incorrectly responded as optimization proceeds, even the overall performance continues to improve. Further analysis reveals that this degradation coincides with a deterioration of the geometry of hidden representations, characterized by reduced intra‑cluster compactness and diminished inter‑cluster separation. Motivated by this observation, we propose Representation Geometry Regularization (RGR), which preserves representation geometry during post‑training by encouraging compactness within cluster and separation across cluster. RGR can be seamlessly integrated into existing OPSD methods without altering their core optimization procedures. Extensive experiments on multiple OPSD methods and benchmarks show that RGR generally alleviates capability degradation while retaining capability improvements. Our code is available in https://github.com/supermachinelearner/RGR.
Authors:David Dinkevich, Nisan Chiprut, Yoav HaCohen, Dani Lischinski
Abstract:
Fewer latent tokens promise cheaper diffusion. Yet aggressive latent compression creates a reconstruction‑generation dilemma: adding channels improves reconstruction but makes generation harder. We introduce APEX, short for Asynchronous Prefix denoising for EXtreme compression, which pairs a hierarchical latent space with asynchronous denoising over channel groups. Our autoencoder orders channel prefixes from structure to detail, while one shared denoiser generates them at staggered local times without adding tokens. On ImageNet‑512, APEX outperforms existing methods at every evaluated channel count under 64x and 128x compression. At matched training and guided sampling compute, it also outperforms a 256‑token model at every evaluated budget, with larger gains at lower compute. Code is available at https://daviddinkevich.github.io/APEX.
Authors:Chun Liu, Yuxin Wang, Zhenxiang Huo, Linlin Zhu, Tao Xu, Henggang Han
Abstract:
UAV thermal‑to‑satellite geo‑localization localizes UAVs by matching real‑time thermal images against georeferenced satellite images, enabling autonomous navigation in GNSS‑denied and low‑visibility conditions. However, substantial cross‑modal disparities in color, brightness and viewpoint hinder conventional appearance‑based methods from establishing reliable correspondences. While existing approaches predominantly address global modality shifts, they overlook the invariant scene structures inherently shared between thermal and satellite views. To bridge this gap, we propose SAMF, a Structure‑Aware Matching Framework that explicitly exploits stable cross‑modal structural cues to improve UAV thermal geolocalization. Specifically, the Directional Structure Enhancement Module (DSEM) strengthens robust structural representations by capturing directional and low‑frequency spatial patterns, and the Modality‑Aware Affine Remapping Module (MARM) subsequently applies affine correction to these enhanced features, further promoting cross‑modal alignment. Additionally, we introduce the Thermal‑UAV‑Paired benchmark derived from the Thermal‑UAV dataset. Extensive experiments on both the Boson‑nighttime and Thermal‑UAV‑Paired datasets show that SAMF achieves state‑of‑the‑art performance. Ablation studies further validate the effectiveness and synergistic complementarity of DSEM and MARM. The code and dataset are released at: https://github.com/YuxinWang879/SAMF.git.
Authors:Jingkun Yi, Shujie Zhang, Zirui Zhou, Haoxiao Wang, Haiyang Sun, Yuhang Dong, Xudong Xu, Xing Shen, Tao Zhang, Yong Liu, Jiangning Zhang
Abstract:
Pretrained vision‑language models (VLMs) already exhibit strong visual, spatial and reasoning capabilities relevant to robotic manipulation. The key challenge in turning pretrained VLMs into generalist manipulation policies is to expose these capabilities through an appropriate action interface. Vision‑language‑action (VLA) models provide a unified trainable policy, but typically introduce learned action tokens, action heads, or action experts whose semantics must be learned from robot data pretraining. In contrast, modular foundation‑model‑based manipulation methods can exploit pretrained VLM capabilities for zero‑shot manipulation, but rely on external planning and control scaffolds that are not end‑to‑end trainable as a unified continuous‑action policy. In this work, we introduce EASG, an Explicit Action‑Space Grounding framework that bridges these two paradigms. By projecting a tool center point (TCP)‑centered action frame into the wrist camera observation and expressing actions in the form of VLM's native textual generation interface, EASG makes the semantics of a continuous robot action space directly observable in the VLM's perceptual context. This allows pretrained VLMs to directly generate executable robot actions using their existing visual‑spatial reasoning capabilities, without robot‑action pretraining or additional action modules. Besides, the same action interface supports direct fine‑tuning with robot trajectories, allowing the models to continuously improve beyond their manipulation capabilities unleashed by EASG. Overall, EASG enables zero‑shot manipulation and generalization across object positions, categories, and robot embodiments, while supporting effective fine‑tuning in the same continuous action space. These results demonstrate that, given an appropriate action interface, pretrained VLMs can themselves serve as native action models for robotic manipulation.
Authors:Joshua Maccoby, Wendy Hui Wang
Abstract:
Unlearning in large reasoning models (LRMs) requires removing forget‑set information from both the reasoning trace and the final answer. Existing approaches achieve this objective at the cost of reasoning collapse, where the model fails to produce a coherent chain‑of‑thought (CoT) and does not properly terminate its reasoning process. To address this challenge, we propose REDaCT (REgenerate via Detection and Corrective Tuning), an inference‑time unlearning framework for LRMs. REDaCT consists of two key components: (1) a binary leak detector that identifies sentences in the reasoning trace that leak forget‑set information, and (2) an embedding optimization module that learns a corrective embedding offline, such that when injected at the first detected leak during inference, it guides subsequent reasoning towards sentences that do not leak forget‑set information. Experimental results show that REDaCT completely avoids reasoning collapse, substantially improving reasoning preservation compared to state‑of‑the‑art unlearning methods while maintaining perfect answer forgetting and near‑perfect CoT forgetting. Our code is available at: https://github.com/jmaccoby/redact.
Authors:Oktie Hassanzadeh, Dharmashankar Subramanian
Abstract:
Semantic query engines evaluate natural‑language filters with a large language model, and every evaluation is a costly oracle call. Optimizers that reduce these calls face a dilemma: cascades that trust a cheap proxy miss their precision and recall targets on many queries, methods with finite‑sample guarantees save little, and all of them discard the oracle's labels when the query ends. We present SemViews, a certified semantic filter that orders tuples by signals that cost no oracle calls (the labels that earlier queries left behind as semantic views, a local embedding, and a helper model's score), refines that order with labels it must pay for anyway, and certifies its answer with fresh stratified oracle samples. The answer meets user precision and recall targets with probability at least 1‑δ for every workload and every catalog, so a stale or misleading view can cost oracle calls but never accuracy. On four workloads replayed against 725,000 recorded oracle labels, SemViews meets both targets on 99.63% of queries and saves 28%‑45% of the oracle calls, whereas the only prior method in our study with the same guarantee needs 1.32x‑1.71x as many calls. When a question returns in different words, views cut its calls by a further 27%‑36%, with both targets met on at least 99.8% of such queries; similarity caches and entailment‑based reuse meet them on only 17%‑47%.
Authors:Davide Wiest, Ahanaf Hasan Ariq, Abdul Monaf Chowdhury
Abstract:
Gradient‑based attribution provides input‑specific information about model parameters, but computing it via backpropagation for many samples is expensive, while static importance scores discard input dependence. We derive KPSA (Kernelized Parameter Sensitivity Atlas) from a block decomposition of the empirical neural tangent kernel. It measures normalized parameter‑group sensitivity, embeds those sources in an explicit query space, and uses a lightweight kernel to estimate sensitivities for unseen inputs. We derive bounds that separate finite‑source from irreducible representation error and give margin‑based top‑k recovery. This paper studies two interpretability applications. Firstly, KPSA identifies parameter groups with high susceptibility to perturbations across vision, forecasting, and protein‑modeling setups. Across a fivefold range of parameter budgets (0.02‑0.1%), its selected groups attain 39‑85% of the perturbation susceptibility of a first‑order backpropagation oracle. In vision and forecasting, these parameter groups form sparse sensitivity circuits that extend across depth and vary systematically with the input. Secondly, we use KPSA on an activation‑attribution problem for MLP features in transformers. It retains 81% of the gap between a backpropagation oracle and a constant‑sensitivity baseline on vision while increasing batched query‑time throughput from 551 to 2,041 images/s and reducing peak GPU memory from 4.55 to 0.44GB. On the protein‑modeling task, it recovers 68% of this gap. An ablation of KPSA's kernel aggregation shows that nearest‑neighbor retrieval in KPSA space can remain competitive. The empirics show that compact kernel atlases can store useful input‑dependent parameter sensitivity while amortizing repeated gradient attribution, a step toward scalable conditional interpretability. Code can be found at https://github.com/Axym‑Labs/kpsa .
Authors:Revanth Krishna Senthilkumaran, Ruwan Wickramarachchi, Alessandro Oltramari, Jonathan Francis
Abstract:
Long‑horizon manipulation requires robots to retain task‑relevant history while revising conclusions no longer supported by physical evidence. Retaining observations alone does not specify which completed conditions should be withdrawn after a disturbance or which obligations require repair. We introduce REMM, Revisable Embodied Memory for Manipulation, a supervisory memory layer for a frozen G0.5 vision‑language‑action executor. REMM separates an append‑only evidence history from revisable task state and explicitly links derived relations and completed conditions to their supporting facts. When accepted evidence changes a supporting fact, it withdraws dependent conclusions and schedules targeted recovery at verified skill boundaries without updating the executor. Across 51 physical trials, including operator‑assisted runs, REMM improves mean recorded task progress over the strongest baseline from 57% to 91% in intervention recovery and from 43% to 70% in sequence recall. Complementary video diagnostics show improved grounding of retained history into the next instruction, including comparisons with π_0.5‑compatible high‑level planner adaptations. These results support revisable memory as a practical interface for continued manipulation without policy retraining. Project: https://revanthsenthil.github.io/remm/.
Authors:Mustafa S. Aljumaily, Ali Ataeemh Allami, Mustafa N. Mnati, Nawar S. Alseelawi, Shahab Abdulla
Abstract:
Federated learning allows dispersed oilfield sites to train a shared intrusion detector without pooling network traffic, yet its aggregation step forces a choice between privacy and robustness. Byzantine‑robust rules must read each update in the clear, whereas secure aggregation hides updates from the checks that detect poisoning. We present PVBRF‑ID, a two‑server protocol in which sites secret‑share their updates and the servers jointly test each update's norm and its cosine similarity to a reference computed on a small clean root set, revealing one accept bit per site. Commitments, Merkle roots and a homomorphic audit make the inclusion of accepted updates publicly verifiable. On a 15‑class Edge‑IIoTset task with ten sites holding differently distributed data and ten random seeds, PVBRF‑ID matched FedAvg on clean data (macro‑F1 0.708 vs. 0.706) and withstood sign‑flip attacks (0.696 vs. 0.091). Against an adaptive attacker controlling 30% of sites, PVBRF‑ID with secure clipping reached 0.641, whereas FedAvg fell to 0.046; plaintext multi‑Krum remained higher by 0.052. An update‑inference attacker recovered each site's class mix from plaintext updates (Spearman correlation 0.81) but not from one server's view. Backdoors passed the filter, and noise‑based mitigation gave only heuristic protection. The implementation uses Python 3.13, NumPy 2.3, 30 rounds and 10 seeds.
Authors:Zhengyan Huan, Peter Y. Lu, Shuchin Aeron
Abstract:
We consider the problem of learning to generate data while satisfying a prescribed \emphsample‑wise constraint with a user‑specified probability of constraint satisfaction. Starting with a population‑level constrained negative log‑likelihood minimization and using a flow‑matching (FM) parameterization, under regularity assumptions and sufficient expressivity of the velocity field family, we derive guarantees on the probability of constraint satisfaction for the corresponding primal‑dual optimization algorithm. We provide a practical implementation of this approach, called Primal‑Dual Flow Matching (PDFM), which has direct control over constraint satisfaction rate and requires only binary membership oracle feedback for constraint satisfaction, avoiding the need for differentiable distances, projections, or special structure of the constraint set, such as convexity. We show that PDFM improves constraint satisfaction over vanilla FM and achieves competitive performance against constraint‑aware baselines while maintaining distributional fidelity. Our code is available at https://github.com/ZhengyanHuan/PDFM.
Authors:Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros Potamias
Abstract:
Vision‑language‑action models (VLAs) offer a promising route toward general‑purpose robotics by transferring semantic knowledge from large‑scale vision‑language pretraining into language‑conditioned control. However, most VLA evaluations emphasize aggregate manipulation success under changes in objects, instructions, or environments, leaving unclear how reliably these policies ground spatial descriptions in the physical scenes where they act. We therefore ask a sharper question: how reliably can current VLAs translate spatial descriptions into appropriate real‑robot behavior? To investigate this, we introduce Spa‑Bench, a real‑robot benchmark that evaluates spatial grounding across six tabletop tasks covering Object State, Relative Size, Referential Description, Relational Placement, Ordinal Position, and Counting. Our primary evaluation compares in‑distribution (ID) and out‑of‑distribution (OOD) spatial configurations while retaining training support for the relevant objects, concepts, and actions, reducing object and motor unfamiliarity as alternative explanations for failure. Four diagnostic conditions further probe physical execution, sensitivity to wording, object‑task associations, and recognition that no action is required. We release 4,092 episodes collected on the SO‑101 robot arm: 1,200 human‑teleoperated demonstrations and 2,892 real‑robot evaluation rollouts from five state‑of‑the‑art VLA policies. Across the three fully evaluated policies, success falls from 67.5‑73.3% on ID spatial configurations to 40.7‑52.0% on OOD spatial configurations, while matched direct instructions improve success by 28.3‑40.0 percentage points. These results show that the evaluated VLAs can often execute the required manipulations but lack robust spatial grounding. Project materials, including videos, code, and benchmark data, are available at https://justintiensmith.github.io/Spa‑Bench/.
Authors:Khoi-Nguyen Tran
Abstract:
Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning‑preserving variants routinely move model answers. Studying this systematically requires applying perturbations one at a time, reproducibly, and identically wherever they run, yet existing tools couple them to a benchmarking stack rather than packaging them as a reusable transform layer. We present perturber, a stateless service and Python library that applies exactly one semantics‑preserving perturbation per call, drawn from a first‑party catalog of 68 perturbations across 11 families. It turns on a single primitive, with composition left to the caller chaining calls, so the service and the library produce identical output by construction. We evaluate the tool itself, its correctness, reproducibility, and cost, rather than any model's robustness, measuring meaning‑preservation per perturbation rather than assuming it. The demonstration adds a live playground, an introspectable catalog for benchmark selection, and a one‑command reproducible evaluation.
Authors:Zekai Gu, Peng Wang, Jiahao Lu, Yongsen Mao, Cheng Lin, Yuan Liu
Abstract:
Autoregressive video diffusion models have become a leading approach to real‑time interactive video generation and video world modeling. However, existing methods typically allocate computation uniformly across space when generating the next video chunk, updating all patches with the same denoising schedule and number of function evaluations. This ignores the spatial heterogeneity of video content: static, simple, or recurring regions are often easier to denoise, while other regions require further refinement or additional context. A uniform budget thus leads to redundant computation on easy patches, whereas reducing the fixed step count can leave difficult patches insufficiently refined. We introduce \methodname, a general framework that dynamically allocates denoising steps to individual patches: easy patches stop early to improve efficiency, while difficult patches receive more refinement steps. This design accelerates generation with negligible quality degradation. Our method is compatible with most existing few‑step autoregressive diffusion models, requiring only a lightweight routing module with modest training. Experiments demonstrate that \methodname reduces the total number of denoising steps applied across video patches by about 50% while maintaining visual quality comparable to the four‑step baseline.
Authors:Jiuming Liu, Mengmeng Liu, Guangming Wang, Lihao Liu, Michael Ying Yang, Yunpeng Zhang, Francesco Nex, Hao Cheng, Hesheng Wang
Abstract:
Prior outdoor LiDAR registration works are commonly limited by the pair‑wise input paradigm, which neglects temporal correlations inherent within streaming sequences. In this paper, we propose a novel multi‑frame outdoor point cloud registration network for streaming LiDAR scans enhanced with pose‑related memory buffers. The key observation is that long‑term temporal LiDAR sequences can provide rich global contextual information to complete sparse measurements, filter outliers, and address low‑overlap problems, thereby boosting registration performance. Specifically, two hybrid memory buffers are designed, including an implicit memory feature buffer and an explicit memory pose buffer, to store and dynamically update pose‑related temporal features. Moreover, a novel dynamic history weighting module is developed to adaptively fuse current and history pose‑related features. Extensive experiments on three outdoor datasets, including KITTI, nuScenes, and Apollo‑Southbay, demonstrate state‑of‑the‑art performance of MemReg, surpassing all previous pair‑wise methods and also multi‑frame SLAM systems. Our method also generalizes surprisingly well to multiview indoor registration scenarios with rather competitive performance on 3DMatch, 3DLoMatch, and ScanNet. Code will be released at https://github.com/IRMVLab/MemReg.
Authors:Dmitry Osin, Egor Surkov, Petr Mokrov, Igor Udovichenko, Ivan Karpukhin, Evgeny Burnaev, Egor Shvetsov, Alexander Korotin
Abstract:
Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecasting occurs in finance, healthcare, business process intelligence, and other fields. In this work, we study multi‑horizon forecasting of event sequences as a conditional generative modeling problem. We evaluate forecasts not only for prediction accuracy, but also for distributional realism. To this end, we present a benchmark comprising metrics for both quality aspects, strong statistical baselines, recent models adapted from adjacent fields, and curated real‑world datasets. Careful evaluation reveals that no method excels at both accuracy and realism, highlighting a fundamental trade‑off between the two. Finally, we introduce Wasserstein BaryBooster, an algorithm that aggregates samples from a realism‑strong diffusion model into a state‑of‑the‑art predictor, demonstrating that distributional realism can be converted into prediction accuracy. Code is available at https://github.com/xaosina/event‑sequence‑forecasting.
Authors:Pratik Maitra, Lauren Kwon, Qi Li
Abstract:
Event ontology is the foundational framework for defining events in a dataset. Event type induction generally involves domain experts who define guidelines for creating event types and their names based on the dataset's domain and scope. However, this process is resource‑intensive in terms of annotation costs and is prone to human biases. To alleviate the need for manual event ontology creation, we propose the CLUE framework for event type induction, naming, and annotation. Our main contributions in this work are: 1) designing a novel joint embedding representation for expressing both the verb‑centric view and the semantic view of events; 2) event type induction using contrastive loss guided clustering; 3) event naming and refinement. Experiments demonstrate the effectiveness of our method. Our code is available at https://github.com/PratikMaitra/CLUE.
Authors:Sofiène Boutaj, Pierre Marza, Varun Belagali, Dimitris Samaras, Maria Vakalopoulou, Stergios Christodoulidis
Abstract:
Multi‑teacher distillation allows transferring knowledge from multiple "teacher" networks to a single "student" network. This method is promising in fields such as digital pathology, where many powerful foundation models were proposed recently. However, as we show in this paper, the standard multi‑teacher distillation approach reacts poorly to an increase in the number of teachers used to train a student encoder. We demonstrate that learning teacher‑specific representations is a key to scaling in MTD. Importantly, different design choices, such as learnable teacher tokens, a tailored attention scheme, additional mixture‑of‑experts layers and a contrastive loss, are proposed to better learn such teacher‑specific representations. We show that our method better scales with respect to the number of considered teachers, allowing us to train compact student encoders, up to 10x more efficient than larger teacher foundation models, while matching their performance and even outperforming them on a large set of tile‑level and slide‑level tasks. We conduct a thorough empirical validation, evaluating more than 10 foundation models on 39 tasks spanning tile and slide levels. As a result, we release a new collection of strong and efficient foundation models, named OMNI, trained from the knowledge of 10 state‑of‑the‑art foundation models. We finally conduct an analysis of the learned teacher‑specific representations, highlighting their complementarity and explaining why they can be easily aggregated at downstream time.
Authors:Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang, Pengfei Zhou, Wangbo Zhao, Fangyuan Kong, Pengfei Wan, Kun Gai, Yang You
Abstract:
Online reinforcement learning (RL) fine‑tunes diffusion models with black‑box rewards, but reverse‑process methods such as Flow‑GRPO are limited in efficiency by poor credit assignment, while forward‑process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential‑moving‑average (EMA) off‑policy training. We show that NFT‑style objectives contain a hidden forward‑‑reverse consistency force that joint positive‑‑negative training cancels only partially, and that EMA stabilizes training through restoration rather than off‑policy rollouts. We introduce DiffusionART (Diffusion Reinforcement with Aggregation‑First Reward Tilting), which aggregates each prompt's rewards into a shared reward‑improving direction for reverse‑process states, removing the consistency force by construction, and adds a trust‑region‑masked second update for drift‑adaptive restoration without EMA tuning. It requires no differentiable reward model, remains effective with fewer samples per prompt, and supports pairwise reward models (DiffusionART‑Pairwise). Across image and video generation, DiffusionART trains efficiently and stably without per‑task tuning. In particular, on HunyuanVideo‑13B with equally weighted visual and motion quality rewards, DiffusionART reaches the late‑stage mean reward levels of DiffusionNFT and Flow‑GRPO‑Fast with 12.02 and 11.72 times fewer GPU hours, respectively, while achieving 28.8% and 53.0% higher mean training rewards.
Authors:Xinzhe Wang, Changjian Jiang, Kaiwen Song, Xudong Li, Kerui Ren, Ran Yi, Lizhuang Ma, Chunhua Shen, Linning Xu, Tao Lu, Mulin Yu
Abstract:
Feed‑forward 3D priors enable high‑quality reconstruction from image streams but often lack reliable metric scale and suffer from long‑term trajectory drift. In contrast, LiDAR‑‑inertial estimation provides metrically accurate motion but lacks the image‑space consistency needed for high‑fidelity reconstruction. We propose MIRAGE, which anchors feed‑forward visual geometry to LiDAR‑‑IMU metric motion and regulates online pose corrections according to correspondence support and cross‑modal agreement, jointly enabling accurate metric pose estimation and high‑fidelity Gaussian reconstruction. Experiments on Oxford Spires, M2DGR and INS show that MIRAGE achieves accurate metric trajectory estimation, outperforming COLMAP, a widely used but computationally expensive reference for vision‑only benchmarks. Motivated by the scarcity of long‑horizon monocular benchmarks for large‑scale indoor reconstruction, we further build a multimodal data‑collection platform and use MIRAGE to establish accurate 6‑DoF reference trajectories for a new vision‑only benchmark. We benchmark recent monocular large‑scale reconstruction methods on this dataset, providing a systematic evaluation of their performance in long‑horizon indoor scenes.
Authors:Shu Ishida, Aliasghar Khani, Tianyu Zhang, The Cong Luong, James Seale Smith, Adam Gaier
Abstract:
Autoregressive decoding is less suited to non‑causally ordered data, e.g., cross‑references or future dependencies, since their outputs cannot be revised post hoc, unlike iterative human reasoning. Diffusion LLMs partially mitigate this limitation but still cannot flexibly insert or remove arbitrary numbers of tokens from a pre‑generated context. We design AutoIndexer as a generative model with a novel training and decoding mechanism that simulates a Chain of Edits, allowing the model to flexibly modify its prior outputs in hindsight via token insertions, substitutions, and deletions. The model learns to predict marker tokens for opening and closing edits, simultaneously moving a cursor to positions in the existing token sequence, analogous to a text editor. Training sequences are obtained by perturbing ground‑truth labels with inverse chained edit operations, and the model learns to edit by undoing those inverses. The perturbation training allows AutoIndexer to be robust to erroneous context and adaptable to editing tasks such as code correction. We demonstrate that our method exhibits post‑generation context‑editing capabilities that most autoregressive and diffusion models lack without compromising performance on standard LLM benchmarks. The code is available at https://github.com/AutodeskAILab/autoindexer.
Authors:Yuhang Xie, Zhiyi Shi, Zeyi Zhang, Zhisheng Qi, Yushun Dong, Yu Wang, Xiongye Xiao
Abstract:
Large language models (LLMs) are increasingly used in molecular discovery and structure elucidation, where decisions depend on physical observations. Existing evaluations primarily focus on end‑to‑end molecular reconstruction, where correct predictions do not necessarily indicate reliance on the supplied properties. We introduce MolP2SBench, a benchmark for molecular property‑to‑structure (P2S) reasoning in LLMs. Each task presents a masked molecular scaffold with given properties and asks the model to select the missing structure from plausible candidates. We shuffle or replace properties to test reliance on correct property‑structure pairing and responsiveness to counterfactual property changes. A channel‑wise leaderboard uses Property‑Grounded Recovery (PGR), giving equal weight to source and counterfactual recovery. Evaluating seven models, we find that LLMs can use physical properties for structure prediction, but performance varies substantially across property types. Higher accuracy with a property does not necessarily imply greater responsiveness to it: a model may benefit more from ^1H NMR than from ^13C NMR, yet be less likely to change its prediction when the ^1H data is replaced. LLM performance also need not reflect the relative difficulty of these properties for human chemists. Although UV‑Vis spectra provide less direct structural information than NMR spectra, Astra achieves higher accuracy with UV‑Vis data than with NMR peaks. MolP2SBench supports studying how LLMs extract and use physical‑property information, revealing patterns that may differ from conventional chemical intuition and providing a foundation for more reliably property‑grounded models for molecular and quantum‑chemical inference. Code and dataset: https://github.com/DUAL‑Xiao/MolP2SBench.
Authors:Fabien Furfaro
Abstract:
Small language models trained on a single local machine leave little room for architectural trial and error. Kairos combines bidirectional gated‑DeltaNet linear attention and sliding‑window attention over shared projections (LiZAttention2), block‑wise Attention‑Residual aggregation, a sparse Mixture‑of‑Experts (MoE) feed‑forward layer and a multi‑scale byte‑level codec, trained with a uniform‑noise denoising objective over one byte space shared by text, image, audio, video, lidar and control streams; this report asks whether the combination optimizes at all before asking whether it is useful. We give code‑derived algorithms for each component, including a cross‑session recurrent‑state memory gate, then run convergence sanity checks: overfittinga single 440‑byte sentence at 1‑‑15M parameters, text only, with three seeds for codec and attention choices and single‑run sweeps for seven further factors. The two attention variants tested at 3\,000 steps (LiZAttention2 and window attention alone) reach a minimum loss near 10^‑3. After 600 steps, the mean minimum loss ranges from 0.018 (patch codec) to 0.354 (depthwise convolutional codec) with an identical ranking on three seeds. An apparent advantage of shared over separate attention projections closes with more steps or more diverse data and reverses when the projections are frozen: it is a convergence‑speed effect, not a capacity effect. These checks suggest that the implementation optimizes; they say nothing about generalization, multimodal training or comparison with existing models, which a planned pretraining run is meant to address. The Discussion sizes a 200M‑total/25M‑active target analytically; none of its throughput figures is measured. Code is available at https://github.com/fabienfrfr/Kairos . Python package at https://pypi.org/project/kairos‑fm/ .
Authors:Chinmay Deo, Danish Mahajan, Soudeep Ghoshal, Himanshu Buckchash
Abstract:
Audio‑visual zooming on mobile devices is fundamentally constrained by limited microphone count, compute resource, unknown array geometry, and weak assumptions on real‑world acoustic conditions. We present RDSE: Real‑Time Directional Speech Enhancement Framework, a hybrid pipeline that combines traditional signal processing and modern machine learning methods for dual‑microphone low‑end smartphones. Unlike tightly coupled AV systems that continuously use visual feeds or embeddings, RDSE decouples visual interaction from feature inference, using camera input solely to parameterize the steering vector for spatial filtering. The proposed system uses four modules: dereverberation network, a beamformer with a temporal convolutional network, and a MetricGAN based denoiser. Extensive evaluation on benchmark data and real‑world recordings demonstrates effective zooming without precise geometry calibration. Comparative experiments show competitive performance to existing methods. Proposed system is rigorously tested for statistical validity, ablation study, latency profiling, and practicality through real‑time live‑streaming.
Authors:Nhan Tran, Neal Wadhwa, Abe Davis, Stefan Stojanov
Abstract:
On a film set, the camera move is committed during a take. Generative video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept hallucination. We demonstrate the workflow through a mobile augmented reality application for on‑set planning, capture, and review, and an offline pipeline for existing video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination.
Authors:Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
Abstract:
Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white‑box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes surpass black‑box monitors on agentic coding benchmarks, achieving 98.3% AUROC in SHADE‑Arena and 98.0% in BashArena (vs. 97.7%, 97.6% resp. for action‑only text monitors), and show improved efficacy as the underlying model is scaled up. We further test probes on several cases where deception cannot be determined from the context alone; in these cases of introspective deception, the ground truth can only be determined through careful elicitation or knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUROC of up to 99.7%. Our probes also detect deception on prominent open‑weight models which lie about politically sensitive topics when put under pressure. We release our training dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.
Authors:Hongxing Li, Jinyue Su, Dingming Li, Wenqi Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Abstract:
Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input. Yet real‑world spatial intelligence demands predictive spatial reasoning: constructing a scene from observations, anticipating how an intervention changes it, and reasoning about the unseen outcome. We introduce SpaceCast‑Bench, the first benchmark to directly and diagnostically evaluate this capability. Built around an observe‑transform‑infer framework, its 3,862 questions from 182 real‑world scenes span 16 task types at three levels: static perception, local prediction, and global prediction, progressively requiring scene understanding, spatial state updating, and relational inference over unobserved outcomes. Evaluating 21 models exposes a stark gap: the strongest model reaches only 58.0% against 87.2% human performance, while spatially specialized models remain near random chance. Controlled analyses further reveal that bridge views are critical for integrating distributed observations, and that explicit 3D evidence benefits models more reliably than generated outcome images or videos. Fine‑tuning on our programmatically generated data lifts Qwen3‑VL‑4B from 34.0% to 65.7% with macro‑average gains across six out‑of‑domain benchmarks.
Authors:Qun Dai, Liangjian Wen, Jiang Duan, Yong Dai, Dongkai Wang, Maolin Wang, Mingjie Wang, Jianzhuang Liu, He Yan, Zhao Kang
Abstract:
Self‑supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross‑modal interactions. Unlike the shared information across individual modalities, synergy arises when task‑relevant signals emerge only from the joint configuration of multiple modalities and cannot be recovered from any modality in isolation. This work focuses on how to preserve the information capacity for such synergistic signals in multimodal representations. The key observation is that synergistic information is reflected in higher‑order statistical dependence among modalities, which provides a principled target for explicitly modeling joint interactions. Motivated by this insight, we propose Higher‑order Representation and Information Learning (HRIL), which constructs an empirical cross‑moment tensor over modality embeddings to represent multi‑way interactions. HRIL employs Tucker decomposition to obtain a core tensor, complemented by a synergy‑aware regularizer that prevents energy concentration and preserves higher‑order coupling capacity for synergistic information capture. Experiments on the controlled synergy task and real‑world benchmarks demonstrate consistent improvements over existing multimodal contrastive methods, with notable gains on tasks dominated by synergistic interactions. Code is released at https://github.com/brightest66/HRIL.
Authors:Jiawei Chi, Shangchen Miao, Zhiyuan Shi, Kailu Wu, Hanyang Wang, Weiliang Chen, Qiyu Dai, Chengyang Li, Jinshan Ren, Jun Gao, Mingsheng Long, Yueqi Duan, Jiangran Lyu, Jialong Wu, Fangfu Liu
Abstract:
Interactive virtual worlds allow agents to learn through exploration and interaction. What agents can learn is bounded by the environments they practice in, which must be faithful, with consistent state, rules, and dynamics, and realistic, with observations that follow the real‑world visual distributions. Achieving both across diverse worlds remains a bottleneck. We introduce AgentGarten, a framework that couples simulators and game engines with a shared neural renderer to build real‑time interactive environments. Its simulation backends maintain persistent world state and execute program‑defined interaction rules, while the renderer generates visual observations from structured conditions exported through a common interface. To build the neural renderer, we adapt a pretrained video model to geometry conditions, distill it with our proposed Adversarial Forcing, and optimize inference for real‑time interaction. Adversarial Forcing makes history prefilling differentiable through exact replay, so that losses on later predictions update how the renderer encodes prior observations, and adds real‑data adversarial supervision to improve its visual quality. In AgentGarten, agents perceive the world through visual observations, interact with it in real time, and improve by distilling each round of experience into playbooks that subsequent agents inherit and refine. Our empirical study demonstrates a substantial gain in learning efficiency, with agents learning from just 4 rounds compared with millions for a conventional reinforcement learning counterpart. As new worlds can be written as code and rendered through the same interface, environments can scale in both number and difficulty alongside their agents, a step toward agents that keep evolving through interactive experience.
Authors:Yonghan Jung, Shu Yang
Abstract:
Randomized controlled trials (RCTs) identify treatment effects without confounding but are often small, whereas observational studies (OBS) are large but may be confounded. Many estimators combining a small RCT with a large OBS have been developed for the average treatment effect (ATE) and the conditional ATE (CATE). However, existing ATE estimators either make assumptions on the OBS or do not borrow enough power from them. The CATE has been studied less than the ATE. Existing CATE methods either assume the OBS are unconfounded, rely on a model of the confounding function, or accept bias in exchange for lower variance. We therefore propose a framework that, without special assumptions on the OBS, fuses the OBS and the RCT by preserving the unbiasedness of RCT‑based estimation while borrowing power from the large OBS to boost precision. Applying this principle, we build an ATE estimator, AIPW‑Fusion, with closed‑form weights and confidence intervals, and two CATE learners, DR‑Fusion and R‑Fusion. Experiments corroborate our findings.
Authors:Tianyu Fu, Tengxuan Liu, Ruoxi Wang, Yixin Dong, Yi Ge, Yichen You, Yu Wang
Abstract:
Large language model (LLM) routing distributes inference work across different models, advancing the cost‑quality Pareto frontier of LLM serving. While coarse‑grained routing at the session or query level has been widely adopted in production systems, recent algorithmic work shows that fine‑grained token‑level routing can yield substantial efficiency and quality gains. However, efficiently serving token‑level routed inference poses significant challenges to existing systems. Built on single‑LLM assumptions, current systems suffer from severe step desynchronization and frequent batch admission delays under token‑level routing, and they also impose high implementation complexity on developers. To address these challenges, we design TokenRouter, an efficient and developer‑friendly serving system for token‑level routed LLM inference. TokenRouter follows the principle of request‑centric programming, model‑centric execution: developers describe routing logic from the perspective of a single request, while the runtime launches a subserver for each LLM and dispatches requests asynchronously. Each subserver employs a delayed‑batching scheduler, whose optimal hyperparameters are derived from a mathematical throughput model of the system. Across diverse routing algorithms, workloads, and model pairs, TokenRouter achieves 2.01‑64.15x higher decoding throughput than existing systems, substantially advancing the serving efficiency of token‑level LLM routing. Our code is available at https://github.com/thu‑nics/TokenRouter.
Authors:Mohamed Readh Fentazi, Mazene Ameur, Adlen Ksentini
Abstract:
Test‑time adaptation (TTA) methods for time‑series forecasting update a deployed model, or a small adapter around it, from incoming ground truth. But the label of an H‑step forecast exists only H steps later, and real data pipelines add further delay. We build a leakage‑free harness in which the label of forecast origin s is released for updates only at step s+d with d \ge H, and enforce this rule inside the released code of four recent TTA methods (TAFAS, COSA, PETSA and DynaTTA), run on their own backbones and checkpoints across five benchmarks (ETTm1, ETTh2, Weather, Electricity and Traffic). As references we add two closed‑form correctors: a bank of recursive least squares (RLS) filters combined by a per‑coordinate median, with no tunable hyperparameters and 56 microseconds per step on the 7‑channel streams, and an ELF‑style linear corrector. Under causal delayed labels the picture is asymmetric. On ETTm1 every audited method genuinely adapts, yet the RLS bank still beats three of the four at a fraction of their cost; only DynaTTA beats the bank, only at the minimum causal delay, and at roughly 2,500 times the per‑update cost; the ELF‑style corrector beats all four. On the other four datasets, the largest statistically significant improvement any published method achieves over its own frozen checkpoint is half a percent, on all four at least one published method is significantly worse than the frozen model at the minimum causal delay, and on drift‑heavy ETTh2 longer label delays make every adapter that separates from the frozen model, ours included, significantly harmful. Leaky next‑step updates inflate the apparent gains of simple adapters by up to 110%, and the backbone training recipe moves frozen online error by up to a factor of 25, more than any adaptation effect we measure. We release the harness, integration patches and all cached runs.
Authors:Yitong Wang, Fangyun Wei, Jinjing Zhao, Sirui Zhang, Hongyang Zhang, Dong Chen, Bo Dai, Yan Lu
Abstract:
Text prompting is an indirect interface for poster generation, requiring users to encode inherently two‑dimensional composition intent into a one‑dimensional sequence of words. We introduce a Spatial Canvas Interface that enables users to directly compose generation intent in space through four complementary binding types: semantic, identity, text, and pixel, together with Text Specifications for individual elements and global appearance. Based on this interface, we develop Compo, a poster generation model adapted from a pretrained image editing model to understand Spatial Canvas inputs and Text Specifications. Compo supports both direct inference, where users explicitly construct the canvas, and agentic mode, where a high‑level request is automatically translated into a planned Spatial Canvas. To train Compo, we develop a scalable pipeline that automatically constructs supervision data for different binding types and their combinations, enabling efficient adaptation without training a specialized poster generator from scratch. We further introduce a benchmark that evaluates adherence to individual binding types and their joint composition. Experiments show that Compo achieves stronger compositional controllability than both general‑purpose image generation models and dedicated poster generation systems while maintaining high visual quality. By decoupling intent specification from visual generation, our work shifts poster generation from prompting toward composing.
Authors:Jinjing Zhao, Fangyun Wei, Yitong Wang, Xiuyu Wu, Yunuo Chen, Yang Yue, Sirui Zhang, Wenbo Wang, Hongyang Zhang, Dong Chen, Yan Lu, Chang Xu
Abstract:
In text‑guided image editing, describing the desired change is often straightforward, but identifying the intended object or region can be cumbersome, especially when several objects look alike. We introduce a new image editing interface that lets users place spatial marks and optional short notes directly on the image. Together, these annotations form a canvas instruction that specifies where to edit and what to change. Our editor, VibeEdit, follows these instructions to perform object addition, removal, replacement, attribute modification, and movement without a separate text prompt. We construct 1.55 million source‑target edit pairs with object masks and structured edit descriptions, from which we render canvas instructions during training. We adapt Qwen‑Image‑Edit with layer‑decoupled conditioning that separately encodes source images and canvas instructions for image editing. We train the model with region‑weighted supervised fine‑tuning, followed by rubric‑guided reinforcement learning to improve edit completion, local edit quality, and preservation of unedited regions. We evaluate VibeEdit on an independently constructed, human‑curated benchmark of 419 cases emphasizing target selection among similar objects. VibeEdit achieves a VLM rubric score of 79.9 and an outside‑region PSNR of 32.8 dB, compared with 67.4 and 24.0 dB for FireRed, the highest‑scoring text‑instructed baseline in our evaluation.
Authors:Cody Sheltraw, Tsimafei Lazouski, Maani Ghaffari, Alan Papalia
Abstract:
Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim‑to‑real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open‑water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real‑world deployments on a BlueBoat ASV demonstrate successful zero‑shot sim‑to‑real transfer in path following and station‑keeping tasks without prior hydrodynamic and propeller information.
Authors:Jie Chen, Ruofei Bai, Yuxin Cai, Yifeng Zhang, Chengyang He, Jun Li, Wei-Yun Yau, Guillaume Sartoretti
Abstract:
World modeling has emerged as an effective co‑training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states. However, most WAMs predict future states in the native representation space of pretrained visual backbones, resulting in high‑dimensional targets with substantial training cost. We introduce MiniWAM, which instead predicts compact future representations learned from privileged current‑future transitions. To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse‑dynamics supervision with feature reconstruction to emphasize control‑relevant transition information while preserving useful future‑state information. With the learned PRISM encoder frozen, MiniWAM is trained to jointly predict the resulting targets and robot actions from current observations. With 65× fewer native future feature tokens, MiniWAM consistently outperforms native future‑feature prediction with both DINOv3 and WAN2.1 VAE features, while achieving up to an 8× speedup in world‑action training. At 0.25B parameters, MiniWAM is already competitive with substantially larger WAMs on LIBERO, LIBERO‑Plus, and RoboTwin 2.0 simulation benchmarks. Representation analyses further show that PRISM contributes behavioral structure beyond feature reconstruction alone. These results demonstrate that effective world‑action modeling does not require predicting native visual futures, and that compact predictive representations provide a strong and substantially more efficient target for policy learning. The project page is available at: https://j1dan.github.io/MiniWAM.
Authors:Yudi Zhang, Mingyu Cao, Lu Yin, Mykola Pechenizkiy, Shiwei Liu
Abstract:
As claims about recursive self‑improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models have a sense of data, i.e., can they reliably select the right data for training? We introduce DataSense‑Bench to study this capability through the fundamental problem of data selection and performance forecasting in machine learning. We ask AI agents to select and rank candidate training subsets that can be used to fine‑tune a small LLM model. Agents are allowed to inspect the data, write and execute analysis code, and run model forward passes, but can not train the model or access the actual evaluation tasks. We then fine‑tune the base model on each selected subset and evaluate its post‑training performance under a standardized protocol. We instantiate the benchmark in terminal problem solving and tool use, selecting trajectories from OpenThoughts‑Agent and EnvScaler and evaluating on TBLite and BFCL, respectively. We then evaluate the agents along two complementary dimensions: the post‑training performance of the top‑ranked subset, reflecting the ability to identify high‑value training data, and ranking accuracy, reflecting the ability to predict the relative performance of the selected subsets. In our experiments, selection gains over random selection are limited; agents do not reliably rank their selected groups, and ranking ability does not hold consistently across tasks: Astra identifies the best group in all three tool‑use runs but in only one of three terminal runs. Analysis of execution traces on both tasks shows that agents often use similar data signals while interpreting their training value differently.
Authors:Jannik Wiese, Johannes Schusterbauer, Tommaso Martorella, Björn Ommer
Abstract:
Generative diffusion models are well‑suited for probabilistic precipitation nowcasting, but existing approaches often rely on separately trained compression or deterministic forecasting components and remain costly at inference due to iterative denoising. We introduce Just Weather Scoring (JWS), a single‑stage, end‑to‑end diffusion model which addresses both issues by forecasting directly in radar space and enabling few‑step generation. Radar‑space modeling greatly simplifies training and inference and eliminates uncertainty arising from lossy compression. JWS combines Masked Asynchronous Diffusion, a timestep‑sampling scheme that preserves clean context while adapting diffusion training to high‑dimensional spatio‑temporal data, with a simple scoring‑rule objective that aligns training with probabilistic forecasting and unlocks few‑step generation. On the SEVIR and MeteoNet benchmarks, JWS achieves state‑of‑the‑art probabilistic forecasting performance at reduced training and inference cost. Even our smallest model remains competitive using substantially fewer parameters and more than 17x faster inference.
Authors:Yuxin Chen, Senqiao Yang, Zixuan Wang, Jinhui Ye, Changsheng Lu, Pengguang Chen, Shu Liu, Zhuotao Tian, Jiaya Jia
Abstract:
Reliable robotic manipulation requires timely intervention to correct emerging deviations and restore progress after execution errors. However, recovery methods based on repeated vision‑language reasoning or iterative online optimization can incur substantial latency, delaying intervention. To address these challenges, we introduce RESETTLE(Robotic rEcovery through diSagrEement‑Triggered reTrievaL and Efficient Corrective Control), a model‑agnostic framework that provides computationally efficient recovery at the action‑execution interface of frozen robot policies. RESETTLE triggers recovery when two action proposals independently sampled under identical conditioning persistently disagree. It retrieves a same‑task demonstration reference using an adapted V‑JEPA encoder and combines a state‑servo prior with a guarded visual residual to execute one corrective action without online trajectory optimization or additional vision‑language reasoning, then returns control to the base policy. Across six base policies in simulation, RESETTLE achieves up to 8.70%, 6.28%, and 6.83% absolute success‑rate gains on LIBERO‑Plus, Meta‑World, and RoboCasa Tabletop, respectively, with further improvements on four real‑world tasks using two policies. In QwenPI‑based comparisons, its monitoring‑and‑recovery computation latency is 74.04%‑‑93.57% lower than VoLoAgent's monitoring‑and‑planning latency for grasp and place tool calls. It also raises Harness VLA's LIBERO‑Pro Swap success from 42% to 50%, demonstrating compatibility with high‑level agentic planning. Code available at: https://github.com/JIA‑Lab‑research/RESETTLE
Authors:Ming Chen, Rong-Xi Tan, Ke Xue, Yu-Jie Zhou, Taiye Lu, Zhi-Xuan Gao, Peng Xie, Zijun Shen, Chen Lu, Haopu Shang, Chao Qian
Abstract:
Black‑box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited. Recent large language model (LLM) agents offer a new way to approach BBO by combining task semantics, computation, optimization tools, and feedback‑driven decision making, showing great potential due to the integration with mathematically rigorous tools. However, existing agentic BBO studies use different task domains and system configurations, making their results difficult to compare and the effects of individual design choices hard to isolate. We therefore introduce AgenticBBO‑Bench, a cross‑domain benchmark for agentic BBO spanning synthetic functions, hyperparameter optimization, database tuning, chip design, and molecular design under a unified finite‑budget evaluation protocol. In our experiments, agentic BBO achieves higher family‑averaged scores than direct LLM‑based methods in all five domains and outperforms the best numerical optimizers in four. We further study three factors shaping agent performance: optimization tools, task information and prior knowledge, and the role of the LLM during search. Our results show that additional numerical tools do not consistently improve performance, task semantics are broadly useful while more specific priors are less reliable, and numerical optimizers can effectively absorb gains from search trajectories established by the agent. Finally, we introduce a five‑task frontier challenge within AgenticBBO‑Bench and evaluate seven LLMs under the Codex agent harness, where GPT‑6 Astra and DeepSeek‑V4.1‑Flash lie on the Pareto frontier of performance and cost among the evaluated models. Our code is available at https://github.com/lamda‑bbo/agentic‑bbo.
Authors:Xinliang Xiao, Bowen Yang, Wenjing Zhang, Li Yang, Wei Zhou
Abstract:
A robot that assists people must often act on what a person has shown rather than said: which of several identical cartons was pointed at, or which box was handled. The plan is executed from the final scene, whereas the evidence occurs earlier, possibly on objects that have since moved. We propose instance‑anchored interaction evidence (IAE), which registers every object of the final scene to its public identifier, keeps each identity through the video by backward mask propagation, and describes every frame by the geometry between hands, forearms and these instances. An evidence network trained only from task outcomes scores the instances. For pointing tasks, a grammar‑constrained dynamic program trained with a structured loss decodes object‑destination programs; symbolic programs handle reference disambiguation and, without learning, episodic tasks. On 1,255 WatchAct benchmark requests, scored by symbolic execution, IAE reaches 64.2% plan success on implicit‑intent tasks against 27.5% for a 32B vision‑language model (strict success 49.7% against 15.4%), and 46.4% against 27.0% on restoration, reversal and imitation without task‑specific training. Controls with the same perception overlays, the same 32 frames, forward tracking, or a relation model trained on the same labels do not explain the gain. Given IAE's evidence as text with its meaning explained, the same language model reaches 57.4%: most of the gain comes from the instance‑anchored evidence, and the explicit programs add 6.8 points at a fraction of the cost. Pointing remains the hardest case, with 16.9% strict success. The code is available at https://github.com/WeiZhou96/iae‑watchact.
Authors:Dongbin Zhang, Chaoda Zheng, Kangjie Chen, Xiangyu Li, Shijia Chen, Jinhao Deng, Yuqi Zhang, Guangfeng Jiang, Hongbin Lin, Choo Sin Wai, Minqi Wang, Puyi Wang, Jingye Zhang, Yu Zhang, Xianming Liu, Boyang Wang
Abstract:
To stream long videos while maintaining visual quality and temporal consistency, Self Forcing mitigates exposure bias through self‑rollout training on self‑generated histories with key‑value (KV) caching. To keep memory manageable, it detaches historical caches, preserving forward dependencies between chunks but severing the backward gradient paths. We introduce Connected Self Forcing, a training framework that reconnects gradient paths across autoregressive chunks, allowing feedback from later predictions to guide how earlier context is generated. These connections go beyond historical KV‑writing: gradients pass through generated latents into the computations that produced them, linking the generation of earlier context to its use in later predictions. To make this connected training memory‑efficient, we develop shortcut gradient replay, which recovers cross‑chunk gradients without retaining the full rollout computation graph. Integrated with distribution matching distillation, Connected Self Forcing trains historical chunks according to both their direct supervision and their contribution to subsequent generation. Experiments on autoregressive video generation show improvements in long‑horizon visual quality and temporal consistency, without changing the inference procedure.
Authors:Inês Cruchinho Garcia, Mariana Mourão, Francisco Maria Calisto, Carlos Santiago, Jacinto Nascimento
Abstract:
False negatives remain a critical limitation of computer‑aided diagnosis (CAD) systems for breast cancer screening due to delayed detection and treatment. To address this issue, we propose a counterfactual data augmentation strategy that generates healthy mammograms by "erasing" lesions from anomalous images, thereby enriching the training distribution. We train a Denoising Diffusion Probabilistic Model on BI‑RADS 1 (healthy) mammograms and use a RePaint‑based sampling strategy to inpaint realistic normal tissue within annotated lesion bounding boxes. The resulting healthy counterfactuals replace annotated lesion regions with realistic healthy tissue while preserving patient‑specific anatomical structure, as supported by similarity metrics between real and generated images. Image realism was further assessed by radiologists and found to be consistent with the original dataset quality. We evaluate counterfactual augmentation across four representative classifier architectures: a convolutional neural network (ConvNeXt), a vision transformer (ViT), a vision‑language model pre‑trained on mammogram‑report pairs (Mammo‑CLIP) and a multi‑scale attention‑based multiple‑instance learning framework (FPN‑MIL). Experiments conducted on the VinDr‑Mammo dataset show improvements in sensitivity across all architectures, particularly at 80% fixed specificity, contributing towards more reliable CAD systems for breast cancer. Code is available at: https://github.com/ines03garcia/diffusion‑based‑counterfactual‑generation.
Authors:Robert Graham, Yariv Barsheshat, Phil Blandfort, Sabri Alouache
Abstract:
A large body of research measures model coherence based on output variance without adequately considering competing causes. We identify two such causes, ambiguity and indifference, and we introduce a set of 175 questions where contradicting answers cannot easily be explained by either. We then measure incoherence in terms of contradictions when resampling answers to the same question. In contrast to other methods our metric has high specificity, and only ranks models as incoherent when the issues are glaring. Even so, we find narrow finetunes score poorly. Inspecting inconsistencies flagged by our method, we find that model organisms from the literature display severe issues such as identity conflation, introspection failures and rationalizations. These findings suggest that the pathologies induced by narrow finetuning may limit what these models can tell us about coherent misaligned behaviour.
Authors:Jinkai Zhang, Jingyi Xu, Yuanhong Yu, Jiarui Guo, Ruizhen Hu, Hujun Bao, Xiaowei Zhou, Sida Peng
Abstract:
General‑purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine‑tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general‑purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation‑specific fine‑tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent‑oriented Tools for physical interaction, and task‑progress and context management. A unified visual‑point interface connects decision‑making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance‑level, multi‑object, and demand‑driven tasks. Category‑level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/
Authors:Zhanyi Lu, Huan Wang
Abstract:
Knowledge distillation (KD) transfers knowledge from stronger Teacher models to weaker Student models, but most methods require training the Student parameters, thereby binding the distilled knowledge to a specific architecture and checkpoint. This implicit representation is difficult to interpret or reuse across models and limits KD for API‑only or costly‑to‑train models. This paper studies knowledge transfer for large language models (LLMs). We introduce Universal Textual Teaching (UTT), a parameter‑update‑free framework that distills observed Teacher‑Student knowledge gaps into a textual, interpretable, and reusable natural‑language artifact called Primer. Specifically, UTT first identifies representative gap cases through paired evaluations, and iteratively updates the Primer via multi‑role interactions: the Student attempts each task, the Prompter turns evaluation feedback into a teaching instruction, the Teacher provides a targeted demonstration, and the Synthesizer consolidates validated lessons. Empirically, on the challenging math (Omni‑MATH‑2) and code generation (KernelBench) tasks, extensive results confirm the effectiveness of the method: UTT remarkably raises the Student's accuracy from 9.4% to 48.6% and Fast1 accuracy from 9% to 35% on KernelBench, while increasing mathematical reasoning accuracy from 27.6% to 51.7%. UTT also performs better than representative prompt engineering and parameter‑based KD methods. Of note, UTT is shown to be generalizable across different Teachers and Students: a Primer synthesized for one Teacher‑Student pair can generalize to other Students that do not participate in the synthesis.
Authors:Ruixiang Wang, Yongyi Su, Wenlve Zhou, Bo Yue, Hengyan Liu, Dekun Lu, Yuxin Tian, Yihan Fang, Zerui Wu, Xing Hu, Jietao Chen, Yong Guo, Ziyan He, Junbin Yuan, Guiliang Liu, Xiaofen Xing, Kui Jia
Abstract:
World action models (WAMs) aim to answer a coupled physical question: given a task instruction, what motion should the robot execute, and how will that motion change the surrounding world? Most existing WAMs build on pretrained video generators and represent world evolution through images or visual latents. Robotic interaction, however, takes place in metric three‑dimensional space, while images are view‑dependent projections whose pixel distances do not directly encode physical distances. We introduce UNITAS, to our knowledge the first 3D‑native world action model that unifies observations, actions, and scene dynamics in a shared metric 3D frame within each interaction, using a common representation across robot embodiments and human hands. Action flow represents human hands and robot grippers as 3D point trajectories, while scene flow describes scene‑point displacements conditioned on these trajectories. World‑aligned 3D positional embeddings ground visual tokens with or without depth input, and a physical‑time trajectory tokenizer encodes each point trajectory as one token anchored at its current 3D position. This interface supports both direct action execution and action‑conditioned scene prediction. With 1.7B parameters, UNITAS achieves the best action‑conditioned scene prediction on RoboTwin among the compared methods, with up to 49% lower displacement errors than PointWorld, and state‑of‑the‑art manipulation success, including 99.8% on LIBERO and an average of 85% across real‑world tasks. The code is available at https://github.com/DexForce/UNITAS.
Authors:Wenhao Li, Xianjing Meng, Qiangchang Wang, Zhongyi Han, Yilong Yin, Liqiang Nie
Abstract:
Few‑shot learning aims to recognize novel categories from limited labeled examples. Recent studies incorporate textual semantics to compensate for limited visual observations and improve class representations. However, high image‑text agreement may reflect both intrinsic object properties and incidental context, making support prototypes susceptible to contextual contamination. To address this problem, we propose DVLA‑RL++, which extends DVLA‑RL with complementary semantic purification (CSP) and counterfactual reinforcement‑learning gating (CRG). Specifically, CSP generates intrinsic and nuisance descriptions from labeled supports and compares their agreement with each support token. An ambiguity‑dependent rejection margin guides sparse evidence allocation, while an intrinsic semantic anchor fills the unassigned mass to provide a fallback when visual evidence is unreliable. CRG learns layer‑wise semantic fusion strengths using a reward that balances recognition performance and nuisance exposure. An independently executed reference trajectory on the same episode provides a paired learning signal. Theoretical analysis relates retained evidence and anchor quality to prototype stability and establishes conditions for unbiased on‑policy gradient estimation. Experiments on standard, fine‑grained, and cross‑domain benchmarks show state‑of‑the‑art accuracy, with an average gain of 1.4% over DVLA‑RL. The project page is available at https://peacelwh.github.io/TPAMI27‑DVLA‑RLpp/.
Authors:Šimon Soldát, Václav Šmídl
Abstract:
Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation‑based inference to evaluate the posterior distribution of the input parameters based on real‑world observations of the simulator outputs. However, inference becomes challenging when individual simulator evaluations are computationally expensive. In such cases, a Bayesian optimization‑based active learning approach with Gaussian process surrogate models has been used to maximize the information obtained from a limited simulation budget. Recently, gradients of simulator outputs with respect to input parameters have become increasingly available, yet they are rarely exploited for inference. Even though we only need to learn the simulator input‑output relationship, gradient information can provide an additional valuable signal to guide the active learning procedure. This is of particular interest in the case of expensive simulators, when sample efficiency is crucial. In this paper, we demonstrate how incorporating gradient information into the Gaussian process surrogate accelerates Bayesian optimization‑based inference under a limited simulation budget. Our results show significant improvement in convergence speed from using gradient information. For reverse‑mode differentiation, the inference efficiency gains are maintained when accounting for the additional computational cost. In contrast, for forward‑mode differentiation, the inference speed‑up does not outweigh the computational costs. These results indicate that gradient‑enhanced surrogates are beneficial primarily in problems where the number of parameters exceeds the output dimensionality, where reverse‑mode differentiation is efficient.
Authors:Chenghao Jia, Mengdi Liu, Hong Chang, Shiguang Shan, Xilin Chen
Abstract:
Elucidating molecular structures from spectra is a foundational problem in chemical and materials characterization, yet remains challenging due to spectral ambiguity and the vast molecular space. Although recent diffusion‑based generators show strong promise for spectra‑conditioned elucidation, existing methods struggle to learn robust spectra‑structure relationships from limited paired data when relying solely on global spectral representation. Moreover, the repeated full sampling inference strategy incurs substantial computation overhead. To address these limitations, we propose MAST, a Motif‑Augmented diffusion framework with Search Tree, for joint 2D‑3D spectroscopic molecular structure elucidation. MAST introduces explicit, interpretable \emphmotif priors as intermediate evidences throughout denoising, reducing conditional ambiguity and facilitating spectra‑conditioned optimization. We further cast diffusion sampling as \emphreward‑guided tree search to prioritize high‑reward denoising trajectories, yielding a compact set of spectra‑consistent candidates under limited budgets. On the QM9S multi‑spectra benchmark, MAST achieves 94.89% exact recovery and improves 3D fidelity, while preserving high chemical validity and stability. Code is available at https://github.com/Jia040223/MAST.
Authors:Zhaoxin Yu, Qingchao Kong, Dajun Zeng, Wenji Mao
Abstract:
Large language models (LLMs) are increasingly deployed in sociotechnical systems where social attribution, the reasoning process attributing external events to the causes and reasons of agents' social behaviors, plays a critical role. These processes involve judgments of social cause, responsibility, and blame/credit to agents. Although attributional models are well‑studied in social psychology and cognition through Attribution Theory, social attribution remains underexplored in AI, particularly LLM social reasoning. This paper provides the first systematic exploration of LLM social attribution. Our work focuses on responsibility and blame attributions, examining current LLMs' judgments and their underlying internal mechanisms. Guided by attribution theory, we construct a social attribution benchmark consisting of a Vignette subset based on classic scenarios from attribution theory research and a Reality subset based on real‑world social narratives, yielding 7,639 responsibility/blame judgment questions. On this basis, we evaluate 32 representative LLMs and 5 basic non‑LLM baselines. To further explore the internal mechanisms underlying the LLM judgment process, we develop a probing‑based methodology to investigate the latent‑space representations of 5 key attribution dimensions and the consistency of their influences on LLM judgments compared to those in human social attribution. Our research findings reveal that current LLMs exhibit measurable but incomplete agreement with human responsibility and blame judgments, and meanwhile, this agreement is positively correlated with model size. Some attribution dimensions are systematically decodable from specific positions in LLM hidden states, and their influences on the final judgment are consistent with those indicated by human Attribution Theory. The dataset and associated code are available at https://github.com/Yuzhaoxin946/SAB‑Bench.
Authors:Johanna Moser, Christopher Albert, Sascha Ranftl
Abstract:
Physics‑informed machine learning has gained significant attention in recent years. In regimes of limited data, parametrized Gaussian processes have become popular. Existing approaches, however, often face limitations, such as requiring parametrizable (also called controllable) systems or a large number of output tasks. In this work, we introduce a systematic procedure we call "ghost tasking", using auxiliary tasks to circumvent these limitations. We prove that such ghost tasks can render any non‑parametrizable system effectively parametrizable, enabling algorithmic construction of parametrized Gaussian Processes while keeping the number of required tasks (i.e. output dimensions) and latent functions low. We find that ghost tasking performs especially well in an inverse problem setting, even with very few available data. We show the usage and power of ghost tasking in three experiments, providing systematic comparisons to the only other currently available method applicable to all experiments. We provide necessary syntax and explications for two computer algebra programs that compute parametrizations for systems with polynomial or rational coefficients. Our theoretical results extend to systems with meromorphic functions.
Authors:Kanghui Ning, Marin Biloš, James T. Wilson, Yilang Zhang, Kashif Rasul, Dongjin Song, Anderson Schneider, Yuriy Nevmyvaka
Abstract:
Which forms of test‑time compute improve the predictions of strong pretrained tabular foundation models (TFMs)? We systematically study this along three axes: adaptation, aggregation, and context construction. Our evaluation spans modern TFMs across the TabArena benchmark, supplemented by experiments on wide and large‑scale tables from OpenML. For adaptation, we introduce DiagScale, a diagonal query‑key similarity update. It trains only 0.003‑0.03% of model parameters and achieves gains comparable to full fine‑tuning across three independently pretrained backbones. For aggregation, both pool composition and selection strategy matter. TabPFN‑3 already averages predictions from different preprocessing variants of the same data, and adding more such predictions yields diminishing returns. With a broader pool of 96 configurations, greedy selection reduces error by 2.4% relative to the default predictor, but uniform averaging increases error. For context construction, attention‑guided retrieval improves TabPFN‑3's predictions on some large tables and supports source pools beyond the full context memory limit. The context expansion methods we test yield no consistent improvement. Taken together, our results suggest that adaptation and selective aggregation yield consistent benchmark‑level gains. The benefits of context construction depend more on the task and data regime. Adaptation and aggregation over the same backbone yield further gains when combined, but require substantially more computation than default inference. These trade‑offs motivate choosing strategies according to the available computation budget. Code is available at https://github.com/kanghui‑learning/test‑time‑compute‑for‑tabular‑foundation‑models.
Authors:Yupeng Xie, Zhenyang Wang, Jiayi Zhu, Yinghao Tang, Zhouan Shen, Yiyu Chen, Yuyu Luo
Abstract:
Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared with general video understanding, data video understanding places greater emphasis on accurately reading data from animated charts, integrating evidence across charts and time, and understanding how narrative organization and visual design communicate information. Yet existing benchmarks target either general videos or static charts, and data video understanding has not been systematically evaluated. We present DataVista, the first benchmark for data video understanding, containing 961 real‑world data videos and 6,775 evaluation questions organized under a three‑level progressive capability framework (data perception, temporal reasoning, narrative understanding) with 10 fine‑grained question types across five topic domains. Systematic evaluation of 19 mainstream MLLMs shows that the best‑performing model, Gemini‑3.1‑Pro, achieves 70.0% overall accuracy, still far below human expert performance, with models performing worst on Causal Reasoning and Narrative Structure. Increasing frame counts and adding subtitles mainly benefit data perception and temporal reasoning, with limited gains in narrative understanding. Further analysis of model responses identifies typical failure modes in chart reading, evidence judgment, and instruction understanding. The benchmark is available at https://github.com/HKUSTDial/DataVista.
Authors:Avihai Giuili, Rotem Atari, Avishai Sintov, Maya Bechler-Speicher
Abstract:
Learned simulators for deformable linear objects (DLOs) such as cables have to predict the motion of cables they were not trained on and stay stable over long rollouts. Most of their error occurs where the cable touches itself or the floor. Attention over all pairs of cable segments can represent contact between parts of the cable that are far apart along its length, but attention has no notion of geometry. A cable has two pairwise distances, which agree only while it is straight: the arc‑length distance along the cable, which governs elastic forces, and the Euclidean distance in space, which governs contact. We add a physical attention bias, an additive term on the attention logits with a learned rate, and ask which distance it should use. We compare no bias, each distance alone, and both distances on disjoint sets of heads, keeping the rest of the model and the training protocol fixed. A physical bias improves prediction on unseen cables. The gain is largest when attention is the only mechanism that connects distant segments: there, the arc‑length bias reduces prediction error by 15% and more than halves the drift in segment length. The Euclidean bias alone stays close to unbiased attention, while assigning both distances across heads is best or near‑best on every metric we report. Code and per‑run records: https://github.com/avihaig/dlogps.
Authors:Panagiotis Kiousis, Kuangyi Chen, Jun Zhang, Friedrich Fraundorfer
Abstract:
Localizing an event camera against a pre‑built LiDAR map can be cast as dense optical‑flow estimation between a rendered depth view and an event image, followed by a Perspective‑n‑Point (PnP) solver over the induced 3D‑2D correspondences. Existing pipelines rely on geometric consensus during pose estimation, but do not explicitly model the reliability or pose informativeness, i.e., how strongly a correspondence constrains the camera pose, of individual correspondences. We show that the natural way to learn it ‑‑ using the per‑correspondence error to constrain the learning of confidence ‑‑ suffers from a depth‑dependent bias: small pixel errors reside predominantly at large depths and do not lead to high pose informativeness. Instead, in our method (CELL), we learn a per‑correspondence confidence end‑to‑end through the pose, using a differentiable probabilistic PnP whose log‑partition term encourages weight configurations that yield a better‑constrained pose distribution. The learned confidence is used in three ways: (i) it reweights the flow supervision in a decoupled training scheme that keeps pose gradients out of the flow/edge backbone; (ii) it drives a probabilistic correspondence selection at test time; and (iii) together with the network's edge‑probability it weights a final edge‑matching refinement. We further design a partial‑completion depth representation that adds signal without hallucinating across large gaps. On M3ED and DSEC our full system improves over the LEAR baseline on the majority of the evaluated sequences: it reduces the median translation error by up to 26.9% and the median rotation error by up to 15.8%.
Authors:Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Zhiyuan Gao, Deyuan Qu, Max Gandyra, Yanxiang Zhan, Mehreen Naeem, Andrew Melnik, Jeroen Schafer, Qing Yang, Michael Beetz
Abstract:
A generated video of a task the robot is about to perform is useful guidance only if it depicts the phase the robot is actually in. We show that temporal misalignment can turn a task‑consistent generated future into actively harmful guidance. On CALVIN, a five‑frame early shift nearly erases the benefit of generated futures, reducing success from 81.3% to 54.8% against 54.0% without futures; imposed timing shifts reduce it even further to 34.2%, 19.8 points below the future‑free policy. We introduce Reliability‑Aware Future Conditioning (RAFC), which treats this as a control problem rather than a generation problem. At every step, RAFC estimates how far to trust the received clip and which nearby temporal hypothesis to prefer, falling back toward a static branch when neither fits, and it learns both from task reward alone without shift labels or alignment supervision. RAFC sits on top of Future‑Experience Conditioning (FEC), which builds the clip once from task grounding, a robot‑free digital‑twin rollout, and mask‑free video diffusion. Under deliberately off‑grid phase shifts and rate mismatch, RAFC substantially improves success under temporal mismatch. Candidate ensembling accounts for most of the recovery near alignment, while learned reliability adds a further 7.0 percentage points over uniform averaging of the identical candidate bank under off‑grid shifts. The gain holds on the evaluated task sets and survives on a Franka under natural timing mismatch nobody imposed, where aggregate success rises from 26.7% to 56.7%. All resources will be made publicly available. https://future‑condition.github.io/.
Authors:Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu, Haijier Chen, Xianxin Lai, Shuyu Pan, Zhen Yang, Xiaoquan Sun, Wenze Cui, Zhongliang Jiang, Shaopeng Liu, Jiayu Chen
Abstract:
Collecting tactile demonstrations on robots is costly and slow, motivating the use of lower‑cost human tactile gloves for scalable data collection. However, human capacitive/piezoresistive gloves and robotic tactile sensors differ fundamentally in transduction principle, sensor layout, spatial resolution, and dynamic response, making alignment of raw sensor channels ill‑posed. To address this problem, we present TACROSS, a scalable system for learning from human touch and transferring it to robots that bridges this heterogeneity by aligning tactile streams at the level of contact events rather than raw sensor values. The hardware component of TACROSS integrates a piezoresistive glove with five layers and a cost of USD 10.86 with 285 sensing points. To align contact semantics, we design canonicalizers and residual adapters that map heterogeneous signals into a shared tactile latent with 256 dimensions via a temporal Transformer with attention across fingers. We further introduce a robot‑grounded policy learning scheme in which robot demonstrations provide the sole source of ground‑truth action supervision, while human demonstrations support tactile representation learning and provide confidence‑weighted auxiliary supervision through valid retargeted hand targets. We evaluate our system on four contact‑rich manipulation tasks. Compared to conventional teleoperation, our proposed system achieves a 3.5‑fold efficiency improvement while reducing demonstration acquisition equipment cost by 95.7%. We will open‑source the TACROSS hardware and software system and publicly release a tactile dataset comprising over 150 hours of recordings. Project page: https://tacross‑touch‑project.github.io/.
Authors:Jierui Lei, Wenjian Zhang, Qingyi Yang, Yuyang Hong, Fangzheng Chen, Zhengbo Zhang, Haina Tang, Shiming Xiang
Abstract:
Media‑bridged time series forecasting is expanding to encompass traditional "multivariate" and emerging "multimodal" (e.g., through textual assistance). Existing Time Series Forecasting (TSF) models still rely on paradigm‑specific relation, fusion, and temporal modules, hindering a common forecasting backbone across numerical and pre‑aligned narrative‑flow settings. To explore this, we propose the Multimedia Identity‑Aware Prism Network (MIDAPN), a unified spatiotemporal forecasting backbone based on media‑general graph adaptation and automatic temporal learning: (1) Following media pre‑alignment, our Multimedia Identity‑Aware Graph (MIDAG) revisits identity through static essence, dynamic behavior, and latent commonality, inducing affinities that extend variable‑specific dependencies across media. Contextual Identity Modulation (CIM) further refines discriminative aggregation. (2) We develop Spectral Prism Convolution (SPConv) to automatically perform hierarchical temporal analysis, balancing coarse trends and fine‑grained details. Meanwhile, its Adaptive Search Guidance configures a scale‑efficient architecture for temporal‑dimension reconstruction. These decoupled yet synergistic components jointly address media identity disentanglement and temporal‑scale mismatch. Comprehensive evaluations involving 16 SOTA TSF models across 13 "multivariate" and 12 "multimodal" datasets, alongside targeted long‑context comparisons against 14 time series foundation models and fused pretrained language models, demonstrate MIDAPN's consistent superiority and broad shared backbone compatibility. The code is available at \hrefhttps://github.com/leijieruilq/MIDAPN/tree/mainhttps://github.com/MIDAPN.
Authors:Jia Li, Yichao He, Yangchen Yu, Qiankun Li, Xinyi Li, Baiyi Ye, Zhenzhen Hu, Richang Hong, Erik Cambria
Abstract:
Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue‑label associations, giving rise to the Clever Hans effect. Such shortcuts become unreliable when affective cues are implicit, conflicting across modalities, linguistically misleading, or obscured by redundant details. In contrast, human emotions are shaped by how individuals interpret and evaluate surrounding events beyond observable cues. Inspired by appraisal theories of emotion, we formulate multimodal emotion understanding as a progression from perception to cognitive appraisal, and introduce a dataset, a model, and a benchmark to support this novel paradigm. CogEmo‑40K is a large‑scale instruction‑tuning dataset constructed through a perception‑to‑appraisal pipeline to elicit evidence‑grounded reasoning across six cognitive appraisal dimensions underlying emotion. CogEmo‑MoE is a compact sparse MLLM that introduces interleaved MoE blocks for appraisal‑specific adaptation, enabling effective appraisal reasoning at a substantially smaller scale than typical emotion MLLMs. CogEmo‑Bench introduces an Appraisal Evidence Quality Score (AEQS) to assess cognitive‑affective understanding across six complementary appraisal dimensions, addressing the limitation of conventional emotion metrics that evaluate what emotion is predicted but not why it arises. Extensive experiments show that our paradigm not only leads CogEmo‑Bench, but also exhibits strong cross‑domain generalization. Our findings suggest that perception‑to‑appraisal reasoning can move beyond surface‑level cue‑label associations toward more reliable multimodal emotion understanding and closer cognitive alignment between MLLMs and humans.
Authors:Jingyi Pan, Dan Xu, Qiong Luo
Abstract:
3D scene inpainting aims to recover missing or occluded regions in edited 3D scenes, while ensuring geometric and textural consistency. Existing approaches, however, typically require accurately calibrated camera poses, which restricts their applicability in casual, in‑the‑wild scenarios and introduces additional preprocessing overhead. To overcome this limitation, we present FreeInpaint, a novel feed‑forward framework that generates complete and 3D‑consistent scenes directly from unposed multi‑view images with masked regions. At its core, FreeInpaint extends a 3D foundation model to propagate masked regions from a reference view to other unposed views, bridging 3D reconstruction and scene inpainting while preserving the model's native ability to recover camera poses and scene geometry. Our method addresses two key challenges in adapting feed‑forward 3D foundation models to masked inputs. First, masked regions can corrupt cross‑view correspondence reasoning, degrading pose estimation and geometry recovery. To address this, we introduce a Learnable Mask Attention mechanism that preserves the spatial anchoring of reliable observations while allowing masked regions to progressively absorb useful context in deeper layers. Second, under severe occlusions, a single forward pass often lacks sufficient appearance evidence for high‑fidelity completion. Therefore, we propose a Support Token Refinement strategy, which injects diffusion‑generated support evidence as confidence‑weighted auxiliary tokens to refine under‑observed regions while preserving the original spatial anchor. Extensive experiments across diverse datasets demonstrate that FreeInpaint achieves superior inpainting quality, eliminating the reliance on pre‑computed camera poses while keeping a fast inference speed. The project page is https://rorisis.github.io/FreeInpaint/.
Authors:Hexuan Deng, Zihao Yan, Xuebo Liu, Shuo Nie, Yue Wang, Chen Wang, Zhaohua Zhang, Tianwen Jiang, Qiuyong Xiao, Jihong Zhang, Min Zhang
Abstract:
Reinforcement learning (RL) methods such as GRPO substantially improve large language model reasoning but often suffer from policy entropy collapse: the loss of sampling diversity weakens exploration and limits further improvement. Existing methods address this issue either through algorithm‑level interventions, such as reward modification and entropy/KL regularization, or through token‑level reweighting. We investigate a complementary perspective: entropy collapse can also be mitigated by changing which generated rollouts contribute to policy updates. Under the same sampling budget, not all rollouts contribute positively to an update, and selectively excluding some can improve learning. To address this, we propose GRPODropout: before the standard update, we use a simple strategy that selectively removes a small number of high‑probability positive‑advantage rollouts and recenters the retained advantages. To motivate this design, we develop a rollout‑level theoretical analysis that guides method design and threshold selection. The method changes only rollout usage, and adds negligible computational overhead. Experiments show higher accuracy than original GRPO and higher actor entropy while using fewer rollout samples for updates, illustrating "less is more." This work provides insight into RL rollout usage: removing some rollouts can improve performance. Code is available at https://github.com/hexuandeng/GRPODropout/.
Authors:Varun Gumma, Navonil Majumdar, Soujanya Poria
Abstract:
The rapid proliferation of Large Language Models (LLMs) necessitates reliable watermarking techniques to identify AI‑generated text and ensure appropriate attribution. While token‑based watermarks are vulnerable to paraphrasing, a central challenge for semantic watermarking is to turn sentence meanings into a stable, well‑calibrated document‑level signal. To this end, we introduce SemField, a simple, training‑free semantic watermark that embeds a continuous, linear signal directly into the sentence embedding space. Using a shared secret key to define a specific Gaussian direction, SemField iteratively evaluates candidate sentences and selects those that maximize the alignment of the cumulative document aggregate with this targeted direction. We also propose SemField‑PL, a polarity‑locked variant that first determines the optimal orientation from the initial sentence and continuously reinforces it throughout the generation process. The document‑level aggregated formulation guarantees exact invariance to sentence reordering and provides theoretical bounds against structural tampering, such as sentence insertion and deletion. Lastly, with extensive evaluations across three models, we demonstrate that both variants outperform 12 recent baselines. Across clean detection and four paraphrasing attacks, both variants achieve a mean True Positive Rate (TPR) of 88.2% to 90.4% at a 1% False Positive Rate (FPR), all while maintaining comparable perplexity and naturalness as human‑generated content. We open‑source our implementation at https://github.com/declare‑lab/SemField
Authors:Tjaš Ajdovec, Marko Robnik-Šikonja, Simon Šuster
Abstract:
Spin in clinical trials includes reporting practices that distort the presentation of results. This is particularly critical in medicine, where spin is present in more than 50% of randomized controlled trials that fail to reach statistical significance. The comparison of primary and reported outcomes is crucial for detecting several types of spin, including outcome switching. We used 300 pairs of outcomes labeled with semantic similarity to develop a system for automatic detection of outcome switching. We evaluated baseline text similarity models and open‑source LLMs using generated similarity scores and the Youden index to determine the classification threshold. The proposed approach involves prompt engineering, classification based on token probabilities, and majority voting for the final decision. The results on the test set of 2,496 examples with an F1 score of 0.78 and an accuracy of 0.90 outperform baseline text similarity models but trail behind fine‑tuned versions of BERT. We used LLMs to generate natural language explanations for the classified instances and manually assessed their quality.
Authors:Tejasvi Singh Tomar
Abstract:
The rank of the tensor of 3× 3 matrix multiplication over the field with two elements is at most 23 by Laderman's algorithm, and Rudich and Rousseau recently proved that it is at least 22. We prove that it equals 23. Hence Laderman's algorithm uses the fewest multiplications among all bilinear algorithms over \mathbbF_2 and among all bilinear algorithms with integer coefficients. The proof uses the substitution method in the form developed in recent work of D'Ambrosio, Wang and Yang et al.: a subspace S of the space of first factors contains at most r‑R(S) first factors of a decomposition of length r, where R(S) is the rank of the tensor modulo S. We raise the known lower bounds on R(S) for 111 of Wang's 496 symmetry classes of subspaces. One of these bounds, R(S)\ge 21 for a point spanned by a matrix of rank one, forces the 22 first factors of a decomposition of length 22 to be distinct. A 27× 27 flattening of the tensor gives further constraints on the ranks of the first factors, and a separate enumeration shows that, when at least 14 first factors have rank one, no line in a certain orbit of lines contains two first factors. A computer search then lists, up to symmetry, all sets of 22 matrices that satisfy these constraints, and an exact completion search shows that none of them is the set of first factors of a decomposition. The computation emits certificates, which are checked in the Lean 4 proof assistant by checkers whose soundness is proved in Lean. The largest checks are evaluated as compiled code, so the proof relies on the Lean compiler in addition to its kernel.
Authors:Uddipan Basu Bir, Vincent Christlein, Andreas Maier, Mathias Zinnen
Abstract:
While massive, closed‑source Vision‑Language Models (VLMs) set strong benchmarks for document understanding, their dependence on commercial APIs limits adoption in institutional archives due to data autonomy concerns, recurring costs, and the environmental footprint of hyperscale computing. This is especially acute in heritage digitization, where documents include historical handwriting, domain‑specific terminology (e.g., jewelry, prehistory, architecture), and non‑standard layouts requiring high‑dimensional structured extraction. We present a comparative study of eight open‑source lightweight VLMs (up to 7B parameters) for Optical Character Recognition (OCR)‑to‑structure across three university heritage collections. Given a document image, models must extract text and generate schema‑compliant JSON, enabling automatic validation and downstream use. We evaluate models under a constraint‑aware protocol across zero‑shot, few‑shot, and fine‑tuning settings, measuring extraction fidelity and structured‑output quality using Character Error Rate (CER), Approximate Normalized Levenshtein Similarity (ANLS), and mean Average Precision F1 (mAP‑F1). Against a fine‑tuning baseline, we further test the independent impact of (i) hyperparameter optimization, (ii) classical image preprocessing (illumination flattening, denoising, and CLAHE), and (iii) multi‑stage training. Finally, we analyze the trade‑off between dataset‑specific fine‑tuning and a single multi‑dataset checkpoint, where joint training enables one model to operate across collections but can shift performance between datasets. Overall, we show that carefully adapted VLMs with up to 7B parameters can provide a sustainable, private, high‑performing alternative to manual transcription or commercial black‑box systems, and we offer actionable guidance for heritage institutions seeking institution‑controlled OCR‑to‑JSON extraction.
Authors:Yubo Sun, Chunyi Peng, Yukun Yan, Zhenghao Liu, Zhipeng Xu, Sen Mei, Linlin Xin, Zheni Zeng, Maosong Sun
Abstract:
Dense retrievers have made significant progress on text and image corpora, but whether these capabilities extend reliably to mixed corpora containing text, image, and fused text‑image documents remains unclear. In this paper, we systematically examine retrievers across architectures and find that their performance is highly sensitive to modality composition. As image documents are progressively replaced with semantically corresponding text representations, retrieval performance follows a pronounced V‑shaped curve, remaining strong on single‑modality corpora but degrading substantially when modalities coexist. In particular, irrelevant text causes more severe degradation than an equal number of irrelevant images, a phenomenon we term Chaos in the Text. Further analysis reveals modality preference, whereby text representations receive systematically higher similarity scores, allowing irrelevant text to outrank relevant images. To mitigate this bias, we introduce Trident, which constructs text, image, and fused text‑image views of each document as co‑equal positives and jointly optimizes relevance discrimination and positive‑view balance through Multi‑Positive View InfoNCE. Experiments across visual document and natural image benchmarks show that trident improves mixed‑modality retrieval on both CLIP‑based and VLM‑based architectures, reduces sensitivity to modality composition and text distractors, and increases average single‑modality retrieval performance.
Authors:Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai, Jingwen Ye
Abstract:
Existing approaches to multi‑view spatial reasoning operate largely on sparse input views. Vision‑language models (VLMs) are thus restricted to understand a scene and infer spatial relations within these fixed views, leading to fragile cross‑view alignment and geometry‑to‑language bottleneck. To address these issues, we formulate a novel Seek‑and‑View reasoning approach to find implicit cross‑view spatial evidence by locating a question‑relevant view to support the spatial reasoning. To realize this approach, we propose Vantage, a training‑free model‑agnostic reasoning framework that pairs a VLM with a 3D foundation model: a viewpoint‑grounded reasoning stage for question analysis and view planning, followed by a geometry‑grounded evidence augmentation stage to effectively synthesize and incorporate visual evidence into the final reasoning. Comprehensive experiments on six VLMs demonstrate consistent improvements on five benchmarks without fine‑tuning. Overall, by revealing spatial evidence through view‑grounded reasoning, Vantage can largely reduce reliance on language‑based cross‑view alignment and improve multi‑view spatial understanding. Our code is available at https://github.com/q1xiangchen/Vantage.
Authors:Nicolás Vera Zúñiga
Abstract:
Byte‑level language models such as the Byte Latent Transformer (BLT) group bytes into patches and run their large global model once per patch. BLT starts a patch where a small model's next‑byte entropy is high, so global compute goes where the next byte is hard to predict. We show that this rule has a systematic blind spot: positions whose type is predictable but whose value must be computed, such as the number after "=" in a worked math solution. Under tight patch budgets, entropy‑triggered layouts skip these positions, and accuracy on them collapses. In Meta's BLT‑1B with patch starts on 10% of bytes, the entropy rule puts a patch start at 16% of the computed results in GSM8K solutions and gets 19.0% of them exactly right; a boundary after each "=" at the same patch count gets 51.8%, and entropy combined with a label‑free boundary‑dependence signal gets 67.1% (default layout at 26% of bytes: 76.8%). The gap survives adapting BLT‑1B to the budget with low‑rank fine‑tuning (32.9% vs 72.7%, three runs per rule, paired p < 1e‑200) and grows with model size in byte models trained from scratch at a 10% budget: at 1M, 12M and 50M parameters, boundary dependence beats entropy on final answers by ‑1.6, +10.1 and +19.8 points, and at 50M it gets 35.9% of computed results against 13.9% (3 seeds each). BLT's entropy‑jump rule helps neither target at 50M. The entropy trigger of Scratchpad Patching is likewise indistinguishable from random scratchpads on final answers (5.6% vs 5.9%, 5 seeds), while answer‑start scratchpads give 38.1%. The effect is specific to computed values: copies and lookups gain little, and values the model cannot compute gain nothing. Boundary dependence, the rise in the model's own loss when a patch start is removed, measured per two‑byte context, finds these positions without labels: combined with entropy it beats the hand‑written rule on computed results.
Authors:Youwei Feng, Yitong Zhang, Yuetong Liu, Jia Li
Abstract:
Guard models are increasingly used to safeguard LLM‑based agents, primarily by identifying actions that agents are forbidden to perform. However, identifying forbidden actions alone is insufficient to ensure agent safety. In this paper, we argue that agent safety also depends on identifying required yet unperformed safety‑critical actions, which we call obligations. Our preliminary study on a popular benchmark for evaluating safety shows that 56.92% of GLM‑5.3 trajectories contain unfulfilled obligations, compared with only 30.00% containing forbidden actions. This finding reveals unfulfilled obligations as a major and previously overlooked source of safety risk. However, to our knowledge, no existing benchmark evaluates whether guard models can identify these obligations. To close this gap, we introduce ObligationBench, the first benchmark for evaluating the capability of obligation identification, comprising 240 expert‑validated trajectories covering issue resolution, feature development, and terminal operations. Our evaluation of 14 representative models reveals substantial limitations: the highest recall and exact‑match rate are only 48.97% and 10.00%, respectively. To address these limitations, we develop ObligationGuard using 40,000 synthetic training examples. ObligationGuard achieves 57.52% recall and an exact‑match rate of 21.67%, surpassing all evaluated models on both metrics. We call on the community to incorporate obligation identification into the design and evaluation of future guard models to improve agent safety.
Authors:Haitong Jiang, Chunlin Liu, Sihan Tang, Chan Wu, Xiaoqing Su, Yuhong Feng
Abstract:
Safety evaluations of large language models commonly summarize harmful‑output behavior with attack success rate (ASR). Yet the same non‑harmful outcome can arise for very different reasons. A model may recover a harmful task and refuse it, fail to recover the task, or respond to something else entirely. Distinguishing these cases becomes especially important under intent‑obscuring prompts, where a low ASR does not reveal whether the evaluated task was actually engaged. To make this distinction explicit, we pair ASR with operative understanding rate (UR), which measures whether a response both identifies the evaluated task and treats it as the task to be answered. Across interfaces, this paired view reveals substantial variation hidden by ASR: similar ASR values can correspond to sharply different recovery rates. Controlled English reconstructions show that recovery consistently improves as compressed prompts become more explicit, whereas ASR does not follow the same pattern. A complementary contrast comes from FormalLogic, where high recovery can still coincide with frequent harmful assistance. Together, these results show that non‑harmful outcomes are not equally informative about model safety, motivating the joint reporting of intent recovery and ASR in LLM safety evaluation. Code and experiment inputs are available at https://github.com/kevinjiang0121‑cyber/IRIS.
Authors:Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr, James Zou
Abstract:
Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no‑thinking", has received far less attention. We study LLMs' no‑thinking behavior along two axes. a. How to measure no‑thinking? Prior work typically defines no‑thinking through proxies such as a disabled thinking mode or the absence of long traces. These proxies are unreliable: disabled thinking modes may still emit reasoning, while long traces may contain filler rather than genuine inference. We instead normalize each response into a pre‑answer trace and final answer, and evaluate it at three levels: (i) Empty‑Thinking Rate for strict answer‑only compliance; (ii) instruction‑aware Question‑Pre‑answer Relevance for similarity between the question and pre‑answer trace; and (iii) LLM‑as‑judge Explicit Inference Rate for visible explicit inference. Together, these metrics distinguish answer‑only output, relevant but non‑inferential text, and explicit inference. b. How does no‑thinking vary across tasks and models? We evaluate six prompting interventions on six LLMs across Boolean, multiple‑choice, and open‑ended questions. We find that explicit no‑think controls cannot reliably eliminate visible inference. Models instead exhibit "Thinking Inertia": explicit inference persists even under strict controls and becomes more prevalent as the answer space opens. Accuracy remains stable on Boolean and multiple‑choice tasks, whereas open‑ended tasks reveal a trade‑off between answer‑only compliance and task accuracy. Rewriting the same questions across answer spaces shows that supplying candidate answers makes answer‑only responses easier to produce. These findings establish no‑thinking as a non‑trivial capability: stopping explicit reasoning cannot be assumed from model settings or instructions alone and deserves systematic evaluation alongside reasoning ability.
Authors:Frederikke Isa Marin, Panagiotis Antoniadis, Dionysia Danai Brilli, Andreas Bjerregaard, Rachael DeVries, Yan Li, Ole Winther, Wouter Boomsma
Abstract:
The genome holds the blueprint that governs the biological properties of the cell. Consequently, advancing our knowledge of genomic function is crucial both for a broader understanding of biology and for continued biomedical advances. The success of large language models on natural language and protein sequences has motivated similar efforts on genomic data. However, standard genomic language models (gLMs) often require extremely large computational resources and still fall behind traditional methods on some downstream tasks. Recently, MSA‑based pretraining has been proposed as an efficient alternative, but existing models are limited to short input contexts, restricting their use to short‑range tasks, such as variant effect prediction. In this work, we present RAGenome, the first retrieval‑based gLM that scales pretraining to longer contexts (100× longer than existing MSA‑based gLMs), allowing it to capture both across‑species evolutionary relationships and within‑species longer‑range interactions. Trained on whole‑genome alignments from 100 vertebrates, RAGenome substantially improves the long‑range capabilities of MSA‑based gLMs, raising gene finding performance from 0.45 to 0.60, while remaining competitive on purely evolutionary‑based tasks like prioritizing pathogenic variants. RAGenome provides competitive gLM performance at a fraction of the training cost, unifying evolutionary modeling and long‑range capabilities within a single, flexible, scalable framework. Code is available at https://github.com/PanosAntoniadis/RAGenome.
Authors:Arijit Ghosh, Lucas Degeorge, Paul Couairon, Alexei A Efros, Vicky Kalogeiton, David Picard
Abstract:
Co‑denoising pretrained representations such as DINO can substantially improve the training speed and quality of flow matching models, but it introduces a second denoising trajectory and requires carefully designed schedules. We propose a simpler alternative: predict the pretrained representation directly, then condition the model on its own prediction. This removes the need for a second ODE and any representation‑specific denoising schedules, while retaining the benefits of representation guidance. Our approach converges substantially faster and achieves better generation quality as measured by FID score. On ImageNet, it outperforms the state of the art in latent space at 2x fewer epochs than prior methods; in pixel space, it improves FID over comparable prior methods by more than 20%. These results support a simple principle: do not denoise what you can predict. Our code is openly available at https://github.com/arijit‑hub/dino_forcing.
Authors:Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou
Abstract:
Tree‑based speculative decoding verifies multiple draft continuations in one target‑model pass, but finite trees built from draft scores face a fundamental draft‑target mismatch. We ask whether better exact verification can increase acceptance on a fixed tree and how target feedback can improve the tree itself. Through a target‑flow view, we identify a canonical exit law and prove that one plus target coverage sharply bounds the expected output‑block length, including the bonus token, of any exact path verifier. All optimal verifiers share the same exit and bonus‑token law, already attained by representative predraw‑and‑follow and sequential residual verifiers. This yields Tree Exit Verification (TEV), an exact, level‑parallel procedure using one exit‑node decision and one bonus‑token decision. The exit law also identifies missing target probability, providing node‑level feedback for Exit‑Guided Draft‑Tree Training (ExitTrain) on inference‑time draft trees. Experiments across dialogue, code, and mathematical reasoning validate fixed‑tree equivalence: ExitTrain increases average output‑block length by 13%, while TEV reduces verifier‑stage latency by 15%, yielding a 14% end‑to‑end speedup over DDTree. Our results distinguish two opportunities: better draft trees for higher acceptance and more direct verification for lower latency. Code: https://github.com/hsj576/TEV.
Authors:Yawen Shao, Yubo Zhu, Ziyun Dai, Zixun Fang, Kai Zhu, Zeyinzi Jiang, Yufeng Ai, Siyang Sun, Haolan Xue, Yu Shang, Yuxiang Bao, Zoubin Bi, Jingming Luo, Jie Xiao, Chaojie Mao, Zhehan Kan, Hongchen Luo, Yu Liu, Sheng Zhong, Wei Tong, Xueyang Fu, Yang Cao, Wei Zhai, Zheng-Jun Zha
Abstract:
Modern video generators can realize increasingly complex visual narratives, positioning the prompt enhancer (PE) as a critical bridge from concise user instructions and multimodal references to structured cinematic plans. However, existing PE evaluation relies on rendered videos, imposing substantial computational and human costs, slowing PE training and iteration, and conflating PE quality with downstream generator behavior. To address this gap, we introduce PEBench, the first unified benchmark for direct PE evaluation across text‑to‑video, image‑to‑video, and reference‑to‑video prompt enhancement. It comprises 1,100 expert‑verified cases and 1,005 visual assets, spanning 35 fine‑grained tasks with diverse temporal, cinematic, audiovisual, and multi‑reference requirements. In addition, we develop PEBench evaluation, an evidence‑grounded framework that combines modality‑aware fact extraction with rubric‑based assessment across 24 criteria. Our systematic evaluation of representative open‑ and closed‑source PE methods reveals an emerging shift from fine‑grained descriptive expansion toward intent‑preserving cinematic planning, while the caption‑reconstruction and forward‑refinement methods show complementary strengths in cinematic coverage and semantic fidelity or internal coherence, respectively. Human validation shows that PEBench scores align closely with expert judgments of enhanced prompts and downstream videos from Wan3.0 and MiniMax‑H3, indicating that prompt‑level evaluation reliably reflects downstream utility.
Authors:Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov
Abstract:
Standard weight decay treats each weight matrix as a vector and ignores its spectral structure. We introduce spectral weight decay, a post‑step decoupled nuclear‑norm update that applies additive rather than multiplicative spectral shrinkage. We connect the update to approximate proximal descent and show that its sensitivity to update order can exceed that of conventional \ell_2 weight decay near rank deficiency. Across LLaMA models with 124M to 500M parameters, spectral weight decay lowers effective rank and improves SVD‑LLM compression at matched validation loss. At 500M and a 4% distortion budget, it reaches 1.89× compression and 1.18× GPU inference speedup, compared with 1.14× and 1.01× after standard weight decay. Under fixed‑horizon training with 60% label noise, it also improves final mean clean‑test accuracy over matched \ell_2 regularization by up to 17.8 points on MNIST and 4.6 points across four BERT‑base tasks. Code is available at https://github.com/brain‑lab‑research/SpectralWD.
Authors:Hartwig Grabowski
Abstract:
The Spec Growth Engine anchors AI‑assisted software development in a graph of specifications that the code is coupled to. This paper describes its implementation, which serves two tasks and keeps them apart as two layers. The first layer prevents spec‑code divergence: a deterministic engine validates the spec graph, compares it with the code's import graph, earns a node's verified status from recorded test evidence, and classifies every change by what it can break ‑‑ without calling a model. The second layer grows the spec with agents: an intent author, a planner and a coder, each played by its own model, extend the graph in rounds, and a deterministic rule decides after each round whether the run goes on. How much of the human's judgement is delegated is set by three independent switches ‑‑ a draft gate, a delegation for breaking changes, and the run mode ‑‑ which, with two ways of laying a project's floor, give eighteen ways to run a project. We describe each of them, the gates, requests and waivers through which agents and the human communicate, and the spectrum of operation from entirely manual work to an unsupervised run whose decisions the human reviews afterwards. Throughout, one claim holds the design together: an autonomous run is worth only as much as the deterministic instance that measures it.
Authors:Chenchen Zhou, José Matias
Abstract:
Distributed control requires choosing how far information travels and how long it is retained. For networked linear quadratic regulation (LQR), we use System Level Synthesis to bound these resources for a specified performance loss per node relative to centralized control. For locally coupled systems under uniform regularity assumptions, including stabilizability and detectability, we prove that the centralized optimal state and input responses to disturbance impulses satisfy exponential decay bounds in both time and spatial distance. On networks with polynomial neighborhood growth, sufficient communication ranges and memory horizons grow logarithmically with the inverse tolerance, independently of network size. Truncating these responses gives local finite‑memory filters with stability and performance guarantees under a computable small‑gain condition. Under an additional uniform local feedback condition, we also obtain controllers that confine nominal disturbance effects in space and eliminate them in finite time. Numerical experiments illustrate how performance and disturbance containment requirements guide architecture selection.
Authors:Tobias Ritschel, Yang Zhou, Nick Milef, Mikhail Dereviannykh, Chen Liu, Christophe Hery, Carl Marshall
Abstract:
We suggest a method to generate time‑varying Gaussian noise with controlled variance and controlled temporal correlation. This noise is used in several downstream tasks for temporal control and temporal coherence. The core technical idea is to phrase this problem as joint Monte‑Carlo estimation of both a classic pixel reconstruction and estimation of variance using the concept of "sketching" from the database literature. We demonstrate that our method allows temporal control for downstream tasks with simpler and faster code than previous methods.
Authors:Yuchen Yang, Xin Wang, Lufan Wang, Yinghong Pan, Yujuan Feng, Yuqing Yang
Abstract:
Generating ultrasound reports from multiple images requires aggregating clinical evidence across views, yet archived key frames capture only part of the dynamic examination. Raw‑report imitation is therefore misaligned with visual supervision: content that is clinically valid for the full examination may be unverifiable from the images available to a model. This gap creates a clinical behavior alignment problem. A model must preserve visible findings, avoid diagnostic reversals and unsupported completion, and not collapse into conservative templates. We propose CAMEO, a Clinically Aware Multi‑image Evidence‑grounded Orchestration framework for ultrasound report generation. Stage I learns ultrasound visual‑language primitives; Stage II performs Cross‑View Evidence Grounding by distilling trusted visible report points into multi‑image QA and report‑style supervision; and Stage III performs Clinically Aware Preference Alignment using clinical‑error‑oriented preference pairs. From USReport, we construct USReport‑Distilled with 17,670 evidence‑grounded paired‑image training instances and USReport‑Pref with 21,869 preference pairs; we additionally use 25,631 PubMedVision‑US ultrasound instruction samples for domain adaptation and multi‑image instruction tuning. On the primary USReport‑Distilled benchmark, CAMEO improves over EchoVLM from 0.25 to 0.40 BLEU‑1, 0.28 to 0.45 ROUGE‑1, and 0.27 to 0.43 METEOR, while raising ClinicalScore from 55.02 to 74.20. These results underscore the value of evidence‑grounded supervision, clinically aware alignment, and clinically structured evaluation for reliable ultrasound report generation.
Authors:Matěj Kripner, Milan Straka
Abstract:
We introduce NanoProof, to our knowledge the first factorized execution‑guided theorem prover in Lean 4 whose training data, extraction tooling, training pipeline, and weights are all released, making it end‑to‑end reproducible using open‑source resources. To this end, we build and release a dataset of structured proof trees, as well as a tool for programmatic interaction and data extraction within the Lean 4 formal verifier. To support sustainable research, we focus on compute efficiency to facilitate accessible training and evaluation. NanoProof achieves 50.8% pass@16 on MiniF2F‑Test, exceeding the two closest systems of its class, HyperTree Proof Search and ABEL, at roughly 90x and 7x less compute, and using more than four orders of magnitude less compute than AlphaProof. Stronger open‑weight provers exist, but they are fine‑tuned from large pretrained language models and release neither training data nor pipeline; NanoProof shows that the factorized execution‑guided class of provers can be rebuilt from scratch with modest resources.
Authors:Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu
Abstract:
In this paper, we argue that looped Transformers need their own residual connections to prevent performance degradation as the number of iterations grows. We observe that increasing loop iterations can reduce reasoning accuracy: noisy state updates overwrite correct intermediate deductions and even undo completed solutions. This leaves subsequent iterations to recover lost information from an already degraded representation: once an error arises in an earlier loop, often as a result of long‑range propagation through the recurrence, later loops find it difficult to correct. In this paper, we introduce InfiLoop, a loop‑native residual connection that learns which past computations to retain and how much to accept from each new update. InfiLoop combines content‑based weighting with learned temporal decay to maintain a running summary of recurrent states. An exact streaming recurrence keeps its persistent aggregation memory constant as the loop count grows. The resulting adaptive update suppresses unreliable proposals and preserves useful intermediate states. Across extensive reasoning tasks, a 7M‑parameter InfiLoop model outperforms existing recursive architectures, reaching 97.9% exact accuracy on Sudoku‑Extreme, and 13.6% pass@2 on ARC‑AGI‑2. Notably, on Sudoku‑Extreme, InfiLoop continues to improve with test‑time looping beyond 20,000 effective steps, showing that added depth translates directly into stronger reasoning. Our code is available at https://github.com/pixeli99/InfiLoop.
Authors:Meilin Chen, Hongyuan Bao
Abstract:
Existing Open‑Ended Deep Research (OEDR) systems primarily generate reports from scratch, making them inefficient for scenarios where research reports need to be continuously maintained as new information emerges. We introduce Incremental Open‑Ended Deep Research (Incremental‑OEDR), a research setting that treats a report as an evolving research state and incrementally updates it by preserving valid knowledge, revising outdated or incomplete content, and incorporating newly available information. To support this setting, we propose Structured Harness, which represents reports as structured collections of outlines, sections, and supporting evidence, and provides structured retrieval, a persistent structured evidence pool, and structured generation for selective report updating and evidence reuse. We further establish a temporal evaluation framework spanning ten years, with \emphSingle‑Step Task and \emphLong‑Chain Task to evaluate incremental updates over both individual transitions and long‑term update chains. Extensive Experiments on DeepResearch Bench and DeepConsult under both the Open‑source Configuration (OC) and Proprietary Configuration (PC) show that Incremental‑OEDR maintains competitive report quality while substantially improving report continuity and reducing research costs. As shown in Figure~\reffig:profile, it achieves up to 0.51 higher content‑level ROUGE‑L F1, 0.63 higher outline‑level EM F1, 33% lower token consumption, and 61% fewer search calls than OEDR on DeepResearch Bench. For more details, please refer to our project page: https://ioedr‑project.github.io/.
Authors:Yisen Gao, Yue Guo, Qing Zong, Yiwen Guo, Yangqiu Song
Abstract:
Large language model agents can invoke tools fluently, but enterprise workflows demand more than selecting the right tools: actions must strictly comply with organizational policies, tool feedback often conceals hidden side effects under partial observability, and long‑horizon tasks require persistent state tracking across multiple records. To address these challenges, we introduce E‑Ledger, a multi‑agent harness for safe and persistent execution. E‑Ledger employs a code approval layer that checks every proposed action against policy before execution, and maintains a world ledger of verified hidden rules alongside evidence‑backed dynamic state. Because hidden rules are typically unknown a priori, we further propose WorldAbduct, an abductive, world‑model‑driven harness evolution framework. WorldAbduct diagnoses execution trajectories across four complementary views (state consistency, world‑observation gap, policy‑gate correctness, and goal judgment) to hypothesize latent rules, and verifies them through targeted abductive interactions before integrating them into the ledger. On the enterprise benchmark World of Workflows, E‑Ledger with WorldAbduct improves safe task completion across four LLM backbones, outperforming the strongest evolution baseline by 5‑‑15 percentage points. Experiments in ScienceWorld and DiscoveryWorld further show that abductive harness evolution carries over to scientific environments. Our code is available at https://github.com/HKUST‑KnowComp/E‑LEDGER‑WorldAbduct.
Authors:Hanjun Kim, Chiyun Noh, Sangwoo Jung, Jaehyung Jung, Simon Boche, Cedric Le Gentil, Stefan Leutenegger, Ayoung Kim
Abstract:
Legged robots offer superior mobility in unstructured environments, but reliable operation in such conditions requires robust state estimation. To address the vulnerability of proprioceptive estimators in rough terrain, recent methods have incorporated radar to provide velocity measurements. However, their limited yaw observability still leads to drift, and failure‑aware fusion for adverse environments remains underexplored. In this letter, we present RAGNAROK, the first radar‑visual‑kinematic‑inertial SLAM designed for robust operation in challenging environments. It integrates slip‑ and rolling‑contact‑aware leg velocity estimation, a kinematics‑aware radar factor, and degradation‑aware image enhancement. We further incorporate a B‑spline‑based radar‑aided proprioceptive backbone, adaptive weighting, and online extrinsic calibration. Extensive experiments on public and self‑collected datasets demonstrate that RAGNAROK achieves robust performance under challenging conditions and outperforms state‑of‑the‑art baselines. The source code and dataset are available at https://github.com/hanjun815/RAGNAROK.
Authors:Shutong Zheng, Sijia Chen
Abstract:
Federated reinforcement learning (FRL) enables distributed agents to collaboratively train decision‑making policies, but its decentralized training process also exposes global policy learning to Byzantine manipulation. Existing poisoning attacks primarily focus on how to construct malicious updates, while trajectory‑level intervention timing remains largely implicit. In sequential decision making, however, where an intervention is applied can alter subsequent trajectories and learning signals. Through controlled experiments, we find that changing the selected trajectory states materially alters attack efficacy even when the malicious‑update construction is fixed. We therefore identify when as a distinct attack dimension and introduce the Viability‑constrained Behavioral Steering Attack (V‑BSA), which uses local policy uncertainty to select sparse intervention states and applies envelope‑constrained behavioral steering. Across discrete‑action benchmarks, V‑BSA achieves substantial degradation against robust aggregators and ensemble defenses with only a fraction of the interventions used by dense poisoning, while revealing task‑ and aggregation‑dependent boundaries. Overall, our results highlight intervention timing as a distinct dimension of sequential robustness in FRL. The code is available at https://github.com/Yodeesy/V‑BSA
Authors:Pengxin Guo, Shuang Zeng, Zonggen Li, Weiying Zheng, Mengting Liu, Liangqiong Qu
Abstract:
Large Language Models (LLMs) have shown strong reasoning capabilities when fine‑tuned with reinforcement learning (RL), particularly through Group Relative Policy Optimization (GRPO). However, existing GRPO methods assume centralized access to training data, which may not hold in practice due to privacy or regulatory constraints. To this end, we propose Fed‑GRPO, a federated GRPO training framework that addresses these privacy constraints by enabling collaborative reasoning training without sharing raw data, which leverages the reward statistics naturally produced during GRPO training as zero‑cost signals to guide aggregation, local training, and communication. Fed‑GRPO contains three reward‑signal‑driven mechanisms: (i) \emphsignal‑weighted aggregation that weights clients by their reward standard deviation, prioritizing clients with stronger learning signals; (ii) \emphglobal reward calibration that re‑weights per‑prompt objectives based on the local‑global reward gap, steering each client toward its relative weaknesses; and (iii) \emphadaptive sparse communication that allocates bandwidth based on the informativeness of each client's update. Extensive experiments on mathematical reasoning tasks demonstrate that Fed‑GRPO achieves the best performance among all federated methods, clearly outperforms FedAvg and approaches centralized training performance, while losslessly reducing communication by 32× and supporting up to 621× compression under tight bandwidth budgets with only graceful accuracy degradation. Our code is available at https://github.com/HKU‑HealthAI/Fed‑GRPO.
Authors:Lan Feng, Peter Karkus, Maximilian Igl, Julius Berner, Yuxiao Chen, Shuhan Tan, Alexandre Alahi, Boris Ivanovic, Marco Pavone
Abstract:
Video diffusion and flow models require many sequential evaluations, making generation computationally expensive. Few‑step distillation reduces this cost but poses a capacity allocation problem: a student must match the teacher's iterative generation with far less sequential computation. Existing trajectory methods ask the student to reproduce teacher transitions that are highly curved at high noise, which can exceed its capacity and degrade fine detail. We introduce Parametric Trajectory Distillation (PTD), which lets the student parameterize teacher trajectory segments as polynomials and learn from teacher guidance along its own predicted path. PTD is designed to let the learned curvature adapt to the backbone's predictive capacity, preserving motion and diversity. The curvature head is used only in training; inference keeps the original backbone architecture. On Wan2.1‑14B, four‑step PTD sets a new state of the art for trajectory distillation, significantly improving dynamic quality and naturalness over PDD, the best‑performing trajectory‑only method on this model, under the same training setting. On the 33B audio‑video MiniMax‑H3, LoRA‑trained PTD significantly improves diversity and naturalness over the state‑of‑the‑art LightX2V Turbo. Blinded human votes give PTD 55.1% and 63.4% preference shares against PDD and LightX2V Turbo. Project page: https://alan‑lanfeng.github.io/PTD/.
Authors:Chenghao Yue, Siming Xing, Shuran Liu, Angran Li, Yuanlong Zhang
Abstract:
Two‑photon calcium imaging is a standard tool for recording large neural populations in vivo, yet inferring spikes accurately across the growing diversity of calcium indicators remains an open problem. Existing supervised methods achieve reasonable in‑domain accuracy but generalize poorly to unseen indicators, because different indicators induce distinct fluorescence kinetics and signal statistics while existing architectures remain relatively simple generic temporal regressors without dynamics‑matched inductive bias. We propose SpikeSSL, a universal spike inference framework whose temporal backbone is a bank of bidirectional IIR state‑space layers broadly motivated by calcium dynamics. A multi‑modal conditioning encoder maps indicator identity, sampling rate, and trace‑level signal statistics into a global conditioning vector that modulates the backbone via Adaptive Layer Normalization, while a heteroscedastic variance head provides calibrated per‑frame uncertainty. On a benchmark with five fixed evaluation splits built from 33 public ground‑truth datasets, SpikeSSL achieves state‑of‑the‑art performance in both in‑domain and zero‑shot leave‑one‑indicator‑out settings. We also develop a biophysical simulation pipeline capable of generating paired fluorescence‑spike traces with systematically varied kinetic parameters, spike statistics, response nonlinearities, baseline drift, and noise. Using this pipeline, we synthesize approximately 11,000 simulated traces. Augmenting training with these data effectively closes the cross‑indicator domain gap and improves zero‑shot generalization. Code is publicly available at https://github.com/detimage123/SpikeSSL.
Authors:Jorge García-Carrasco, Sergio García-Carrasco, Alejandro Maté, Juan Trujillo
Abstract:
Large language models (LLMs) are increasingly deployed as coding agents that edit files, run builds and tests, inspect execution results, and repair software iteratively. Embedded firmware is a demanding target because correctness depends on closed‑loop behavior under sensing, timing, and safety constraints, not only on static source quality. Yet embedded‑agent evaluation remains limited and often emphasizes one‑shot synthesis or offline correctness. We present a benchmark for closed‑loop evaluation of embedded coding agents. Each task provides a plain‑text engineering description, constrained workspace, and visible build‑and‑runtime surface. The agent must translate requirements into implementation and self‑verification steps, then iterate until the required device behavior is achieved. The suite contains five embedded‑control tasks and four feedback scenarios: one‑shot generation, realistic self‑verification, CI‑style red/green feedback, and oracle‑style detailed feedback. The implementation targets simulated ESP32 firmware for reproducibility. We evaluate seven GPT‑family and Qwen‑family configurations across five tasks and four scenarios, with three repetitions per condition for 420 runs. gpt‑5.4 has the highest pass rate among evaluated configurations but does not saturate the benchmark; qwen3.5‑27B is the strongest observed local model; and smaller local models degrade sharply in pass rate and search efficiency. These results suggest that capable local embedded coding agents are emerging.
Authors:JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu, Xiao Cao, Zhenxin Zhu, Guang Chen, Hangjun Ye, Bing Wang, Yanye Lu
Abstract:
Video world models aim to generate explorable, 3D‑consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long‑horizon generation. However, these auxiliary memory pathways operate outside the model's internal generative dynamics, preventing the model from intrinsically learning when and what historical information should be retrieved. We propose to internalize memory retrieval into the generation process, allowing retrieval to emerge as an intrinsic behavior of the video world model rather than relying on an external memory system. Based on this principle, we introduce Learning‑to‑Retrieve (L2R), which repurposes the model's persistent internal state as a memory for historical context. A camera‑conditioned retrieval gate selectively accesses relevant historical information from this state, determining what to retrieve, while a retrieval trigger determines when to retrieve. We further supervise the trigger with a 3D re‑visibility signal, activating retrieval when previously observed content re‑enters the current view while otherwise preserving the existing context. Together, these components enable the model to intrinsically acquire memory retrieval behavior and incorporate relevant historical observations into generation without a separate retrieval pathway. Across multiple base models and camera‑revisit benchmarks, L2R improves long‑term scene consistency while eliminating the need for an external memory bank or 3D conditions. https://jkhu29.github.io/l2r
Authors:Yiwei Liu, Luwei Yang, Shunbo Lei
Abstract:
Many dynamical systems generate influences whose consequences are not fully exhausted in the realized trajectory at the moment they arise. Such consequences are often treated as absent, delayed or statically stored, leaving unclear how unrealized influence retains future relevance as the system evolves. Here we formulate causal‑fate dynamics, in which generated influence may be realized, remain latent, or be transformed by subsequent dynamics, and give an exact finite‑transport representation when the relevant maps are specified. A connectome‑constrained Caenorhabditis elegans model first motivates the biological hypothesis that unresolved inter‑neuronal influence may persist and contribute to later propagation; it does not establish such a mechanism in living animals. We next examine operational Internet routing, where a dynamically updated cross‑observer history retains predictive information beyond the current local route state. We then use the representation to construct a Transformer architecture that explicitly transports and selectively realizes latent contextual influence while retaining language‑modeling function. The three studies distinguish a model‑motivated scientific hypothesis, an observational phenomenon compatible with future‑relevant history and an executable construction for carrying unrealized influence through subsequent computation.
Authors:Hao Li, Jinye Zhang, Bobo Li, Mong-Li Lee, Wynne Hsu, Zheng Wang, Hao Fei, Min Zhang
Abstract:
Affective computing has progressed from categorical emotion recognition to open‑ended affective analysis with large multimodal models. Yet affective science describes emotion as an unfolding process shaped by appraisal, regulation, and social interpretation, which remains underexplored computationally. We propose TRACE, a cognition‑oriented framework that formalizes an affective episode through three interrelated stages: Condition, Affect, and Effect, integrating observable cues with cognitive factors such as internal stance and regulation of emotional display. Based on this formulation, TRACE‑Bench evaluates multimodal models in real‑world social scenes through five tasks spanning grounded affect recognition, regulation decoding, cause reasoning, effect reasoning, and full‑chain reconstruction, with 3,746 structured question‑answer pairs over 646 videos. A matched human‑model comparison reveals a substantial performance gap, while affect‑specialized models also generally lag behind general‑purpose MLLMs. Model outputs show recurring failures, including treating displayed behavior as genuine feeling and fabricating unsupported events during long‑chain generation. We further propose TRACER, a cognition‑grounded structured reasoning method that couples each inference with explicit premises from factual observations, cognitive appraisals, and established upstream conclusions, forming a traceable graph of intermediate and target conclusions. TRACER outperforms all evaluated model baselines on each of the five tasks. Project page: https://cogaffc.github.io/TRACE
Authors:Wentao Wang, Lifeng Han, Zining Ren, Hengyu Zhong, Guangyu Zou
Abstract:
Public mobile and wearable mental‑health datasets often provide summarized feature tables rather than synchronized raw sensor streams. In these releases, each anchor corresponds to a survey or label time and may combine phone or wearable summaries, prior symptom scores, demographics, clinical variables, and source‑availability indicators. We propose the Modality‑Conditioned Temporal Recursive Context Model (MC‑TRCM), which preserves each feature source as a separate token and incorporates missingness as part of the input context. Observed sources are encoded with values and missingness summaries, absent sources use learned absence tokens, dataset and task embeddings condition fusion, and a recursive prediction head refines each output over validation‑selected steps. We evaluated MC‑TRCM on six predefined endpoints from DepreST‑CAT and Prediction of Severity Change‑Depression (PSYCHE‑D) using participant‑level splits and validation‑only model selection. MC‑TRCM achieved the lowest mean absolute error on DepreST‑CAT Patient Health Questionnaire‑9 (PHQ‑9) and Generalized Anxiety Disorder‑7 (GAD‑7) severity, improving over the best tabular reference by 0.181 and 0.217 scale points. Classification endpoints showed task‑dependent behavior: MC‑TRCM matched the best rounded GAD‑7 category balanced accuracy, was numerically highest by 0.002 balanced‑accuracy points on PSYCHE‑D multiclass prediction, and remained close to the strongest references on PHQ‑9 category and PSYCHE‑D binary prediction. Ablations support Feature‑wise Linear Modulation, absence tokens, missingness projections, and recursive refinement, while calibration and feature‑source controls characterize endpoint behavior. Our code is available at https://github.com/Botwwt/MC‑TRCM.
Authors:Kai Ding, Yang He, Ruijie Quan, Yi Yang
Abstract:
World Action Models (WAMs) enable generalist robot manipulation by conditioning an action expert on representations from a pretrained video Diffusion Transformer (DiT). In closed‑loop control, the video DiT runs at every chunk to encode the current observation into layerwise key‑value (KV) pairs that the action expert queries. This prefill dominates the per‑chunk computational cost, yet existing training‑free accelerations leave it fully dense. We present WAM‑Cache, a training‑free framework that retains layerwise key‑value representations across chunks and recomputes only a sparse refresh set of tokens. Crucially, we find that the intuitive heuristic of refreshing visually drifted tokens plateaus far below the dense baseline, even with an oracle predicting ground‑truth KV drift. Downstream action accuracy is instead governed by where the action expert attends, not by what moved. WAM‑Cache therefore selects the refresh set by uniting the action expert's cross‑attention with visual latent surprise, complemented by a strict age bound that suppresses compounding error. On Fast‑WAM, WAM‑Cache cuts video DiT prefill FLOPs by 32‑42% across RoboTwin 2.0, LIBERO, and real‑world experiments, while staying within 0.7‑1.8 percentage points of the dense policy in simulation and 2.5 points on a real robot.
Authors:Chihun An, Ikbeom Jang
Abstract:
Multimodal Alzheimer's disease (AD) diagnosis benefits from integrating heterogeneous clinical, imaging, genomic, and biomarker evidence, but clinical cohorts frequently suffer from irregular modality missingness. Existing fusion methods often synthesize absent inputs, risking the introduction of artificial surrogates, or pool available signals into uninterpretable latent spaces. We present CoPoE (Disease‑Coordinate Product‑of‑Experts), a disease‑coordinate framework that maps multimodal evidence into a structured latent space partitioned into four distinct biological and clinical axes: genetic Risk, molecular Pathology, Neurodegeneration, and clinical Stage (R/P/N/S). Each observed modality parameterizes a diagonal Gaussian expert over the full RPNS vector, and a masked Product‑of‑Experts architecture fuses only the available modalities. Consequently, absent modalities add no factor to the fusion path, allowing the network to preserve a robust, decomposable posterior for any non‑empty modality subset without synthetic imputation in the RPNS path. Through extensive missing‑modality experiments on the ADNI dataset, CoPoE achieves the best all‑modality performance and the highest mean AUROC across all 15 observed‑subset evaluations among standardized missing‑modality fusion baselines under a shared non‑PET ADNI embedding benchmark, while substantially improving raw‑probability ECE, Brier score, and NLL. Furthermore, PET‑supervised probing shows evidence enrichment within the pathology (P) block under full modalities, with tau‑related signal retained even when direct fluid biospecimen inputs are withheld. Our code is available at https://github.com/labhai/CoPoE.
Authors:Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam
Abstract:
Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back‑project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel‑space diffusion decoder with a backbone‑agnostic latent adapter, DiT‑based denoiser, and support mask predictor. CRISP replaces video‑VAE and LiDAR‑native decoders alike while keeping the encoder and latent generator fixed. Across KITTI‑360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR‑native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero‑shot replacement improves FSVD by 15.5%, narrowing the sim‑to‑real gap.
Authors:Rena Gao, Yue Dai, Hao Guan, Shengxiang Gao, Wangyang Wu, Yixin Shen, Jey Han Lau
Abstract:
Personalized conversational shopping requires maintaining preference consistency over multi‑turn interactions, where users reveal constraints gradually. Existing approaches often rely on static profiles and do not explicitly control long‑horizon interaction behavior. We propose a multi‑agent, multimodal Retrieval‑Augmented Generation (RAG) framework that decomposes dialogue state tracking, recommendation retrieval, preference‑aware reasoning, and response generation, while integrating product metadata, product reviews, image‑derived descriptions, and user historical reviews. To evaluate interaction‑level quality, we adopt a trajectory‑level protocol with four dimensions: Global Preference Consistency, Cumulative Information Synthesis, Interaction Trajectory, and Tone Consistency. On an Amazon Reviews 2023 benchmark, retrieval‑enabled variants outperform a no‑RAG baseline on automatic trajectory metrics (average 4.82 vs. 3.74). In a small real‑user study (n=5), the Full variant achieves the highest mean overall rating (4.60 vs. 2.20 for Baseline), providing exploratory evidence that role decomposition plus user‑centric retrieval improves perceived personalization.\footnoteCode and dataset are available at: https://github.com/RenaGao/Multimodel_RAG_Indexing
Authors:Zidan Wang, Yaqian Li, Xiaokai Zhang, Kaiwen Long, Kun He, Hanpeng Liu
Abstract:
CLIP serves as a foundational vision‑language model and the de facto vision encoder for downstream VLMs such as LLaVA. Post‑training offers a lightweight route to refine CLIP, but recent work argues that the standard contrastive loss is unsuitable for post‑training due to catastrophic forgetting under small batches, motivating designs that abandon the contrastive objective in favor of distillation. We revisit this premise and find that, for the InfoNCE objective, the reported forgetting is driven primarily not by insufficient negatives but by an inappropriate magnitude of the contrastive temperature τ: with τ set sufficiently small, contrastive post‑training improves rather than degrades the pretrained CLIP, which we explain through the temperature dependence of the InfoNCE gradient. Building on this finding, we propose ComCLIP, a lightweight single‑epoch post‑training recipe that freezes CLIP's text encoder‑‑‑so the refined vision encoder is a drop‑in replacement with unchanged architecture and inference cost‑‑‑and trains the vision encoder with a properly‑tempered contrastive loss, an MSE anchoring loss against the original CLIP, and a relational distillation loss from DINOv2. Over multiple seeds, ComCLIP matches the self‑distillation baseline CLIP‑Refine on zero‑shot classification while significantly improving the transferability of visual features, measured by linear probing (48.99 vs.\ 42.28 on ViT‑B/16), and on ViT‑L/14 it also improves MMVP over CLIP‑Refine (24.20 vs.\ 19.01); CLIP‑Refine remains stronger on image‑text retrieval. Used as a drop‑in vision encoder for LLaVA‑1.5‑7B without re‑aligning the projector or LLM, ComCLIP yields no net change across 8 VLM benchmarks, i.e., the refinement does not break downstream compatibility. Code and models are available at https://github.com/showstarpro/ComCLIP.git.
Authors:Zhi Rao, Yucheng Zhou, Qianran Sun, Yiqing Huang, Longcan Yuan, Jiayi Hou, Chengwen Yao, Lin Cheng, Donghui Sun, Xiaoxin Chen, Jun Wan
Abstract:
Contemporary decoder‑only large language models (LLMs) have demonstrated strong capabilities across a wide range of domains. However, existing pretraining paradigms for gloss‑free sign language translation (SLT) are largely designed around conventional encoder‑decoder pretrained language models, which limits their direct applicability to decoder‑only LLMs. To address this limitation, we propose SignRAG, a unified framework combining hierarchical pretraining, target‑domain retrieval augmentation, and retrieval‑aware reinforcement fine‑tuning. Hierarchical pretraining first learns linguistically grounded sign representations and then jointly aligns the sign encoder with an LLM, mitigating cross‑modal optimization imbalance. For downstream adaptation, SignRAG complements parameter‑based fine‑tuning with a target‑domain retrieval gallery that provides instance‑specific translation cues. To ensure that retrieved contexts are used appropriately, we further introduce Retrieval Utility‑Guided Reinforcement Fine‑Tuning (RUG‑RFT), which combines translation‑quality and retrieval‑utility rewards to encourage beneficial retrieval use while suppressing harmful reliance. Experiments on multiple SLT benchmarks establish new state‑of‑the‑art performance. In particular, to the best of our knowledge, SignRAG is the first gloss‑free approach to outperform gloss‑supervised methods across all reported metrics on CSL‑Daily. Our code has been released at \hrefhttps://github.com/shahelaojieraozhi/SignRAGGitHub, together with models of different sizes to support future academic research.
Authors:Cary Chang, Jialin Zhou
Abstract:
AI services incur inference costs during execution, while revenue may arrive later. Changing API prices, limited upfront capital, and service failures can limit operators' ability to sustain or expand their services. Beyond reducing per‑request costs, operators need to plan future spending, fund execution before revenue arrives, and obtain compensation for specified losses. This requires clear agreements across services with different pricing and execution conditions. These agreements must distinguish rights to consume services from rights to receive payments, define obligations under uncertain costs and income, and specify which failures qualify for compensation and how much can be paid. We present TokenBank, a financial infrastructure that represents these commitments through structured contracts. It supports service‑consumption rights, agreements that settle API‑price differences in cash (forwards), financing through limited rights to future service revenue, and protection claims for specified service failures. Contracts specify participants, covered services, validity, ownership, fulfillment conditions, and settlement rules. Evaluation combines replay of 899,441 API requests, real model‑driven agent execution, and contract API tests. In a zero‑discount rising‑price resampling scenario, forwards reduce mean expenditure by USD 304.88 but increase its standard deviation from USD 1,152.45 to USD 1,190.82. A controlled replication with five portfolios per capital condition finds mean contribution differences between financing and self‑funding of +1.0635, ‑0.1406, and ‑0.2962 experimental USD under low, baseline, and ample capital, respectively. The evaluation distinguishes contract correctness from economic effectiveness under declared economic and failure assumptions; supplier invoices and commercial revenue are unavailable.
Authors:Chuanrui Zhang, Zaijia Yang, Duomin Wang, Lu Shi, Daquan Zhou, Ruihua Zhang, Ziwei Wang
Abstract:
Geometrically faithful and functional articulated 3D assets are essential for real‑to‑sim robot manipulation, where policies trained in simulation must transfer to physical objects. Recent mesh‑based methods learn to infer articulation from annotated 3D assets, but deployment remains challenging when real‑world objects fall outside the training distribution or their meshes are incomplete or corrupted. To address these limitations, we formulate articulated asset reconstruction as programmatic modeling grounded in partial geometric evidence and introduce USDCraft, a framework in which a pretrained LLM writes and revises executable programs for simulation‑ready articulated assets without task‑specific training. We propose source geometry analysis, which converts the source mesh into a metric textual description that distinguishes observed surface from unknown space, and iterative geometric rechecking, which re‑encodes each candidate in the same representation so that discrepancies point to program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance complete the modeling process, which produces articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. Experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real‑to‑sim‑to‑real robot manipulation.
Authors:Zirui Liao, Zhengxian Wu, Zhuohong Chen, Yunyao Yu, Xiaoyu Liu, Yifan Xu, Haoqian Wang
Abstract:
Large Language Models (LLMs) serving as long‑term dialogue agents require memory systems that support reliable reasoning over extended interactions. However, existing Retrieval‑Augmented Generation (RAG) frameworks typically treat memory as passive storage, making it difficult to distinguish source‑attributed beliefs from unattributed event/fact records and to connect evidence dispersed across sessions. We introduce CogMem, a cognitive memory architecture based on the PEC^2F (Person‑Event‑Concept‑Claim‑Fact) graph schema. Dedicated Claim nodes preserve the source and target of subjective statements, while Fact and Event nodes represent semantic and episodic knowledge. Dialogue turns are incrementally converted into provenance‑aware graph records, consolidated into higher‑level facts, and reconciled into temporally scoped Claim views when the same source provides conflicting updates. For retrieval, a rule‑based controller driven by LLM intent parsing composes four deterministic graph operators‑‑‑anchoring, traversal, intersection, and evidence grounding‑‑‑to reconstruct query‑relevant context. Experiments on LoCoMo and LongMemEval show strong performance, especially on multi‑hop, temporal, and knowledge‑update tasks. Ablations and a semantic‑collapse probe support complementary contributions from epistemic separation, consolidation, and agentic retrieval. Code: https://github.com/Silent‑Rain02/CogMem.
Authors:Kyeong-Rae Kim, Sungnyun Kim, Tae-Hyun Oh
Abstract:
While 3D spatial reasoning in dynamic egocentric environments is crucial for embodied intelligence, audio‑visual large language models (AV‑LLMs) lack explicit mechanisms to process and internalize global geometry directly from raw sensory streams. Existing approaches either require costly fine‑tuning or underutilize the model's cross‑modal reasoning capacities. In this paper, we propose FloorSAV, a novel framework that explicitly grounds spatial audio‑visual context by rendering a dynamic 2D floormap. By integrating 3D point clouds, camera trajectories, spatial audio cues, and semantically grounded object landmarks, we inject this floormap into the AV‑LLM as a synchronized stream with an egocentric video. AV‑LLMs utilize their multi‑modal capabilities to jointly reason over visual, auditory, and geometric cues in a single inference with floormap interpretation guidance. We further introduce SAVED‑Bench (Spatial Audio‑Visual Egocentric Benchmark with Dynamic Agents), constructing essential tasks of spatial capability in real‑world scenarios: dynamic relativity, regional, and path reasoning QAs. FloorSAV improves AV‑LLMs' spatial reasoning on various tasks from both SAVED‑Bench and SAVVY‑Bench. Studies with ground‑truth floormaps demonstrate the substantial potential of FloorSAV with accurate spatial information.
Authors:Yingge Hu, Gautham Ramesh Babu, Mostafa Milani
Abstract:
Blowfish privacy relaxes differential privacy (DP) by protecting only the attribute‑value substitutions a data owner specifies as edges of a policy graph. A sparser policy can reduce the noise required by a mechanism, but only when the released statistic changes less across protected substitutions than across arbitrary DP neighbors. We study this question for evolutionary, nearest‑neighbor DP synthesizers such as Private Evolution (PE) and its tabular instantiation Tab‑PE, which score private records against a candidate population and release a noisy vote histogram. Their hard vote is constant inside each candidate's decision region and jumps at its boundary. Its policy‑specific sensitivity therefore equals the full worst‑case value whenever at least one protected substitution crosses a boundary, regardless of how short that substitution is. Because every round we examined contained such a substitution, the policy graph gave no reduction in noise. We propose BF‑Soft, a temperature‑smoothed soft vote whose response changes gradually with distance. Its sensitivity has a tight closed‑form bound in the policy graph's reach and the temperature, independent of the number of candidates, and the bound can be computed once before synthesis. It also predicts from the policy alone when policy‑aware smoothing cannot substantially reduce noise: protecting a flat categorical or binary attribute drives the reach to its maximum. On real and synthetic datasets under narrow numeric policies, BF‑Soft reduces error relative to hard voting at strong privacy budgets, while the advantage reverses at weaker budgets. A public‑data pilot predicts when soft voting is beneficial without spending private budget.
Authors:Wei-Xiang Mao, Zhi-Kai Chen, De-Chuan Zhan, Han-Jia Ye
Abstract:
Agents combine reasoning with tools to interact with external systems and complete real‑world tasks. Early agents typically interleave reasoning and actions along a single execution chain. On complex tasks, this chain becomes unreliable because growing histories obscure intermediate dependencies and allow early planning errors to propagate. Recursively decomposing a complex task into smaller subtasks offers a natural solution, yet effective decomposition must account for the system's capabilities so that each subtask can be executed by the available tools. In realistic systems, however, tool libraries can be too large to expose in full. Injecting every tool description consumes substantial context while making relevant tools harder to retrieve and useful task boundaries harder to identify. We propose tool‑aware recursive decomposition, which organizes tools by functional relationships into a hierarchy of capabilities. During execution, the agent discovers tools on demand and uses the hierarchy to recursively decompose a complex task into a subtask tree whose levels are aligned with the capabilities required at each stage. Experiments on complex real‑world tasks show that the proposed method improves end‑to‑end task success rate by up to 40 percentage points over the compared baselines. The implementation of TaReD is available on GitHub: https://github.com/WeiXiang‑Mao/TaReD.
Authors:Kang Yang, Tianci Bu, Peng Wang, Deying Li, Yongcai Wang
Abstract:
Given the inherent unpredictability of packet loss in vehicular wireless communications, V2X collaborative perception can yield practical benefits only if agents can achieve reliable collaboration under lossy and low‑bandwidth communication conditions. Existing dense BEV feature fusion methods depend on redundant BEV feature exchange, which is infeasible in low‑bandwidth scenarios, while compact‑communication methods aggressively compress messages but can hardly recover the missing feature content after packet loss. In this paper, we present LR‑V2X, a loss‑resilient, latent‑space reconstruction framework that converts corrupted received latents (even under severe 90% packet loss) into a spatial prior and then reconstructs the missing BEV information from this informative prior and using ego context as condition. Notably, the model can be trained under complete communication conditions and can be directly applied to lossy conditions at test time, eliminating the need for training under numerous lossy conditions. Experiments on DAIR‑V2X and V2XREAL show that LR‑V2X delivers the strongest robustness under severe packet loss and preserves reliable collaboration as communication quality degrades. And it reduces communication overhead by 64× compared to dense BEV feature fusion baselines. Code will be released at https://github.com/sidiangongyuan/LR‑V2X.
Authors:Kyoungin Baik, Youngwoon Lee
Abstract:
Large‑scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real‑world data collection. While simulation offers a scalable alternative, its potential for sim‑to‑real VLA learning in mobile manipulation remains largely underexplored. We introduce SimVLA, an end‑to‑end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation. SimVLA is first pre‑trained on two complementary simulation‑derived datasets: SimAction, a large‑scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask‑level visual‑language supervision. We further post‑train SimVLA on a mixture of SimAction and SimDeploy, a dataset collected from policy rollouts across diverse simulated environments. We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero‑shot transfer to real‑world mobile manipulation, including real home environments. SimVLA outperforms policies trained on 50 in‑domain real‑world demonstrations, suggesting that simulation can enable scalable sim‑to‑real mobile manipulation. We further demonstrate the value of multiple complementary forms of supervision for effectively leveraging simulation in VLA training.
Authors:Lei Zhao, Qichao Zhao, Bowen Zuo, Qishi Zhan
Abstract:
On‑policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generation and mathematical reasoning, OPD with larger‑scale teachers exhibits early loss plateaus, with an average final loss reduction of 25.1% after 200 updates, compared with 96.2% for self‑RL teachers, obtained by further reinforcement learning (RL) training of the initial student. To understand this difference, we analyze OPD as an idealized continuous‑time dynamical system in the small‑learning‑rate limit. Our training‑log diagnostics associate these plateaus with an early decline in a gradient‑based learning‑signal proxy while substantial loss remains; these measurements do not establish why the underlying gradient weakens. We further prove a local recovery guarantee for teachers sufficiently close to the initial student in a shared parameterization under regularity conditions, offering a conditional explanation for the success of self‑RL teachers in our experiments. Across runs with and without loss plateaus, we observe small relative parameter changes (0.025‑0.098%) and high similarity between the student's representations before and after OPD (linear CKA >0.98 across layers). These observations suggest that limited representation adaptation may contribute to learning‑signal collapse, a hypothesis that remains to be tested. Code is available at https://github.com/leizhao7/opd‑learning‑signals.
Authors:Xie Zhang, Chengxiao Li, Xuan Liu, Chenshu Wu
Abstract:
Human body point clouds are a versatile representation for AI‑enabled human sensing. However, existing methods using LiDAR, radar, and depth cameras suffer from inherent drawbacks in high cost, sparse reconstruction, and privacy concerns, etc. In this paper, we exploit low‑cost thermal arrays and present TAP3D, the first system to reconstruct 3D human point clouds from body heat signatures, offering significant advantages in cost, density, human sensitivity, and privacy. To overcome major challenges in depth estimation, thermal interference, and multi‑person separation, we propose a novel physics‑informed design, which integrates a forward thermal physics model with two distinct modules: multi‑primitive estimation for self‑supervised joint recovery of depth and other thermal properties, and geometric perspective fusion for suppressing interference and disentangling multiple people. We implement TAP3D using a single commodity thermal array sensor and build a large‑scale dataset (160K samples, 8 environments, 11 users) for evaluation. TAP3D achieves remarkable accuracy for dense point cloud generation, enabling downstream tasks like fall detection (91.46%), indoor tracking (21.86 cm MAE), and human mesh recovery (4.87 cm error). By transforming body heat into point clouds for the first time, TAP3D pioneers a new paradigm for privacy‑first, fully passive human sensing for many applications. TAP3D is open‑sourced at https://github.com/aiot‑lab/TAP3D.
Authors:Volodymyr Shcherbyna, Linh Kästner, Duc Anh Do, Hoang Tung, Huu Giang Nguyen, Maximilian Ho-Kyoung Schreff, Tim Seeger, Eva Wiese, Ahmed Martban, Huajian Zeng, An Tran, Nguyen Quoc Hung, Jonas Kreutz, Vu Thanh Lam, Ton Manh Kien, Harold Soh
Abstract:
Building upon the foundations laid by our previous work, this paper introduces Arena 5.0, the fifth iteration of our framework for robotics social navigation development and benchmarking. Arena 5.0 provides three main contributions: 1) The complete integration of NVIDIA Isaac Gym, enabling photorealistic simulations and more efficient training. It seamlessly incorporates Isaac Gym into the Arena platform, allowing the use of existing modules such as randomized environment generation, evaluation tools, ROS2 support, and the integration of planners, robot models, and APIs within Isaac Gym. 2) A comprehensive benchmark of state‑of‑the‑art social navigation strategies, evaluated on a diverse set of generated and customized worlds and scenarios of varying difficulty levels. These benchmarks provide a detailed assessment of navigation planners using a wide range of social navigation metrics. 3) Extensive scenario generation and task planning modules for improved and customizable generation of social navigation scenarios, such as emergency and rescue situations. The platform's performance was evaluated by generating the aforementioned benchmark and through a comprehensive user study, demonstrating significant improvements in usability and efficiency compared to previous versions. Arena 5.0 is open source and available at https://github.com/Arena‑Rosnav.
Authors:T. Y. Tsui, Jiatao Gu, Lingjie Liu
Abstract:
Diffusion and autoregression (AR) have long been seen as different categories of generative models, with diffusion specialising in continuous fields and AR specialising in discrete tokens. Recent work seeks to combine the advantages of the two models, and each hybrid fixes its decoding schedule by design. In this paper, we ask whether the performance of decoding schedules of one model can be predicted before decoding at a fixed number of steps. We describe diffusion, AR, and models in between as paths on one corruption lattice, and define the cost of a schedule as the dependence its parallel steps discard. The cost shows that the fewest steps of a zero‑cost schedule are set by the geometry of the data, in the same way for tokens and for continuous fields. In particular, for data that are Markov on a graph and dependent along its paths, the fewest steps equal the graph's treedepth, which is logarithmic in the length of a sequence and linear in the side length of a grid. With fewer steps than the treedepth, every schedule pays a positive cost, whose ranking we predict before decoding with a kernel of pairwise dependence estimated from pretrained weights. Across text generation, image generation, and video generation, we verify most of the predictions about the rankings of different schedules under different metrics and benchmarks. This work therefore provides a design principle for decoding for future AR models, diffusion models, and anything in between. Our code is available at https://github.com/TSUITUENYUE/The‑Lattice‑of‑Transition‑Laws.
Authors:Fahd Seddik
Abstract:
Agent skills are SKILL.md instructions and scripts that AI coding agents such as Claude Code and Codex run with the permissions of their user. Developers share skills by copying them between repositories, which makes them a software supply chain without a registry, versions or provenance. The origin of a copied skill, the reach of a security fix and the repositories that warrant review are therefore unknown. Studies that record which repositories hold a skill at a single point in time cannot reveal who copied it from whom. We contribute the first dated copy network of agent skills, built from the git history of every SKILL.md in GitSkills and covering 2,193,119 skill adoptions across GitHub, together with an interactive viewer. A few repositories are the source of almost all copies, and GitHub stars do not identify them. Skill copies almost never change with their source, and a fix at the source therefore rarely reaches them. We fit a model of which repositories others copy from and use it to rank repositories for audit. Reviewing the 100 repositories it ranks highest prevents 14.9% of later adoptions of high‑risk skills, against 0.5% for the 100 most starred, which gives security engineers a short list to check before a skill spreads. Platforms should therefore distribute versioned references rather than copies. Project Website: https://fahdseddik.github.io/Skill‑Constellations/
Authors:Guo Cheng, Changlong Lv, Jingyi Hou
Abstract:
Multivariate long‑horizon forecasting is critical to electricity load scheduling and traffic flow management, and to financial risk control. Existing deterministic backbones output a single trajectory, masking heterogeneous prediction difficulty across horizons and channels and providing no localized reliability signal. We present CARE (Corrective branch with Aligned context and Relative‑error Estimation), a lightweight plug‑in that enhances any deterministic forecaster without architectural redesign. Operating in parallel with the base model, CARE resamples historical context to match the forecast horizon, learns residual correction patterns from this aligned history, and applies scale‑aware bounded updates modulated by per‑coordinate sigmoid risk gates. A multi‑objective loss jointly optimizes forecast accuracy, residual tracking, risk alignment, and base‑model anchoring. Across eight benchmarks with three representative backbones, CARE improves accuracy with marginal parameter and latency overhead. Its risk gates reliably identify high‑error regions: on Weather, the highest‑gate tertile exhibits nearly four times the error of the lowest‑gate tertile, offering planners an interpretable per‑step trust signal. Code is available at https://github.com/CG‑BNYC/CARE.
Authors:Jinbo Hu, Hang Su, Lichun Fan, Heinrich Dinkel, Gang Li, Zhanchen Dai, Yiru Zhang, Chang Liu, Peng Wang, Junnan Wu, Jian Luan, Cong Zou, Heng Qu
Abstract:
Large audio‑language models (LALMs) have achieved strong performance in general audio understanding, yet most are designed for monaural input and discard the inter‑channel cues essential for spatial perception. In contrast, existing spatial audio‑language models are purpose‑built for spatial tasks and fail to capitalize on the general understanding capabilities of monaural LALMs. We present MiDashengLM‑Spatial, the first open‑source end‑to‑end unified audio‑language model, to our knowledge, which supports both general audio understanding and spatial awareness within a single architecture. It extends MiDashengLM with a spatial audio encoder, Spatial‑Dasheng, integrated through a hierarchical semantic‑to‑spatial conditioning module that injects intermediate semantic representations into the spatial branch at multiple depths while preserving the original semantic pathway. To provide spatial audio‑language supervision at scale, we develop a data synthesis pipeline that renders diverse spatial acoustic scenes with scene‑level spatial descriptions and question‑answer pairs. Experiments show that Spatial‑Dasheng achieves strong performance on sound event localization and detection in real‑world scenes, and that MiDashengLM‑Spatial substantially outperforms existing LALMs on spatial understanding and reasoning benchmarks. Meanwhile, it remains competitive with state‑of‑the‑art 8B‑scale LALMs on diverse monaural benchmarks, demonstrating that spatial awareness can be acquired without compromising general audio understanding. The source code and model checkpoint are available at https://github.com/xiaomi‑research/midashenglm‑spatial and https://huggingface.co/mispeech/midashenglm‑spatial.
Authors:Hsing-Hang Chou, Yun-Shao Lin, Ching-Chin Sung, Chi-Chun Lee
Abstract:
Cochlear implants (CIs) restore speech access but weaken cues needed for vocal emotion recognition. Prior CI‑oriented enhancement requires parallel normal/strong recordings and intensity labels. We propose SEER, a retrieval‑based framework that learns which same‑emotion reference helps each source remain recognizable after CI processing. A source‑conditioned retriever learns CI‑aware utility from sampled emotional voice conversion outcomes, while uncertainty‑guided exploration avoids exhaustive pair evaluation; neither parallel recordings nor intensity labels are required. SEER improves Source macro‑F1 at N8 by 7.30 points on RAVDESS and 11.66 points on ESD, with significant ESD gains across N4/N8/N16. Sixteen‑listener RAVDESS gains are significant across all conditions. Exhaustive analysis finds an aggregate benefit from stronger references but little effect from matching gender or content.
Authors:Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng
Abstract:
Most human‑agent interaction today remains text‑based. Natural language can impose cognitive overload, ambiguity, information chaos, and slow input for complex tasks; ephemeral generative UIs can present structured information and guide users toward task completion. We propose GenUI‑Harness, a multi‑agent harness pairing a Tool Agent for information retrieval and task execution with a GUI Coder Agent that identifies ambiguities and generates front‑end code for structured interfaces. Training the coder with reinforcement learning is challenging: verifiable rewards for interactive UI generation require costly execution, while LLM‑as‑a‑Judge rewards are prone to reward hacking. We address the first challenge with Dynamic UX, a lightweight package for dynamic interaction and reward collection in a single sandbox, and the second with Reward Auditor, a meta‑reward mechanism that monitors reward distributions and distills diagnostic patterns into a shared rubric and scoring specification. We introduce UI‑TAU Bench, a benchmark for active human‑agent interaction through generated UI code, built on 10 real‑world domain databases constructed from public data sources and based on Tau‑Bench tool‑use settings, with Lite (300 tasks) and Full (1,000 tasks) splits. GenUI‑Harness achieves an average Pass@3 gain of 4.48 percentage points over smolagents on Lite. Training with GenUI‑Harness improves a 4B backbone from 9.33% to 58.00% Pass@3, outperforming larger frontier models such as Claude Opus 5 (46.67%). GenUI‑Harness also remains robust on ambiguous and non‑ambiguous queries. In a reviewer survey comparing communication channels, generated UIs reduce average dialogue rounds from 3.4 to 1.2. These results show that data‑aware generative interfaces can support effective task completion and reduce dialogue rounds in evaluated database‑backed workflows.
Authors:Zeyu Ye, Yanchun Li, Sibei He, Meng Xie, Hangtao Zhang, Xianlong Wang, Li Zeng, Jiahao Chen, Yichen Wang, Junhui Wang, Ziqi Zhou
Abstract:
Image‑generation models can now produce text‑rich, natural‑looking visual artifacts that are hard to distinguish from real‑world evidence, such as news reports and textbook pages. Yet, the same capability introduces a new risk: these models can just as easily fabricate visual misinformation. Even commercial models (e.g., GPT‑Image‑2) readily produce it. Curiously, we find that these models can recognize a claim as false when asked, yet still render that very claim as credible visual evidence. This discrepancy points to a blind spot in current alignment: safeguards judge what an image shows, not what it asserts; however, existing red‑teaming benchmarks target conventional harmful content, such as violent or explicit imagery, and say little about where the alignment boundaries lie for visual misinformation, especially in commercial models. To fill this gap, we introduce EpiReal‑Bench, the first systematic benchmark for evaluating visual misinformation risks in commercial image generators, comprising 10k false‑claim prompts and 10k corresponding generated images that span 10 real‑world claim categories and 10 credible visual formats. We further introduce EpiReal‑Attack, a skill‑guided black‑box optimization framework that uses Pareto‑based selection and multimodal feedback to identify commands that bypass alignment safeguards while preserving visual realism, textual legibility, and semantic fidelity. Experiments on four commercial models reveal that more than 70% of false‑claim prompts elicit images that faithfully depict the corresponding misinformation, and EpiReal‑Attack pushes this rate to 95%. Most worryingly, these models are only a click away, and their outputs are cheap to spread yet hard to disbelieve, leaving this dimension of alignment largely unguarded.
Authors:Jihoon Jang, Hyunju Park, Jebin Kim, Seokhie Hong, Suhri Kim
Abstract:
Post‑quantum TLS at an IoT gateway must protect many device connections without large increases in handshake delay, CPU cost, or network traffic. HQC provides code‑based diversity beyond ML‑KEM, but its computation and ciphertext sizes can increase these costs. We optimize HQC for x86 processors with AVX2, AVX‑512, and the Galois Field New Instructions (GFNI), and integrate the resulting implementations into TLS 1.3. We extend branch‑free Toom‑Cook/Karatsuba multiplication to AVX‑512, accelerate Reed‑Solomon decoding with GFNI, improve Reed‑Muller decoding, accelerate SHA3‑512 and fixed‑weight sampling, and port Frobenius additive FFT (FAFFT) multiplication to AVX‑512 + GFNI for HQC‑5. On an Intel Core i5‑1135G7 processor, our AVX2 implementation reduces decapsulation by 11.1‑21.5% over the fastest prior AVX2 results. Our AVX‑512 implementation reduces key generation by 24.9‑29.5%, encapsulation by 10.4‑11.1%, and decapsulation by 20.6‑26.1% relative to Cabral et al. across the three HQC parameter sets. We evaluate the effects of these implementations on TLS 1.3 handshake latency and server and client CPU costs, as well as the effects of round‑trip time (RTT) and bandwidth. In local loopback TLS measurements, our AVX‑512 HQC‑5 implementation reduces handshake latency from 3.22 ms to 2.92 ms compared with Cabral et al. On a constrained path with 1 Mbit/s bandwidth and an added RTT of 50 ms, the HQC‑5 handshake takes 243 ms, compared with 87 ms for ML‑KEM‑1024. This indicates that HQC public‑key and ciphertext sizes, rather than implementation speed, determine the remaining handshake cost on constrained links.
Authors:Haozhe Yang, Zhiyang Dou, Zekai Gu, Cheng Lin, Wenping Wang, Yuan Liu, Taku Komura
Abstract:
Generating high‑quality and controllable camera motion is essential for AI‑assisted cinematography, video synthesis, and 3D scene understanding. We introduce TKCAM, a Text‑ and Keyframe‑conditioned CAMera‑motion synthesis framework based on generative masked modeling. We represent camera dynamics using a 12‑dimensional kinematic feature comprising position, velocity, and a continuous rotation representation and discretize them into hierarchical motion tokens via a Residual Vector Quantizer (RVQ). A two‑stage masked transformer architecture then learns to reconstruct and refine these tokens, utilizing explicit self‑ and cross‑attention modules for multimodal conditioning. A central feature of our framework is sparse visual keyframe conditioning: users can provide free‑form text prompts together with RGB observations at selected timestamps, which provide temporally localized visual guidance for generating coherent in‑between trajectories. Furthermore, to advance evaluation standards, we curate RealEstate10K‑Cap, a large‑scale text‑camera dataset, and establish a cross‑domain benchmark with a Universal CLaTr Evaluator. Extensive experiments demonstrate that TKCAM surpasses recent state‑of‑the‑art baselines on Fréchet distance (FID), text‑motion matching scores, and retrieval metrics (R@K), while additional analyses evaluate temporal smoothness and cross‑domain generalization. Code is available at https://github.com/linearalgebrayhz/TKCAM.
Authors:Boa Jang, JunGyu Lee, Gwanho Lee, Jinwook Choi, Young-Gon Kim
Abstract:
Accurate segmentation of thin curvilinear structures is vital for various real‑world applications, from vessel analysis to road extraction. Yet their intricate geometry makes even minor pixel‑wise errors enough to break the global topology, and this structural fragility turns severe domain shifts into catastrophic failures. The difficulty is most acute under cross‑modality gaps, where the imaging process itself differs fundamentally between source and target. While test‑time adaptation (TTA) offers a practical source‑free remedy, existing methods adapt feature statistics and confidence, neither of which constrains connectivity, and thus degrade under such extreme gaps. To address this, we propose Skeleton‑Guided Progressive Test‑Time Adaptation (SGP‑TTA). Progressive Batch Normalization (ProgBN) shifts normalization from frozen source statistics toward current target estimates under a sample‑count schedule, so that the source‑target balance follows the stage of adaptation rather than a fixed coefficient. Consensus Skeleton Recall (CSR) then derives a structural target from geometrically aligned multi‑view predictions and updates only the BN affine parameters to preserve connected structures. Extensive experiments show that SGP‑TTA consistently outperforms existing TTA methods in topological connectivity, with the largest margins under cross‑modality shift. The project page is available at https://boa‑jang.github.io/SGP‑TTA.
Authors:Dominik Blain
Abstract:
Translating COBOL to a modern language can change what a program computes, and LLM translations carry no guarantee of equivalence. We present IRONPROOF, which parses COBOL into an intermediate representation, generates Python, encodes both as Z3 formulas over shared inputs, and emits either a machine‑checkable equivalence certificate (UNSAT) or a counterexample (SAT). On 2,345 COBOL files (GnuCOBOL tests, NIST CCVS85, open‑source collections, and programs we generated or wrote), 782 enter the checking path: 606 (77.5%) are proved equivalent, 101 are partially verified, none is refuted, and 75 fail inside our pipeline and stay in the denominator. On independently authored programs the rate is 52.6% (153 of 291), against 92.3% on programs we wrote. Of the 153 independent proofs, 37 cover a single execution of a program that reads input the encoder does not model, and across all 606 proofs only 14 quantify over an input that a proved output depends on. On two public business‑application corpora (AWS CardDemo and IBM GenApp), the encoder models no program end to end. An April 2026 LLM‑only baseline failed on 54 of 94 independently authored programs, counting translations our encoder could not verify. PIC‑bounded overflow detection flags an out‑of‑range output in 22.1% of the programs it can analyze, an upper bound. An earlier confrontation with GnuCOBOL 3.2.0 found 24 of 49 executable independent proofs disagreeing with the runtime on a variable in the proof's domain. A proof establishes that the generated Python computes what our intermediate representation says the COBOL computes, not equivalence to a COBOL implementation.
Authors:Deepak Sridhar, Yi Li, Kartikeya Bhardwaj, Shuangjun Liu, Taotao Jing, Yuan Li, Shuai Zhang, Jiancheng Lyu, Dashan Gao, Nuno Vasconcelos
Abstract:
Prompt learning is a popular method for adapting foundation models, but learned prompts are typically task‑specific and fail to generalize to new classes, domains, or compositions of tasks. In this paper, we introduce a Diffusion Meta‑Prompt (DMP) model , a framework that models the distribution of learned prompts using diffusion models. Given a repository of previously learned prompts, DMP is trained and sampled without access to the original task examples or task losses, and synthesizes new prompts conditioned on natural language task descriptions. To improve the sampling stability, we introduce a test‑time steering strategy for DMP, which uses the best training‑selected prompt in the repository as a latent anchor during diffusion sampling, without retraining the DMP or accessing test classes. DMP improves generalization across classification, retrieval and text‑to‑image generation tasks, supports concept composition and negative prompting without explicit training. It reduces storage and inference costs by over 90% compared to prompt retrieval methods. For composite classification, DMP achieves upto 2.0% average gain over prior meta‑learning methods across 55 pairs of datasets with gains as high as 8.5% on specific pairs such as Eurosat and Flowers. DMP also enhances cross‑task generalization with ~2‑9% improvement for hierarchical classification task. We further provide a theoretical guarantee bounding the expected task loss of prompts sampled from a DMP. Code is available: https://github.com/DeepakSridhar/dmp
Authors:Koya Sakamoto, Daichi Azuma, Shuhei Kurita, Naoya Chiba, Yusuke Iwasawa, Yutaka Matsuo, Taiki Miyanishi
Abstract:
Active robot vision requires controlling the camera to reveal task‑relevant information that is hidden from the current viewpoint. For example, determining what is inside a box may require raising the camera and looking down into it. For viewpoint‑dependent question answering, the challenge is to select camera motions that expose the visual evidence needed to answer the question. Although vision‑language models (VLMs) can interpret observed images, selecting such motions requires anticipating the usefulness of unseen views. We quantify this usefulness as answerability, a VLM's estimate that a view suffices to answer the question, and present Rendering‑Free Lookahead (RFL), a viewpoint‑selection policy that ranks candidate camera motions by predicted future answerability. RFL transfers visual lookahead from deployment to offline training. At training, a privileged teacher renders candidate future views in 3D Gaussian Splatting (3DGS) scenes and uses a frozen VLM to compute one‑ and two‑step answerability targets. Through two‑stage distillation, a student learns to predict these action values from the question, recent visual observations, and a candidate camera motion. At deployment, RFL uses these predicted values to select camera motions without rendering future views. On 377 E3VS‑Bench test episodes in unseen environments, RFL improves the mean judge score by 43% over a direct‑action baseline using the same VLM. These results support learning camera‑control policies from privileged visual lookahead for viewpoint‑dependent question answering.
Authors:Feng Wang, Zijie Li, Ceyuan Yang, Alan Yuille, Peng Wang
Abstract:
Recent image editing systems have achieved impressive semantic understanding, visual fidelity, and instruction‑following ability, while video editing remains substantially more difficult and costly. In this paper, we present a simple alternative to end‑to‑end video editing: instead of training a monolithic video editor, we transform a strong image editor into a video editor through anchor‑based generation. Our key insight is that video editing can be decomposed into two subproblems: editing a sparse set of keyframes and propagating those edits across time. Based on this observation, we propose Anchor‑based Video Editing (AVE), a two‑stage framework in which a powerful image editor first performs composed editing on selected keyframes, and a motion‑guided image‑to‑video diffusion model then generates the final video by treating the edited keyframes as fixed anchors. This design directly inherits the strengths of modern image editors while avoiding expensive end‑to‑end video editing training. Experiments on IVEBench and VIE‑Bench show that AVE achieves strong performance in instruction following, temporal consistency, and content fidelity. Further ablations reveal that final video editing quality is strongly correlated with the quality of the image editor, suggesting that future progress in video editing may come from stronger image editing foundations and lightweight transfer to video. Code is available at https://github.com/wangf3014/AVE.
Authors:Sota Sugawara, Yukihiko Okada
Abstract:
Multiple institutions may each hold their own private knowledge edits and wish to integrate them into a single large language model without sharing raw edit requests. Null‑space‑constrained editing methods such as AlphaEdit mathematically guarantee that each update leaves unrelated knowledge intact, while collaborative frameworks such as CollabEdit aggregate edits from multiple clients without data sharing. Combining the two appears trivial. However, we show that this naive combination fails structurally, and we identify its cause. Guided by this analysis, we propose FedAlphaEdit. To our knowledge, this is the first collaborative knowledge editing framework that aligns both local editing and the server‑side merging rule under a single null‑space principle for preserving existing knowledge. FedAlphaEdit builds on null‑space‑aligned merging, in which clients share projected statistics and the server provably recovers the result of editing everything in one place under a one‑shot idealization. Empirically, the proposed method repairs the collapse and brings edit success and preservation simultaneously close to the level of centralized editing across two architecture families. FedAlphaEdit thus lets institutions that cannot share raw edit data, such as hospitals and financial firms, jointly maintain a shared model that closely approximates editing all facts in one place.
Authors:Ian de Holanda Cavalcanti Bezerra, Vivek Trivedy, Lucas Pascotti Valem, Longin Jan Latecki
Abstract:
Image retrieval methods often rely on a single global semantic descriptor extracted from an image, e.g., the [CLS] token in vision transformers. However, trying to squeeze all the semantic information of an image into a single descriptor can hurt downstream retrieval performance, especially for fine‑grained retrieval tasks. In this work, we augment the semantic tokens in the newer visual transformers, the global [CLS] token and the four register tokens, with a carefully selected collection of spatial tokens, aiming to capture the spatial region representation that characterizes the contents captured in each of the semantic tokens. We leverage the DINOv2‑reg model, which includes register tokens that emergently learn object and part‑based representations. For each "cue" token ([CLS] and each register token), we find a "buddy" image patch token and extract an N x N patch region to produce a set of localized ROI tokens. Our approach automatically captures important regions of interest without any external bounding boxes or saliency modules, purely by matching semantic tokens with their spatial representation regions. Furthermore, we incorporate these tokens into a multi‑vector retrieval framework inspired by ColBERT, enabling fine‑grained matching via a per‑token alignment mechanism while avoiding the large storage cost of keeping all patch embeddings. Through extensive experiments, we find that (1) register tokens encode useful fine‑grained details that can complement the [CLS] token; (2) automatically pooled ROI tokens further improve fine‑grained discrimination; and (3) multi‑vector retrieval with a small set of tokens improves over a DINOv2‑reg single‑vector baseline while remaining tractable for large‑scale search. The code is available at https://github.com/IdhcbIan/Augmenting_CLS_with_ROI_tokens.
Authors:Youngmin Oh, Jihwan Won, Yuntae Park, Bosun Hwang, Suwan Kim
Abstract:
Analog transistor sizing, finding design parameters that simultaneously satisfy multiple performance specifications, is a labor‑intensive bottleneck in circuit design. To support analog circuit experts, various automation methods have been proposed, including Bayesian optimization (BO), reinforcement learning (RL), and others. Yet existing methods are oblivious to the topological structure of a feasible design space, which can fragment into disconnected regions due to operating‑regime transitions, conflicting specification trade‑offs, and nonconvex device physics. This topological blindness causes the optimizer to converge within a single feasible region while missing others that may contain superior designs. To address this limitation, we propose ATLAS. a BO framework utilizing Topological Data Analysis (TDA). At each iteration, a Mapper graph is constructed over a surrogate‑predicted feasible region to estimate connected regions, enabling topology‑aware exploration from the very first iteration without any observed feasible points. A topological sensitivity score classifies candidates as bridge, frontier, or interior points, injecting a targeted exploration bonus into the acquisition function. Experiments on four analog circuit benchmarks in the GF180 and SKY130 processes demonstrate that \coin finds feasible designs with significantly fewer simulations than baselines, including RL and BO methods. To the best of our knowledge, this is the first work to apply topological data analysis to analog circuit design automation. The official implementation is publicly available on https://github.com/youngmin0oh/atlas.
Authors:Gaston Besanson
Abstract:
When a field that an authority contract needs exists only in unstructured evidence, a model can sense it. We admit the model's output only as an observation record with a score. An admission policy, with thresholds fitted on a held‑out split at a declared false‑positive ceiling, maps each score to true, false or unknown. Unknown denies. A deterministic, sufficiency‑checked contract decides. The probability that sensing changes the verdict is bounded by the sum, over the contract's sensed fields, of the admitted‑wrong and unknown rates. This is an instantiation of union‑bound reasoning, indexed by the contract. Minimising the estimated bound is a valid cost model for choosing among sufficient contracts. In a registered study on two constructed domains with two sensor families (36,000 model calls), no cell refuted the bound. Deny‑to‑allow changes from sensing appeared for the first time in this programme: 13 of 21,000 test verdicts, all from 3 contradictory records; each flip in a cell with a registered bound lay under it. Sensing‑aware selection picked the lower‑exposure contract in 4 of 4 registered tests. Both sensors' scores were informative but not calibrated. Correctness is relative to the declared loss model, candidate representation and reachable states; all domains are constructed.
Authors:Biao Xiang, Ali Eshragh, Yuexing Li, Kai Wang
Abstract:
Off‑policy evaluation (OPE) estimates the value of a target policy from logged data, but limited behavior‑policy coverage can force high‑variance reweighting or reward‑model extrapolation. Counterfactual annotations can add evidence about unobserved actions, yet practical sources, including domain experts and large language models (LLMs), may be costly, biased, or noisy. We study budgeted acquisition of such annotations for contextual‑bandit OPE. Given source‑specific costs and error profiles, we formulate an integer allocation problem over context‑action pairs and annotation sources to minimize the component of estimator variance that depends on the annotation plan. We characterize when annotations are valuable through a first‑annotation threshold and local annotation‑value regimes. For the coupled multi‑source problem, we develop a majorization‑minimization algorithm with dynamic‑programming subroutines that monotonically improves the objective. Experiments in synthetic clinical and LLM‑annotated education bandits show that our allocation method reduces fixed‑profile mean squared error (MSE) by 20.58% and 10.77%, respectively, relative to no annotation.
Authors:M. Mikail Demir, M. Abdullah Canbaz
Abstract:
Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supported by the cited source. We introduce PARCEL, a benchmark for checking whether a legal claim is supported by the underlying authority. Using recent New York State Court of Appeals decisions, we build a dataset of 3,396 parenthetical‑style claims labeled as Supported, Refuted, or Not Found. We cast this task as a three‑way natural language inference problem and evaluate several state‑of‑the‑art LLMs in a zero‑shot setting. Although the strongest models reach up to 0.97 accuracy, the results also show an important weakness: models still incorrectly mark unsupported claims as supported, even when the full opinion text is provided. Across models, missing support is harder to detect than direct contradiction, and fabricated but plausible citations cause the largest drop in performance. Overall, PARCEL provides a practical benchmark for testing claim‑level groundedness in legal RAG systems.
Authors:Vincenzo Guarino, Emanuele Musumeci, Vincenzo Suriani, Daniele Nardi
Abstract:
Agentic AI based on Large Language Model generalization capabilities offers a wide range of potential applications, including planning for embodied tasks. For example, embodied agents based on Foundation models can generate plausible plans in autonomous robotics scenarios. Due to limited context windows or hallucinatory phenomena in the next‑token prediction formulation, behaviors may be generated without establishing whether the deployed robot and the observed environment actually support the requested operation, in what we call a "grounding failure". Thanks to the recent improvements in reasoning capabilities of foundation models, autonomous robot behavior generation problem can be formulated as a code generation problem. We present iAm.md, a Markdown standard and generation framework, that allows anchoring this process in complementary forms of deployment evidence. Through open‑vocabulary semantic mapping, we combine local vision‑language detections and object segmentation and refer them to persistent object records in this intermediate standardized representation, allowing agentic introspection. We then study this new technique on a simulated TIAGo, on navigation‑and‑manipulation tasks, showing how this standardized representation jointly supports skill self‑assessment and executable task generalization.
Authors:Xi Chen, Yachi Zhang, Linghao Chen, Minghua Liu, Hao Su, Zexiang Xu, Xiaoshuai Zhang
Abstract:
We present LVSPM, a generalizable model that jointly estimates camera poses and synthesizes novel views from uncalibrated image collections. Trained with only RGB images and pose supervision, LVSPM avoids dense 3D ground truth and employs test‑time training (TTT) layers to scale seamlessly to hundreds of input views. On RealEstate10k, Co3Dv2, and DL3DV, LVSPM surpasses VGGT in pose estimation across 16‑256 views, with especially large margins at strict thresholds. For novel view synthesis under a practical protocol where more views cover larger scenes, LVSPM achieves state‑of‑the‑art pose‑free quality‑‑‑surpassing even pose‑dependent models in PSNR‑‑‑and still maintains high quality as scene scale grows, while baselines collapse. The code is available at https://burningdust21.github.io/Projects/LVSPM .
Authors:Fan Wang, Hubert Wagner, Rezaul Chowdhury, Chao Chen
Abstract:
In recent years, persistent homology has seen rapid adoption in deep learning, yet its computation remains a major bottleneck in network training. This paper introduces TopoGPU, a GPU streaming pipeline that computes persistence diagrams of cubical complexes induced by 2D and 3D images. TopoGPU streams the input image chunk by chunk, processing each chunk with massively parallel GPU kernels on a grid of GPU blocks; the resulting boundary relations are accumulated in host memory, where the CPU performs the boundary matrix reduction. TopoGPU introduces a stratification‑aware discrete Morse matching that provably preserves persistent homology under streaming, together with a parallel topological sorting algorithm and a parallel V‑path parity algorithm for deriving Morse boundaries on the GPU. TopoGPU outperforms Cubical Ripser, a state‑of‑the‑art method for persistent homology computation, on every benchmark evaluated, achieving an average end‑to‑end speedup of 53.24x and a maximum of 198.01x. We further integrate TopoGPU into a topology‑preserving deep network, demonstrating that it substantially reduces the cost of persistent homology computation during network training. TopoGPU is open source, with pre‑built binaries, Google Colab notebooks, and Docker images available at the project's GitHub page: https://github.com/seravee08/GPU‑Computation‑of‑Persistent‑Homology‑for‑Image‑Data.
Authors:Yue Zhao, Xiyang Hu, Zuobin Xiong, Zhangyu Wang, Ruolin Li
Abstract:
Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire tasks, with and without grounding? Bare means a model receives the task input alone. Grounded means it also receives one task‑specific addition: for smoke detection, a smoke‑free reference frame from the same camera. AI4Fire runs six core models bare and grounded on five wildfire tasks, zero‑shot; a sweep adds 29 more. Our literature search on fire tasks found 138 works; none combines this roster, task coverage, and paired bare and grounded runs. We report three findings. (1) Grounding helped most where the addition carried the answer: a read‑only SQL tool lifted every core model's database accuracy from at most 16 to at least 88 percent. (2) Simple rules were hard to beat: no core model outperformed repeating today's staffing count, and two open‑weight models mostly copied the median of similar earlier fire‑days, a worse forecast. (3) Public releases carry hazards: a fire‑danger column separates the holdout perfectly, and 67 aerial fire frames carry smoldering or fire‑free labels read from a clipped thermal maximum. We release prompts, responses, scores, code, and the survey record.
Authors:Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart, Igor Gilitschenski
Abstract:
Offline goal‑conditioned reinforcement learning (GCRL) has produced a diverse set of goal‑reaching algorithms, yet no single algorithm performs best across environments, goals, and even different phases of the same task. Rather than deploying only the best‑performing policy, we ask whether a set of frozen goal‑conditioned policies can be used collectively as a portfolio, deciding at every state which policy should act. Choosing a policy at each state is not straightforward. The policies' own value functions cannot be compared directly: they may use different scales, and some policies have no value function. We need to judge each policy by the states it is likely to reach, even though we can execute only one policy at a time. We also need to avoid switching so often that control becomes unstable. To address these challenges, we introduce World‑Model Policy Arbiter (WMPA), a test‑time framework that, given a set of frozen policies as input, rolls out each frozen policy in a learned state‑space world model, evaluates the imagined futures with a shared goal‑conditioned value function, and executes the highest‑scoring policy for a short commitment interval before the next round of arbitration (policy selection). WMPA assumes access to a bank of frozen goal‑conditioned policies and requires neither policy retraining nor privileged task‑specific knowledge. Under the official OGBench evaluation protocol on 18 state‑based datasets spanning maze navigation as well as cube, scene, and puzzle manipulation, WMPA improves the macro‑average success rate from the 44% achieved by the best policy selected per dataset to 58%, with statistically significant gains on 12 datasets. These gains include +33 percentage points on cube‑double‑play and +36 percentage points on scene‑play.
Authors:Ting Mao, Yanming Shao, Ziheng Wang, Haoyu Liu, Yiqun Wang, Xuanye Wu, Yao Mu
Abstract:
Monocular videos of human manipulation provide abundant dexterous demonstrations, yet reconstructing hand‑object interaction from a single view and transferring it to robot hands remain difficult, limiting their direct use for robot execution. Prior methods either require task‑specific RL training, limiting scalability, or assume clean motion‑capture trajectories and thus cannot operate directly on video. We present OmniHOI, a pipeline that turns an RGB video of hand‑object interaction into an interaction‑faithful trajectory on dexterous hands. The key idea is to enforce physical consistency using the evidence available at each stage: image evidence during reconstruction, contact geometry during retargeting, and dynamics during physics‑in‑the‑loop refinement. Each stage optimizes the corresponding representation directly, correcting errors before they propagate downstream or must be absorbed by a learned policy. Across 150 motion‑capture trajectories transferred to each of five dexterous hands with 6 to 22 DoF, we achieve 39‑89% success, compared with at most 31% for prior transfer methods. On 60 monocular video clips, we achieve 53% success, compared with 28% for the best prior video‑to‑robot pipeline. Its trajectories also execute on a real bimanual robot across diverse tasks.
Authors:Erik Skalnes, Layne C. Price, Raviteja Anantha, Michael Oberst
Abstract:
Instance‑specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale, prompting research into the generation of rubrics with large language models (LLMs). However, even when expert rubrics are available as references, it is unclear how to productively evaluate the quality of generated rubrics at scale. In this paper, we introduce a method for the evaluation of rubric generation, which we call Interventional Transfer (IT), based on the idea that two rubrics are similar if they move together when a response is perturbed to pass/fail one of them. In contrast to existing approaches for evaluation of rubric generation, we argue that different forms of interventional transfer can be used to evaluate the utility of generated rubrics for different tasks. For instance, we apply this approach in a case study on HealthBench, where we demonstrate an asymmetry in rubrics generated by Qwen3.8‑27B, Deepseek‑V4‑Flash, and Opus‑5, used to evaluate responses from GPT‑5.6‑Terra. Perturbations that degrade responses according to the generated/expert rubric transfer into lower scores on the corresponding expert/generated rubric, but perturbations that improve on one rubric do not reliably transfer into higher scores on the other. We argue that this finding has implications for the usage of LLM‑generated rubrics for performance monitoring and hill‑climbing. We contrast our approach with existing approaches for rubric evaluation, which do not surface the same asymmetry that we observe.
Authors:Gengze Zhou, Yicong Hong, Jiazhao Zhang, Xunyi Zhao, Jian Zhou, Zixing Lei, Zun Wang, Chongyang Zhao, Xionghui Chen, Stephen Gould, Anton van den Hengel, Qi Wu
Abstract:
Language models trained with long‑horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, raising the ceiling on what an embodied agent can understand and decide. Physical interaction, however, remains the domain of action policies, which provide dense, low‑latency control. We present NavGPT‑3, a harness that connects the two models, with an OS‑like runtime built above it: reasoning, acting, and monitoring run as threads with their own context, tools, and permissions, while the runtime schedules them and decides which thread controls the robot's motion, so that the robot can react to sudden real‑world events through interruption and thread switching. Beneath it, our action policy NavGPT VLA, trained on 19.28M examples, allocates visual tokens using codec allocation, in proportion to scene change; its 8B model alone reaches 74.51 SR on R2R‑CE and leads RxR‑CE with 78.19 SR. With the complete harness, NavGPT‑3 sets the state of the art on R2R‑CE (81.51 SR) and, for the first time, brings an autonomous agent to human level: on RxR‑CE it matches human followers in success (90.43 vs. 90.4 SR) and path fidelity (78.47 vs. 77.7 nDTW) at 1 min 22 s per episode, versus roughly 3 min for a human. We comprehensively ablate the harness design and the interaction between the two models, showing how tools and the action policy shape the path from language‑model reasoning to physical control: when NavGPT VLA executes the route, the reasoning loop shortens and the system's minimum reaction time falls from 3‑19 s per language‑model decision to 0.5‑1 s per action‑policy step (1‑2 Hz). These results show that designing this embodied interface is central to connecting frontier language‑model intelligence with low‑level physical control. We will release all models, code, and evaluation records.
Authors:Bingfan Zeng, Zhisheng Chen, Chenbo Sang, Zhengwei Xie, Jinpeng Wang, Xiangchen Guan, Rui Qian, Zheng Lu, Jingwei Song
Abstract:
Long‑term agents face growing storage demands as they accumulate experience. World models capture reusable regularities that can reduce the information stored for each experience. We formulate the problem of memory allocation conditioned on a world model and introduce MemoWM, a framework that uses shared predictions to compress retained information and reconstruct omitted content. Its task‑aware allocation rule balances the expected impact of reconstruction errors against storage cost, retaining information with downstream value beyond the predictive prior. Across five long‑term agent‑memory benchmarks, MemoWM achieves 42.42% average answer accuracy, exceeding the strongest baseline by 2.62 percentage points, while reducing average experience‑specific storage by 53.9% relative to MIRIX, the most storage‑efficient baseline. Further analysis shows that stronger world models reduce per‑experience storage at comparable task quality. Accounting for model parameters reveals a trade‑off between shared model capacity and recurring storage costs, with the capacity that minimizes total storage increasing as more interactions are retained. Our code is available at https://github.com/Feld‑maxiu/MemoWM.
Authors:Tristan Wu, Daniel Chin, Liwei Lin, Junan Zhang, Gus Xia
Abstract:
Audio generation models can translate natural‑language descriptions into sound, but their outputs are typically waveforms. Their audio quality is constrained by audio compression, and their outputs do not readily support direct edits to notes, timbral parameters, or modulation relationships. We present AutoSynth, which represents MIDI performance events, fixed synthesizer parameters, and variable‑length modulation routes as a unified sequence for a synthesizer, and learns their dependencies with an audio‑conditioned autoregressive model. A single model supports both tasks. Given reference audio, the model directly predicts a synthesizer program; given text, it uses a pretrained audio generation model and converts the generated audio into a program. Training consists of two stages: supervised learning on large‑scale audio‑program pairs automatically constructed from a small set of native presets, followed by group‑relative policy optimization with a mixed reward combining semantic similarity, pitch‑related features, acoustic similarity, and sound usefulness. The pipeline requires neither paired text‑target‑program annotations nor a differentiable synthesizer. Experiments show that AutoSynth produces complete, editable synthesizer programs and achieves competitive results in both synthesizer inversion and text‑driven generation. Audio demos and source code are available at https://auto‑synth.github.io/.
Authors:Jiuming Liu, Jianing Li, Mengmeng Liu, Hongyang He, Hesheng Wang, Per Ola Kristensson
Abstract:
Scene flow can capture low‑level 3D motion displacements in dynamic scenarios. Early pairwise estimators relying on instantaneous two‑frame motion lack long‑term temporal correlation and also struggle with poor extrapolation ability in future prediction. Although some recent methods attempt to explore multi‑frame scene flow estimation in a sequence‑to‑sequence manner, they typically suffer from heavy computational overhead with increasing input frames and long‑horizon prediction degradation due to ineffective motion propagation. To address these problems, we propose a novel memory‑enhanced sequential scene flow pipeline, called MESSENGER. To sufficiently mine long‑term temporal dependencies naturally within consecutive sequences, a memory buffer is designed by explicitly storing multiple history flow estimates and latent states. For each input frame, the temporally stored flows and states are correlated and retrieved to predict the current initialized flow in a next‑frame forecasting manner. Furthermore, we develop an uncertainty‑aware reweighting module to filter unreliable retrievals and mitigate accumulated errors. Extensive experiments on nuScenes and Argoverse 2 demonstrate state‑of‑the‑art performance of our MESSENGER, reducing EPE3D by 71.6% on nuScenes and 67.7% on Argoverse 2 in long‑horizon future extrapolation. This superiority can be attributed to our designed autoregressive forecasting paradigm, which naturally forces the network to progressively learn the next‑frame distribution based on history observations. Code will be released at https://github.com/liujiuming123/Messenger.
Authors:Christos Plachouras, David Robinson, Marius Miron, Gagan Narula, Paul Laisné, Anthony L. T. Fine, Benno Weck, Ellen Gilsenan-McMahon, Diane Kim, Laura Hay Mack, Maddie Cusimano, Sara Keen, Lukas Rauch, Benjamin Hoffman, Emmanuel Chemla, Emmanouil Benetos, Johan Pauwels, Milad Alizadeh, Matthieu Geist, Olivier Pietquin
Abstract:
Bioacoustics and ethology encompass a wide range of audio understanding tasks, many of which stand to benefit from recent advances in large audio‑language models. However, progress in the field has so far been assessed on a narrow set of tasks, primarily centered on label‑centric biological category recognition, such as species and call‑type classification. In this work, we introduce BEANS‑Next, a benchmark grounded in a taxonomy of bioacoustics tasks spanning acoustic perception, biological category recognition, scene understanding, and in‑context learning. Using BEANS‑Next, we show that existing models exhibit limited performance beyond the task families emphasized by existing evaluations, constraining their usefulness for broader bioacoustic applications. To support progress on this broader task space, we also introduce ROOTS, a large‑scale training resource built from expanded curated real‑world data and previously underused behavioral and acoustic metadata, supplemented by audio‑derived information and scalable synthetic generation where labeling is insufficient. We demonstrate that training on this dataset yields substantial progress across all task groups of BEANS‑Next, moving audio‑language models closer to their potential as general‑purpose assistants for bioacoustics and ethology. To accelerate progress in the field, we open‑source our benchmark, dataset, and data pipelines.
Authors:Jan Dubiński, Anna Sztyber-Betley, Jan Betley, Owain Evans
Abstract:
In subliminal learning (SL), a teacher model passes on a trait to a student model by distillation on data semantically unrelated to the trait. So far, SL has been demonstrated for only a limited range of traits, including preferences for animals (e.g., owls) and malicious personas. These traits can also be elicited with simple prompts or with steering. Can SL transfer a wider range of traits, including more complex ones? If so, distillation might transfer subtle forms of misalignment (e.g., reward‑seeking, scheming, and secret loyalties) without detection. To this end, we test whether SL can transfer a novel capability: predicting the outputs of a randomly initialized MLP. After distilling on unrelated text, the student achieves substantial performance on the task, while falling short of the teacher. We find that a directly optimized steering vector matches SL in distribution but generalizes worse out of distribution. Next, we test whether SL can transfer backdoors. We finetune the teacher to answer in French when the prompt contains a female name, then distill on number sequences containing neither names nor French. The student partially acquires the backdoor, responding in French on 23.5% of prompts with female names versus 0.0% with male names. Finally, we test whether SL can transfer a propensity to hack in an agentic chess environment. We finetune the student on number sequences from a steered hacker teacher. The student hacks in 58.3% of episodes, compared with 10.9% for the unfinetuned model. Thus, we show SL can transfer capabilities, backdoors, and hacking propensities. The amount of transfer is sensitive to the setup. In several experiments, it is made stronger by using logit distillation or by restricting LoRA to the attention layers.
Authors:Zihao Sheng, Pei Li, Zilin Huang, Yen-Jung Chen, Yuhao Luo, Zhengyang Wan, Steven T. Parker, David A. Noyce, Sikai Chen
Abstract:
Work zones are critical yet hazardous components of transportation infrastructure, requiring carefully designed Transportation Management Plans (TMPs) to ensure safety and mobility. However, TMP preparation remains labor‑intensive and heavily dependent on practitioner expertise. This paper proposes a Large Language Model (LLM)‑assisted framework to automate TMP content generation, leveraging the WisDOT WisTMP system as the application context. The framework fine‑tunes multiple open‑source LLMs across different model scales and deploys them locally to ensure data security. To support model training, we construct a domain‑specific dataset from historical WisTMP documents by converting PDF files into structured question‑answer pairs in JSON format. Experimental results show that fine‑tuning significantly improves performance across standard text generation metrics. Further section‑wise and strategy‑level analyses reveal that, while LLMs achieve strong overall performance, they tend to over‑generate strategies and struggle to produce project‑specific justifications and accurate cost estimates. In addition, scaling from 7B/8B to 14B yields limited gains. These findings demonstrate the potential of LLMs to improve TMP preparation efficiency while highlighting remaining challenges in LLM‑assisted TMP development. The source code and demo videos will be publicly available at https://zihaosheng.github.io/TMP‑LLM/.
Authors:Kaisong Zhang, Haotian Fang, Junmeng Zhou, Hang Lv, Yulan Pan, Yanchao Tan
Abstract:
Large language models (LLMs) offer promising potential for next‑visit diagnosis prediction, owing to their ability to integrate longitudinal clinical evidence and reason over it in natural language. However, reinforcement learning for LLM reasoning commonly rewards each trajectory according to the correctness of its final answer. In next‑visit diagnosis prediction, multiple diagnoses can be simultaneously valid, but independently rewarding one diagnosis per trajectory does not distinguish repeated hits from coverage of different diagnoses. The policy can therefore concentrate on a few correct diagnoses, leaving others uncovered. Meanwhile, LLM tokenizers can split ICD codes into several generic tokens with limited clinical meaning, requiring multiple decoding steps to predict each diagnosis and hindering reasoning over a large disease vocabulary. To address both challenges, we propose CARing, a framework that represents diagnoses with compositional Semantic IDs (SIDs) and optimizes reasoning trajectories for multi‑label coverage. Concretely, we first encode ontology‑enriched disease semantics into compact SIDs through residual quantization, and ground the resulting SID tokens in natural language and longitudinal EHR contexts through multi‑task alignment and reasoning‑enriched training to unlock transferable LLM reasoning. CARing further improves unordered multi‑label prediction through a coverage reward for reinforcement learning and multi‑positive supervision. At inference time, the model supports both efficient direct constrained decoding and multi‑chain reasoning with rank fusion. On MIMIC‑III and MIMIC‑IV, CARing exceeds all EHR‑trained baselines in weighted F1 and attains the highest top‑k recall at every reported cutoff, including R@30 of 46.04% and 46.52% in reasoning mode. Our codes and logs are available at https://github.com/zmlxzyh/CARing‑Codes‑Logs.
Authors:Ravil Akhtyamov
Abstract:
Self‑improving LLM agents can adapt a credit pipeline to a changed rule, but an agent that rewrites itself destroys the artefact a supervisor reviews: a named change, a recorded test, an approval. We argue that self‑evolution is reviewable only if it is confined to the runtime harness (instruction text, tool‑call logic and primitive composition) while model weights stay fixed, so that every adaptation is a diff with a cause and a test attached. We give a dual‑loop engine built on that bound, with one admission gate that writes a hash‑chained record before deployment, and we measure the gate in simulation, with a simulated agent and a seeded‑search proposer rather than language models. Across three families of supervisory re‑interpretation at three severities, 10 seeds each, the gated loop admitted 144 of 7,449 candidate changes, none of which worsened error on held‑out history, and restored the false‑positive rate to the oracle level without raising missed flags in every low‑ and mid‑severity cell. With the gate replaced by the check an unbounded system applies (fewer errors visible in recent traces), the same loops admitted 309 harmful changes and left missed flags above 10% in 49 of 90 runs: false positives fell because the screen was loosened. Evaluated on pre‑shift labels, the gate rejected every candidate, so a re‑interpretation must be encoded as a rule that relabels history. Parametric and scope shifts were repaired locally, a structural one only by primitive replacement; at the highest structural severity the gate's fixed tolerance blocked the correct replacement in half the seeds. We map the mechanisms to the EU AI Act's provisions for high‑risk credit scoring and note that the April 2026 US model‑risk guidance excludes agentic AI from its scope.
Authors:Kevin Riehl, Shaimaa K. El-Baklish, Fan Wu, Anastasios Kouvelas
Abstract:
Deep reinforcement learning has achieved substantial performance gains over classical control approaches. Yet, a central challenge to learning in real‑world applications is acquiring costly samples. Kolmogorov‑Arnold Networks are a recently proposed architecture that can learn physical relationships in control problems effectively, with significantly higher parameter efficiency and interpretability when compared to Multi‑Layer‑Perceptron architectures. In this work, we systematically study sample‑efficiency using computational experiments, covering the Feynman dataset and the Gymnasium RL benchmark. The results show that similar performance can be achieved with 40% fewer samples using the Kolmogorov‑Arnold architecture, and that relative performance improvements up to 50% occur during the training process. The observed gains are robust to varying levels of noise in rewards. These results highlight the potential of the Kolmogorov‑Arnold architectures for more sample‑efficient reinforcement learning. Code: https://github.com/DerKevinRiehl/neurips26_kan_training
Authors:Krish Bakshi
Abstract:
Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision‑language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close‑up contents. We present WorldBench, a benchmark and judge for open‑ended, LLM‑generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code‑only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT‑6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench
Authors:William Guey, Rashik Jahangir, Pierrick Bougault, Vitor D. de Moura, Wei Zhang, José O. Gomes
Abstract:
When an assistant encounters a message for another AI, does it tell its user? Four fixed model‑provider deployments perform simulated source tasks in 1,280 ordinary‑note and 128 enhanced‑note sessions. Harmless and harmful messages have matched plaintext and ROT13 versions, with no‑message controls. Observers receive no decoder or decoded meaning; a requested reference code incentivizes inspection. Asking for reports increases rule‑detected notifications identifying another AI as recipient by 53.1 percentage points for harmless ROT13 messages and 54.7 for harmful ones. This is a joint inspection, recognition, and notification effect; missing‑response bounds are 38.3‑‑77.3 and 36.7‑‑78.1 points. Model‑based trace checks identify eleven ordinary plaintext cases where agents interpret the message but do not notify their user. Seven encoded omissions are verified with enhanced notes; ordinary encoded omissions remain unverified. Seven simulated filename disclosures coexist with accurate review‑status answers, and two answers use a planted false count. Interpretation, notification, and authorized task performance are distinct outcomes.
Authors:Jie Liao, Simeng Qin, Wenqi Ren, Wei Zhou, Junhao Wen, Ranjie Duan, Yang Liu, Xiaojun Jia
Abstract:
Agent skills combine instructions with executable resources, giving third‑party packages access to an agent's runtime. Existing skill scanners inspect documentation and visible source, but Python may execute a bundled bytecode cache with different behavior. We study this gap between inspection and execution through PyCache Trap, which pairs benign source with a substituted cache accepted by the loader and connects it to a task‑relevant invocation. Scanner‑guided rewriting changes the invocation wording while preserving the cache body, separating package admission from recognition of the concealed behavior. Across 100 skills and seven scanners, PyCache Trap achieves 94‑100% attack success, with no semantic recognition of the cache‑resident behavior. We propose execution‑aware validation (EAV) to connect inspected instructions, scripts, imports, and runtime artifacts in a typed execution graph. EAV combines grounded behavioral analysis with trusted reproduction of compiled artifacts. It detects all 100 evaluated source‑present cache substitutions and reaches 92.8% Recall at 10.0% FPR across five attack families and 200 benign skills. The results support checking the executable artifacts a runtime can select as part of skill admission, within the supported loaders and code‑object normalization. The code is released at https://github.com/leo0481/PyCacheTrap.
Authors:Linxuan Lu
Abstract:
Immersive VR180 video is increasingly produced with professional stereo fisheye cameras, yet public VR180 research resources are mostly collected from online platforms such as YouTube: already stitched, projected and compressed by unknown pipelines, and without lens calibration. We present a firsthand‑captured stereo VR180 dataset recorded with two Blackmagic URSA Cine Immersive cameras. It contains 1,211 samples ‑‑ 636 stereo video clips (2,220.8 s, mostly 90 fps) and 575 stereo stills ‑‑ each released as camera‑native Blackmagic RAW, separate‑eye native fisheye HEVC (8160x7200 per eye) and half‑equirectangular HEVC (7200x7200 per eye), together with the factory lens calibration, portable fisheye/half‑equirectangular conversion tools and AI‑generated scene and visual‑challenge annotations. Re‑encoding the released fisheye and half‑equirectangular renders with x265 over 24 clips, both eyes, four rate points and nine viewing directions, native‑fisheye coding needed more bitrate than half‑equirectangular coding at equal viewport quality for all 24 clips (median +38%), in every part of the field of view. Data: https://huggingface.co/datasets/lulinxuan/VR180 ; code: https://github.com/lulinxuan/vr180‑dataset‑tools
Authors:Ankush Checkervarty
Abstract:
Activation probes that monitor deployed language models are trained on synthetic conversations, and how many a probe needs is open. We trace learning curves over 10‑590 synthetic samples for three monitoring concepts, high‑stakes situations, replies harmful to a person, and replies that do not follow the user's instruction, on fourteen held‑out evaluation distributions and four probe models, varying the generator LLM and the prompt's detail. The need is set by what is monitored: probes for high‑stakes and harmful are within a few hundredths of their plateau from 80 samples on Gemma‑3‑27B‑IT, instruction probes need several times as many, and the ordering holds on three smaller probe models and on real samples (from dev set). Prior work advises spending a generation budget on breadth, more kinds of data, over depth, more of each kind. We read the depth a concept needs as the half‑gain size of a fitted curve, the number of samples at which half the gain is in hand. Concept and distribution account for 42‑45% of its variance, the generator, probe model, and prompt detail for under 10%. What sets the value of the half‑gain size is coverage, not per‑kind difficulty: the number of samples of its own kind a distribution needs to saturate. Every kind, one per evaluation distribution, has a median half‑gain size of 7‑11 own‑kind synthetic samples under all three concepts alike. What differs is how far samples of one kind transfer to the concept's other kinds, almost fully under high‑stakes, less under harmful, and least under instruction, which accounts for most of the gap between concepts on generated and real samples. Breadth therefore pays differently by concept: many kinds are necessary under instruction, where no kind covers another, and nearly redundant under high‑stakes, where one kind covers the rest. We release the evaluation suites, dev sets, and generated sets.
Authors:Szymon Kocur
Abstract:
Historical Polish is well documented as a language but annotated in machine‑readable form only to about a million words for the period this paper covers; the rest sits behind optical character recognition of variable quality. We present Wieszcz‑XIX, a corpus of 6.75 billion tokens (about 3.1 billion words) in 294,369 documents, most of them periodical issues, of Polish published from 1800 to 1918, assembled from Wolne Lektury and the Internet Archive by a pipeline that filters, deduplicates, audits for post‑1918 leakage and splits at the document level. It is over three orders of magnitude larger than the annotated corpus of the same period, and we quantify its defects: recognition corruption against a false‑positive floor, near‑identical duplication, which is removed, and post‑1918 leakage, which is excluded from the training corpus itself down to a known residue of 0.04 to 0.38% of its bytes, found in the transcribed source, so the published corpus is the trained one document for document. On a hand‑corrected sample the character error rate is 0.68% where the text is legible, and 45% of the sampled passages cannot be corrected. On it we train a ladder of decoder‑only models from 47M to 349M parameters from scratch, and measure their temporal boundedness. Against two modern Polish base models, one far larger, the 349M shows a crossover, as does the 107M against the comparator of its size: post‑1918 vocabulary costs them about 3.1 bits per byte more than period vocabulary, a gap the comparators do not show, and period vocabulary costs them fewer bits than it costs the comparators. Shown period text, the models keep its spelling and the comparators only partly. Adding parameters gains about twice as much as a second pass over the data. We release the corpus, code and weights. Content warning: the models reproduce period prejudice, including antisemitic statements.
Authors:Jan-Peter Franke
Abstract:
Tool‑using agents repeatedly carry observations whose useful content can be much smaller than their original payload. We study agent‑controlled forgetting: the acting model selects previously observed tool results, replaces each with a short note at its original position, and retains the exact original in a recoverable archive. A Python harness exposes batch archival and explicit recovery without task‑specific model training, while protecting user instructions and assistant messages from these operations. In an exploratory OpenTelemetry debugging case followed by an unrelated implementation task, the method ended with 231,951 provider‑reported prompt tokens versus 912,492 under retained history, used 50% fewer cumulative input tokens, and had an estimated API cost of USD 1.28‑1.44 versus approximately USD 4.38. Both arms passed the two‑case primary behavioral oracle; neither fully satisfied the follow‑up evaluation. The method made more requests and took 17% longer. A contrasting application‑development pair produced no context or cost saving, and an earlier continuation exhibited lower manually assessed quality despite reduced context. These observations demonstrate substantial resource savings in noisy tool‑use trajectories and identify workload dependence as a central consideration for reversible context management.
Authors:Zekui Xue
Abstract:
Dynamic‑point filters are routinely added to feature‑based visual SLAM, and several recent systems argue that removing dynamic features can leave too few static features in low‑texture regions. So far, these systems have been evaluated only on texture‑rich benchmark sequences. We present a controlled study that isolates this interaction. We render synthetic indoor sequences in which surface texture (four levels, quantified by FAST‑corner density and image‑gradient entropy) and scene dynamics (three levels) are varied factorially along identical camera trajectories, with stereo, RGB‑D, ground‑truth poses and dynamic masks. On this grid we compare ORB‑SLAM2 without filtering, with an optical‑flow and epipolar‑residual filter (FLOW), and with a multi‑view depth‑consistency filter (GEOM), and report trajectory error, tracking completeness and surviving static features over five runs. Because the masks give per‑keypoint ground truth, we also measure each filter's dynamic‑point precision and recall and its static‑feature false‑removal rate, so that mechanistic explanations can be tested directly. We do not propose a new filter. On 720 runs over 24 sequences, filtering helped mainly in the most dynamic cells; the benefit did not decline monotonically with texture, but at the lowest level filtering reduced tracking completeness, and ORB‑SLAM2 never initialised in static L3 scenes. Contrary to our hypothesis, GEOM discarded more static keypoints than FLOW (median FRR 6.8% vs. 1.5% for RGB‑D, 19.6% vs. 1.5% for stereo); its RGB‑D advantage tracked dynamic‑point recall and vanished in stereo mode. Data and code are available at https://github.com/felixxxue/texture‑dynamics‑slam.
Authors:Albert Gao, Bing Xue, Andrea Zanette
Abstract:
Multimodal large language models (MLLMs) often answer visual reasoning questions by relying on linguistic priors rather than task‑relevant visual evidence. Textual chain‑of‑thought reasoning can partially mitigate this issue by encouraging models to decompose visual questions into intermediate evidence‑seeking steps, but generating these steps autoregressively increases inference cost. Latent reasoning avoids explicit rationale generation, but existing approaches provide limited control over what intermediate states encode, making it difficult to impose separate supervision for planning, grounding, and evidence selection. We propose Structured Latent Visual Reasoning (SLVR), a training framework that bridges explicit chain‑of‑thought and latent reasoning by organizing multimodal reasoning into typed latent stages for planning, grounding, evidence selection, and reasoning integration. SLVR first trains the model to rely on the image by masking answer‑revealing text and contrasting the correct answer with visually plausible distractors. It then organizes reasoning into latent stages for planning, grounding, evidence selection, and integration, supervising each stage with the corresponding signal: plans, boxes, visual evidence, and final rationales. This gives latent reasoning an explicit functional structure while avoiding generated textual chains at inference time. Built on Qwen2.5‑VL‑7B, SLVR improves consistently across multimodal reasoning benchmarks, with absolute gains of +9.4 on MMVP and +14.2 on BLINK Relation, as well as improvements on V, MathVista, and ChartQA. These results suggest that structured latent supervision can improve fine‑grained visual reasoning without the decoding overhead of textual CoT. Project page is available \hrefhttps://bogao‑code.github.io/SLVR/here.
Authors:Yipeng Li, Ashutosh Hathidara, Jane Lo, Harshavardhan Abichandani, Gunraj Singh, Atin Ghosh
Abstract:
Tool‑calling agents have become central to enterprise AI, yet training and evaluating them at scale remains severely constrained due to business and legal restrictions on enterprise systems, data, and database schemas. Tabular data synthesis offers a natural alternative, but its effectiveness is fundamentally limited by structural validity and schema availability, while procedure‑based approaches yield the opposite weakness, typically lacking distributional fidelity without per‑domain authoring. We introduce Synthesis Through Simulation (STS), a schema‑‑free data synthesis paradigm in which an LLM agent generates data by executing operations against policy‑enforcing APIs within simulated enterprise environments. Because data is generated through the same environment that defines what is valid, STS guarantees structural validity by construction while decoupling validity enforcement from distribution modeling, allowing each to be addressed independently. The Generalist Populator (GP), STS's domain‑agnostic agent, addresses the remaining challenges of distributional fidelity and synthesis scalability: GP achieves 0.88 average marginal fidelity and 100% constraint satisfaction across all ten environments without access to DB schemas, while statistical synthesizers are inapplicable to seven due to necessary seed data requirements, and schema‑privileged agents fail 82% of trajectories on airline environment's tightly coupled workflows due to brittle task composition. We open‑source the full framework, all ten environments, and generated datasets at https://github.com/SAP/synthesis‑through‑simulation.
Authors:Yinling Zhang, Langchen Liu, Dongbin Xiu, Xueyan Zou, Xu Kuang, Mengdi Wang, Shilong Liu
Abstract:
Language‑model agents are increasingly asked to carry out open‑ended scientific research, yet their results are usually graded against a known answer, a rubric, or a language‑model reviewer, none of which can tell whether a new scientific model is valid. The AI Science Exam for El Nino‑Southern Oscillation (SciExam for ENSO) is a benchmark in which agents build low‑order stochastic models of ENSO, the dominant mode of interannual climate variability, from real observations. Within a six‑hour budget, agents process the observations, write their own diagnostics, which are then frozen, and develop a model using only these diagnostics as feedback. Hidden graders then test whether the model reproduces ENSO's statistics, recovers unobserved variables, and forecasts held‑out years, and score a published model in the same way. Across twelve agent systems, six produce models that score higher than the published model, mainly through better reconstruction and forecasting. The simplified forms of the stronger models are each compatible with one of the two competing explanations of ENSO's warm‑cold asymmetry, an open debate that the task never mentions. Controlled runs of the top system under varied information suggest that its scores do not come from recalling the dated observational record and that the information it receives shapes how it builds its model. SciExam for ENSO can thus evaluate agent research where no answer is known, and the results suggest that agents can already build competitive models whose structures bear on questions that scientists still debate.
Authors:Zihan Su, Junhao Zhuang, Yaowei Li, Siwen Lu, Haoran Li, Lingen Li, Haoyu Wu, Weiyang Jin, Songchun Zhang, Haoyang Huang, Chun Yuan, Zeyue Xue, Nan Duan
Abstract:
Autoregressive video generation requires denoising the current frames while writing their key‑value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We introduce Self Gradient Forcing Plus (SGF+), which assigns separate parameters to context writing and denoising while preserving their interaction through causal attention. Both roles are jointly optimized using the original generation objective without auxiliary losses, with context writing supervised through its contribution to future predictions. This simple change improves visual quality and long‑horizon consistency over the evaluated baselines in both framewise and chunkwise generation, without additional video training data or a longer training horizon. Trained on only 5s rollouts, SGF+ supports continuous generation for up to 24 hours without long‑video fine‑tuning. These results highlight role‑specific parameterization as an effective design principle for high‑quality autoregressive video generation and native long‑horizon extrapolation.
Authors:Maverick Morales, Tomáš Dominik, Vermut Gao, Katrina Shirey, Paulius Rimkevičius, Aaron Schurger, Uri Maoz
Abstract:
Monitoring the chain‑of‑thought of reasoning artificial intelligence (AI) models remains a key approach to detecting deception and other forms of misbehavior in such models. However, semantic chain‑of‑thought monitoring depends on reasoning traces being legible and sufficiently faithful to the underlying computations that produced the model's behavior, not to mention accessible. Moreover, there is increasing evidence that chain‑of‑thought outputs may soon become illegible or unfaithful, if they even remain accessible. Based on cognitive load theory, we investigate a lower‑bandwidth signal‑‑the number of reasoning tokens generated‑‑which does not require access to the content of the reasoning trace. Three reasoning‑capable large language models answered 210 multiple‑choice questions‑‑across analytic, descriptive, and normative reasoning types as well as moral and non‑moral domains‑‑under system prompts instructing them to respond truthfully, falsely, or without regard for truth. Across all three models, truth‑directed responses elicited fewer reasoning tokens than both lie‑directed and truth‑indifferent responses. In addition, in one model, untruthful instructions increased the likelihood of reasoning at all. These findings show that explicitly prompted untruthful response policies can produce robust group‑level differences in test‑time reasoning‑token use. While not yet establishing reasoning‑token count as a detector of spontaneous deception or general misalignment, our results are a proof of concept that it can serve as a simple, content‑independent black‑box candidate signal for differentiating untruthful from truthful model behavior when raw reasoning traces are unavailable or unreliable. Future work should test instance‑level detection rates, out‑of‑distribution generalization, learned deceptive policies, hidden objectives, and robustness under adversarial pressure.
Authors:Miao Shang, Yabin Wang, Xiaopeng Hong
Abstract:
This paper proposes a novel crowd counting approach, the Gaussian Density Splatting Network (GDSNet). Unlike methods that rely on conventional, grid‑based density maps and are sensitive to spatial resolution, GDSNet represents a crowd as a superposition of continuous 2D Gaussian primitives. Our approach is built upon two key contributions. First, a control‑point‑based fitting mechanism is introduced to structure the prediction of Gaussian parameters. A set of control points is adaptively allocated to define local regions, from which features are pooled to regress each primitive's parameters. Second, we adapt a differentiable Gaussian Splatting framework to the counting task by parameterizing each primitive with geometric parameters and a scalar density mass. This formulation allows the network to be trained end‑to‑end via spatial matching of differentiably rendered density maps, naturally providing both local density supervision and global count optimization. Extensive evaluations on four standard benchmarks show GDSNet consistently outperforms the state of the art. The code is available at https://github.com/infinite0522/GDSNet‑Gaussian‑Density‑Splatting‑Network.
Authors:Markus Gross, Andreas Greiner, Taehyoung Kim, Sivasubiramaniam Subbiah, Tomaž Cotič, Conrad Christoph, Oussema Dhaouadi, Simon Zieher, Surya Vijaya Kumar, Gordon Elger, Henri Meeß, Olaf Wysocki, Paul Spannaus, Daniel Cremers
Abstract:
We introduce MultiFly, a real‑world, low‑altitude UAV dataset for semantic perception across RGB, thermal, LiDAR, and radar modalities. MultiFly provides 17,272 synchronized samples from four suburban scenes with frame‑wise annotations for 15 semantic classes, together with calibration and GNSS‑RTK/IMU measurements. To avoid costly and inconsistent modality‑specific annotation, we propagate labels from only 115 manually annotated RGB images through shared geometric representations to all four modalities. This approach generates semantic labels for 17,157 additional RGB images, 17,272 thermal images, 840M LiDAR points, and 3.4M radar points. Transferred annotations achieve 89.93% average agreement with held‑out manual annotations, and 90.94% average semantic consistency across all six modality pairs. We further establish semantic segmentation benchmarks for all four modalities, revealing distinct architectural behavior for dense LiDAR and sparse radar data. Taken together, MultiFly provides a scalable foundation for multimodal aerial perception and, to the best of our knowledge, the first public real‑world low‑altitude aerial benchmark that combines consistent frame‑wise semantic annotations for RGB, thermal, LiDAR, and radar. Data at https://github.com/markus‑42/multifly.
Authors:Jingyu Li, Xiaoxiao Xiang, Yiwen Guo
Abstract:
Deploying a joint audio‑video diffusion transformer for real‑time, interactive generation normally requires two essential modifications: block‑autoregressive attention, so frames can be emitted before the whole clip is finished, and few‑step sampling, so each block is cheap. Conventionally, the streaming video literature obtains both capabilities from a chained pipeline. It first distills a bidirectional teacher into a causal student, then into a few‑step one, or proceeds in reverse order. Each stage of such a chain fine‑tunes the weights the previous one produced, so a later objective can undo an earlier capability. Following the idea of model merging, we show that on a packed audio‑video backbone the two capabilities can be acquired in parallel. A causal adapter is trained against the frozen backbone, and an off‑the‑shelf few‑step adapter provides the few‑step capability. As the two edit different functional axes, we predict, and then verify, that their weight‑update directions are near‑orthogonal, without any explicit orthogonality constraint during training. Orthogonal updates should combine without interfering, so parallel composition is a direct sum. The two adapters are simply added at inference, with no joint training, yielding few‑step, streaming audio‑video whose image quality tracks the bidirectional teacher. Compared to the chained baselines, the composed model matches or beats them on most metrics, making parallel composition a practical approach. The resulting streaming system generates joint audio‑video in real time, \approx26 fps at 480×832 without quantization, and sustains 30 s of continuous generation with stable image quality. Demo Page: https://pac‑demo‑2027.github.io/demo/
Authors:Hui Wei, Hao Yu, Hui Kuurila-Zhang, Guoying Zhao
Abstract:
Face de‑identification (FDeID) has emerged as a critical privacy‑preserving technology, yet its evaluation remains fundamentally fragmented. Existing protocols rely on inconsistent metrics, heterogeneous datasets, and partial annotation coverage, so methods targeting different utility dimensions, such as landmark versus expression preservation, are reported on different benchmarks under different metrics, rendering cross‑method comparison infeasible. We revisit FDeID evaluation from both the data and metric perspectives. On the data side, we introduce UtilFace, a curated, demographically balanced benchmark with high identity diversity, assembled from four large‑scale face datasets through identity‑aware cleaning, resolution enhancement, and stratified filtering. On the metric side, we propose HiFD, a Hierarchical Face De‑identification metric that unifies identity suppression, multi‑level utility preservation, and image quality under a single consistency‑based paradigm: every component is computed from pretrained estimators' outputs on the original face and its de‑identified counterpart, directly quantifying how much identity is suppressed and how much downstream‑perceivable utility survives. HiFD organizes facial signals into a three‑level utility hierarchy spanning macro cues (L1), micro cues (L2), and imperceptible cues (L3), and aggregates the five resulting components into a single interpretable score via weighted harmonic mean, with configurable application‑specific profiles. Using this unified protocol, we conduct a comprehensive comparative study spanning adversarial, GAN‑based, and diffusion‑based methods, surfacing trade‑offs and failure modes that remain invisible under existing protocols. We release the benchmark and evaluation toolkit to foster systematic and reproducible research in privacy‑preserving human face analysis.
Authors:Yingchen Xu, Edward Grefenstette
Abstract:
Token‑based world models enable fine‑grained latent planning, but repeatedly processing large spatial token grids makes action search expensive. We introduce COSTGRAD, a training‑free, goal‑conditioned selector that ranks spatial tokens by the gradient norm of the planning cost with respect to each input token. By deriving importance from the downstream control objective, COSTGRAD targets tokens that matter for planning rather than merely for prediction. On AdaLN‑conditioned predictors at 50% sparsity, COSTGRAD matches or exceeds full‑token planning on three of four continuous‑control benchmarks, while giving a measured 2.6× wall‑clock speedup per environment planning step. Combining token sparsity with reduced CEM search increases this to a ~ 5× total speedup while still exceeding the full‑token baseline. We also identify an architecture‑dependent failure mode: in a matched AdaLN‑vs‑concat comparison, concat maintains comparable full‑token performance but pure COSTGRAD loses its advantage over random selection. This difference tracks action‑pathway drift: gradient‑selected removal produces less drift than random removal on AdaLN, but more on concat. These results highlight selector‑architecture compatibility as a design axis for sparse world‑model planning. Project page and demos: https://ycxuyingchen.github.io/costgrad/
Authors:William Chen, Prem Seetharaman, Ke Chen, Oriol Nieto, Kevin Duarte, Siddharth Srinivasan Iyer, Mamshad Nayeem Rizve, Zhiwen Cao, Shinji Watanabe, Yuanjun Xiong, Jianming Zhang, Zeyu Jin, Justin Salamon
Abstract:
Multimedia editing requires generative models to interpret complex, compositional instructions and modify only the desired elements across modalities, a capability largely beyond existing systems. Current approaches are typically limited to simple, single‑attribute edits within one modality, since diverse, high‑quality editing pairs are hard to obtain, especially for cross‑modal tasks where aligned audiovisual (AV) data is scarce. We present CrossEdit, a unified omni‑modal editing model for images, audio, and video that performs AV movie scene edits zero‑shot through cross‑modal transfer. Our key observation is that complex instruction‑following, once learned in any modality, generalizes to others. We exploit the additive nature of acoustic signals to procedurally generate large‑scale audio editing pairs with compositional instructions, and show that fine‑tuning on this synthetic data and self‑supervised AV masked reconstruction, alongside a targeted set of curated cross‑modal tasks, induces instruction‑following that transfers zero‑shot to unseen modality and instruction combinations. To evaluate this capability, we release CrossEditBench, a human‑annotated benchmark of AV edits on movie scenes, and propose AV‑FES, a metric that jointly scores instruction following and consistency. We show that our proposed techniques improve performance on zero‑shot AV movie scene editing, while maintaining or improving performance on lip‑synced speech editing and standard image, video, and audio editing benchmarks. Demos are available at https://wanchichen.github.io/crossedit/.
Authors:Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li
Abstract:
While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low‑data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable decision trees under a few‑shot learning setting. Instead of directly prompting the LLM to generate full trees, which is often unstable and inefficient, we develop a three‑stage paradigm that prompts the LLM to generate rules and organize the rules into a tree. Experiments on multiple real‑world tabular datasets demonstrate that our method achieves superior accuracy and interpretability with significantly lower prompting overhead compared to existing baselines.
Authors:Haoyu He, Basile Tessier-Cloutier, Yang Wang, Mahdi S. Hosseini
Abstract:
Whole slide image (WSI) analysis in computational pathology follows a multiple instance learning (MIL) pipeline where patch embeddings are extracted independently and aggregated for slide‑level prediction, but within‑slide variance from staining, scanner, and local texture can overwhelm the discriminative signal. We propose Masked Feature Encoding for Multiple Instance Learning (MFE‑MIL), a feature‑space masking framework that trains a lightweight MLP adapter jointly with a window‑based masked reconstruction branch and a MIL classification head. The two objectives are complementary. Classification guides the adapter to suppress within‑slide patch variance, while window‑based masked reconstruction provides an auxiliary regularizer for the adapted features without using patch coordinates, coordinate graphs, or segmentation preprocessing. The raster patch‑extraction order is used only as a weak implicit prior. At inference, the decoder is removed, leaving only the adapter and MIL head. Across CAMELYON16/17, PANDA, and TCGA‑BRCA with four diverse encoders, MFE‑MIL improves ACC/F1 for nearly all tested aggregator‑encoder settings and AUC in most, outperforms coordinate‑based spatial methods (CAMIL), and achieves higher AUC than 2DMamba on three of four datasets (UNI). On five TCGA survival cohorts it improves the average concordance index for every aggregator tested, its most consistent gain. Code is available at https://github.com/AtlasAnalyticsLab/MFE‑MIL.
Authors:Nicholas Tan Jerome, Fangnian Wang
Abstract:
A natural strategy for inverse problems with scarce labelled data is to transfer relational structure learned from abundant forward‑simulation data. We show this strategy fails systematically, even when it satisfies the standard theoretical justification for why structure should help. We prove that approximate structure provides estimation‑error benefits whenever the edge error satisfies Δ< n^2 ‑ kn, reducing sample complexity from O(n^2) to O(kn+Δ). Structure learned via Neural Relational Inference (NRI) from dynamics prediction satisfies this condition, yet on a source‑localisation task across 180 CFD‑simulated hydrogen‑leak scenarios and 180 acoustic scenarios, it degrades performance by 116% and 201% relative to a flexible, task‑optimised attention baseline, while a physics‑based prior (Green's function) degrades by only 69‑72%. Four independent lines of evidence show this is not a tuning failure: NRI improves only 0.5% when given 18x more training data (versus 16.6% for the task‑optimised baseline, p<0.001); performance is insensitive to the NRI edge threshold across a wide range; the dynamics‑learned graph overlaps the task‑optimal graph on only 6% of edges; and two further dynamics‑derived structure estimators (correlation‑ and mutual‑information‑based) show no measurable benefit over a structure‑free baseline, with the correlation‑based estimator performing markedly worse. We formalise this gap as a statement about approximation error that the edge‑accuracy condition cannot control, and we provide a lightweight transferability test (Jaccard similarity against a partially‑observed target‑task graph) that separates successful from failed transfer in all four domain/structure pairs we evaluate, using under an hour of computation and 15‑20% of target‑domain data; we present this as a heuristic calibrated on few cases, not a validated general threshold.
Authors:Zhuo Chen, Yihua Cheng, Aleš Leonardis, Hyung Jin Chang
Abstract:
Accurate contact modeling is fundamental to understanding hand‑object interaction, yet existing contact representations are typically restricted to object surfaces and rely on hand‑crafted rules to recover contact details, leading to severe penetrations and implausible results. To better exploit the rich detail in motion‑capture data, we introduce Volumetric Contact (VolCo), a representation that expands surface points to a set of 3D volumetric grids. VolCo encodes 3D contact that allows precise hand part recovery, and is organized in an inherent hierarchy: local contact details within each volume and global hand geometry across all volumes. Our framework, VolCoDiff, employs two modules to capture local and global features following this hierarchy. For local contact details, we use a 3D variational autoencoder to model the possible hand configurations conditioned on the local object signed distance field (SDF). For global hand geometry, we design a prior‑guided diffusion model that learns the distribution of compressed latent features aggregated from the volumetric grids. We evaluate our method on two benchmark datasets and demonstrate state‑of‑the‑art performance in penetration and stability, indicating the capability to generate tight grasps with much less severe penetrations. Our code is available at https://github.com/chzh9311/volco.
Authors:Salma Galaaoui, Nermin Samet, David Picard
Abstract:
LiDAR point clouds of humans are extremely expensive to collect and annotate, thus represent a scarce resource that hinders the development of human analysis using this modality. To alleviate this scarcity, prior work relies on simulated human LiDAR, but such samples do not fully reflect the geometry and sensing characteristics of real observations. In contrast, we introduce HuLiGen, a generative model that generates human LiDAR point clouds from a parametric body model, using a point transformer trained with a flow‑matching objective. We show that our generated point clouds are closer to the real capture distribution. Using HuLiGen to generate synthetic data, we propose a synthetic‑only pretraining scheme for LiDAR‑based HPE that achieves state‑of‑the‑art performance, with even larger gains in low‑annotation and low‑data regimes, where MPJPE is reduced by up to 50%. Code, models and generated samples are available at https://github.com/valeoai/HuLiGen.
Authors:Nicolas Brosse, Arnak S. Dalalyan
Abstract:
We analyze the first‑order deterministic diffusion sampler of Karras et al. (2022), termed EDM, in 2‑Wasserstein distance by separating two sources of error: local discretization error and its amplification by subsequent learned steps. We prove that local error admits a universal bound: for any data distribution with finite second moment, the one‑step discretization error is quadratic in the step size, with an explicit constant that does not depend on the data distribution. Error propagation, in contrast, depends on the learned network. At high noise levels, we exploit the network parametrization of EDM to derive an explicit contraction criterion. At low noise levels, we measure propagation through the amplification realized on the distributions transported by the sampler; this realized amplification can be arbitrarily smaller than the worst‑case Lipschitz constant. This analysis yields an O(e^Λ_K/K) global discretization error for K sampling steps, where Λ_K is the low‑noise log‑amplification. Experiments on a one‑dimensional Gaussian mixture show how measured amplification accounts for slower error decay on finite sampling grids. Diagnostics on a pretrained CIFAR‑10 model illustrate related stability mechanisms without certifying the global assumptions.
Authors:Yifei Lu, Cheng Liu, Dianzhi Yu, Hui Xiang, Ji Zhang, Yuanchu Xiao, Rong Liang
Abstract:
Large language model agents can improve across tasks by retaining reusable skills distilled from prior interactions. Recent work jointly optimizes task execution and skill extraction, enabling the policy and skillbank to co‑evolve. However, as the actor continues learning, rewarding skill proposals through their reuse in subsequent training steps may conflate skill benefits with actor improvement, while directly testing each proposed skill requires costly additional actor rollouts. In this paper, we introduce UniSkill, which uses a shared policy to interact with the environment and propose skillbank edits (Add, Update, or No Edit) from the resulting trajectories. Specifically, the actor learns from environment rewards, while contrastive action feedback guides skill proposal learning. This feedback provides an actor‑alignment signal by measuring how replacing the retrieved skill with a proposed skill changes the current actor's action log‑likelihood gap between previously collected successful and failed trajectories from the same task, thereby avoiding new rollouts for each proposal. Since proposal‑level feedback may suppress an otherwise appropriate edit operation when the proposed skill content scores poorly, we further apply skill‑edit support regularization to preserve exploration. Empirically, UniSkill achieves strong performance, reaching 98.4% success on ALFWorld and 84.7% on WebShop while maintaining stable joint training. Further ALFWorld experiments show that UniSkill remains effective when the shared policy uses a smaller backbone. Our implementation is available at https://github.com/LimOkii/UniSKill.
Authors:Anas Filali Razzouki, Killian Steunou, Khalil Guetari, Thomas Kling, Mounîm El-Yacoubi, Yannis Tevissen
Abstract:
Linking people's appearance and actions to character identities is essential for understanding video narratives. We present a framework for identity‑aware video captioning and person‑centric question answering that combines automatic character identification, explicit spatial grounding, and task‑specific adaptation. Starting from LSMDC v2 movie clips, our pipeline matches detected faces to actor reference images, tracks characters across frames, and builds inputs with identity‑linked bounding boxes. A strong vision‑language model generates identity‑aware captions and questions, which are manually verified and filtered to create a benchmark of 750 captioned clips and 3,000 person‑centric questions. We study five grounding strategies combining textual coordinates with visual face or estimated person boxes across Video‑MLLM families at roughly 2B, 4B, and 8B parameters and larger frontier models. Combining visual face boxes with textual coordinates yields the most consistent performance across scales and significantly improves overall performance over coordinates alone. Smaller models tend to over‑assign known identities when the queried person is not grounded, while larger models better recognize such UNIDENTIFIED cases. We introduce BAC by LoRA fine‑tuning Qwen models at 2B, 4B, and 8B scales on about 32K identity‑aware captioned clips. Across all scales, BAC outperforms every other evaluated model family of comparable size. BAC‑8B reaches 93.20% overall QA accuracy, ranking behind only GPT‑5.6 Sol among the frontier models evaluated in our study. Overall, explicitly communicating who is where, together with lightweight task‑specific adaptation, substantially improves identity‑aware video understanding without changing the underlying architecture. We release the benchmark, training data, code, and BAC checkpoints at https://github.com/momentslab/beyond‑anonymous‑captions.
Authors:Leon Mayer, Lucas Luttner, Patrick Godau, Kai Fritzsche, Annika Reinke, Leonie Boland, Jule Brandt, Janne Heinecke, Chloe K. Nobuhara, Niklas Holzwarth, Evangelia Christodoulou, Marcel Knopp, Dominik Michael, Pascale Piermarco, Saliq Neyaz, Korhan Derin Özarslan, Jakob Hennighausen, Carlos Aumente-Maestro, Tim Rädsch, Dheeraj Baji, Peter Maximilian Full, Finn Aichholz, Justus Veit Erpenbeck, Linus Finn Schott, Bastian Winkelhausen, Claas de Boer, Bianca Güttner, Anneli Hummel, Gregor Just, Max Kirchner, Chenyang Li, Rozenn Raffaut, Ariel Rodriguez, Danush Kumar Venkatesh, Kevin Wang, Jinjing Xu, Mona Sheikh Zeinoddin, Salman Khan, Thomas M. Pausch, Stefanie Speidel, Danail Stoyanov, Daniel A. Hashimoto, Fiona R. Kolbinger, Thomas G. Weiser, Lena Maier-Hein
Abstract:
Recent advances in Vision‑Language Models (VLMs) have led to rapid progress in video understanding across a wide range of benchmark tasks. However, existing evaluations largely focus on short‑term reasoning, failing to assess a critical capability: maintaining cumulative temporal consistency over extended time horizons. To close this evaluation gap, we introduce HeiCo‑FOCUS, a clinically grounded dataset for evaluating long‑context video understanding through the task of Foreign Object Contextual Understanding in Surgery. Built on a dataset of Heidelberg Colorectal surgeries, this task requires models to continuously track multiple objects as they are inserted, manipulated, occluded, and removed over procedures lasting up to hours. HeiCo‑FOCUS comprises 30,000 visual question answering (VQA) pairs covering five core capabilities: object recognition, temporal grounding, aggregation, event and procedural understanding, and complex reasoning. The dataset was constructed through a rigorous multi‑stage annotation pipeline involving large‑scale crowd annotation and 39 surgical domain experts to ensure high quality and clinical relevance. To systematically probe model behavior, we introduce a multi‑track evaluation framework that progressively increases temporal and contextual demands from single frames to full procedures. Experiments with ten frontier VLMs show that HeiCo‑FOCUS tasks are far from solved: only around half of the models clearly outperform a text‑only baseline. Across the video tracks, models perform best on event and procedural understanding (mean Accuracy: 56.5% across all models), while temporal grounding remains particularly challenging for all evaluated models (mean Accuracy: 19.7%). We therefore expect HeiCo‑FOCUS to serve as a catalyst for the development of models capable of reliable, temporally consistent reasoning over hours‑long videos.
Authors:Xiangtao Kong, Shuaizheng Liu, Rongyuan Wu, Lingchen Sun, Zhengqiang Zhang, Jinxin Zhao, Yuhui Wu, Lei Zhang
Abstract:
Real‑world low‑quality images suffer from complex mixed degradations, including but not limited to noise, blur, atmospheric effects, etc. Recent agentic methods usually model real‑world image restoration (Real‑IR) as a sequential tool calling problem over task‑specific single‑degradation restoration models. This paradigm, however, is fundamentally limited because complex real‑world degradations cannot be cleanly undone degradation by degradation, and the tool used for task‑specific models caps the capability of the agent system. In this work, we present HarnessIR, an agentic framework for Real‑IR by harnessing a multimodal foundation model (MFM) as the executor. HarnessIR consists of five stages: perception and diagnosis, on‑demand tool invocation, prompt composition, execution, and verification‑driven refinement. Unlike prior agentic Real‑IR methods that rely on tool chains assembled from task‑specific models, HarnessIR feeds the restoration requirements, the perceptual diagnosis, and the evidence into an MFM that performs restoration in a single pass, followed by verification stages to determine whether the result warrants further processing. Under our harness, off‑the‑shelf MFMs handle restoration tasks remarkably well, achieving state‑of‑the‑art results on the widely used MiO100 synthetic benchmark. More importantly, by exploiting the strong generalization ability of MFMs, HarnessIR delivers compelling restoration quality on challenging real‑world scenes where previous agentic IR systems often struggle. Codes is available at https://github.com/PolyU‑VCLab/HarnessIR.
Authors:Huishan Ji, Hua Xu, Weiming Zhang, Qirui Ye
Abstract:
Long‑horizon reasoning demands access to earlier information at a manageable generation cost. Full‑history attention incurs growing storage and computation, while recurrent compression can lose precise details. Therefore, we present YANchor‑4B, a general‑purpose recurrent model that preserves crucial memory as ANchors for retrieval during subsequent reasoning. Beyond O(N)‑time generation and O(1) memory, YANchor enables effective long‑horizon reasoning through its multidimensional memory mechanism. For example, on challenging math problems, it achieves 82.93% mean pass@1 on AIME 2024‑‑2026 and 63.64% on HMMT, substantially outperforming linear‑time, constant‑state counterparts, including larger models. It also delivers several‑fold higher batched long‑generation throughput than Transformer and hybrid baselines on H100. Furthermore, evaluations across dozens of benchmarks demonstrate YANchor's superiority in general‑purpose capabilities.
Authors:Hyungjoon Juen, Minwoo Shin
Abstract:
The Fourier Neural Operator (FNO) learns solution operators of partial differential equations (PDEs) through Fourier‑space kernel parameterization, but frequency truncation can limit the learning of high‑frequency variations. AM‑FNO and SirenFNO generate kernels for all grid modes from spectral coordinates using shared networks, making coordinate encoding and generator design important. Recent work on implicit neural representations (INRs) has proposed constructing frequency interactions through explicit feature composition rather than relying on subsequent MLPs to form them implicitly. Building on this approach, we propose CAFE+FNO, which incorporates Content‑Aware Frequency Encoding+ (CAFE+) into Fourier kernel generation. CAFE+ combines Fourier‑‑Chebyshev features through parallel affine branches and a Hadamard product, forming interactions within and across the two feature families. A kernel MLP maps the resulting representation of each normalized spectral coordinate to a complex channel‑mixing matrix. Each layer shares its generator across all stored modes, making the number of trainable parameters independent of the number of modes for a fixed architecture. We compare CAFE+FNO with existing FNO variants on five PDE benchmarks and conduct ablation studies on basis configuration, multiplicative composition, and bandwidth learnability. Code and experimental configurations are available at https://github.com/fabsk101/CAFEPlusFNO.git.
Authors:Sean Lesmana, Theodore Tjugiarto
Abstract:
Start with the number 2. At each move, combine two numbers already made, but the operations must be used in the fixed repeating order +,×,‑,÷. We ask for the fewest moves needed to make every integer from 1 to N. Since 2 is already one of the numbers we want and each move makes at most one new number, at least N‑1 moves are needed. We show that N‑1 moves are also enough for every N\ge 9. Conventional induction cannot work, because a division that comes right after a completed interval \1,\dots,P\ produces numbers already made. Instead we extend a completed interval \1,\dots,P\ to \1,\dots,3P\ all at once, which counting shows is the smallest multiplicative extension P\to kP that can work, and then adjust the last few moves to reach every other N. For 9\le N\le 33 we give explicit sequences, found by computer search.
Authors:Sangmin Lee, Youngju Na, Chanmi Lee, Sung-eui Yoon
Abstract:
Offline MARL increasingly relies on generative policies to model multimodal joint behavior, typically by distilling a centralized teacher into decentralized one‑step actors under the CTDE. We identify a failure mode at the teacher training stage: standard flow‑based teachers pair noise with replay targets independently, so nearby noise samples can be routed toward conflicting coordination modes. The teacher then produces samples between valid modes, and because the distillation loss regresses each local actor onto the conditional mean of the teacher's output given local input, this error is not absorbed but propagated to the student. To remove this teacher‑side artifact, we propose Mode‑Support Semi‑Discrete Optimal Transport (MoSDOT), which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi‑discrete optimal transport to assign each noise sample to a single mode before teacher training. We additionally study a shared‑randomness variant that uses a shared noise component at execution to expose the residual gap intrinsic to strict‑product execution. On controlled diagnostics and offline MARL benchmarks, MoSDOT improves endpoint quality and routing consistency, particularly on datasets exhibiting multimodal joint behavior.
Authors:Ran Zhou, Xueming Han, Jiaheng Liu, Yuyao Zhang, Fanyu Meng, Junlan Feng, Yuxiang Ren
Abstract:
LLM‑based agents face challenges in personalized interactive tasks due to heterogeneous, multi‑typed, and implicitly constrained long‑term traces. Existing memory mechanisms struggle with accurate routing and retrieval, especially on‑device where personalization is critical. Most methods use single‑vector representations, blurring type distinctions and relational structure. We propose HGP, a hybrid graph memory framework. HGP employs a lightweight self‑enhancement classifier for personalized memory routing and constructs episodic, semantic, and procedural memories as graphs. It also extracts working memory as a state trajectory to capture current state and implicit constraints, ensuring reliable decision‑making. The classifier reduces large‑model calls, enabling on‑device deployment, while graph storage enables accurate retrieval and incremental user profile refinement. Experiments on two benchmarks show that on PAL‑Set solution selection, HGP achieves an S‑score of 35.58, nearly 7 points above the strongest baseline. Code and data are at https://github.com/Ouan6/HGP‑.git.
Authors:Obada Kraishan
Abstract:
Reasoning models allocate extra computation at inference time and present their answers as the product of deliberate thought. If this deliberation works the way dual‑process accounts of human cognition suggest, longer thinking should weaken the classic decision biases that fast, intuitive judgment produces. Using 30 vignettes covering six biases (anchoring, framing, loss aversion, escalation of commitment, availability, confirmation) from an established benchmark, we run a dose‑response study across four model families, pairing each reasoning model with a matched non‑reasoning sibling and requesting thinking ceilings of 0, 1,024, 4,096, and 8,192 tokens, for 12,350 API calls. Because a requested ceiling is not the same as realized deliberation, we use the reasoning tokens each call consumed as the dose. First, reasoning models are not less biased than their siblings; the point estimate leans the other way in every family, but the item‑level pooled contrast is not reliable (Delta = +0.031, t(29) = 1.45, p = .157). Second, bias magnitude does not reliably fall as realized deliberation grows: no slope is significantly negative, and where anything moves it is the signed score drifting further from the human direction. Third, anchoring is the only bias in the human direction (d = 1.89). Four of the other five lean the opposite way in all seven models; with five items per bias, that reversal is reliable for framing and directional for escalation of commitment, confirmation, and loss aversion, while availability is absent. A one‑line instruction to restate the anchor before answering lowered anchoring on all five anchoring items, which no amount of additional thinking did, although the effect does not reach significance (p = .057). The results argue against treating test‑time reasoning as a rationality guarantee and for auditing deployed models bias by bias.
Authors:Yuyang Zhao, Lizi Liao, Leyang Shen, Xiaoyan Zhao, Yang Zhang, Fuli Feng, Xiangnan He
Abstract:
Large language model (LLM) agents can improve their performance by reusing knowledge distilled from past interactions. However, curating new experiences into a knowledge bank that becomes more useful as it grows remains challenging. Effective knowledge accumulation should limit redundant overlap among entries and ensure that new knowledge contributes beyond what the bank already provides. Yet training a curator with Group Relative Policy Optimization (GRPO) on standalone task success can reinforce general guidance even when it duplicates existing knowledge. Therefore, we propose Knowledge Weaver, a reinforcement learning framework that trains a language model to curate reusable knowledge from agent trajectories. We couple feedback inspired by token‑wise mutual information (MI) with marginal success rewards to guide knowledge accumulation. Together, these signals encourage the curator to preserve distinct information from experience and produce entries that improve task success when added to existing knowledge. Standalone success rewards also favor entries that are useful on their own. On ALFWorld and WebShop, Knowledge Weaver achieves mean success rates of 54.0% and 42.0% with k=10 retrieved entries, exceeding GRPO by 16.9 and 18.7 percentage points, respectively. Its knowledge banks also outperform the evaluated prompt‑based and established banks, including human‑written banks, in overall ALFWorld success rate and WebShop score with the executor frozen. Our codebase is available at https://github.com/LaoKuiZe/Knowledge‑Weaver.
Authors:Amirhossein Hajavi, Hanhee Lee, Pushya Jain, Sky Qiao, Emmanuel Ko, Yuanhao Yu, Irina Kezele
Abstract:
Language‑queried audio source separation (LASS) enables extracting any sound source using natural language. However, adapting LASS models to application‑specific sound classes is challenging due to noisy training data and limited semantic coverage of the CLAP‑based control signals. We propose a framework comprised of an automated data engine for training‑data curation and a two‑stage optimization process for class‑specific CLAP control signals. Our objective evaluations across seven sound classes show that data refinement and control signal optimization consistently improve source separation performance. Subjective evaluation with 17 participants further demonstrates perceptual improvements of the model trained with optimized control signals over baseline and similar commercial models.
Authors:Joonghui Cho, Minchan Kang, Daeshik Kim
Abstract:
Can a biological wiring diagram serve as a useful computational substrate beyond the behaviors for which it evolved? We use the publicly released MaleCNS v1.0 connectome, reconstructed from a single adult male Drosophila specimen, as the fixed recurrent topology of an artificial network. We train separate models for bounded addition and for a controlled grounded relational language task built from a fixed 100‑word lexicon. In both models, one scalar is learned per anatomical edge. The anatomical graph reaches 92.77% mean accuracy on held‑out addition, compared with 67.93% for directed degree‑preserving rewires. On the strict paired language endpoint, which matches original and order‑reversed scenes to their corresponding descriptions, it reaches 61.59% across four fixed interfaces, compared with 44.17% for matched rewires. At the canonical interface, it ranks first in a fixed 21‑graph comparison. On the matched 48‑group intervention subset, shuffling task‑defined sensory features reduces its score from 60.94% to 19.27%. Together, these results show that higher‑order MaleCNS wiring provides a reusable inductive bias for bounded addition and grounded relational language.
Authors:Piotr Marszalik
Abstract:
We study the randomness required to share possibly correlated secret bits among parties connected by a graph. A dealer places shares on the edges so that each party can recover its own secret from its incident shares and learn nothing about the others beyond what its own secret reveals. Anilkumar et al. completely determined the minimum randomness required for three binary secrets. We extend this study to four secrets and obtain results for arbitrary numbers of secrets on general graphs. For four parties on a complete graph, we determine the minimum number of random states for every set of permitted secret combinations: the possible values are one, two, three and four. We also characterize when one random bit suffices on an arbitrary graph.
Authors:Michael Ofengenden, Maksym Andriushchenko
Abstract:
An essential control of AI agents is their ability to manage runtime. This ability requires a sense of time‑awareness, to predict and estimate wall‑clock time and to control their own actions. Prior work has focused on time‑awareness, but duration‑following and control in native agent harnesses remain unexplored. We present AgentTime, a benchmark for testing whether agents can work for a requested duration, predict their runtime, and estimate elapsed time afterward. It comprises 222 tasks from 18 sources spanning coding, computer use, agentic work, and automated research. Duration‑following experiments append a single instruction specifying how long to work, with requests ranging from about a minute to multiple days. Accuracy on these instructions varies substantially: Fable 5.1 in Claude Code deviates from requested runtimes by a typical factor of 2.9×, compared with only 1.2× for GPT‑6 Astra in Codex. However, matching the requested runtime does not, by itself, establish continued work on the task. Among 158 reviewed Astra runs with classifiable transcripts, 14 explicitly slept after appearing to finish. In forecasting experiments, predictions tend to overestimate natural runtimes. In retrospective experiments, removing temporal information more than doubles deviation for Sol and Astra and nearly doubles it for Fable. An agent's ability to complete a task does not guarantee that it can control its own time or work for the whole requested duration. For agents to run reliably, safely, and autonomously over long horizons, we require the evaluation of both.
Authors:Bojun Yang, Haochen Zhou, Zhifang Zhang, Haobo Wang, Songze Li, Lei Feng
Abstract:
Large vision‑language models (LVLMs) are increasingly deployed in safety‑critical applications, yet they remain vulnerable to backdoor attacks. Defending against such attacks remains costly, as existing methods require either extensive retraining on clean data or per‑query intervention at inference time. To address this limitation, we propose OrthoPurify, a more efficient method to purify backdoored model weights via one‑step orthogonal projection. Specifically, through structural analysis of backdoor weight updates, we find that the backdoor is encoded by diverting a small number of weight update directions from task adaptation to backdoor shortcut encoding, a phenomenon we term direction hijacking. However, identifying these hijacked directions requires a benign reference model, which is typically inaccessible to the defender. We show that a pseudo‑benign model, obtained by fine‑tuning the pretrained weights on only a small set of clean samples, provides a sufficient approximation, as the dominant update directions stabilize within the first few gradient steps. OrthoPurify uses this pseudo‑benign reference to isolate the hijacked directions and removes them through a single projection on the weight update. Extensive experiments show that OrthoPurify reduces the attack success rate to near zero while preserving the original performance across diverse benchmarks, without retraining the backdoored model or introducing inference‑time overhead. Our code is publicly available at https://github.com/womeimingzi/OrthoPurify.
Authors:Wenhua Huo, Fenglei Han, Wangyuan Zhao, Jialin Wu, Jiayi Han
Abstract:
Ship‑form design combines smooth geometric representation, local shape editing, and constraints on the resulting hull. We present the Natural‑Language‑to‑Hull Framework (NL2Hull Framework), which formulates ship‑form editing as a typed discrete decision problem and connects language decisions to numerical geometry. Its Constrained Free‑Form Deformation Engine (CFFD Engine) represents hull waterlines with non‑uniform rational B‑splines (NURBS), applies free‑form deformation (FFD) to their control points, reconstructs the hull, and checks geometric constraints. We construct the Ship Design Decision Dataset (SDD Dataset) with 134,558 cleaned records and evaluate compared models on its subset Ship Design Decision Benchmark (SDDBench), containing 5,000 records and 43,496 typed questions. We propose Chip, a constrained ship‑design decision model for processing natural‑language requests. Chip reaches 95.90% question accuracy and 99.32% FFD exact match, with a negative log‑likelihood of 0.0951, an expected calibration error of 0.0032, and a Brier score of 0.0551. The NL2Hull Framework provides a reproducible interface for evaluating language‑based ship‑form decisions while identifying the geometry and continuous‑control components that require further development. Our code and dataset is available at https://github.com/wenhuahuo/NL2Hull.
Authors:Hanyu Meng, Zhanhong He, Zixun Guo, Yaolong Ju
Abstract:
Single‑instrument automatic music transcription (AMT) has advanced substantially, yet choral applications require soprano, alto, tenor, and bass (SATB) to be transcribed as separate parts. Recent note‑level choral AMT instead produces a single merged note track, limiting rehearsal, education, and score reconstruction. To address this limitation, we introduce Part‑aware Choral Transcription (PawCT), to our knowledge the first end‑to‑end neural framework that identifies active SATB parts from choral audio and transcribes each into a separate note‑level track. PawCT combines part‑specific onset, offset, and frame prediction with part‑presence estimation, union‑level supervision, and structured training targets using a range prior (RP) based on SATB pitch ranges and its ordered‑continuity (OC) extension, which adds within‑part melodic continuity and cross‑part pitch ordering. On YouChorale, PawCT‑RP‑OC achieves a macro part‑aware note F1 of 0.225 at a 50‑ms onset tolerance, outperforming an adapted choral baseline (0.165) by 36.4% relative and a two‑stage post‑hoc assignment pipeline (0.175). Its part‑agnostic variant, PagCT, achieves a 50‑ms onset F1 of 0.382, compared with 0.237 for the previous state‑of‑the‑art choral AMT model. Cross‑dataset evaluations on CSD and Cantoria further assess performance under dataset shift. These results demonstrate the benefit of jointly modeling note transcription and vocal‑part assignment. Code and demos are available at https://hanyu‑meng.github.io/Paw_Choral_AMT_Demo/.
Authors:Yifei Zhu, Yangyang Cai, Mingyi Shi, Miao Cheng, Lin Gu, Taku Komura, Yoshifumi Kitamura
Abstract:
Holistic co‑speech animation is prone to averaging in both motion representation and speech conditioning. In coordinate‑space diffusion, slow body posture, mid‑frequency gesture strokes, and fast hand or facial details are entangled in one prediction target, often producing low‑variance, over‑smoothed motion. Meanwhile, dense rhythmic and acoustic cues can dominate sparse content‑specific information under fixed multimodal fusion. We present DynaConTalk, a wavelet‑constrained diffusion framework for long‑form and controllable holistic co‑speech motion generation. Diffusion operates in stationary wavelet transform (SWT) coefficient space, whose temporally aligned bands separate coarse posture evolution, gesture strokes, and fine expressive details. Our dynamic gating network preserves HuBERT and speaker identity as a base and selectively adds rhythm, mel, and transcript features through motion‑state‑ and noise‑aware residual gates. Attention pooling and learned depth routing deliver complementary conditions to each denoising stage, while a frame‑resolution rhythm path preserves precise timing. A signed proposal‑consensus update then reconciles these conditions with the evolving motion state. Matched‑noise constraint injection uses the same sampling interface for history continuation and localized keypose repair, and extends to reference‑guided control. Separate body‑hand and facial denoisers, followed by inverse SWT and a pose‑driven root regressor, produce holistic motion. Experiments evaluate generation quality, facial accuracy, temporal continuity, and controllable editing. Code, models, and the interactive editing interface are available at https://github.com/zhuyifeiabcd1/DynaConTalk.
Authors:Bjørn Leth Møller, Bulat Ibragimov, Christian Igel
Abstract:
The adoption of machine learning for socially relevant tasks requires effective explainable artificial intelligence (XAI) methods to better understand the behavior of machine learning models. Attribution methods are a popular XAI approach in which input‑output relationships are characterized by heat maps that reflect the relative importance of input features for a particular prediction. The quality of such maps is often assessed by measuring faithfulness based on the area under insertion and deletion curves, which measures changes in the model output as features are added and removed. In this study, we derive an objective function from this notion of faithfulness and a way to approximate its gradient. We establish the connection between insertion curves and top‑k feature selection, which leads to a loss function measuring the quality of attributions. Randomization of the loss allows us to efficiently approximate its gradient. To show the effectiveness of the general approach, we combine the loss function with the neural explanation mask framework. The resulting method, termed Ra‑NEM, can be used with any differentiable model without affecting the model's performance. Experiments demonstrate that Ra‑NEM provides accurate attributions robustly and efficiently. Compared to other algorithms, the attributions have not only higher faithfulness but also perform well in terms of other XAI metrics. The high inference speed of Ra‑NEM makes the method suitable for online applications. The code is available online: https://github.com/baerminator/Ra_Nem
Authors:Raneem Mahajne, Toviah Moldwin
Abstract:
We present a framework for building and interpreting minimal transformer models. By constraining a transformer's embedding dimension and head size to 2, we enable full two‑dimensional visualization of its internal representations. Embeddings, query/key/value transforms, attention outputs, residual streams, and decision boundaries can all be seen directly. Our central claim is that the learned geometry implies an algorithm; the arrangement of points and boundaries in R^2 can be read as a step‑by‑step procedure. We train a transformer on a simple task where it must produce the most recently observed even number whenever the '+' operator appears in a sequence of digits. Once trained, we visually walk through every step of the transformer's computation. We show how the model embeds the tokens and their respective positions in the sequence, transforms them via the Q, K, and V matrices, uses the dot product between the Q and K representations to form the attention matrix, and uses the attention matrix to select values that move the representation of each input token to the region of the domain of the output layer that will correctly predict the next token. We introduce a suite of interpretability visualizations that make the algorithmic interpretation of this procedure explicit. Our framework offers a pedagogical and experimental testbed to explore how transformers use informational geometry to implement next‑token prediction.
Authors:Giovanni Affatato, Sara Mandelli, Paolo Bestagini, Stefano Tubaro
Abstract:
Video deepfakes targeting a specific individual, the Person‑of‑Interest (POI), are the most harmful ones, and, since a public figure is abundantly recorded, a detector can be built from genuine footage of that individual. Such detectors commonly describe a subject through a 3D Morphable Model (3DMM) and adopt its coefficients as a whole, so which part of that description carries the signal has never been measured. We dissect it, holding the encoder, the training corpus and the enrollment protocol fixed and varying only what the encoder observes. The groups of coefficients prove largely redundant, since the shape block alone recovers almost all the accuracy of the full vector, and their temporal evolution contributes a real but bounded amount. We further show that the dense surface the same fit returns, which these detectors discard, carries identity information that the coefficients do not, and that it helps precisely where they are weakest. We assemble the best configuration into MOTIF, a visual‑only detector trained on real videos only, with no manipulated video and no POI‑specific data. It improves on both state‑of‑the‑art POI detectors in every dataset and manipulation of our benchmark and at two quality levels. Our experimental code will be released at https://github.com/polimi‑ispl/MOTIF.
Authors:Deyuan Liu, Yihao Hu, Jingxuan Zhang, Xingying Li, Jun Xie, Jiacheng Liu, Jungang Li, Yu Huang, Xuanyi Liu, Yue Ding, Zecheng Wang, Lei Zhao, Mingda Wang, Zhenglin Cheng, Peng Sun, Tao Lin
Abstract:
Dense visual text requires image generators to reproduce long strings across multiple regions with correct placement and legibility. As short‑string rendering improves, evaluation must test sustained performance across more demanding scenes. We introduce UltraText Bench, a bilingual benchmark for prompt‑only generation of dense visual text. It contains 432 prompts spanning 24 real‑world scene categories and three difficulty levels, split equally between English and Chinese. Each human‑reviewed prompt supplies exact strings for four to twelve text regions, paired with structured references for their content, placement, and visual attributes. We use the Q‑Judger vision‑language model to assess each image against the complete reference, reporting text fidelity, text clarity, spatial quality, and scene quality. Across 24 model configurations, these dimensions reveal different strengths: Z‑Image‑Turbo gains 3.81 clarity points over Z‑Image‑Base while losing 14.76 fidelity points under the reported settings. Performance also varies with workload; Qwen‑Image‑2512's English composite falls from 86.50 at L1 to 42.86 at L3. Ten participants took part in human evaluation of the automatic scores. Repository: https://github.com/LINs‑lab/UltraText_Bench.
Authors:Keke Yang, Erqi Wang, Sainan Guan, Hongliang Ren
Abstract:
World models can enable autonomous ultrasound scanning by predicting the outcomes of probe motions from local observations. Learning this action‑‑observation relationship typically relies on synchronized video‑‑pose pairs, which are costly to collect at scale and largely unavailable in routine clinical recordings. Reliable action following further requires modeling ultrasound's cross‑sectional sampling geometry. We present UltraWorld, a self‑distillation recipe that transfers priors from clinical ultrasound videos into interactive world models without real action annotations. Starting from clinical videos, we adapt a video foundation model into an ultrasound generator conditioned on reference images and anatomical masks. Anatomical masks sampled along programmable trajectories through 3D anatomy provide spatial guidance for synthesizing action‑‑video pairs. We then use these synthetic pairs to self‑distill the generator into a world model that predicts future observations from local observations and actions, without requiring anatomical masks or other 3D assets at inference time. To further improve action following, we introduce the Acoustic Sampling Map (AsMap), which represents probe poses and imaging settings as pixel‑wise 3D sampling positions, beam directions, and depths. Experiments demonstrate improved prediction fidelity and action following. Across nine simulated closed‑loop local planning episodes, UltraWorld reduces the mean final distance to the goal and orientation error by 29% and 38%, respectively, compared with visual servoing. Project Page: https://ultraworld‑project.github.io/.
Authors:Ruoshi Xu, Mingqi Gao, Shengda Luo, Jingkun Chen
Abstract:
Reliable liver segmentation in contrast‑enhanced MRI is essential for quantitative hepatic assessment, treatment planning, and longitudinal disease monitoring. However, limited annotated data and scanner‑ or vendor‑dependent intensity variations can cause overfitting and poor generalization to unseen acquisition domains. Moreover, simultaneously achieving robust global localization and precise boundary delineation remains challenging, while predictions may contain isolated false‑positive regions outside the main liver component. To address these challenges, we propose CIRSeg, a coarse‑to‑fine, intensity‑robust liver segmentation framework based on nnU‑Netv2. CIRSeg combines 3D CutMix with stochastic intensity transfer using either Nyul augmentation or histogram matching to improve robustness to heterogeneous MRI intensities. Its cascaded architecture decouples low‑resolution anatomical localization from full‑resolution boundary refinement. At inference, source‑free test‑time adaptation based on confidence‑filtered predictions and probability‑prior regularization further improves robustness to out‑of‑distribution inputs. As a final deterministic post‑processing step, largest connected component filtering removes isolated false‑positive regions. On the CARE 2026 test set, CIRSeg achieves Dice scores of 97.13% and 97.93% on the in‑domain and unseen‑domain subsets, with corresponding HD95 values of 20.18 mm and 11.30 mm, respectively. These results demonstrate consistently accurate segmentation across both in‑domain and unseen acquisition settings. The code is available at https://github.com/jingkunchen/MICCAI_CARE_2026
Authors:Mahmoud Selim, Cristina Cipriani, Karl Henrik Johansson
Abstract:
Offline reinforcement learning enables reward‑driven policy improvement from fixed datasets without requiring online exploration, making it particularly attractive in safety‑critical domains. A central challenge, however, is distribution shift: policy optimization may favor actions that are weakly supported by the offline data, rendering value estimates unreliable. Existing approaches primarily control this shift in the policy's own action space. In interactive environments such as autonomous driving, this can be insufficient: a candidate ego trajectory may remain well supported under the marginal behavior distribution while being poorly supported jointly with the surrounding‑agent behavior observed in the logged interaction. We refer to this degradation in interaction support as \emphinteraction distribution shift (IDS), and introduce \emphInteraction‑Constrained Drive Policy (ICDP), an offline reinforcement learning framework that explicitly controls interaction‑level distribution shift. Starting from the joint data distribution over ego and surrounding‑agent futures, we show that joint‑support degradation decomposes exactly into an ego‑support component and a residual interaction‑support component. We recover the latter through contrastive density‑ratio estimation, isolating interaction compatibility without explicit joint‑density modeling, surrounding‑agent prediction, or rollouts in reactive simulators or learned world models during policy optimization. Closed‑loop evaluations on nuPlan, Interplan and real‑world truck experiments show that ICDP suppresses high‑value yet interaction‑unsupported trajectory selections and improves performance in interaction‑critical driving scenarios. Project webpage: https://mahmoud‑selim.github.io/ICDP/
Authors:Ahmad Abbas, Tamara Fakih, Nour Fakih, Ammar Mohanna
Abstract:
Classical Arabic poetry generation requires simultaneously satisfying semantic, linguistic, and fine‑grained prosodic constraints. Existing systems typically control broad poetic attributes but do not jointly model semantic intent, meter subform, and poem length. We present Shaer, a controllable Classical Arabic poetry generation framework jointly conditioned on natural‑language descriptions, meter subforms, and target hemistich counts. To support this task, we construct an enriched corpus of 116,032 classical Arabic poems derived from Ashaar, containing normalized meter‑subform labels and automatically generated, validated semantic descriptions. We then adapt Yehia‑7B using QLoRA‑based supervised fine‑tuning with a completion‑only objective. Our evaluation combines automatic assessment of base‑meter conformity, requested‑subform adherence, and length control with three LLM judges, blinded human evaluation, and memorization analysis. Shaer achieves 95.17% base‑meter accuracy, 91.75% poem‑level meter‑subform accuracy, and 83.40% exact count accuracy. Relative to its untuned foundation model, these results represent gains of 68.68, 57.77, and 38.93 percentage points, respectively; Shaer also attains the highest base‑meter accuracy among all evaluated systems. Multi‑LLM evaluation and a blinded human assessment of top‑ranked outputs further indicate competitive semantic and literary quality. Finally, analysis of all 3,481 test generations finds no exact copies from the training corpus or paired source poems. Code, models, and datasets are publicly available.
Authors:Jacobus Arthur, Ahmad Sait, Batool Hani, Merey Ramazanova, Jan Held, Marc Van Droogenbroeck, Bernard Ghanem, Anthony Cioppa, Silvio Giancola
Abstract:
Refereeing decisions in professional soccer remain inconsistent because referees cannot easily compare a contentious foul against similar past cases. We cast this as a retrieval problem and introduce SoccerNet‑FoulRet, the first benchmark for semantic foul retrieval. Given a query foul, the task is to retrieve past fouls judged to be relevant precedents, regardless of camera angle, teams, or appearance. This differs from prior video‑to‑video retrieval, which matches clips by visual similarity or a shared event. Here, relevance is defined by refereeing interpretation. We build the benchmark from the SoccerNet‑MVFoul dataset and evaluate retrieval ability of zero‑shot video and vision‑language embedders together with a task‑specific fine‑tuned baseline on 693 human‑verified queries and category‑relevance labels. Semantic foul retrieval remains challenging. The strongest zero‑shot model achieves under 5% HitRate@10 on human‑verified precedents, while category‑supervised fine‑tuning improves category relevance but transfers only modestly to precedent retrieval. We release SoccerNet‑FoulRet to establish semantic foul retrieval as an open problem: https://github.com/SoccerNet/sn‑foulret.
Authors:Hong-Son Nguyen, Thi-Ngoc-Hanh Le
Abstract:
Region‑based neural style transfer enables fine‑grained artistic control by allowing independent stylization of semantic image regions. However, compositing these regions often leads to boundary artifacts, degrading visual quality. We propose Interior‑Guided Boundary Repair (IGBR), a lightweight and model‑agnostic method that improves boundary handling in multi‑region stylization. IGBR repairs boundary pixels using interior‑guided propagation and applies inward, distance‑based blending restricted to object‑background boundaries, preventing inter‑object style leakage. The method is derived from a region‑wise constrained formulation with a closed‑form solution and can be seamlessly integrated into existing stylization pipelines without retraining. To evaluate efficiency of our IGBR, we introduce quantitative metrics that measure boundary consistency, gradient artifacts, inter‑object leakage, and interior preservation without requiring annotated stylized images. Our experiments and evaluations demonstrate that the proposed IGBR consistently produces plausible boundaries, outperforming prior blending techniques in boundary consistency, gradient stability, and interior preservation. The code is available at https://github.com/Son‑SDT/IGBR.
Authors:Zihao Zhang, Haochen Tian, Tianyu Li, Changhui Jing, Jingliang He, Naisheng Ye, Ziyuan Pu, Zhenjie Yang
Abstract:
Visual foundation models (VFMs) are increasingly integrated into end‑to‑end autonomous driving for their powerful representations, yet it remains unclear when these representations improve driving performance. To investigate this question, we introduce ViRA, a planner‑agnostic visual representation alignment framework that keeps the planner architecture and inference cost unchanged. Our study reveals three findings: (1) VFM‑guided visual representations consistently improve driving performance across diverse end‑to‑end planners, with gains extending to zero‑shot closed‑loop evaluation. (2) The choice of VFM target matters for planning performance, and alignment to a different VFM can further benefit planners with pre‑trained VFM encoders. (3) Auxiliary perception supervision reduces sensitivity to VFM target selection, narrowing the EPDMS spread across five targets from 2.7 to 0.5 points and potentially compensating for less effective VFM targets. Guided by these findings, we develop ViRA‑Diffusion, a diffusion‑based planner trained without auxiliary perception supervision, which achieves 92.3 EPDMS on NAVSIM v2 navtest, outperforming recent methods in our comparison by at least 1.9 points. The results motivate jointly considering target selection and planner supervision when integrating VFMs into end‑to‑end autonomous driving. The results and demo are available at https://github.com/OpenDriveLab/ViRA.
Authors:Gian Luca Bailo
Abstract:
Dual‑process agents pair a fast policy with a slow deliberative model. In real‑time settings the slow model usually runs continuously; in turn‑based agents and robot planners it is invoked on events such as uncertainty or a detected failure. We study a fast learned actor that takes every decision and hands control to a reasoning vision‑language model only when a gate opens, while the game keeps running. We use closed‑loop Doom and the new open "System One" typed‑decision models, served through a common llama.cpp interface. On 900 held‑out questions, (i) zero‑shot decision models from 0.15B to 9B parameters choose to collect items 1.6‑1.8 times more often than chance among their errors, in any option order, although the order changes some models' accuracy; (ii) accuracy, calibration and sensitivity (how well confidence separates right from wrong answers) are distinct: models of similar accuracy differ widely in AUROC, and the confidence of the most sensitive one tracks which kinds of situation it fails, not which answers are wrong; (iii) offline, deferring the least confident 30% of decisions to a reasoning model gains over random deferral in proportion to the actor's AUROC (rank correlation 0.87); with actor and rate chosen on held‑out games the gain is +0.13 [0.08, 0.18] with doomLaya's option order and +0.08 [0.02, 0.14] with shuffled options, and reasoning carries about half of it; (iv) in closed loop (33 games, three seeds) no variant reaches the exit. Committing to plans, the reasoner's or a fixed explore rule's, opens more doors and makes an actor that stands still play; with the rule the agent dies more often. Told that some doors need keys, the reasoner takes ordinary doors for locked ones, which the state cannot tell apart; without that knowledge it goes back to collecting. We release code, prompts, data and logs.
Authors:Shouhei Hanaoka, Takahiro Nakao, Atsushi Takamatsu, Takeharu Yoshikawa, Osamu Abe
Abstract:
Unsupervised anomaly detection, which learns only from normal images, is a central task in medical image analysis and remains an open problem. Reconstruction‑based methods pass an image through an encoder‑decoder network trained on normal data and detect anomalies from the residual between the image and its reconstruction. This works only if the information passed from the encoder to the decoder is limited; otherwise the network learns an identity mapping and reconstructs anomalies too. This limit is usually imposed through architectural choices, tuned per dataset, that cannot be stated in bits. We introduce the quasi‑binarizing (QB) layer, which squashes each latent element into [0, 1] and adds Laplace noise of scale 1/epsilon. Each element is then epsilon‑locally differentially private, and the mutual information between an image and its reconstruction is bounded by a quantity that depends only on epsilon and the number of QB elements, whatever the encoder and decoder. Placing a QB layer on every encoder‑decoder path, including all skip connections, we build QBAE, a seven‑level attention U‑Net with 32,768 QB elements. On the seven datasets of the MedIAnomaly benchmark, QBAE with one architecture and one configuration reaches a mean image‑level AUROC of 0.828, the highest among methods that do not adapt to each dataset, and the best reported results on BraTS2021 (AUROC 0.911, pixel‑level AP 0.838). The noise is kept at test time, so that every reconstruction satisfies the bound. Without input corruption, the bottleneck alone prevents identity collapse (mean AUROC 0.805 vs. 0.590). Code is available at https://github.com/hanaokalog/MedIAnomalyQB.
Authors:Tianle Wang, Jiayu Liu, Ruizhi Zhao, Ning Miao
Abstract:
On‑policy post‑training can improve large language models by learning from their own trajectories, but requires costly rollout generation. We ask whether fixed demonstrations can support competitive off‑policy learning through better supervision. Our premise is that their usefulness depends not only on the training trajectories, but also on whether supervision provides informative preferences among continuations and connects this guidance to the reasoning decision being learned. We introduce Step‑Aligned Privileged Distillation (SAPD), a rollout‑free self‑distillation method that turns demonstrations into step‑aligned distributional supervision. Its key insight is to use the known progression of a reference solution to associate each reasoning transition with targeted privileged guidance, rather than treating the solution as undifferentiated context. On mathematical reasoning benchmarks, SAPD outperforms supervised fine‑tuning and label smoothing on average while remaining competitive with on‑policy reinforcement learning and self‑distillation. Analyses support both the value of context‑dependent distributional guidance and the benefit of aligning privileged information with the current step. SAPD also largely preserves out‑of‑domain coding performance and achieves approximately 2x training‑loop speedups over the on‑policy baselines. These findings suggest that carefully constructed supervision can make fully off‑policy post‑training a competitive and computationally efficient alternative. Our code is available at https://github.com/Miaow‑Lab/SAPD.
Authors:Yuta Sahara, Kento Kawaharazuka, Kei Okada
Abstract:
This study proposes a multi‑objective optimization method for tendon‑driven arm design, addressing the inherent trade‑off between joint torque and arm thickness through tendon wrapping and shortcut effects. We formulate the problem to simultaneously maximize torque and minimize thickness, solved using NSGA‑II. The algorithm optimizes tendon routing, pulley configurations, and attachment points. Results reveal Pareto‑optimal designs with effective moment arm expansion via strategic shortcuts, providing practical guidelines for compact high‑torque arms. The optimized configurations demonstrate non‑trivial patterns beyond conventional design intuition, offering new insights for engineering efficient robotic systems.
Authors:Jiawei Dong, Zhikun Zhang, Quan Yuan, Zhe Liu, Yunjun Gao
Abstract:
The widespread application of graph data inevitably brings significant privacy risks, as its unprotected use can lead to the leakage of sensitive information. These risks are particularly acute in the setting with an untrusted curator, where the data remains decentralized and each user only holds the connections to their neighbors. To mitigate such privacy risks, we adopt local differential privacy (LDP) to collect users' private information and generate a synthetic graph. However, existing methods suffer from either excessive noise injection by perturbing the local adjacency lists or significant structural information loss due to the simplistic graph encoding process. To address these issues, we propose LDPGraph, an effective graph synthesis algorithm that takes one step further by exploiting neighborhood structures under LDP. To obtain neighborhood statistics beyond degrees, LDPGraph aggregates a noisy global view from perturbed adjacency lists and combines it with projected local connections to estimate node‑level triangle counts. To correct the structural inconsistency caused by separately perturbing degree and triangle count, LDPGraph jointly refines them into feasible degree‑triangle targets. To reconstruct a global graph from these estimated targets, LDPGraph adopts a triangle‑first strategy that first preserves local clustering structures and then fulfills remaining degree requirements. Extensive experiments on four real‑world datasets and multiple commonly used graph metrics validate the superiority of LDPGraph. Source code is available at https://github.com/ZJU‑TrustAID/LDPGraph.
Authors:Taejun Kim, Wonil Kim, Jongmin Jung, Hyeongseok Wi, Sangeun Kum, Keunhyoung Luke Kim, Taehyoung Kim, Dongjoo Moon, Seungsoon Park, Taewan Kim, Virginie Berger, Juhan Nam, Jongpil Lee
Abstract:
How can we verify whose music contributed to an AI‑generated output? This paper demonstrates how input‑based attribution can provide verifiable evidence of which audio sources were used in a generation and whether they shaped the output. To do so, we condition the generation solely on audio without any text input, then trace the inputs behind each output, and establish their musical effect. In prompt adherence tests and controlled input swaps, the stems generated by our generator, MixAudio, follow the prompt audio in timbre and the context audio in harmony. Yet these outputs may still reproduce training data not supplied as inputs. We therefore audit memorization with our musical version identification model, musicDNA, and find few reproductions outside the input records. On human‑judged cases within the flagged pool, it achieves higher precision and recall than the other tested memorization detectors. The two evaluations suggest that input records and output analysis provide complementary evidence for attribution, on which rights‑holder reporting and compensation can draw as the AI music economy takes shape. Audio examples are available at https://neutune.github.io/attr2027demo/
Authors:Xijie Gong, Tingxu Han, Jiahao Zhang, Wei Song, Ziqi Ding, Hanqi Yan, Youcheng Sun, Lijie Hu
Abstract:
Tool calling, invoking external tools on demand, is central to agentic LLMs, yet the mechanism that decides whether a model calls a tool or responds directly remains poorly understood. Agentic prompts are long and heavily scaffolded, combining role instructions, tool schemas, format templates, and the user's request across hundreds of tokens, creating a noisy, highly entangled context in which no single controllable variable for mechanistic analysis is obvious. To obtain such a variable, we propose a method that converts complex agentic prompts into minimal contrastive pairs in which a single request verb determines the tool‑call decision: replacing an execution‑verb (e.g., write) with an analysis‑verb (e.g., discuss) reliably flips the decision, suggesting it is mediated by a compact internal state. We construct 500 such paired prompts across Python, Java, and C++ (300 for mechanistic analysis, 200 held out for evaluation). We trace the decision to a vector, μ_Δ, that is both causally necessary and sufficient and generalizes beyond the discovery prompts to native multi‑turn τ^2‑Bench trajectories and verb‑free requests. Behavioral ablations show that the scaffold establishes a tool‑call prior; Transcoder decomposition then reveals that analysis verbs suppress this prior through features signaling that tool use is unnecessary, whereas execution verbs largely leave it intact. Downstream scaffold‑reading attention heads and MLP features read out the resulting state, and the same mechanism recurs across seven models from the Qwen, Mistral, and Granite families. Our code is available at https://github.com/XijieGo/MI4ToolCalling.
Authors:HyeonSeok Lim, SeungWoo Song, Inho Won, Hoyun Song, Jihyo Kim, KyungTae Lim
Abstract:
Skeleton‑based reasoning prompting is a promising training‑free approach for structuring LLM reasoning, but prior work largely assumes an English‑centric setting. We propose the Language‑Aware Skeleton Exploration Framework (LASEF) to study skeleton‑language choice in multilingual mathematical reasoning. Across math benchmarks, model scales, and languages, we show that English skeletons yield a small positive tendency on average, most visible for smaller models and low‑resource languages. However, few language‑level gains remain significant after correction, and English is not universally optimal. Combining greedy decoding, multi‑rollout evaluation, translation ablation, and cross‑benchmark validation, we further find three patterns of skeleton‑language effects: directionally consistent, evaluation‑ and benchmark‑dependent, and asymmetric negative. These effects cannot be fully explained by generation quality alone. Overall, skeleton language is a context‑dependent design variable that requires multi‑level exploration. All resources are released at https://github.com/lhsstn/LASEF.
Authors:Yi Yang, Xiang Fei, Lehong Wang, Zilin Dai, Ruogu Li, Jiting Cai, Liyao Chang, Xinyi Yang, Henry Kou, Ruijie Fu, Lu Li, Howie Choset
Abstract:
Goal‑conditioned dynamic manipulation of deformable linear objects has mainly specified goals as positions for a rope tip to reach. Many tasks, however, depend on how the tip arrives. We therefore study single‑swing rope striking with goals that specify the tip's 3D position and arrival direction, across the workspace and on different ropes. This is challenging because rope dynamics are hard to model, no demonstrations exist, distinct swings reach the same goal with different reliability, and the sim‑to‑real gap extends beyond the rope. To address these challenges, we extend the state‑of‑the‑art DLO simulator DeformX with GPU acceleration, a stable Cosserat rod solver, and a cross‑flow aerodynamic model, yielding DeformX2.0, which is more than 20,000× faster. We then propose TRACE (Trace‑rooted Adaptive Cross‑Entropy), which generates striking data by warm‑starting each new target from the stored swing whose tip path passes closest to it. Its cost penalizes rope bending and abrupt tip motion to favor repeatable swings. A conditional flow‑matching policy trained on this data reaches 92.1% accuracy in simulation. Finally, we propose RECAP (Residual Calibration Policy), which fits the simulator's rope and rig parameters to a few calibration swings and adapts actions with a correction policy trained in simulation. On a real robot, across three ropes, RECAP raises success within 5cm from 72% to 87% for position goals, and within 10cm and 10° from 50% to 79% for goals that also specify the arrival direction.
Authors:Leizhen Wang, Peibo Duan, Zhenlin Qin, Yancheng Ling, Jian Xu, Yue Wang, Hao Wang, Zhenliang Ma
Abstract:
Effective traffic signal control (TSC) requires policies that respond to changing traffic demand and network conditions while meeting different control objectives. However, adapting existing strategies often involves repeated manual design and adjustment, making it difficult to systematically explore better control rules for a target network. Large language models (LLMs) can automate this process, but directly using them to select signal phases leaves decision rules embedded in black‑box models and incurs recurring inference costs and latency. This paper formulates TSC as a modular program design problem and proposes EvoSignal, an LLM‑guided evolutionary framework using traffic knowledge and performance feedback. The modular representation separates traffic feature extraction, local phase prioritization, and optional network‑based priority adjustment. Starting from several established strategies, EvoSignal improves programs through feedback on congestion and signal operation, retaining strategies with different performance trade‑offs. The resulting programs operate without online LLM inference. Simulation experiments across five scenarios on two real‑world road networks show that the selected default EvoSignal program reduces waiting time by 16.8‑‑49.2% relative to the lowest waiting time achieved by the 20 conventional, reinforcement learning‑based, and LLM‑based baselines in each scenario. A program prioritizing travel time and queue length outperforms all 20 baselines on all three metrics in the search scenario and remains among the top three on each metric when transferred unchanged to the other four scenarios. These findings support automated design of inspectable control programs that transfer across the evaluated road networks and traffic demands.Code is available at https://github.com/georgewanglz2019/EvoSignal.
Authors:Samuel Margolis, Paul Schmiedmayer, Alan Huang, Ethan Chen, Ishan Bhattacharjee, Atman Shah, Ben Viggiano, Fang Cao, Shriya Reddy, Roger Xia, Jack O'Sullivan, Daniel Katz, Matthew Wheeler, Euan Ashley, Bruna Gomes
Abstract:
Drug target discovery requires distinguishing molecules that causally drive disease from those that are merely associated with it. Training and evaluating AI agents to perform this workflow end‑to‑end is difficult because real world biobanks lack known causal ground truth and participant‑level data is access controlled. We introduce DrugTargetWorld, a framework that procedurally generates simulated biobanks, or "worlds," with known but concealed causal structure. Each world contains genotypes, proteins, health records, outcomes, and synthetic magnetic resonance imaging (MRI) for 54,000 participants. Agents must construct a disease phenotype, identify causal driver proteins, infer the beneficial direction of modulation, and optionally conduct virtual 'wet lab' experiments. We evaluated nine agents in 540 episodes across 20 cardiovascular worlds and three experimental budgets. Opus 5 and GPT‑5.6 Sol achieved the highest mean composite scores, 39.98 and 35.38 of 100, respectively, and both recovered 64% of causal drivers on average. However, no agent reliably distinguished misleading non‑causal proteins, and performance remained limited by the integrative judgments required to connect phenotype construction, causal evidence, and intervention decisions. By making each world's causal structure known to the evaluator but hidden from the agent, DrugTargetWorld turns end‑to‑end drug target discovery into a scalable training and evaluation problem with verifiable reward.
Authors:Vaibhava Lakshmi Ravideshik, Mayank Kejriwal
Abstract:
Entity Alignment (EA) identifies equivalent entities across knowledge graphs and is critical for knowledge base integration and ontology merging. Evaluating EA systems at scale requires expensive expert annotation, making systematic assessment across diverse domains practically infeasible. LLM‑as‑judge evaluation offers a potentially scalable alternative, yet its reliability for structured prediction tasks like EA remains unstudied. We present the first systematic benchmarking study across three frontier models, three datasets, and four EA systems, using perturbation bias diagnostics, meta‑evaluation across all dataset‑judge‑prompt combinations, and counterfactual label‑flip tests. We identify anchor bias, a failure mode in which judges invert discrimination when the system's decision label is visible. Label exposure causally collapses judge discrimination (J‑ROC‑AUC 0.12‑0.87), while a label‑free protocol recovers near‑ceiling capability on distinctive‑name datasets (0.93‑1.00) and significant recovery on biomedical pairs (0.93‑0.95). Counterfactual experiments confirm causality (FSR 53‑99%) and reveal a frontier model paradox: stronger judges exhibit greater label sensitivity, not less. A blinded two‑annotator human evaluation (102 pairs, Cohen's kappa=0.902) confirms this mechanism directly. We release the first biomedical EA benchmark (MeSH‑SNOMED CT, 15K pairs) and a reproducible auditing framework for LLM judge reliability in EA. Code and data are available at https://github.com/vaibhavalakshmiravideshik/llm‑as‑a‑judge‑entity‑alignment.
Authors:Yulin Wang, Mengting Hu, Hongli Li, Jianghao Zhou, Chen Luo
Abstract:
Real‑world applications require 6D pose estimation to be accurate, fast, and scalable to unseen objects. This paper introduces WAPR, a zero‑shot wide‑angle pose refinement model that refines candidate poses with rotational deviations up to 90 degrees. With as few as 12 candidate poses per detected object instance, WAPR supports fast inference within 1 s per frame and reaches a pose‑estimation throughput of up to 25 detected object instances per second. To support wide‑angle training for rotationally symmetric objects, WAPR uses rotational symmetry priors to canonicalize symmetry‑equivalent pose targets before loss computation. We further construct SA6D, a large‑scale 6D training dataset with such priors. SA6D obtains KASAL‑assisted rotational symmetry priors for 944 GSO scans and expands them through geometry and texture augmentation into about 50K augmented object instances and about 2M rendered RGB‑D images. In addition, an angle‑balanced loss stabilizes learning across different angular ranges by reducing the influence of uninformative large‑error cases. Experiments on seven BOP core datasets show that WAPR achieves state‑of‑the‑art performance in unseen‑object 6D pose localization and detection under both fast and unconstrained inference settings. Project page: https://github.com/WangYuLin‑SEU/WAPR.
Authors:Mengxin Zhang, Yulin Wang, Chen Luo, Yongzhe Li, Yijun Zhou
Abstract:
Rotational symmetry is an important prior in 6D pose estimation, improving pose accuracy and supporting symmetry‑aware evaluation. However, current symmetry annotations for 3D objects remain largely manual or semi‑automatic, often requiring predefined types or orders, which limits scalability. This work introduces a fully automatic, reference‑free framework for symmetry‑type classification, rotational‑order identification, and full‑axis localization across all eight canonical 3D rotational symmetry types. The method localizes a dominant high‑order axis, infers its rotational order through self‑consistency analysis, and reconstructs the complete symmetry structure under a hierarchy‑guided formulation. A texture‑aware extension further models appearance‑induced reductions in rotational order while preserving axis orientations. Experiments on idealized and real‑world datasets demonstrate strong accuracy and generalization, achieving 94.75% accuracy on 438 symmetric objects in GSO. Training FoundationPose with these priors improves accuracy by up to 0.9% across five BOP datasets, showing that automatically estimated rotational priors improve downstream 6D pose estimation. Code is available at https://github.com/WangYuLin‑SEU/KASAL.
Authors:Jackson Eshbaugh, Jorge Silveyra
Abstract:
We introduce a calibration‑first framework that produces supervision scores without access to ground‑truth labels or a shared annotation space. Our framework aligns subset‑specific scorers using a synthetic ordinal reference space before fusion. This reference space is constructed from ordered calibration features that represent the latent concept, providing a common scale on which otherwise incomparable scorer outputs can be aligned. Because our calibration procedure uses the reference space rather than training samples, it is independent of the training set's empirical distribution. Across three benchmark datasets, our framework consistently outperforms uncalibrated averaging and achieves higher primary‑metric point estimates on the evaluation metrics than the best individual scorer. Performance relative to sample‑dependent baselines varies by domain, with absolute differences below 0.02 on Ames Housing and below 0.01 on Breast Cancer Wisconsin and Wine Quality. After Bonferroni correction, differences remain significant for all three comparisons on Ames Housing and one on Breast Cancer Wisconsin. Additionally, we show that using fewer calibration levels per feature can closely approximate higher‑resolution results at substantially lower computational cost. Together, these results support our framework as a viable approach to construct supervision scores when neither ground‑truth labels nor a shared annotation space is available.
Authors:Runze Li, Hanchen Wang, Ying Zhang, Wenjie Zhang
Abstract:
Schema matching increasingly uses generative language models to rerank retrieved column candidates, although the underlying task is a bounded correspondence decision. We present JevNexus, which combines typed pairwise decisions with schema/instance evidence and invokes listwise refinement only when the evidence disagrees and the fused margin is small. The evaluation covers 561 cases from six benchmark families. JevNexus obtains dataset‑macro MRR and Hits@1 of 0.930 and 0.909, compared with 0.926 and 0.903 for Magneto, while reducing mean latency from 123.452 to 15.929 seconds (7.750). Paired analysis finds no statistically significant difference in either MRR or Hits@1. The gate invokes listwise refinement for only 5.665% of source columns and avoids the degradation caused by unconditional refinement. Code and experimental artifacts are available at https://github.com/RazeenLI/JevNexus.
Authors:Joohwan Seo, Xiaofeng Guo, Jinkun Cao, Roberto Horowitz, Rocky Duan, Guanya Shi, Koushil Sreenath
Abstract:
Precise end‑effector tracking during humanoid whole‑body motion is challenging due to floating‑base oscillations, gravity, dynamic coupling, and locomotion‑induced disturbances. We propose ResGAC, a whole‑body humanoid controller for precise end‑effector pose tracking that combines geometric admittance control (GAC) with residual reinforcement learning. GAC provides structured \SE task‑space feedback and generates nominal arm joint‑position targets, while residual RL compensates for unmodeled dynamics and coordinates locomotion and balance in the shared joint‑position action space. The left‑invariant geometric formulation allows the same GAC law to be used across manipulation reference frames. This enables the use of a ground‑attached heading frame that preserves planar locomotion while removing pelvis roll, pitch, and heave from the manipulation reference, thereby reducing reference‑induced end‑effector motion during locomotion. ResGAC is validated on a real Unitree G1 humanoid. Across four standing end‑effector tracking benchmarks, ResGAC consistently outperforms representative baselines, including SONIC, achieving lower translational and rotational errors. Real‑world experiments further demonstrate reduced propagation of pelvis motion to the desired end‑effector pose using the proposed ground‑attached heading frame. ResGAC achieves 90% success in a standing peg‑in‑hole task compared with 50% for SONIC, and accurate world‑frame \SE end‑effector pose tracking during lower‑body motion. Experimental videos are included in the supplementary material and are also available on the project website: https://resgac.github.io/ResGAC‑website/.
Authors:Rohit Kumar Sen, Anik Chowdhury
Abstract:
Political discourse often uses rhetorical and persuasive language to frame narratives, influence public opinion, and mobilize audiences. While Bangla natural language processing has made progress in sentiment analysis and opinion mining, systematic benchmarking of transformer models for fine‑grained rhetorical and persuasion technique detection in Bangla political speech remains largely underexplored. This paper presents a benchmark study of transformer‑based models for detecting rhetorical form and persuasive intent in Bangla political discourse. Using BanglaRhet, a manually annotated corpus of 30,289 Bangla political speech segments collected from publicly available political news sources, we formulate two supervised single‑label classification tasks: rhetorical technique detection (contrast, repetition, exaggeration, metaphor, rhetorical questions) and persuasion technique detection (blame assignment, call to action, unity call, moral, emotional, and logical appeals). We evaluate four transformer‑based models, BanglaBERT, BanglaBERT‑Base, SahajBERT, and XLM‑RoBERTa‑Base, against classical TF‑IDF baselines. BanglaBERT achieves the highest performance, with 65.40% macro‑F1 for rhetorical technique detection and 66.46% for persuasion technique detection, outperforming the best tuned classical baseline by 19.2 and 13.8 macro‑F1 points, respectively. Class‑level analysis indicates that errors are mainly associated with semantic overlap among labels, figurative language, and class imbalance. The results provide initial benchmark baselines for Bangla rhetorical and persuasion‑aware political discourse analysis and highlight the need for context‑aware and multi‑label modeling.
Authors:Zirun Zhou, Jingfeng Zhang, HaoChuan Xu, Xizhe Zhang, Elliott Wen, Jing Sun, Hong Jia
Abstract:
Backdoored vision‑language‑action (VLA) policies can preserve benign task performance while producing malicious actions when a trigger appears. Detecting such activation is difficult because malicious behavior can comprise individually plausible actions, while unfamiliar tasks introduce legitimate changes in observations and behavior. We introduce TMT, a runtime backdoor detector based on Token Manifold and latent Transition modeling. Trained on benign rollouts, its two branches assess input‑token structure and prediction errors in adjacent‑layer latent dynamics. A suspicious rollout identified by the token manifold branch, once confirmed through latent deviations, guides transition selection for subsequent monitoring. We further explore policy purification through self‑distillation: a frozen copy of the backdoored policy provides benign‑input actions to supervise a student on paired benign and triggered observations, without requiring a separate clean reference policy. For evaluation, we adapt traditional backdoor detectors and repurpose anomaly and failure detection methods as VLA backdoor detectors. In a post‑hoc comparison with ten baselines, TMT achieves state‑of‑the‑art backdoor detection performance on unseen tasks across three VLA backdoor attacks. Our project page is available at https://zzr42.github.io/tmt/.
Authors:Seungjun Moon, Subin Jeon, Sangwoo Kim, Hanbyul Joo, Jinwoo Shin
Abstract:
Recently, approaches that leverage human video datasets for robot policy training have become increasingly prevalent. However, most existing hand trackers regress pose from cropped frames with limited priors on hand motion and object interaction, resulting in inaccurate and physically inconsistent estimates. Moreover, the lack of physical cues, e.g., contact and force, limits the use of human videos for robot policy training. To this end, we propose RLHND, a video foundation model‑based hand tracking model that jointly estimates hand pose and realistic tactile information from monocular egocentric videos. RLHND turns the pre‑trained Cosmos 3 video diffusion backbone into a deterministic clip‑level feature extractor via clean‑latent conditioning, carrying its learned priors on hand motion and hand‑object interaction into tracking. For pose estimation, RLHND (i) predicts hand poses with anatomically plausible joint angles and (ii) enables optional conditioning on the shape parameter to maintain consistent hand shape within the same video and even across videos recorded by the same actor. For tactile estimation, a separate tactile expert stream, trained with the pose stream frozen, predicts dense contact and force over the hand surface. We further adopt LBS‑based feature spreading to enable vertex‑wise feature extraction without costly per‑vertex attention. RLHND achieves state‑of‑the‑art performance across various benchmark datasets for pose estimation, while also achieving state‑of‑the‑art performance in contact and force estimation. Moreover, we demonstrate the utility of RLHND for robot learning through retargeting results and real‑world robot experiments. The code will be publicly available at https://seungjun‑moon.github.io/rlhnd/.
Authors:Yeonseo Lee, Hyosup Shin, Guebin Hwang, Sungho Jo
Abstract:
Vision‑Language‑Action (VLA) models are effective at understanding what task to perform, but provide limited control over how it should be executed, such as moving quickly or slowly. We introduce TempoBridge, a lightweight framework that uses frozen VLA representations to modulate actions according to tempo cues in the instruction at each task phase, without additional tempo‑conditioned robot demonstrations or tempo‑specific base‑policy fine‑tuning. TempoBridge extracts tempo cues from contextual VLM representations, aligns them with task progress through a causal phase router, and modulates nominal motion commands during execution. Across LIBERO tasks, TempoBridge improves Tempo Success Rate from 52.6% to 89.7% under canonical tempo instructions while retaining high task success. It also preserves near‑baseline performance when no tempo cue is present and generalizes to unseen tempo expressions without additional training. Experiments on a physical robot further demonstrate language‑conditioned tempo modulation in real‑world manipulation.
Authors:Hanqiu Li Cai, Chema Garabito
Abstract:
Pixel‑space diffusion models avoid the lossy VAE of latent models, which suggests an advantage on downstream tasks where fine‑grained detail matters. We test this claim along both routes to a pixel‑space backbone. We pretrain Iris‑3B, a 3B‑parameter pixel‑space text‑to‑image transformer, from scratch through a 256\to512\to1024 curriculum, after first ablating the prediction target and representation alignment at 256^2 to decide what to scale. We also convert a pretrained latent model, FLUX.2 Klein base 4B, to pixel space. We fine‑tune both families for monocular depth estimation and for image restoration/super‑resolution. We find no significant improvement from using a pixel‑space generative prior. Fine‑tuned for depth with one matched direct‑regression recipe, Iris‑3B is level with the latent FLUX.2 Klein and the converted pixel FLUX.2 Klein falls behind it, and on 4× DIV2K restoration neither pixel model beats a latent FLUX.2 Klein fine‑tune, the converted one trailing it slightly. We document the recipes, the failure modes and the remaining confounds behind this negative result. Nevertheless, Iris‑3B shows that pixel‑space pretraining with the pixel‑transformer (PiT) head of PixelDiT scales to 3B parameters and to text‑to‑image quality competitive with latent models, matching Qwen‑Image on OneIG under the official evaluators at 1024^2. We release its weights and training code in the hope that they help pave the way for further work on pixel‑space generation.
Authors:Benjamin Wilcox, Dawei Gao, Pradeeban Kathiravelu, Douglas Causey, Kewei Sha, Yunhe Feng
Abstract:
Large language models (LLMs) should abstain from scientific multiple‑choice questions when no option is valid, but frequent abstention alone does not demonstrate sensitivity to answer availability. We introduce ArcticQA, a dataset of 194 questions derived from primary Arctic research, with automated checks of answer support and distractor contradiction against source evidence. We further develop ArcticAbstain, a paired benchmark comparing answer‑present and answer‑absent conditions, with the correct answer replaced by a distractor in the latter and an explicit abstention option in both. We evaluate eight models from the Gemini, Claude, and ChatGPT families at high reasoning effort, with three trials per condition, yielding 9,312 recorded responses. Answer‑present abstention rates range from 0.0% to 63.0%, whereas replacing the correct answer increases abstention by 5.05 percentage points on average. These findings highlight substantial baseline differences and the need to evaluate abstention frequency and responsiveness jointly. The dataset and benchmark are available at https://github.com/BenWilcox8/arctic‑qa.
Authors:Jeonghwan Kim, Sofia Stoica, Jiwan Chung, Ansel Blume, Hyeonjeong Ha, Zhenhailong Wang, Xin Luna Dong, Heng Ji
Abstract:
Pre‑trained vision encoders contain layer‑wise visual representations that differ in spatial granularity, semantic abstraction, and sensitivity to local details. However, most Multimodal Large Language Models (MLLMs) rely on only the final or penultimate vision encoder representations or fixed aggregation rules, making visual abstraction largely query‑agnostic and limiting access to fine‑grained cues such as small objects, spatial details, text, and subtle visual attributes. In this work, we propose Mixture of Layers (MoL), an instruction‑conditioned layer routing approach at the visual patch level that dynamically aggregates query‑relevant latent representations from intermediate vision encoder layers. Given a text query, MoL predicts routing probabilities over vision encoder layers and performs a top‑k sparse aggregation over selected hidden states at either the image level, patch level, or through a hybrid routing mechanism. In doing so, MoL enables query‑adaptive access to layer‑specific visual features for fine‑grained visual reasoning. Our experiments across 7 fine‑grained visual reasoning tasks demonstrate substantial performance improvements, especially across fine‑grained visual grounding and understanding tasks such as +18.9% improvement on V in overall accuracy, +4.5% on HRBench4K, and +16.3% on CharXiv compared to the baseline MLLMs, without resorting to multi‑resolution inputs, simple interleaving of multiple vision encoders, or increasing the number of patch tokens. We study vision encoders' receptive field scales across different layers and their sampling behaviors to provide an in‑depth analysis of why layer‑wise sampling is helpful, demonstrating that conditional visual representations are a key step towards better visual perception and reasoning in MLLMs. Our project page is available at https://wjdghks950.github.io/mol.github.io/.
Authors:JunGyu Lee, Jisu Shin, Seunghyun Shin, Hae-Gon Jeon
Abstract:
Crowd simulation plays a central role in robot navigation, autonomous driving, and urban planning. For these applications, realistic simulation requires crowds to adapt their behavior to environmental changes and user objectives. However, existing methods that rely on predefined control settings have limited flexibility in accommodating new user‑specified objectives. To address this limitation, we propose Ctrl‑CWM, a multi‑agent Controllable Crowd World Model that integrates crowd generation and run‑time control. Our key idea is to adapt the world‑model principle of planning using imagined futures to crowd simulation. To this end, Ctrl‑CWM consists of an encoder that learns a representation of human motion dynamics, an actor that proposes pedestrian displacements, a critic that evaluates imagined crowd trajectories, and a planner that selects actions. We first learn human motion dynamics through trajectory prediction on real‑world pedestrian videos and then freeze the encoder to preserve them. Using this representation, the actor generates imagined crowd trajectories through repeated state updates, and the planner combines the critic's scores with user costs to select actions. Repeated planning advances the simulated crowd, while additional user costs introduce new control objectives without retraining. We extensively evaluate crowd generation under varied agent arrival conditions and run‑time control across avoidance and attraction scenarios. Ctrl‑CWM outperforms the state‑of‑the‑art method on most crowd realism and collision metrics, and adapts crowd behaviors to user‑specified objectives introduced during simulation. The project page is available at https://jungyu0413.github.io/Ctrl‑CWM
Authors:Xiaomeng Yang, Yushu Wu, Yi Gao, Yuhao Lei, Xuan Zhang, Pu Zhao, Yanzhi Wang
Abstract:
World‑Action Models (WAMs) utilize future visual prediction as an intermediate reasoning process to guide action generation. However, existing WAMs typically structure visual imagination according to predefined temporal intervals, without explicitly accounting for the different roles of task‑critical interactions and connecting transitions. We argue that effective visual foresight should align directly with task‑relevant interactions and their corresponding reasoning demands. To this end, we introduce an event‑aligned visual action reasoning framework that organizes visual‑action prediction around interaction events. Through event‑aligned visual‑action supervision, WAM learns to generate event‑aligned visual context in each imagined rollout, placing greater emphasis on critical state changes that inform action generation. This shapes the visual reasoning granularity according to the underlying interaction dynamics, with detailed reasoning around task‑critical events and coarser progression through connecting transitions. Furthermore, we introduce an execution validity head that identifies the valid portion of each predicted action sequence, avoiding redundant actions during chunked inference. Experiments demonstrate a 10.26 percentage point improvement in DOMINO success rate over baseline and competitive performance on RoboTwin 2.0. It also transfers from DOMINO Level 1 to Levels 2 and 3 without target‑level adaptation.
Authors:Guillaume Brouillette, Faustin Kagabo, Usef Faghihi, Nadia Ghazzali
Abstract:
Retrieval diversification is widely available in retrieval‑augmented generation (RAG) frameworks, yet prior studies disagree on whether it improves retrieval and answer quality. We show that its effectiveness varies primarily with candidate‑pool redundancy, in a pattern consistent with the number of distinct evidence pieces a query requires. Using controlled near‑duplicate injection and production‑style overlapping chunking, we find that diversification harms relevance, evidence coverage and answer quality on clean pools, but becomes beneficial on multi‑evidence tasks when redundancy causes nearest‑neighbor retrieval to select repeated passages. We therefore introduce a query‑adaptive rule that diversifies only when the effective number of distinct documents in the nearest‑neighbor top‑k selection falls below the query's evidence requirement. Computed from existing embeddings, the rule captures most of the achievable gain, transfers across datasets and encoders and automatically reduces to nearest‑neighbor retrieval for single‑evidence queries. We also introduce RNG‑Score, a geometric reranker with an exact nearest‑neighbor fallback whose margin indicates duplicate structure. Overall, we conclude that diversification should be used selectively, based on observable redundancy and evidence requirements.
Authors:Dominik Schnaus, Thomas Dagès, Daniel Cremers, Xi Wang, Phillip Isola
Abstract:
Multimodal representations enable zero‑shot classification and retrieval, but aligning independently trained models usually requires large amounts of paired data. Yet, the Platonic Representation Hypothesis suggests that models trained on different modalities may converge spontaneously toward a shared representation geometry. But then, do we even need paired examples for cross‑modal alignment? Remarkably, we show that paired examples are unnecessary for coarse cross‑modal alignment. Our simple Wasserstein Procrustes method with a coarse geometric initialization aligns two disjoint embedding sets by estimating a single orthogonal map without seeing any pairs. Across datasets, modalities, and unimodal models, we show that we can consistently align independently trained representations without pairs, and standard geometric alignment metrics accurately predict when this is possible. Nevertheless, we can naturally benefit from paired examples. In the very few‑pair regime, our method substantially outperforms existing ones, while staying competitive with pair‑based methods with more added examples. Finally, we demonstrate that the resulting alignments can enable text‑to‑image generation without paired examples. These results show that independently trained models often share enough geometry to establish cross‑modal correspondence with little or no paired data.
Authors:Yu Ren, Hwee Kuan Lee, Tat-Jen Cham, Jonathan Yap, Khung Keong Yeo
Abstract:
Sparse‑view 3D coronary reconstruction commonly relies on cross‑view correspondence and triangulation, which are vulnerable to vessel overlap and foreshortening, or on volumetric prediction followed by vascular‑graph extraction, which does not directly provide centrelines and radii. We introduce PPCAR‑Net, a projection‑refined parametric coronary artery reconstruction network that directly predicts a branch‑structured centreline‑and‑radius representation without explicit point matching, triangulation, or an intermediate volume. Given a variable number of segmented views, a coarse predictor combines frozen VGGT features with learned branch queries to estimate branch presence, B‑spline centreline trajectories, and dense radius profiles. Projection‑guided geometry and radius refiners then sample local evidence from the input views and apply residual corrections learned with 3D supervision. We evaluate representation fidelity and sparse‑view reconstruction quantitatively and qualitatively. On simulated angiographic masks generated from CT‑derived coronary anatomy, PPCAR‑Net produces better connected artery reconstructions and achieves strong centreline accuracy, particularly for RCA, while maintaining competitive volumetric overlap. Coarse‑to‑fine inference takes 121 ms, enabling real‑time reconstruction.
Authors:Ruochi Li, Jianzhe Lin, Haoxuan Zhang, Haihua Chen, Junhua Ding, Edward Gehringer, Yang Zhang
Abstract:
Real‑world documents distribute evidence across text, tables, figures, and captions within complex page layouts. Answering complex questions over such documents therefore requires more than retrieving relevant passages: systems must recover the evidence topology that connects heterogeneous evidence units. Existing GraphRAG evaluations remain largely text‑centered, while multimodal document RAG benchmarks assess cross‑modal retrieval and generation without directly evaluating recovery of the intended evidence topology. We introduce TOPOGRAPHRAG‑BENCH, a layout‑grounded benchmark for multimodal evidence reasoning in GraphRAG, comprising 2,024 questions over 201 long, visually rich documents. Questions are constructed bottom‑up from text, figure, and table evidence units under three controlled topologies: single‑hop retrieval, bridge‑chain reasoning, and multi‑source synthesis. To ensure that questions preserve their intended structure, we apply counterfactual validation for shortcut resistance, modality necessity, and evidence necessity. We evaluate text‑only GraphRAG, page‑level visual retrieval, and multimodal GraphRAG systems using retrieval, generation, and topology‑aware reasoning metrics. Multimodal GraphRAG systems achieve the strongest overall performance, but still fail when visual‑textual evidence alignment or multi‑unit composition is incomplete. Text‑only GraphRAG struggles when key dependencies are grounded in figures or tables, while page‑level visual retrieval lacks the fine‑grained structure needed for topology recovery. These findings motivate GraphRAG systems that move beyond text‑derived entity relation graphs to explicitly model document layouts, cross‑modal evidence alignment, and the reasoning roles of evidence units. Code and data are available at https://richardlrc.github.io/TopoGraphRAG‑Bench/.
Authors:Yufeng Li, Shuxin Li, Zhenhua Xu, Junxian Li, Peng Zeng, Sheng Yao, Changting Lin, Gaolei Li, Ran Bi, Meng Han
Abstract:
LLM agents that interact with a user across many sessions accumulate histories that exceed their context window, so they store past interactions in an external memory and answer each question from a small set of retrieved records. Existing memory systems rank records by lexical or embedding relevance, yet the top‑ranked memories can each be relevant while jointly omitting a complementary fact that the answer requires, especially for multi‑session and temporal questions. Drawing on the distinction between relevance and sufficiency in legal evidence scholarship, we recast memory retrieval as constructing a sufficient memory set. To operationalize this view, we introduce a blinded LLM judgment over the retrieved set, together with Gold Hit and Turn Hit as evidence‑coverage proxies. We then propose Budgeted Flat Reconstruction (BFR), which builds sufficient sets over a fixed flat memory store in two stages. Specifically, we first apply Formal Concept Analysis for Memory Selection (FCA‑MS) to decompose the question into information requirements and select a compact candidate subset that jointly covers them. Then, we repeatedly acquire unseen records through deeper text search or complementary entity and session views, stopping when the budget is exhausted. Experiments on LoCoMo and LongMemEval‑S show that BFR outperforms same‑store adaptations of recent agent‑memory systems in both answer quality and evidence coverage. Specifically, on LongMemEval‑S it raises judged accuracy from 72.4% to 82.2% and Turn Hit to 91.4%.
Authors:Bohan Lu, Kenneth R. Brown
Abstract:
We construct CSS codes with three logical qubits whose logical CCZ gate is implemented by transversal physical T/T^\dagger gates and whose distance is at least 3. These codes start with 8‑divisible X‑stabilizers defined by projective geometry, which guarantees d_Z\geq 3. A SAT solver then finds three X‑logical operators compatible with the transversal CCZ. We build codes at thirteen block lengths from n=48 to n=496, with d_Z=3 or 4, and illustrate the method with the [[48,3,3]] code Q_48. Additionally, we prove that no CSS code with three logical qubits, Z‑distance at least 3, and a quasi‑transversal CCZ gate (transversal T with a diagonal Clifford correction) exists below n=39. The [[47,3,3]] code of Jacinto et al. serves as an upper bound, leaving 39\leq n\leq46 open.
Authors:Bingxuan Li, Yiwen Song, Xueqing Wu, Yanzhou Pan, Yang Li, Kuang Su, Jingyun Liu, Sebastian Ko, Huan Zhang, Tong Zhang, Nanyun Peng, Tomas Pfister, Yale Song
Abstract:
Video interactive storytelling enables viewers to actively steer how a video unfolds. However, once we allow viewers to intervene during generation, a new challenge arises: The viewer's request can have latent dependencies on both the grounding source and the current rendered video state. These dependencies may not be explicitly stated in any individual input, but emerge only when the source, rendered history, and new viewer intent are considered jointly. Existing interactive video generation systems primarily emphasize following viewer instructions, while source‑grounded video generation methods focus on aligning generated content with an external narrative or knowledge source. This leaves a fundamental question underexplored: What context should a generation model ground on during interactive continuation, and how can heterogeneous, unstructured inputs be transformed into such grounding context? In this work, we formulate contextual grounding as the process of transforming heterogeneous input context into an executable constraint model for video generation. To address this challenge, we introduce VIS‑Ground, which performs Structured Context Abstraction to recover grounded states and cross‑context dependencies, Generation Constraints Induction to project relevant dependencies into candidate‑specific constraints, and Constrained Video Generation to enforce these constraints through planning, verification, revision, and rendering. Across three video generation backbones, VIS‑Ground consistently achieves the highest overall composite score, reaching an average absolute improvement of 10.3 points over the strongest per‑backbone baselines. Detailed analysis further shows gains across both narrative and knowledge grounding, and reveals remaining challenges in dependency extraction, and faithful realization during video rendering.
Authors:Zhanhong He, Hanyu Meng, Yaolong Ju
Abstract:
In music beat tracking, model‑predicted activations must be decoded into a discrete, musically coherent event sequence. Direct peak picking closely follows local evidence but can retain spurious peaks or miss weak beats. Dynamic Bayesian networks (DBNs), a widely used structured post‑processor, improve sequence consistency under predefined global tempo, meter, and transition constraints, but their behavior can depend strongly on these settings. We introduce CASM, a context‑aware semi‑Markov decoder that instead conditions its temporal constraint on local activation evidence. CASM also accounts for ambiguity among competing periodic interpretations, including half‑ and double‑tempo alternatives. Deterministic safeguards prevent implausible outputs and preserve beat‑downbeat consistency. Applied to fixed activations from three neural beat trackers (BeatThis, MSCNN, and TCN), CASM improves temporal continuity while preserving event‑level F1 across the GTZAN and SMC datasets, without backbone retraining or dataset‑specific retuning. Further analysis shows that CASM is less sensitive than the DBN baseline to the composition of the calibration data.
Authors:Jeonghyeok Do, Munchurl Kim
Abstract:
Synthetic aperture radar (SAR) and electro‑optical (EO) imagery provide complementary observations: SAR enables day‑and‑night, weather‑resilient sensing, whereas EO provides rich appearance and fine‑grained semantic cues. We introduce SAREO‑FM, which avoids forcing a single token stream to serve two distinct roles: modality tokens preserve how each sensor observes the scene through masked reconstruction, while learnable semantic queries capture what the scene contains under guidance from a pretrained vision foundation model (VFM). By jointly encoding these queries with SAR and EO tokens, the queries acquire modality‑grounded semantic context, while the modality‑token outputs remain the explicit targets of masked reconstruction. This design assigns semantic and reconstruction supervision to separate token streams while preserving their interaction within the shared encoder. Pretrained on the million‑scale SAR‑1M corpus, SAREO‑FM achieves strong unimodal transfer for both SAR‑only and EO‑only inputs, while delivering substantial gains from joint SAR‑‑EO observations on tasks that benefit from complementary sensing.
Authors:Junzhe Shi, Yuan Gan, Shida Jiang
Abstract:
Vision‑language models (VLMs) often miss small objects in large images. We ask three questions: what limits them, which of these limits better models can remove, and whether the classical way of handling large images, local decomposition, still has a future. We answer them with a theory built on two quantities of the image interface: S, the number of visual tokens across an object's side, and L, the content a call must cover. The limits: a W x H image sent whole within N tokens gives an object of side m at most msqrt(N/(WH)) tokens per side. Doubling the token budget N therefore raises the largest S a whole image can reach by only 41%, and seeing an image at the S an object needs costs at least order S^2 tokens whatever the model. If recognition improves gradually with S, any search strategy, zoom agents included, obeys a recall‑cost frontier. What better models can change: the S an object needs and how much one call can carry, measured in bits per object found; the coverage cost remains. Decomposition: yes. Assuming only that more tokens per object and less content per call do not hurt on average, splitting an image cannot lower recall if no view zooms out relative to the whole image and views overlap by one object. Neither part of this condition can be dropped; we bound the cost of every such decomposition, and a simple rule approaches the bound as the image grows. We test the theory in about 177,000 requests on 797 images. On controlled images, none of 40 orderings predicted for 8 VLMs is violated. Checked after the fact on drawings, floor plans, natural and synthetic images, 61 of 89 implied orderings hold significantly and 4 fail, all for OpenAI models given more pixels than their default path. On construction drawings the rule never significantly lowered recall relative to the whole image and raised it by up to 0.28. Code and data: https://github.com/shijunzhe/vlm‑small‑objects
Authors:Tzu-Yu Chuang, Ching-Hsiang Chang, Yi-Hsiu Lee, Yi-Ting Chen, Min Sun, YuanFu Yang
Abstract:
Generative world models provide rich predictions of how manipulation scenes may evolve toward task objectives, yet those futures do not directly expose the compact task variables required by control. When training supervises future prediction alone, terminal goal accuracy is not an explicit learning objective, even when geometric recovery is available. We present Entity‑Level Goal Readout, a learned prediction‑to‑execution interface that makes the executable terminal goal an explicit output of a 3D trace world model. It combines object‑centric pose prediction with translation grounded in observed depth to produce a compact goal in SE(3). A shared Pose‑Native Executor consumes this fixed goal with online object‑pose feedback for closed‑loop control without rerunning the world model. Across five manipulation tasks, the pipeline achieves a mean success rate of 79.69%. Goal diagnostics directly measure terminal goal accuracy, while controlled translation perturbations characterize how execution degrades under goal error. Zero‑shot deployment on a Franka arm achieves 73.33% success on nominal StackCube, 66.67% with distractors, and 75.00% on PickPlate with a target unseen during policy training. These results support treating the prediction‑to‑execution interface as an explicit learned component of world‑model planning rather than incidental post‑processing in the control pipeline itself. Project page: https://claire0730.github.io/executable‑goals/
Authors:Richie R. Suganda, Bin Hu
Abstract:
Safe navigation in dynamic environments requires robots to plan under obstacle predictions whose errors are uncertain, non‑stationary, and can induce rare but safety‑critical failures. Existing control‑barrier‑function safety filters can reject immediately unsafe controls, but they provide little guidance on when the current finite‑horizon planning mode itself is becoming unsafe as prediction uncertainty evolves. We propose Conformal Event‑Triggered Risk‑Certified Replanning (\emphCERT‑Replan), a framework that uses calibrated barrier risk as an early‑warning signal for replanning. CERT‑Replan calibrates horizon‑indexed obstacle‑prediction residuals online and uses the resulting uncertainty radii to evaluate dynamic‑obstacle safety margins. A one‑step safety filter protects the next applied control, while a horizon‑level risk monitor evaluates the upper‑tail CVaR of predicted barrier‑violation losses along the current MPC rollout. When this risk exceeds an allocated budget, CERT‑Replan rejects the current planning mode and selects a lower‑risk alternative, such as a different speed profile, corridor, or homotopy class, rather than repeatedly correcting the same nominal plan. In a non‑stationary benchmark, CERT‑Replan achieves an \(83.3%\) collision reduction relative to the safety‑filter‑only baseline \textcolorblackand a \(77.8%\) reduction relative to simple replanning triggers, while reducing average safety‑filter intervention by \(32.4%\). \textcolorblackWith Trajectron++, CERT‑Replan achieves \(96%\) collision‑free operation. Hardware experiments and onboard runtime profiling demonstrate computational feasibility.
Authors:Yuhan Wen, Jiawei Wang, Qixuan Zhang, Calvin Zhang, Yusen Qin, Lan Xu
Abstract:
Robot co‑design couples morphology search with policy learning, yet training every new design from scratch discards acquired control experience. We present LACE‑CRAFT, which compares continued learning on the current robot with policy adaptation to new morphology‑reward pairs. LACE resumes the incumbent's full learning state and initializes compatible challengers with its actor parameters and observation statistics. A fixed task metric selects among both branches and the frozen incumbent. CRAFT coordinates Feedback, Morphology, Reward, and Integration roles through shared experimental records and behavioral replays to generate and cross‑review paired proposals. A generative extension converts generated meshes into editable articulated models with configured joints, actuator interfaces, and consistently updated simulation assets. Across five locomotion benchmarks, mean scores over three evaluation seeds are 6.4‑91.9% higher than D2C. Both methods train 30 new morphology‑reward pairs over five rounds; LACE additionally uses four continuation training units. Five‑task ablations examine policy inheritance and replay‑derived feedback. A fabricated prototype demonstrates indoor walking and illustrates the geometry‑to‑hardware workflow.
Authors:Huang Huang, Wensi Ai, Ziyu Chen, Youhui Wang, Zijian Du, Yang Liu, Jiaolong Yang, Li Fei-Fei, Jiajun Wu
Abstract:
Simulation enables large‑scale, low‑cost robot data generation, but policies trained in simulation often fail to transfer to the real world due to the sim‑to‑real visual discrepancies. Existing approaches often rely on intermediate representations, which can discard rich semantic information or require additional perception modules at deployment. We address this visual sim‑to‑real gap with RoboRender, a framework that converts simulated trajectories into photorealistic RGB videos for policy learning. RoboRender trains a robot‑oriented video generation model conditioned on simulated depth videos, language instructions, and robot RGB mask videos, preserving simulator geometry, robot motion, and action labels while synthesizing realistic textures, backgrounds, and distractors. The generated RGB videos are paired with simulator‑provided states and actions to train policies for zero‑shot real‑world deployment. On robot video test sets, our video model outperforms depth‑conditioned video generation baselines in generation quality. In real‑world experiments across pick‑and‑place, articulated‑object manipulation, and mobile manipulation tasks, policies trained on RoboRender‑generated data achieve a 71% average success rate, outperforming raw simulation renderings and conventional visual domain randomization by approximately 7.1x and 3.6x, respectively. We further show that policy performance improves with more generated videos per simulation trajectory, increasing opening‑task success by 65 percentage points. These results demonstrate that generative video rendering mitigates the visual sim‑to‑real gap for zero‑shot policy transfer. Project website: https://robo‑render.github.io/.
Authors:Weicheng Dai, Shantanu Ghosh, Kayhan Batmanghelich
Abstract:
Reconstructing 3D Computed Tomography (CT) images from a few X‑ray projections is a highly ill‑posed inverse problem due to the loss of volumetric information. We propose PhyDiCT, a training‑free framework that integrates a differentiable Physics‑based forward model, grounded in the Beer‑Lambert law, with a text‑conditioned Diffusion as a strong prior to reconstruct 3D lung CT images. We refer to our approach as training‑free since the prior model is used without fine‑tuning, and our goal is to steer the denoising procedure to generate samples consistent with X‑ray observations. We guide the diffusion generation using Split Gibbs sampling to jointly optimize for projection fidelity (reward) and consistency with prior knowledge. Also, we introduce a test‑time refinement step that enhances image realism and anatomical coherence. We extensively evaluate our method on publicly available 3D CT datasets using both perceptual and semantic metrics, demonstrating that it surpasses existing plug‑and‑play diffusion and fully trained reconstruction approaches. Our findings highlight that combining a strong generative prior with the underlying physics of image formation substantially improves reconstruction quality, e.g., 7.5% improvement on SSIM compared to full training methods. Code will be released at https://github.com/batmanlab/PhyDiCT.
Authors:Wanjing Han, Levi Taiji Li, Mu Zhang, Yue Jiang, Guanhong Tao
Abstract:
Modern web agents built on large vision‑language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red‑teaming approaches use adversarial visual content to manipulate this process. However, they primarily target model inference and do not explicitly account for structured input processing or action post‑processing. Consequently, model‑level success does not establish control over browser execution and cannot reliably characterize end‑to‑end agent robustness. To address this gap, we formulate red teaming for vision‑grounded web agents as an end‑to‑end grounding‑to‑execution problem, and introduce WebMirage, a framework that crafts localized visual perturbations that cause agents to select attacker‑controlled content and execute the corresponding browser action across varying webpage renderings. It uses a role‑slot abstraction and webpage recomposition to capture competition among webpage elements, and dataflow analysis to align optimization with action post‑processing. We evaluate WebMirage across four agent configurations and six VLM backbones on 2,250 tasks covering 13 public websites and a sandbox benchmark. WebMirage achieves an average attack success rate of 91.9%, compared with 17.4% for the strongest baseline, and remains effective against three agent‑level defenses.
Authors:Zhengming Zhu, Yang Liu, Xiao Gao, Aude Billard
Abstract:
Robot throwing has emerged as a promising technique for improving efficiency in logistics and warehouse automation, by enlarging the workspace and speeding up the process. To significantly increase the throwing system's throughput, we develop strategies for throwing multiple objects in one swipe. Such multi‑object multi‑target throwing (MOMT) leverages the large degrees of freedom of anthropomorphic hands. The key is to quickly generate fast and feasible throwing motions, which involves a complex trade‑off between short trajectory duration and short planning time. We solve this problem in two stages. Offline, we build a model of the feasible set by combining object's inverted flying dynamics and the robot's kinematics and dynamics. Online, we generate feasible throws through fast solution matching and filtering of object's valid detach state and robot's feasible state that can compose sequences of throws in less than 5 ms. We validate the framework on a 7‑DoF manipulator equipped with a multi‑fingered hand. In simulation, coordinated two‑object throwing reduces execution time by up to 46% compared to independent single‑object planning, and this improvement is maintained when scaling to three objects. Real‑world experiments with two objects confirm a 29% reduction; the remaining gap to the theoretical 50% is attributed to inter‑throw transition overhead. When target positions are randomly changed mid‑execution, the system re‑plans and successfully reaches the new targets within 100 ms latency without stopping the robot. These results establish the first unified planning framework for MOMT throwing ‑‑ demonstrating scalability to multiple objects in simulation and real‑world feasibility on two‑object tasks ‑‑ advancing the frontier of high‑throughput robotic manipulation. A video summarizing the method and the hardware experiments is available at https://liuyangdh.github.io/momt‑video
Authors:Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
Abstract:
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one‑step or few‑step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation‑free and data‑free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow‑map pushforwards to the teacher marginals. On ImageNet 256×256, our method attains an FID of 2.04 with a single function evaluation (1‑NFE) and a 4‑NFE FID of 1.37 within 40 epochs of training, surpassing the state‑of‑the‑art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.
Authors:Saba Sturua, Han Xiao
Abstract:
Visual document retrievers are trained contrastively: each query is matched to one page labelled relevant ‑ the positive ‑ and pushed away from negatives, pages presumed irrelevant. Recent methods distil a vision‑language model (VLM) teacher into the retriever by enriching that positive, transferring the teacher's attention over it or a description of it. We ask whether the teacher is better spent on the other side, judging the candidates the retriever mines as negatives, which the label says nothing about. With student, data, optimizer and evaluation fixed, teacher‑judged hard negatives and score distillation raise ViDoRe v2 nDCG@5 from 55.2 to 62.6 and 63.0; description alignment, as adapted here, gains 2.6 points and attention grounding nothing measurable. Against teacher‑free rules that select four candidates from the same mined pool at identical training compute, the best of which is the positive‑aware threshold current systems use, the teacher's judgement adds 4.1 points on v2 and 1.7 on v3. This is consistent with how incomplete the labels are. Annotators judge about two of a query's four top‑ranked mined candidates relevant, none of them labelled, so training pushes the retriever away from relevant pages treated as negatives. What reaches the student is coarse: under a greedily decoded 0‑100 rating prompt, 82% of the teacher's ratings come back at one end of the scale or the other, and a relevant/irrelevant partition keeps most of the distillation gain. A ten‑annotator audit places the teacher within the range of variation among human annotators, and finds it reliable where a query has a single determinate answer. We release the code, the teacher's 3.3M judgements and page descriptions, the mined pools, the human audit and the trained adapters at https://github.com/elastic/vdr‑teacher‑signals.
Authors:Param Biyani, Krishnamurthy Dvijotham
Abstract:
As autonomous coding agents get increasingly deployed, the risk that accidental or adversarially injected misspecifications in tasks lead to dangerous agent behavior is critical to address. Prior work has shown that agents given such tasks rarely flag the conflict and instead cheat, editing tests or hard‑coding expected outputs, and the actions taken to cheat can cause real damage, such as deleting a security defense to make a corrupted test pass. It remains unclear whether such conflicts can be established with independently verifiable evidence before the agent acts. We present SpecGuard, which detects and formally certifies these conflicts between task intent and tests. Given only the task description and codebase, SpecGuard autoformalizes the intended behaviour into a Lean 4 specification. The tests are formalized independently, and the Lean kernel checks whether any implementation could satisfy both formalizations, producing a machine‑checked certificate when none can. On conflicted SWE‑bench tasks, SpecGuard detects up to 72.8% of conflicts and formally certifies up to 51.1%, with a nearly five‑fold lower conflict miss rate than model‑based judgment. SpecGuard provides a pre‑execution safety check that identifies reward‑hacking opportunities through formal certification of task‑level conflicts, before any agent behavior is observed. Our code is available at https://github.com/prmbiy/specguard.
Authors:Marek Šuppa, Ivan Vykopal, Andrej Ridzik, Kristián Sopkovič, Natália Kňažeková, Jaroslav Kopčan, Miroslav Blšták, Viktória Ondrejová, Daniel Hládek, Michal Gregor, Martin Tamajka, Marián Šimko
Abstract:
Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation. We present sk‑bench, a native‑first Slovak benchmark with 30 datasets (33 scored task variants) across ten skill categories. Eleven resources are introduced or first packaged for generative‑LLM evaluation, including IFEval‑SK with Slovak‑adapted instruction checkers and native Chiby/SKJ1 resources for Slovak grammar and morphology. We evaluate 55 open‑ and closed‑weights models under one harness. The best open model trails proprietary APIs by 12.6 points. Model rankings are similar for native and translated closed‑form data (ρ\geq0.98), though translation separates the strongest models less well. By contrast, human‑authored and LLM‑generated QA questions rank models differently (ρ=0.72). For Qwen3‑14B, continued Slovak pretraining lowers the overall score by 13.9 points. A small instruction set restores three quarters of that loss. Test‑time reasoning improves scores by 8.5 to 12.5 points for models of 9B and above. Together, these findings suggest four design lessons for other under‑resourced languages: use native data where translation fails, plan instruction repair after language adaptation, enable test‑time reasoning before scaling up, and avoid overinvesting in target‑language prompts. We release the data and code at https://github.com/slovak‑nlp/sk‑bench
Authors:Justin Jung
Abstract:
We revisit a standard accepted practice in the continuous diffusion language model literature of fixing conditioning prompt tokens clean during training. We make a very simple modification: also noise the conditioning prompt tokens during training. We demonstrate that under this modified training objective, we achieve better generalization in combinatorial reasoning tasks such as Sudoku and N‑Queens, with the largest gains on harder variants (3.73% \to 24.65% solve rate on Sudoku Hard), and increased diversity of generated solutions (50.60% \to 73.79% coverage on 10x10 N‑Queens). We also show measurable improvements to natural language generation quality in modest dataset regimes with Gigaword summarization, but notably demonstrate that gains do not transfer to all natural language tasks (e.g open ended dialogue generation). Our method is a single line change to the training objective, requires no additional inference costs by default, and provides the flexibility of classifier‑free guidance inspired guided sampling. Our \hrefhttps://github.com/LateralIntelligence/noise‑your‑promptcode is publicly available.
Authors:Jiuyi Xu, Xiao Hu, Meida Chen, Peng Gao, Yang Ye, Yangming Shi
Abstract:
Robot world models support policy evaluation, planning, and synthetic data generation, but these applications require predictions that distinguish successful actions from failures. Across four released checkpoints from two architecture families, we observe weak sensitivity to action changes and success‑like predictions on verified failures. Although recent work incorporates failures into model training, which data can repair released checkpoints without changing their architecture or training objective still remains underexplored. To this end, we introduce CureWM, which constructs alternative actions from successful demonstrations across a severity grid, verifies their outcomes through execution in simulation or on hardware, and fine‑tunes released models on the resulting failures and surviving successes alongside nominal demonstrations. This construction provides controlled action contrasts from shared starting contexts. On 484 held‑out LIBERO failure counterfactuals, optimism falls from 80% after fine‑tuning on the official data to 30‑‑43% across four independently fine‑tuned CureWM models (38% mean). In two separate evaluations on a physical robot arm, failure predictions scored as success‑like by a latent‑distance diagnostic decrease from 90% after fine‑tuning on successful demonstrations alone to 33% with CureWM. With failure counts per task, successful replay data, and training budget matched, counterfactual failures yield a success‑‑failure value gap of 0.124, compared with 0.014 for freshly collected on‑policy failures. These findings support execution‑verified counterfactual replay for post‑hoc repair and show why reduced optimism must be evaluated alongside success‑‑failure discrimination. Code is available at https://github.com/jiuyixu25/CureWM.
Authors:Sanghyun Jo, Chae Yeon Lim, Donghwan Lee, Sihyun Kim, Soo Ye Kim, Kyungsu Kim
Abstract:
Reference‑based object compositing inserts or replaces an object using a background image, a reference image, and a 2D compositing mask. These inputs guide appearance and placement but leave the completed scene's geometry implicit, which can distort object structure or alter the surroundings. Our Depth‑to‑RGB (D2R) framework predicts composite depth for a scene not yet observed in the RGB inputs. It learns reference‑conditioned corrections to a frozen depth estimator using encoder features of paired completed scenes as targets. The unchanged decoder maps the corrected representation to the intended scene's depth, which a separately trained renderer holds fixed during RGB synthesis. Under matched architecture and training, encoder‑feature supervision reduces OOD Stage‑1 AbsRel by 31.4% relative to decoded‑depth supervision. We also introduce AnyInsertion++ with paired in‑distribution and category‑disjoint splits to evaluate generalization beyond compositing training categories. The complete D2R system leads 12 open‑source and 3 closed‑source baselines in estimator‑derived geometry and photometric quality on both paired splits. On category‑disjoint data, D2R reduces AbsRel by 43.7% and improves PSNR by 2.4 dB over the matched RGB baseline. Across three unpaired benchmarks, D2R leads both identity metrics and reduces mean CLIP reference cosine distance by 55% relative to the strongest baseline. Project page: https://shjo‑april.github.io/Depth2RGB/
Authors:Kimia Kazemian, Menghan Xu, John Thickstun, Sarah Dean
Abstract:
Induction heads provide a mechanistic account of in‑context learning in sequential data, but existing theory largely assumes that the context relevant to a prediction forms a contiguous block. In multidimensional data, serialization breaks this assumption by scattering spatial neighbors across distant positions in the token sequence. We study how transformers overcome this routing problem in multidimensional stochastic and deterministic cellular automata, where each trajectory is generated by an unknown local rule and presented as a flattened sequence without an explicit coordinate‑based spatial inductive bias. We introduce spatial induction heads, two‑layer gather‑and‑match circuits in which the first layer reconstructs the relevant spatial neighborhood and the second matches the resulting configuration against earlier occurrences. We give two explicit realizations of the gather and show that the positional dimension required for spatial routing depends only on the local neighborhood and spatial dimension, not on grid volume or trajectory horizon. We further construct a matching layer which implements Bayesian counting. The end‑to‑end circuit can approximate the Bayesian posterior arbitrarily closely for stochastic rules and can predict exactly for deterministic rules. Empirically, trained two‑layer transformers generalize to unseen rules in one and two dimensional settings, achieving near‑perfect deterministic rollouts and less than 0.005 nats KL from the Bayes‑optimal predictor on stochastic rules. Attention patterns and layerwise probes align with the predicted gather‑and‑match computation, providing mechanistic evidence for spatial induction in trained transformers.
Authors:Yikuan Li, Pinyan Lu, Fanghui Liu
Abstract:
Parameter‑efficient fine‑tuning (PEFT) offers many parameterizations, yet their methodological and functional differences remain unclear. We compare six methods in language and diffusion models to examine how their parameterizations relate to task performance, forgetting, and changes in pretrained weight geometry. Motivated by the spectrum‑preserving design of orthogonal fine‑tuning (OFT), we first ask whether spectral preservation is itself important for adaptation and retention. We find that the selected LoRA‑family methods also approximately preserve pretrained geometry, and that restoring their slightly drifted singular‑value spectra largely preserves task performance, questioning the necessity of explicit geometric preservation. Beyond this, we observe that some methods exhibit distinct adaptation‑‑retention trade‑offs that vary across settings: LoRA most consistently limits forgetting at competitive performance, DoRA achieves higher mean task scores than LoRA in most comparisons, while PiSSA often incurs greater retention costs. Further intervention experiments suggest that while performance gains from different PEFT methods can be attributed to modifications in different groups of spectral components, we consistently find that restoring dominant rather than intermediate or trailing components produces the largest mean reduction in general‑text NLL or base‑image drift. Together, these results motivate evaluating geometric constraints through their functional consequences rather than preservation alone. Code is available at https://github.com/Kuaaannn/PEFT_methods.
Authors:Songbo Hu, Qiayuan Liao, Yufeng Chi, Kevin Zakka, Yakun Sophia Shao, Pieter Abbeel, Koushil Sreenath
Abstract:
Humanoid robots still struggle to plan contact‑rich whole‑body manipulation from egocentric RGB and proprioception. Workhorse learns such manipulation from robot‑free human demonstrations. A visual planner predicts five‑link targets: the poses of the torso, both wrists, and both feet. A reinforcement‑learning whole‑body tracker follows them on the robot. Both policies train separately on the same recorded human poses, without retargeting. We augment the training data of each policy to imitate the errors that the other makes at deployment. On a real Unitree G1, Workhorse sorts boxes with its hands and a kick, catches a thrown box, and topples and climbs a suitcase. During box sorting, we show recoveries after a person pushes the robot or takes the box away. In a simulated copy of the demonstration room, the system completes box sorting in 77% of episodes, and in 64% under 40 N.s pushes. With both policies retrained from the same demonstrations, a simulated second humanoid completes box sorting in 83% of episodes without pushes.
Authors:Sebastian Senge
Abstract:
Dasgupta, Stevens and Navlakha (2017) showed that the Drosophila olfactory circuit, modelled as a random sparse projection followed by winner‑take‑all, is a locality‑sensitive hash that beats classical LSH. The projection was random because the wiring was unknown. We test it against four electron‑microscopy connectomes (MaleCNS, hemibrain, FlyWire, BANC; four animals, seven hemispheres). First, the 2017 pattern holds in a reimplementation of its protocol on SIFT, MNIST and odour mixtures (GloVe is near chance at short codes for every method): the fly hash beats k Gaussian projections at short hash lengths (3.1x in AP@200 on MNIST at k = 4). Second, against the tested real‑valued Gaussian baseline that advantage is per active cell, not per operation: Gaussian projections given the same projection arithmetic retrieve better on every dataset and input dimension tested. Third, across four connectomes the measured pairing of glomeruli gives no consistent retrieval advantage over degree‑preserving rewiring: retrieval is slightly lower (median ‑1.6%), and the small odour deficits depend on how missing odour responses are treated. Separately, equalising glomerular fan‑out at fixed connection count improves retrieval in the model in every hemisphere, while equalising inputs per cell lowers it on average. Yet the fan‑out profile is similar across the four sampled animals (median between‑animal Spearman rho = 0.86), structural synapse counts do not offset it, and its relation to odour tuning is weak. In the model that uneven allocation costs retrieval. For practice: measured wiring gives no consistent retrieval advantage over degree‑preserving random wiring, so a fly hash needs no connectome data, and its advantage is per active unit, which may suit hardware where active units rather than arithmetic are the binding cost, a hypothesis we do not test.
Authors:Mojtaba Nafez, Aref Mousavi, Mohammad Ebrahim Mahdavi, Mobina Poulaei, Kiarash Kiani Feriz, Mohammad Hossein Rohban
Abstract:
Many organizations fine‑tune publicly available pretrained Automatic Speech Recognition (ASR) models and deploy them in black‑box settings, assuming limited access provides protection. We show this assumption is fragile: adversarial perturbations crafted on the public base model transfer effectively to fine‑tuned target models, severely degrading performance and posing concerns for safety‑critical applications. We propose TransferBreaker, a unified fine‑tuning framework that suppresses adversarial transfer by integrating Base Adversarial Fine‑Tuning, which restricts adversarial training to base‑effective perturbations; Latent Jacobian Regularization, which enforces latent‑space invariance by suppressing adversarially sensitive directions; and HybridGrad‑AFT, which improves robustness against adaptive attacks by interpolating transferable perturbations from base and target gradients. We theoretically justify all components and evaluate TransferBreaker across three languages and four large ASR models, reducing adversarial WER from 92.6 to 27.8. Our code is publicly available at https://github.com/rohban‑lab/TransferBreaker.
Authors:Shripad V. Deshmukh, Yaswanth Chittepu, Dhawal Gupta, Philip Thomas, Scott Niekum
Abstract:
Policy learning drives many of the most consequential and heavily‑invested applications of reinforcement learning today. Yet the core optimization problem it rests on (maximizing expected return) is notoriously non‑convex, even under a direct policy parameterization, and the field has largely responded by avoiding it: optimizing convex surrogate approximations of the return under trust‑region constraints (NPG, TRPO, PPO, AWR). We show that this seemingly unstructured problem is not actually structureless. In log‑density‑ratio coordinates y := \log[π/π_n], the exact per‑iteration objective, computable via per‑decision importance sampling (PDIS), is a difference‑of‑convex‑constrained difference‑of‑convex (DC‑constrained DC) program. This structure lets us move beyond surrogate approximations: it recovers CPI, NPG, TRPO, and AWR as special cases along interpretable axes, and it opens a multi‑step axis k that couples consecutive decisions. We solve the per‑iteration program with sequential convex programming (SCP), the standard solver for difference‑of‑convex problems, and give convergence guarantees under mild conditions, bridging the difference‑of‑convex optimization and RL literatures. Empirically, multi‑step Convex‑Concave RL (CCRL) wins on diagnostic MDPs where credit must propagate across a horizon (its advantage growing with the dependency length), is competitive with a tuned PPO on classic control, and on a realistic, stochastic, mid‑horizon healthcare domain converges markedly faster than tuned PPO to the same near‑optimal survival, with an 11.3% higher area under the training curve.
Authors:Tobias Braun, Nils Loose, Alexander Herzog, Virginia Ceccatelli, Marcus Rohrbach, Thomas Eisenbarth, Lorenzo Cavallaro
Abstract:
Large language models are informing decisions with ever‑higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust an individual answer? Yet recognizing when to defer remains difficult because language models can present incorrect conclusions with fluent explanations and an authoritative tone. Uncertainty quantification seeks to address this disconnect by estimating the reliability of individual predictions. However, many existing methods require repeated generations or separately trained components, and their scalar estimates do not reveal where uncertainty arises or how it evolves during reasoning. Recent work has also shown that generation length can be strongly associated with uncertainty estimates and correctness, raising the question of how much of an estimator's predictive power comes from uncertainty‑specific information rather than output length alone. Mechanistic interpretability offers a way to address these limitations by connecting human‑interpretable concepts to intermediate model states. Building on this capability, we introduce the U‑Space, a low‑dimensional subspace that makes a model's evolving uncertainty measurable and interpretable. We identify semantic anchors for doubt and certainty, map their unembedding directions back into the residual space, and combine their contrasts into an orthogonal basis. The U‑Lens projects each token state onto these basis vectors, yielding an interpretable token‑level uncertainty map that can be inspected directly or aggregated into a scalar uncertainty score. Our approach requires no correctness labels, repeated generations, or training. Across reasoning benchmarks, its confidence score outperforms established baselines under both standard and length‑controlled evaluation and transfers more reliably than supervised estimators. Code: https://github.com/s2labres/U‑Space.
Authors:Satej S. Soman, Akram Zaytar, Girmaw A. Tadesse, Gilles Q. Hacheme, Muhammad S. Danish, Inbal Becker-Reshef, Rahul Dodhia, Juan Lavista Ferres
Abstract:
Applying deep learning models to satellite imagery from within geographic information systems (GIS) remains high‑friction for remote sensing practitioners. Models arrive in incompatible formats and target different compute environments, from local workstations to serverless cloud services. As a result, every evaluation demands custom deployment, tiling, and georeferencing code before a single prediction reaches the analyst's map. This friction discourages systematic comparison in a domain where model choice directly affects operational outcomes such as field delineation, crop monitoring, and disaster response. We present Anaximander, an open‑source system that unifies model source and compute location choice behind one interactive interface. The system's backend is an inference server that loads models from multiple commonly‑used sources and serves them on any accessible compute backend. The server provides session management and model caching, and streams results back per tile. The backend is paired with a QGIS plugin that drives tiling, result reassembly, georeferencing, and real‑time per‑tile status visualization. An additional user‑interface path injects layer legends as prompts into vision‑language models. We demonstrate the system in a code‑free side‑by‑side comparison of three heterogeneous models on an agricultural field delineation task: gpt‑image‑1 via a cloud API, Segment Anything Model 3 (SAM3) on a remote GPU, and DelineateAnything on a local CPU. The inference backend and protocol are open‑source and available at https://github.com/microsoft/nxmndr.
Authors:Shivansh Aggarwal, Shresth Grover, Divyansh Srivastava, Haiyang Xu, Bingnan Li, Xiang Zhang, Ethan J. Armand, Chuan Li, Jianwen Xie, Zhuowen Tu
Abstract:
Layout‑to‑Image generation has made substantial progress in spatial and object‑level control. However, existing methods still struggle with complex scenes containing many overlapping and interacting objects. We argue that training data is a particular bottleneck: existing datasets lack examples with dense, complex object interactions. To address this gap, we introduce OverLay++, a large‑scale Layout‑to‑Image dataset with structurally complex scenes. OverLay++ contains approximately 500K images with an average of 6.6 objects per image, exceeding existing datasets by 1.67 times in annotation density. Beyond annotation density, OverLay++ provides rich semantic detail with object captions over six times longer than in current datasets. Our dataset generation pipeline is simple and produces dense, overlapping object annotations with rich per‑object captions. Across multiple benchmarks, state‑of‑the‑art Layout‑to‑Image methods trained on the OverLay++ dataset show consistent improvement and faster convergence, demonstrating the importance of dense, overlap‑aware, and caption‑rich supervision for controllable image generation.
Authors:Zuoxu Wang, Xiao Liang
Abstract:
Reliable robotic disassembly requires part‑level representations that distinguish genuine component geometry from scanning and reconstruction artifacts. In point clouds of hard disk drives (HDDs), structured ghost artifacts can resemble valid components locally while remaining inconsistent with the overall geometry, allowing erroneous measurements to receive plausible semantic labels. This creates an engineering information problem: semantic prediction confidence alone does not establish whether the underlying geometry is reliable. We propose GUARD, a geometric uncertainty‑aware framework that performs point filtering and segmentation within a single forward pass by modeling the reliability of learned geometric representations. GUARD combines a multi‑scale geometric transformer with a multi‑bandwidth random Fourier feature Gaussian Process to estimate per‑point geometric uncertainty, complemented by predictive entropy to suppress unreliable measurements while preserving informative structures. Evaluation on 2,745 real HDD point clouds shows that GUARD improves PointNet++ segmentation mean intersection over union from 0.7739 to 0.8318. Additional experiments on ShapeNetPart and ScanNet examine robustness across corruption types, point‑cloud domains, and segmentation backbones. On manually annotated ScanNet samples, geometric uncertainty achieves a corrupted‑point detection F1 score of 0.7931, compared with 0.2212 for predictive entropy. The results demonstrate the value of distinguishing geometric reliability from semantic confidence and reveal a tradeoff between artifact suppression and preservation of informative structures. GUARD contributes a reliability‑aware approach to interpreting imperfect 3D measurements for component identification and subsequent robotic handling. Project website: https://001‑wang.github.io/GUARD_Point_denoiser/.
Authors:Veronique Ziegler
Abstract:
Supervisory governors can interfere with the tool‑using agents they regulate. We study this possibility in a controlled file‑recovery environment where increases in regulatory intensity trigger experimentally imposed tool failures. A cost‑blind governor can turn these failures into persistent blocking that prevents task completion. We compare this governor with a backoff rule that reduces intervention probability using a moving average of known induced events. On a hand‑coded stochastic‑policy agent, the failure pattern appears under both result replacement and execution of corrupted tool arguments. For the persistent policy, adaptive backoff improves completion relative to a fixed weak governor with approximately matched intervention frequency. A Gemini 2.5 Flash experiment comprising 576 episodes across 6 tasks also shows reduced blocking and improved completion under backoff; among the tested settings, intermediate backoff strength achieves the highest observed aggregate success. These results identify an interaction between intervention cost and persistent action blocking, together with a possible mitigation. The cost mechanisms are imposed and their induced events are directly observable to the backoff rule; applicability beyond this controlled environment remains an empirical question.
Authors:Animesh Shaw
Abstract:
Organisations deciding which assets to migrate to post‑quantum cryptography first, and how that work competes with a backlog of classical findings, lack a common unit: post‑quantum scores are dimensionless and quantum‑only, while classical risk is annualised loss. We express both threats as a cost of delay in currency per year on the same asset. The quantum term is the rate at which deferring migration commits irreversible harvest‑now‑decrypt‑later loss, computed from required confidentiality duration, recordable traffic share and data turnover, using a quantum‑arrival law fitted in closed form to published expert anchors. A competing‑risks factor couples the two terms so the same loss is not counted twice. We prove that the construction is not a weighted sum of a classical and a quantum score, give a dominance threshold on the classical hazard that is independent of asset value, and show that the induced order is optimal for sequencing work under constant loss rates. We verify the implementation by simulation, adaptive quadrature and a second implementation. On four public systems with parameters assumed from public documentation, the quantum term reorders the asset list beyond input uncertainty for one system only (signal‑to‑noise 1.36 against 0.14‑0.46), moves specific long‑lived, quiet, recordable assets decisively, and otherwise changes magnitudes. Asset value and turnover explain 73‑91% of the uncertainty in the quantum term; the arrival date explains 3‑17%. The case studies are illustrative and are not validated against outcomes.
Authors:Panagiotis Kasnesis, Christos Chatzigeorgiou, Lazaros Toumanidis, Amalia Contiero Syropoulou
Abstract:
An agentic system issues several structurally different kinds of LLM calls. It routes intent, classifies actions, grounds language in a device registry, plans multi‑agent pipelines and writes the Python code those pipelines run. The difficulty of these call sites varies by an order of magnitude, yet in practice a single model, chosen for the hardest site, serves all of them. In this work, we evaluate 9 models from 0.8B to a frontier hosted model across the five call sites of a deployed open‑source home‑automation framework (Wactorz), using its unmodified production prompts and two real Home Assistant installations (280 cases, 2520 scored calls). We find that capability is not ordered the same way at every site, and that larger models are not uniformly better: one 4B model is worse than its 2B sibling at grounded actuation. Paired testing shows the best local model to be statistically indistinguishable from both hosted models at four of five sites. Only code generation separates them, against a small hosted model (p = 0.039) as well as a frontier one (p = 0.002). Aggregate accuracy also hides a safety failure specific to actuation, where small models resolve the accuracy/refusal trade‑off in degenerate ways: one model (Gemma4 E2B) actuates on 87.2% of requests for devices the site does not own, while another refuses every request it receives. Routing each site to its best local model reaches 91.8% against 95.4% at no per‑call cost. In a live deployment judged by a user, hosting only the two generative sites matches hosting everything (39/43 against 39/43) for 28% of the spend, and the actuation gap the benchmark predicted appears as exactly one case in twenty‑six. Benchmark, harness and all records are released at https://github.com/waldiez/slm‑callsite‑eval.
Authors:Sourabh Kasliwal
Abstract:
Autoregressive Large Language Models (LLMs) frequently struggle with deterministic multi‑step algorithmic tasks such as multi‑digit multiplication and long division. In this paper, we investigate the mechanics of multi‑step arithmetic in compact "Tiny" Transformers (~10.6M non‑embedding parameters, 49.3M total) trained on synthetic data across four basic operations (+, ‑, , /) unrolled as step‑by‑step scratchpads. First, we establish the necessary training foundations: (1) dataloader sequence padding creates an 83% gradient starvation artifact that collapses accuracy from 40% to 1%, remediated via continuous sequence packing; (2) linguistic pretraining is an essential prerequisite (<= 2.0% without it); and (3) modern architectural primitives (RoPE, RMSNorm, SwiGLU) and Sparse Mixture of Experts (MoE) substantially improve additive reasoning over baseline GPT‑2. Second, we demonstrate that algorithmic scratchpad formulation directly dictates success. Introducing a deterministic Digit‑by‑Digit Long Division scratchpad within a 4‑stage Hierarchical Developmental Curriculum dramatically elevates single‑digit division from 4.0% to 86.7% accuracy on a 4,000‑problem held‑out benchmark. In contrast, multi‑digit multiplication remained challenging: detailed error analysis revealed that while the model correctly computed single‑digit sub‑products and place‑value zeros, our FOIL scratchpad failed because it forced a simultaneous summation of up to nine multi‑digit terms in a single step without pairwise intermediate accumulation. Finally, we identify two key boundaries: performance collapses to 0.00% on unseen 4‑digit operands, and unbuffered training induces catastrophic forgetting, collapsing division accuracy from 86.7% down to 0.00%.
Authors:Songyuan Zhang, Oswin So, Eric Yang Yu, Matthew Cleaveland, Peter Crowley-Dolen, Chuchu Fan
Abstract:
While offline reinforcement learning (RL) enables policy optimization from static datasets without costly online interaction, it remains bottlenecked by the risk of executing out‑of‑distribution (OOD) actions. Recent approaches mitigate this by learning a behavior‑cloning policy through flow matching and then performing RL within its constrained latent space. However, naively optimizing the latent policy can easily cause the policy to collapse into a brittle mode or exploit sharp artifacts of the learned critic. In this work, we find that entropy regularization is essential in latent‑space RL for addressing these challenges. We introduce LASER, a novel offline RL algorithm that applies latent‑space adjoint matching to achieve entropy‑regularized latent‑space RL with expressive flow policies while avoiding backpropagation through time. Through comprehensive experiments on 40 challenging OGBench tasks with varying dataset qualities, we show that LASER achieves state‑of‑the‑art performance. Notably, LASER uses fixed method‑specific hyperparameters across all tasks and outperforms the evaluated baselines, including those with task‑ and dataset‑specific tuning, which highlights the robust applicability of LASER. Project website: https://mit‑realm.github.io/laser/.
Authors:Vincent Liew
Abstract:
SAT solvers are empirically known to perform poorly when reasoning about multiplication. Yet for over a decade we have lacked a theoretical explanation for this phenomenon. CDCL SAT solvers implicitly search for resolution proofs, and no lower bound on proof size has ruled out the existence of short proofs that solvers simply fail to find. We give the first lower bound of this kind by showing that general resolution proofs of the associativity of n‑bit multiplication require size 2^Ω((n/\log n)^1/4). This lower bound holds for a broad class of multiplier encodings based on partial product summation, including the standard array and Wallace‑tree multipliers used to bit‑blast multiplication in SMT solvers. This result resolves an open problem of Beame and Liew. The proof constructs a reduction from a perfect‑matching principle on bounded‑degree bipartite expander graphs to multiplier associativity. Itsykson, Slabodkin, and Sokolov proved that this principle is hard for resolution. The lower bound for multiplier associativity follows. The same reduction, when combined with Håstad's recent lower bound for the perfect‑matching principle of the odd grid, yields an exponential lower bound for multiplier associativity in the stronger bounded‑depth Frege proof systems.
Authors:Arnela Hadzic, Franz Thaler, Simon Johannes Joham, Martin Urschler
Abstract:
Generative inpainting of brain MRI volumes is essential for synthesizing healthy tissue in pathological regions, improving the accuracy and reliability of automated downstream brain analysis applications such as image registration, brain extraction, and segmentation. However, standard 3D approaches are computationally prohibitive, while efficient 2D slice‑wise methods suffer from severe inter‑slice discontinuities. Furthermore, traditional models rely on conditional training, requiring task‑specific learning of masked inputs. We propose a zero‑shot brain MRI inpainting framework utilizing 2.5D unconditional flow priors to capture spatial context along the superior‑inferior axis without the overhead of full 3D convolutions. During training, our flow matching model learns the joint distribution of adjacent axial slice triplets, modeling the manifold of healthy brain anatomy while explicitly excluding pathological regions from the loss function. At inference, the model processes the input triplets autoregressively along the depth axis. We employ the Restora‑Flow solver to constrain the unconditional prior using the input mask, achieving accurate zero‑shot inpainting. Evaluations show our 2.5D strategy resolves the structural discontinuities of 2D baselines, synthesizing plausible healthy tissue while maintaining volumetric consistency across the axial, sagittal, and coronal planes. As a final step, we generate and average an ensemble of multiple stochastic reconstructions to form the final prediction. Quantitative results benchmarked on the official BraTS 2026 Inpainting Challenge validation set demonstrate the effectiveness of our proposed approach, yielding an SSIM of 0.816 \pm 0.112, MSE of 0.007 \pm 0.005, and PSNR of 22.923 \pm 4.343. Code is available at https://github.com/imigraz/brats2026‑inpainting.
Authors:Kristian Muri Knausgård
Abstract:
Let N(T) count the nontrivial zeros of the Riemann zeta function up to height T with multiplicity, N_d(T) the distinct ones, and N_0^s(T) those that are simple and on the critical line. We prove liminf N_d(T)/N(T) >= 1645064/1960733 = 0.83900...; earlier work proves 0.83699..., and a report we have not verified claims 0.83805.... Hence more than 67.80% are simple. Also liminf N_0^s(T)/N(T) >= 0.67353...; earlier work proves 0.67250..., and reports we have not verified claim 0.673492. Two estimates of Lamzouri are improved: at least 88.93% of the zeros are simple or on the critical line, and the proportions of simple zeros and of zeros on the line average at least 83.67%. By the unconditional form of Montgomery's pair‑correlation theorem the energy, a sum of a test function over pairs of zeros, is known asymptotically. A zero of multiplicity d contributes d^2 to it and nearby zeros contribute too, so a lower bound for such pairs leaves less energy for multiple zeros. The proof has three steps. First, for zeros at least a fixed fraction of the mean spacing apart, a large sieve inequality lets their pairs be used in full, with multiplicities. Second, their contribution is bounded below by a computer‑assisted inequality for seven or eight consecutive zeros which distinguishes simple and double zeros. Its correction terms telescope, as increments of a storage function. Third, the test function is chosen to make this contribution large, at the cost of more energy. We also prove upper limits for what such inequalities can give with the two main test functions. The lower bounds and these limits are proved in Lean 4, analytic inputs included. The proofs use three axioms beyond those of Mathlib, one per inequality. Each records that a search program returned true; Lean's kernel does not check the run itself. This work is an experiment in AI‑assisted mathematical research.
Authors:Yifan Zhang
Abstract:
Asynchronous reinforcement learning (RL) for large language model (LLM) agents trains one policy on trajectories generated by another: rollouts come from stale checkpoints, and the inference engine's probabilities differ from the trainer's even at identical parameters. Standard remedies either clip importance ratios, which biases the update, or, as in GRPO, sample a group of responses per prompt, which is costly when episodes are long. We propose KL‑Regularized Policy Optimization (KLPO), a framework that anchors the KL regularizer at the sampler. The regularized improvement step then has a closed‑form Gibbs solution, and KLPO fits its log‑ratio optimality condition by least squares on the sampler's own trajectories, so the sampler probability enters through a log‑ratio and no importance weights are needed. Profiling out the regression intercept replaces the intractable log‑partition function with the signal's sampler mean plus a sampler‑to‑trainer KL divergence. For token‑level policy mirror descent targets, we show that the resulting gradient can be computed from terminal returns without a critic, via sampler‑centered scores or a single trajectory residual, even under stochastic tool outputs. We further prove that independent Monte Carlo estimates of the KL term keep these gradients unbiased, derive the exact KL gap of cheaper top‑K and binary approximations, and show that SPPO, GPO, REBEL, and BPO arise as special cases of KLPO. The result is a critic‑free update that uses one rollout per prompt and requires neither a learned normalizer nor a group of responses.
Authors:Yunheng Liu, Ziqi Cai, Siqi Yang, Yimu Wang, Minggui Teng, Jiaming Tan, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi
Abstract:
Language‑guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low‑level actions in perspective views. We propose SPW‑Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360‑degree video in real time from a single panorama. SPW‑Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose‑aligned conditioning keeps translation inputs bounded over long streams, and a multi‑term memory with a few‑step generator continues the scene as instructions change. We also build SPW‑NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW‑Nav outperforms prior panoramic generators in camera‑following accuracy and video quality, and supports on‑the‑fly instruction switching.
Authors:Maksim Plinskiy, Aleksandr Gushchin, Sergey Lavrushkin, Dmitriy S. Vatolin, Anastasia Antsiferova
Abstract:
Robustness of image classification has several benchmarks, but their video counterparts are absent. In video classification temporal dimension introduces additional degrees of freedom for adversarial attacks, defenses, and preprocessing. Temporal sampling, perturbation budgets, and metric aggregation also interact in ways with no direct analogue in the image setting. Therefore, robustness for video classifiers is studied across scattered, incompatible implementations, making reported numbers hard to reproduce and analyze. We introduce VCR‑Bench, a modular open‑source benchmark framework that standardizes video loading, wrappers for classifiers, adversarial attacks and defenses, perceptual metrics, configuration presets, and result logging. VCR‑Bench currently integrates 30 video classification models, 14 adversarial attacks, and 10 defense wrappers under a common evaluation protocol. We evaluate representative video classifiers, attacks, and defenses on Kinetics‑400 subset, reporting clean accuracy, attack success rate, perceptual quality, runtime, and memory usage. VCR‑Bench is released with documented installation, reproducible run presets, component‑extension interfaces, and scripts for reproducing the reported results at https://github.com/msu‑video‑group/vcr‑bench.
Authors:Silong Yong, Anji Liu, Cunxi Dai, Carl Busart, Guanya Shi, Yilun Du, Katia Sycara, Yaqi Xie
Abstract:
A common way for trajectory planning is to leverage generative models trained on large collections of expert trajectories. At inference time, the model generates executable trajectories by conditioning on task goal constraints. However, trajectory‑based methods rely on costly supervision, scale poorly with sequence length, and often generalize poorly to unseen constraints such as novel start‑goal pairs. We propose an alternative to learn the underlying state‑space manifold and use the geometry of the manifold for trajectory planning. This approach requires only state observations and enables generalization to unseen constraints by con‑ structing trajectories on the learned manifold of the state space. Experiments on Maze2D and robotic motion‑planning benchmarks show that Ariadne constructs feasible paths from state‑only supervision and generalizes to unseen start‑goal combinations. On high‑dimensional dual‑arm planning, it remains competitive with trajectory‑supervised and classical planners, while requiring no trajectory data for training.
Authors:Yupei Guo, Jiajun He, Xiaohan Shi, Tomoki Toda, Zekun Yang, Bowen Wang, Yukinobu Taniguchi
Abstract:
The spread of fake news may cause severe social consequences. Existing fake news detection methods mainly focus on stylistic variations or incorporate external information such as explanations. However, news articles are often rewritten under different emotional backgrounds while preserving their underlying factual claims, which may affect the robustness of detection models. In this work, we investigate fake news detec‑ tion under fact‑preserving emotional variations. To study this problem, we construct emotion‑rewritten test sets and generate explanations from the original news articles as stable background knowledge. We then propose a Gated Cross Attention (GCA) framework that adaptively integrates emotionally rewritten news with the corresponding explanations, enabling the model to focus on informative explanation content while reducing potential mismatches caused by emotional reframing. Experiments on PolitiFact, GossipCop, and LUN demonstrate that the proposed method achieves notable improvements under multiple emotional conditions on PolitiFact and LUN, while maintaining competitive performance on GossipCop. We further analyze the effects of explanation guidance and gating mechanisms under different emotional conditions. Our code and data are available at: https://github.com/Flulike/fakenews gca .
Authors:Yue Wu, Qinghe Zhang, Yu Zhang, Jian Huang
Abstract:
Masked diffusion language models (MDLMs) decode by repeatedly committing tokens to masked positions, but these commitments are usually irreversible. A token chosen under sparse, partial context is kept fixed, even when later context no longer supports it. Existing samplers mainly decide when to commit a token, but rarely check whether an already committed token should still be kept, allowing early mistakes to propagate. We trace this issue to confidence drift, where the model's confidence in a committed token drops from its sparse commit‑time context to the denser context available later. Based on this signal, we propose CoDR (Confidence Drift Remasking), a training‑free and sampler‑agnostic refinement pass. CoDR estimates drift for all committed positions in only k forward passes via k‑partition probing, then remasks and regenerates only the tokens the model no longer endorses. Across two backbones, four reasoning and coding tasks, and three base samplers, CoDR improves average accuracy across all evaluated model‑sampler configurations and improves most individual task settings with modest overhead. Controlled experiments show that the gains come from targeted confidence‑drift remasking rather than extra compute alone, and that CoDR uses far fewer forward passes than prior remasking methods. Code is available at https://github.com/YueWu0301/CoDR.
Authors:Houssam Eddine-Othman Lachemat, Shammur Absar Chowdhury
Abstract:
Automatic Speech Recognition (ASR) systems often underperform for children and non‑native speakers, while adapting adult ASR models to child speech can cause adult‑speech forgetting. We study child ASR adaptation with adult retention across Arabic and English. We compare full fine‑tuning, LoRA, and post‑hoc weight‑space merging across encoder‑‑decoder, encoder‑‑CTC, and AudioLLM‑based ASR systems. Experiments use Arabic native and non‑native child speech, English MyST child speech, and adult benchmarks from MGB‑2 and LibriSpeech test‑clean. We evaluate recognition quality with WER and quantify the adaptation‑‑retention trade‑off using Retention Index, Child Adaptation Gain, and Adaptation Recovery. Results show that child adaptation is necessary, especially for non‑native Arabic and English child speech, but direct adaptation often reduces adult ASR performance. Bilingual adaptation is more stable than language‑specific adaptation. Weight‑space merging often improves the trade‑off, especially for encoder‑‑CTC, Whisper, and AudioLLM‑based ASR, with LERP favoring adult retention and TIES recovering stronger child gains. For the encoder‑‑decoder model, direct bilingual fine‑tuning remains strongest in raw WER.\footnoteCode, and models are available at https://github.com/qcri/Child‑ASR‑Adaptation.
Authors:Zhentao He, Ziwei Wang, Dongrui Wu
Abstract:
Electroencephalography (EEG)‑based brain‑computer interfaces (BCIs), particularly steady‑state visual evoked potential (SSVEP) systems, are highly vulnerable to noise and artifacts, which severely degrade decoding accuracy. Although recent denoising approaches have shown promise, they are fitted without paired ground truth, can settle on reproducing their input, and are optimized on waveform distance alone, which says nothing about whether the output stays decodable. To address these issues, we propose DenoFlow, which casts SSVEP denoising as transport: instead of learning a direct map from a contaminated trial to a clean one, a field network regresses the velocity of the straight path between them, following the rectified‑flow formulation, and denoising integrates that field forward from the observation. The field network is an encoder‑decoder that sees the contaminated trial at every layer and the path position at its bottleneck, and a classifier trained alongside it supervises the integrated output. Because the observation itself is both the conditioning input and the starting point of the integration, the model never generates a trial from noise, and training reduces to regression, removing the adversarial min‑max game. To obtain paired data on datasets with no ground truth, we injected physiological artifacts of the recorded electromyography (EMG) and electrooculography (EOG) signals under a controlled signal‑to‑noise target. Experiments on two public SSVEP datasets with five popular SSVEP decoders showed that DenoFlow outperformed seven baseline denoising models on both signal fidelity and downstream decoding accuracy. Code is available at https://github.com/wzwvv/DenoFlow.
Authors:Xiao Wang, Yuxiang Zhang, Dan Xu, Yuehang Li, Shiao Wang, Bo Jiang, Yaowei Wang, Yonghong Tian, Jin Tang
Abstract:
X‑ray image‑based Radiology Report Generation (RRG) constitutes a critical research direction within medical artificial intelligence, with great potential to alleviate clinicians' diagnostic workload and shorten patient waiting periods. Despite substantial advances over recent years, the field faces evident bottlenecks stemming from insufficient standardized benchmarks and inadequate domain adaptation of generic large models. Notably, the newly released CheXpert Plus dataset is provided without accompanying baseline implementations and evaluation results, which impedes standardized training, quantitative evaluation and fair comparison among follow‑up algorithms. To mitigate this limitation, we establish a comprehensive benchmark encompassing prevailing X‑ray report generation models and Large Language Models on CheXpert Plus. This benchmark delivers a reliable comparative foundation for upcoming methods and enables researchers to rapidly identify state‑of‑the‑art approaches within this domain. Beyond benchmark construction, we rethink X‑ray RRG under the paradigm of large models and propose a novel framework termed MambaXray‑PRB. Our framework improves report generation performance and enhances model interpretability via multi‑stage large‑model pre‑training and multi‑modal Chain‑of‑Thought reasoning. The pipeline consists of three successive phases: self‑supervised auto‑regressive modeling, X‑ray‑report contrastive learning, and post‑training optimization for reasoning and report generation. Extensive experiments on IU X‑ray, MIMIC‑CXR, and CheXpert Plus datasets validate the effectiveness of MambaXray‑PRB for radiology report generation. The source code of this paper is available on https://github.com/Event‑AHU/Medical_Image_Analysis
Authors:Ben Gubler
Abstract:
Large language models rely on subword tokenizers whose quality varies across languages, yet no standardized multi‑metric framework exists for broad comparative evaluation. We introduce Tokka‑Bench, an open‑source framework that evaluates tokenizers on five complementary metrics ‑‑ bytes per token, unique token coverage, subword fertility, word‑split rate, and vocabulary composition ‑‑ across 100 natural languages (30+ scripts) and 20 programming languages, using language‑aware segmentation adapted to each writing system. Comparing seven BPE tokenizers (GPT‑2, GPT‑4, gpt‑oss, Llama 3.1, Gemma 3, Qwen3, and Kimi K2) within individual languages, we find that vocabulary allocation strategy matters more than raw vocabulary size, and that programming‑language efficiency has converged among recent tokenizers despite divergent natural‑language profiles. The framework, data, and interactive dashboard are publicly available.
Authors:Liao Ma, Jiayi Song, Yunfeng Wu, Songhua Liu, Peilin Zhao
Abstract:
Diffusion Transformers (DiTs) have achieved strong performance in image and video generation, but the quadratic complexity of full attention makes high‑resolution generation computationally expensive. Window attention offers an efficient alternative, yet existing methods face a practical trade‑off: partitioned window attention typically achieves computational efficiency consistent with its theoretical complexity. However, isolated windows block cross‑window interaction, often introducing visible grid‑like artifacts in the generated results. Fine‑grained sliding‑window attention effectively restores interactions across neighboring windows and improves visual quality. However, its irregular computation patterns create a substantial gap between theoretical and practical speedups and require specialized kernels tailored to each hardware backend. To tackle these challenges, we propose BASA, a backend‑agnostic sparse attention, which brings the best of both worlds: visual quality and practical acceleration. Specifically, BASA replaces visual self‑attention with shifted local‑window attention. By introducing a structured window‑shifting scheme across DiT blocks, we allow tokens divided by window boundaries in one layer to communicate in the following layers, thereby achieving global information exchange and eliminating window‑induced visual artifacts. Notably, our design introduces no additional irregular operators or customized kernels, making it readily deployable on existing attention backends and closing the gap between theoretical sparsity and practical acceleration. Experiments demonstrate that BASA achieves measured speedups exceeding 90% of the theoretical estimates on FLUX and delivers a 4.52× attention speedup on Wan while maintaining competitive generation quality. Codes are publicly available at: https://github.com/lama0110/BASA.
Authors:Jiajun Chen, Haoyu Wu, Mingda Jia, Xihui Liu
Abstract:
Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience‑oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding‑native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI‑based collection. The Reviewer uses carefully designed trajectory‑based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game‑specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state‑of‑the‑art overall performance of 77.89 on GameCraft‑Bench. On GameASG‑Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same‑model baseline, and the highest mean runtime‑check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Project page is at: https://imbaldy.github.io/recursive‑game‑creator/.
Authors:Marcel Crasmaru
Abstract:
CNet is a C++/CUDA framework for building deep complex‑valued neural networks (CVNNs) and optimizing complex functions by gradient descent with Wirtinger derivatives. Complex models are underexplored yet natural where data is intrinsically complex ‑‑ RF/IQ communications, MRI k‑space, radar/SAR, audio spectra ‑‑ and phase carries information real networks discard. CNet is physics‑native: a network is a cascade of complex (often unitary) operations on an amplitude vector, and classification is a Born‑rule measurement p_k = |z_k|^2/||z||^2 rather than a softmax over real logits. Its library of complex layers makes conv(x,k) = IFFT(FFT(x).FFT(k)) learnable via signal‑processing primitives ‑‑ spectral‑padding local kernels (Pad), the inverse DFT, a magnitude nonlinearity |z|^2 (CModulus2), holomorphic powers z^M (CPower), and mean pooling (MeanPool) ‑‑ with GPU kernels for every layer, a GPU true‑Adam optimizer, and a reduced‑memory inference mode. Three studies. (1) A fully complex FNet‑style causal language model, built on a new O(N log N) causal Fourier mixer, matches a param‑matched real‑valued causal FNet on tiny‑shakespeare in under half the steps. We introduce Born‑rule attention: a content‑based causal mixer whose weights are quantum‑measurement probabilities |<Q_k,K_j>|^2 of one token against another ‑‑ the first attention mechanism built on the Born rule. With rotary position embeddings and per‑token complex normalization, a fully complex Born‑rule model reaches 1.52 nats/char on tiny‑shakespeare, matching softmax attention and beating the Fourier mixer by 0.23. (2,3) Two bottleneck analyses ‑‑ radio‑modulation classification (RML2016.10a) and the Fourier phase problem of coherent‑diffraction imaging ‑‑ isolate where CVNNs need new operators. The complex machinery learns the correct structure; open problems are concrete and operator‑level. Code: https://github.com/crasmarum/CNet
Authors:Jiaju Chen, Min Yang, Jinghua Piao, Xiaochong Lan, Xu Xia, Xiangnan He, Yong Li
Abstract:
Multi‑turn LLM agents often receive sparse task feedback across several interactions, while generating each response token by token. This creates two related credit‑assignment questions: which responses helped achieve the outcome, and which generation decisions mattered within each response? Existing methods typically focus on only one level: turn‑level methods evaluate complete responses but do not distinguish the decisions within them; token‑level methods can propagate feedback across turns but do not explicitly model credit for each response. These complementary limitations motivate learning credit at both levels and coordinating it in a single policy update. We introduce VETTA, a credit assignment method that jointly learns turn‑ and token‑level values through separate heads on a shared lightweight critic. VETTA computes advantages along both temporal sequences and combines each turn advantage with a within‑response‑centered token residual for PPO updates. Furthermore, to reduce value‑learning cost, the critic retains only early Transformer blocks from the pretrained checkpoint used to initialize the actor. On two challenging agent benchmarks, ALFWorld and WebShop, VETTA improves success rates over PPO by 37.5% and 22.3%, respectively, with Qwen2.5‑1.5B‑Instruct and achieves success rates of 95.5% and 76.0%, respectively, with Qwen2.5‑7B‑Instruct. Critic‑depth comparisons further show strong task performance with substantially lower critic‑side computation. These results suggest that a compact shared critic can coordinate turn‑ and token‑level credit to improve agent performance while keeping value estimation efficient. Code is available at https://github.com/Jiaju‑Chen/VETTA‑official.
Authors:Kasper Helverskov Petersen, Rasmus Hannibal Tirsgaard, François R J Cornet, Mikkel Jordahn, Mikkel N. Schmidt
Abstract:
Large‑scale self‑supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self‑supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom‑JEPA, a self‑supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom‑level and substructure‑level objectives inspired by joint‑embedding predictive architectures. We pretrain Atom‑JEPA on large‑scale molecular and crystalline datasets and evaluate its transfer performance by fine‑tuning on a diverse set of downstream property prediction tasks. Atom‑JEPA achieves state‑of‑the‑art performance on molecular ADMET and quantum‑chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent‑space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at https://github.com/khelverskovp/atom‑jepa
Authors:Yang Hong, Yajun Yang, Xin Wang, Liping Jing, Qinghua Hu
Abstract:
Large language models (LLMs) have demonstrated strong capabilities in question answering, yet they still frequently suffer from hallucinations on knowledge‑intensive tasks. Knowledge graphs (KGs) provide LLMs with structured, interpretable, and updatable factual grounding, making them a promising external knowledge source for reliable reasoning. However, existing LLM‑guided graph reasoning methods typically rely on hop‑wise greedy or beam‑style pruning during evidence retrieval. Such local decision processes are inherently myopic: evidence that appears weak near the source may become crucial only after deeper graph context is explored, causing answer‑critical branches to be discarded prematurely and making the reasoning chain difficult to recover. To address this limitation, we propose Foresight‑over‑Graph (FoG), a foresight‑aware evidence retrieval framework for knowledge base question answering (KBQA). FoG iteratively constructs a question‑relevant evidence subgraph and uses far‑to‑near feedback to guide path exploration, and maintains a compact memory subgraph to support continued exploration. Extensive experiments on widely used KBQA benchmarks demonstrate that FoG achieves state‑of‑the‑art performance, with a particularly large improvement of 16.58% in Hit on CWQ, while also reducing LLM calls and token usage. Our code is available at https://github.com/yhong7/FoG .
Authors:Zihao Zhao, Ashwath K. Karunakaram, Ali Eshragh, Yuexing Li, Kai Wang
Abstract:
In this work, we consider decision‑focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT conditions of the Bellman equation and require solving a linear system over all state‑action pairs, limiting its scalability. We address this by reformulating the MDP as an occupancy measure‑based linear program (LP), whose feasible region is induced by predicted dynamics, and we derive a closed‑form gradient by identifying the active constraints in the feasible polyhedron via the pivoting algorithm. This occupancy measure‑based LP layer raises two challenges: (1) LP's solution gradient is discontinuous when active constraints change, and (2) the LP backward cost still scales with the state size, which is costly for large or continuous state spaces. We address the challenges with an augmented Lagrangian surrogate and smooth the boundary jumps by random row sketching of the constraints, and a learnable soft state‑aggregation layer and its function‑approximation generalization that scales the LP to large finite and continuous‑state MDPs. Across multiple tasks, our methods reach lower regret than KKT‑based DFL and two‑stage baselines with significantly lower computation cost. The source code for all experiments is available at https://github.com/A‑Eshragh/State_Aggregation_Project.
Authors:Hyeongheon Cha, Young D. Kwon, Sung-Ju Lee
Abstract:
Post‑training quantization is a standard route to fitting vision transformers (ViTs) into edge compute and memory budgets, yet quantized models become especially brittle under distribution shift. Test‑time adaptation (TTA) addresses such shifts without labels, but most existing approaches are poorly aligned with the constraints of quantized inference. Prevailing TTA methods recover accuracy through backpropagation, while backprop‑free methods often still incur overhead from extra forward passes or parameter updates, and lightweight feature‑ or logit‑level methods recover only part of the loss. Across these approaches, a quantization‑specific failure mode that amplifies the drop is not directly targeted: under shift, activations occupy frozen quantizers' calibrated ranges differently, distorting their code distribution. We propose Quantizer‑Aligned Recalibration (QuAR), a single‑pass TTA method tailored to quantized ViTs that neither backpropagates nor updates any model parameters. QuAR recalibrates activations at the input to a frozen quantizer, mapping the test stream's running per‑channel statistics back toward the source calibration. On ImageNet‑C with ViT‑B, QuAR achieves the highest mean accuracy among state‑of‑the‑art backprop‑free TTA methods at 3‑, 4‑, 6‑ and 8‑bit weight/activation precision, outperforming the strongest baseline by 2.28 points at 8 bits and 4.00 at 3 bits, with 46% lower latency and a memory overhead of only 0.17 MB (0.01% of peak inference memory). Analysis and diagnostics trace the gain to a reduced per‑channel mismatch at these quantizers, which restores the code distribution the baselines leave unchanged or distort further. A single fixed configuration remains ahead across continual streams, non‑i.i.d. label shift, seven out‑of‑distribution suites, and three other backbones.
Authors:Jaedong Hwang
Abstract:
Flow‑matching models start from an isotropic Gaussian source, the standard choice when the correlation structure of the data is unknown in advance. For multi‑channel brain recordings, however, part of this structure is known in advance. Electrodes sit at fixed positions on the head, and volume conduction through the skull and scalp makes nearby electrodes co‑vary in a way that is shared across subjects. Existing EEG generative models nonetheless leave the network to learn this from scratch. We put this structure into the source instead. From the sensor coordinates alone, we build a k‑nearest‑neighbor graph and take a graph‑Matérn function of its Laplacian as the source covariance, so the flow starts from spatially coherent patterns rather than channel‑independent noise. The change adds no learned parameters, works with any coupling and any drift network, and uses the same three hyperparameters on every dataset. Across eight EEG datasets and four flow‑matching methods, the graph‑Matérn source lowers the spectral discrepancy between generated and real signals in the five clinical bands (PSD‑KL) on most datasets. PSD‑KL falls by 12% to 17% in geometric mean over datasets depending on the method and by up to 40% on PhysioNet‑MI, the densest montage. We show that the improvement stems from the spatial eigenvectors of the local graph of sensor positions, since randomizing the eigenvectors while preserving the eigenvalue spectrum eliminates the gain. Furthermore, a prior fitted directly to the empirical data covariance performs worse than isotropic noise. The same construction applies unchanged to MEG, intracranial EEG with patient‑specific grids, and a traffic‑sensor network, lowering PSD‑KL for every method on each. https://jd730.github.io/projects/GraphPrior
Authors:Mikkel Abrahamsen, Jacobus Conradi, Asbjørn Lind
Abstract:
The 2026 ACM SIGSPATIAL GIS Cup posed the following geometric challenge: Given a set of simple, pair‑wise disjoint polygons, compute k antennas (points) on polygon boundaries, maximizing the number of polygons with at least a fraction τ of their perimeter visible from the antennas. Contestants had 24 hours to produce the best possible solutions. We describe an approach that separates geometric visibility preprocessing from a portfolio of incremental combinatorial searches. A construction pool provides initial solutions with antennas restricted to polygon vertices; simulated annealing explores one‑for‑one antenna swaps, complemented by exact discrete one‑swap descent, ruin‑and‑recreate, and elite crossover. A secondary objective rewards progress toward the service threshold without overriding the primary score. Candidate antennas are later expanded to include points in polygon‑edge interiors. Our submitted solutions cover between 261 and 16,273 buildings across the nine parameter combinations, and the team was invited to present as one of the top entries. The code is available at https://github.com/JacobusTheSecond/giscup.
Authors:Faraz Shamim, Faris Shamim
Abstract:
Public redispatch records provide empirical data for grid congestion forecasting, but delayed reporting, zero‑inflated distributions, and temporal shift present major modeling challenges. We assess the accuracy and reliability of probabilistic machine‑learning forecasts using published German transmission records under experimentally imposed information‑age constraints. The benchmark evaluates eight daily series of upward and downward intervention energy across four German transmission system operators from 2021 to 2024 (48,242 eligible records; 354 evaluation dates in 2024). We compare seasonal empirical, regularized autoregressive (ARX), quantile LightGBM, GRU, and Transformer models under a minimum seven‑day target‑latency constraint. Neural architectures use a zero‑censored output head to accommodate exact‑zero outcomes. Static, rolling, and adaptive delayed‑feedback calibration are evaluated using normalized weighted interval score (nWIS), empirical coverage, and block‑bootstrap inference. Raw LightGBM achieved nWIS 0.7952, outperforming ARX (1.0604) and the seasonal baseline (0.8739) by 25.0% and 9.0%, respectively (Holm‑adjusted p<0.005). Rolling calibration improved LightGBM to nWIS 0.7767 versus 0.8251 for static calibration (p=0.0092), with 91.81% coverage for nominal 90% intervals. The zero‑censored Transformer achieved nWIS 0.8161, with no significant difference from LightGBM (p=0.260). However, aggregate coverage concealed substantial undercoverage during high‑volume interventions (61.91% coverage among above‑threshold events). These results show that boosted‑tree models with rolling calibration provide accurate probabilistic forecasts of aggregate redispatch volumes under target delays, while nominal aggregate validity does not ensure reliability during extreme congestion events.
Authors:Zhiyuan Zhang, Axel Delaval, Leheng Chen, Jinxuan Chen, Jie Xu, Yuxuan Liao, Jiedong Jiang, Chunlei Liu, Bin Dong
Abstract:
We present an AI‑assisted Lean 4 formalization of the Poincaré conjecture. The project began with limited reusable formal infrastructure for the geometric analysis behind the proof. To organize this work, we combined a proof blueprint prepared by mathematicians with explicit milestone statements. These milestones enabled parallel agent work and gave mathematicians clear points to locate blockers and provide effective mathematical guidance. Our analysis identifies the human interventions and organizational choices behind this workflow. The project provides a starting point toward reusable infrastructure for future formalization projects; such infrastructure, once developed, could eventually reduce the cost of verifying mathematical results in geometric analysis.
Authors:Haozhuo Zhang, Qiang Zhang, Jian Tang, Mingzhe Ni, Michele Caprio, Angelo Cangelosi, Wei Pan
Abstract:
Cluttered indoor environments, where large and heavy objects are scattered across diverse surfaces, require humanoid robots to sequentially navigate, grasp, transport, and accurately place each item at its target location within a single uninterrupted episode. This long‑horizon, whole‑body loco‑manipulation task remains a significant challenge for current methods. Previous approaches often suffer from two main issues: easy‑reward bias, where training overemphasizes early transport stages at the expense of later ones, and catastrophic forgetting, where focusing on later stages leads to a decline in earlier‑stage performance. In this work, we introduce Humanoid Horizon, a unified policy framework designed to overcome these limitations through three interrelated mechanisms. The Parallel Training Strategy organizes N scenes into S concurrent stage streams governed by a shared policy, ensuring all transport stages receive continuous gradient updates and removing the bottleneck of sequential optimization. The Dynamic Starting Mechanism updates each environment's initial state with terminal states from upstream rollouts, gradually broadening transition coverage and enhancing robustness at stage boundaries. Reward Gating sets the reward to zero for the rest of the episode in later‑stage streams when the immediately preceding object is displaced beyond a set threshold, so the shared policy learns not to disturb a just‑placed object and earlier placements are preserved throughout the episode. Collectively, these strategies achieve per‑stage success rates exceeding 80% on the two‑object LHM‑Humanoid benchmark (350 training scenes, 66 held‑out scenes). As the number of sequentially transported objects grows beyond two, success declines with the horizon, but the degradation is graceful relative to the sharp drop seen in all baselines.
Authors:Maoqi Liu, Quan Fang, Yufei He
Abstract:
Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low‑rank adaptation with orthogonal projections (e.g., O‑LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an "Orthogonality Dilemma": rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay‑free method, called Consolidation and Decoupling LoRA (CoDe‑LoRA), for CL of LLMs. CoDe‑LoRA disentangles the learning process into Consolidating Universal Knowledge and Decoupling Task‑Specific Knowledge. To achieve this, CoDe‑LoRA leverages an adaptive null space projection mechanism and semantic routing to balance knowledge accumulation with task‑specific adaptation. Experimental results across four backbones and three CL benchmarks show that CoDe‑LoRA achieves the best average accuracy. Our code is available at https://github.com/Estrellajer/CoDe‑LoRA.
Authors:Tobias Morocutti, Emmanouil Karystinaios, Gerhard Widmer
Abstract:
Music Source Restoration (MSR) seeks to recover original, unprocessed instrument stems from mixed, mastered, and possibly degraded recordings. Unlike conventional source separation, which treats the mixture as a linear sum of clean sources, MSR must additionally invert nonlinear production effects, such as equalization and compression, and transmission‑related degradations, such as codec artifacts. We propose a three‑stage framework built around this distinction: (1) a mixture restoration model that addresses degradation before separation, (2) a single model that separates the restored mixture into eight target stems (vocals, guitars, keyboards, synthesizers, bass, drums, percussion, and orchestra), and (3) stem‑specific restoration experts fine‑tuned on the separator's own residual artifacts. Each stage improves restoration quality over the previous one on the MSR Challenge test set. We release code and models to support future research in MSR at https://github.com/theMoro/music_source_restoration.
Authors:Jingpo Xu, Paul Joe Maliakel, Ivona Brandic, Shashikant Ilager
Abstract:
Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models (LLMs) are increasingly used to support these applications. Yet, due to their high resource demands, LLMs are mostly deployed in the cloud. Layer‑wise edge‑cloud inference lets resource‑constrained edge devices contribute computation to LLMs they cannot host in full. However, heterogeneous split points introduce two coupled inefficiencies. First, edge execution and communication create idle gaps between cloud invocations. Second, requests arriving at different model depths cannot be conventionally batched. We present DySCo, a collaborative runtime that keeps KV caches local and introduces dyForward, a model‑aware layer‑range executor that runs configurable contiguous layer ranges from resident model shards without reloading weights. For multi‑edge serving settings, we introduce depth‑synchronized batching (DSB), which advances heterogeneous requests to the deepest cut and batches their common suffix. Experiments across heterogeneous devices, two model families, and local and wide‑area links show that idle gaps increase the latency of subsequent GPU forward calls even when waiting time is excluded, adding up to 25 ms of additional cloud‑side suffix latency per decoding step in our measurements. At an average concurrency of eight, DSB improves throughput by 275% over FIFO, 48% over exact‑match batching, and 79% over round‑robin interleaving while reducing mean per‑session latency. Together, these results show that requests with different edge‑cloud splits can reuse resident cloud weights and share batched suffix computation. The artifact repository for this work is publicly available at: https://github.com/Large‑scale‑Sustainable‑Computing‑LSC/dysco‑artifact
Authors:Halil İbrahim Kanpak, Sinem Sav, Alptekin Küpçü
Abstract:
Many organizations adapt large pretrained models to their own tasks by fine‑tuning on private data. Several of these parties often hold data for the same task and wish to fine‑tune a model together without pooling that data. Federated learning (FL) enables joint fine‑tuning, but reconstruction attacks on shared intermediate values (the model or its gradients) remain a privacy risk. A one‑shot protocol that exchanges one encrypted contribution exposes no intermediate value. Such a protocol still gives the trained model to every participant, which is not permitted where the model is a regulated or proprietary asset. We present HE‑OFT, the first cryptographically secure one‑shot federated fine‑tuning protocol in which no party receives the trained model. Each client fine‑tunes a low‑rank adapter and a classifier head on a frozen public backbone and keeps the adapter. The client uploads one encrypted head displacement, which the server combines under multiparty CKKS and never decrypts. A quorum of clients returns only the predicted label to the querier. On four text classification tasks and one vision task, HE‑OFT reaches 61 to 79 per cent accuracy, against 20 to 48 per cent for a client training alone. HE‑OFT keeps 85 to 96 per cent of the accuracy of a disclosed model. A test‑time query takes 443.1 to 1713.1 s on one core, or 56.1 to 255.1 s with level restoration on a GPU. Restoring levels at the server cuts the traffic per query from up to 1.6 GiB to 13.5 MiB.
Authors:Neriah Ben David, Ori Meir, Or Ordentlich
Abstract:
NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor‑core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In this work we develop a novel quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4). For each linear projection it uses a diagonal smoothing matrix whose entries are optimized to minimize the squared matrix‑product quantization error under NVFP4, taking into account the rounding procedure that is used (either round‑to‑nearest, or GPTQ‑style successive interference cancellation). This requires performing joint optimization on the smoothing entries as well as the block scales, which is facilitated by analyzing a multiplicative‑dither FP4 quantizer instead of the fixed deterministic one. Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94‑97% of vendor NVFP4 prefill throughput on the measured workloads. Our code is available in https://github.com/neriahbd/OSFP4
Authors:Xinyu Zhang, Sichao Liu
Abstract:
Contextual priors improve neural‑to‑language decoding by reshaping candidate scores. However, confidence is read from the same reshaped scores, so the errors a prior leaves behind can become more confident with no change in accuracy to reveal it. We study how a prior shapes confidence in speech retrieval on MEG‑MASC and MOUS using local decoding scores, a contextual prior combined by additive shallow fusion, and the fused top‑two margin as confidence. Among initially incorrect predictions, we find a confidence‑ordering reversal: a larger margin makes a repair more likely when the correct candidate starts near the top of the local ranking, but less likely when it starts lower. On MEG‑MASC, pooled correctness AUROC is 0.87, yet AUROC separating repairs from residual errors falls from 0.70 at initial ranks 2‑3 to 0.39 at ranks 21‑50. Errors starting beyond rank 20, inside the reversed region, make up 46.6% of all post‑fusion errors. We propose a score‑level account: a repair must first close the correct candidate's initial deficit, limiting its final margin, whereas a residual error can build a large margin between two incorrect candidates. A causal intervention that changes only the fusion weight moves the reversal to deeper ranks as predicted. Under a word‑level LM prior, it keeps moving after accuracy gain peaks, so a weight chosen for accuracy does not settle confidence. Reading local and prior scores separately improves selective decoding: the decoder answers on 74.5% of windows instead of 56.7%, while 92% of output sets still contain the correct candidate. Confidence after contextual fusion should retain the local and contextual evidence behind each prediction, not just the fused scores. Project website: https://confidencereversal.github.io/; Code: https://github.com/AmadeusFake/NeuDecodingConfReversal
Authors:Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi
Abstract:
Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments. Evaluating this ability is therefore important for understanding how effectively agents acquire and use new knowledge. Existing benchmarks have sought to evaluate this ability, but they primarily evaluate tasks whose rules are provided in the instructions or already familiar to pretrained models, making it difficult to distinguish learning from interactions from reasoning with existing knowledge. To address this, we introduce Learn2Play Bench, a benchmark of newly designed text‑based games, whose rules are novel or counterintuitive, requiring agents to acquire knowledge through interaction rather than rely solely on pretrained knowledge. These games provide reproducible feedback and automatic scoring, enabling controlled evaluation of learning across repeated attempts. We also vary game instances to test whether agents can apply what they have learned to new situations. Therefore, we evaluate how backbone models, self‑evolving methods, and agent harnesses affect agents' learning ability, revealing three findings: (1) Experience retention: Retaining complete records of actions and feedback can support more effective learning than summarizing these experiences into rules or strategies. (2) Human agent gap: Top‑performing human players achieve higher peak scores than the evaluated agents. Human explore more varied strategies, and repeat actions less. (3) Harness matters: With the backbone fixed, changing the harness can improve performance while reducing estimated inference cost. Together, these findings provide insights into how LLM agents learn from experience and suggest directions for future work to improve their learning ability. Project website: https://liushiliushi.github.io/learn2play‑bench‑website/
Authors:Geonung Jo, Jongeun Choi
Abstract:
Recent text‑guided audio FX research has focused on translating natural‑language descriptions into parameters or chain configurations within existing FX systems. However, comparatively little attention has been paid to how the internal control space of an individual FX processor can itself be constructed. To address this gap, we propose CTAG‑FX, which functionally reinterprets the roles of 78 parameters in a text‑conditioned synthesizer configuration as controls of a tone‑shaping FX processor. Each synthesizer configuration defines a fixed tone‑shaping processor that can be repeatedly applied to new audio inputs rather than producing only a one‑off rendered output. The text‑conditioned synthesizer configurations are generated using A&R‑CTAG, a retrieval‑enhanced extension of CTAG. We evaluate CTAG‑FX through signal‑level analysis, together with a scrambled‑mapping ablation in which parameter‑to‑control assignments are randomly permuted to assess the contribution of the proposed role assignment. Role‑based mapping produces prompt‑distinct spectral and nonlinear behavior, whereas scrambling reduces this prompt‑specific differentiation and yields a more prompt‑insensitive nonlinear profile. Overall, these results suggest that synthesizer parameter spaces can serve not only as sound‑generation spaces but also as design resources for constructing new audio‑FX control spaces. Audio samples are available at https://taylor3527eg‑hub.github.io/ctag‑fx‑platform‑demo/.
Authors:G. L. John Salvin, Swapnil Hingmire
Abstract:
COMET reports translation quality as a single number, and that number is routinely compared across target languages written in different scripts. Such a comparison assumes Script Invariance: the score should not depend on the writing system that carries the target. We test it on IndicMT Eval by re‑encoding the target into Latin script, which changes orthographic form while holding content and human ratings fixed. Script identity then accounts for 22.9% of native‑script COMET variance, and agreement with annotators falls in all five languages studied. We trace the effect to the tokeniser and measure it with three label‑free diagnostics. The bias is two faults, not one. Scores from different scripts occupy incompatible ranges, and within a single script the metric orders translations less accurately. No order‑preserving transform of the score can repair the second fault. The first is removed exactly by COMET‑QN, which maps the score distribution of each (language, script) pair onto a shared reference. Pooled agreement with annotators rises from 0.300 to 0.399, which is what makes scores from different scripts safe to place on one axis, and every within‑language ordering is provably preserved. A regressor over parity features recovers a further 17.1% of the lost sensitivity. The remainder belongs to the encoder, and no post‑processing can reach it. We therefore recommend publishing the normalised score, the three diagnostics, and the identity of the tokeniser they were computed against, so that a reader can tell how much of a score reflects translation quality and how much reflects the writing system.
Authors:Yongqing Jiang, Haoran Luo, Jianze Wang, Xin Zhou, Kaoshan Dai, Zhiqi Shen
Abstract:
Automated building design must comply with seismic and wind codes and satisfy structural mechanics constraints, yet most existing agents produce visually plausible models without verification grounded in mechanical analysis and code compliance. We introduce the Vibe Building task and propose PE‑Loop (Physics‑Engine‑in‑the‑Loop), an agent in which a deterministic physics engine is the sole source of evaluation signals, mapping code constraints to a physics process reward, while the language model is confined to proposing discrete revisions (section menu, topology, and lateral system). Designs are verified by held‑out seismic and wind time‑history checks and a constructability gate. On VB‑Bench, 3,577 physics‑adjudicated building instances across six code families, PE‑Loop achieves the highest verified success rate under three of four backbone LLMs, the highest held‑out seismic pass rate under all four, and the highest held‑out wind pass rate under three. Replacing the physics verdict with a language‑model judge, all else fixed, leaves 58.43% of accepted designs noncompliant. These results suggest that reliable structural design rests less on a stronger LLM proposer than on an adjudicator the proposer cannot influence, a division of labor for agents whose outputs must hold up in the physical world.
Authors:Zheng Gao, Xiaoyu Li, Zhicheng Bao, Yang Song, Jiaojiao Jiang
Abstract:
AI systems create images and videos with image/video generation models or by writing code and graphics descriptions that are then rendered. These routes can produce similar visible artifacts but expose different representations, intervention points, and provenance evidence. We develop a production‑centered framework that compares detection and watermarking across both routes. An explicit verification specification distinguishes passive inference, message recovery, and authenticated provenance. We organize image, video, source‑code, and rendering‑aware watermarks by production stage. We examine the different requirements of generated images and video, plots and SVG, programmable video, and agent‑composed workflows. Documented Claude, OpenAI, and rendering‑tool interfaces connect the framework to concrete systems. We pose ten scoped research questions on identifiability, observability, fair comparison across stages, recoverable payload, reconstruction, synchronization, composition, hybrid local contribution, and private production‑event authentication. The result is a conceptual research agenda grounded in published methods, inspected interfaces, and elementary boundary examples. It reports no experiments and claims no new theorems; its appendix results are elementary calculations, and documentation and source inspection establish interfaces, not empirical robustness.
Authors:Owen Du, Yang Yue, Jie Zhang, Jiaqi Pi, Chi Bene Chen, Gao Huang
Abstract:
Vision‑Language‑Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real‑time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training‑free heuristics, such as attention scores and motion thresholds, or require costly fine‑tuning of the base VLA model. We introduce VLA‑ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action‑level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full‑context teacher, with ground‑truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real‑world manipulation tasks show that VLA‑ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance‑efficiency trade‑off than existing frozen‑VLA pruning methods and demonstrate the value of action‑level supervision for visual token selection. Code is available at https://github.com/du‑owen/VLA‑ACL.
Authors:Ridwan Arefeen, Ze Li, Rong Tong, Ming Li, Xiaoxiao Miao
Abstract:
Attacker ASV systems for voice anonymization have been studied primarily in English, leaving their behavior in multilingual settings largely unexplored. Conventional ASV has shown that both acoustic and contextual information are important for multilingual speaker verification. Inspired by this, we investigate whether the same holds for attacker ASV on anonymized speech. We evaluate both acoustic‑ and content‑oriented attackers on multilingual anonymized speech and construct a multilingual voice‑converted dataset to improve cross‑lingual generalization. Our results show that attacker effectiveness depends on the linguistic utility of the anonymized speech. Overall, acoustic‑oriented attackers achieve better performance. However, when linguistic information is well preserved, the performance gap between content‑ and acoustic‑oriented attackers narrows compared with conditions involving stronger speech distortion. The multilingual voice‑converted dataset further improves performance and partially reduces the cross‑lingual gap. These findings highlight the need for more comprehensive attacker modeling and evaluation protocols that consider both privacy and utility, rather than relying on a attacker strategy\footnoteFull code and pretrained models and MultiVC Dataset link are available at: https://github.com/monkeyDarefeen/DAST
Authors:Chuan Li, Chengyu Wang, Cen Chen, Ye Lyu, Mingyuan Fan, Ming Gao
Abstract:
Developing reliable models for clinical tasks, such as Medical Question Answering (QA), is severely constrained by the limited availability of high‑quality, expert‑annotated training data. This challenge is exacerbated by stringent privacy requirements and the impracticality of utilizing large open‑source corpora or proprietary cloud APIs within resource‑limited clinical settings. To address these obstacles, we introduce SAGE (Semantic Anchor‑Guided Evolution), a novel data synthesis framework that enables small, locally deployed models to generate high‑quality medical training data. SAGE leverages lightweight, publicly available taxonomies such as MeSH as semantic anchors, imposing a structured prior to effectively guide and ground the data generation process. At its core, SAGE iteratively interleaves atomic (individual concept‑based) and associative (relation‑based) synthesis, bootstrapping training data from minimal seeds. This approach eliminates the need for large collections of medical documents or reliance on external APIs, providing a practical solution for on‑premises data creation. Extensive experiments across multiple medical question‑answering benchmarks demonstrate that models fine‑tuned with SAGE‑synthesized data consistently outperform those trained using self‑derived or conventional document‑based paradigms, highlighting tangible improvements in data efficiency and resource utilization for medical LLM development. Code is available at https://github.com/DIaacKr/SAGE.
Authors:Linda Wermelinger, Simone Lionetti, Fabian Gröger, Nipun Ranasekara, Philippe Gottfrois, Ludovic Amruthalingam, Labelling Consortium, Marc Pouly, Alexander A. Navarini
Abstract:
The clinical integration of AI systems in digital dermatology relies heavily on human trust. Clinically interpretable visual concepts can act as intermediate representations enhancing trust and reliability. However, research in this domain is currently limited by scattered, heterogeneous dataset annotations. In this work, we introduce SkinLex, a harmonized dataset of 48 clinical morphological attributes across four public datasets (SkinCon, DermaCon‑IN, MM‑Skin, and PASSION) for a total of 20,411 records. Supervised nine‑partition classification of skin conditions shows that limiting features to specific visual groups, like shapes or colors alone, reduces diagnostic accuracy. Bootstrapped backward elimination reveals that the set of 48 visual concepts has some degree of redundancy for algorithmic nine‑partition diagnosis on the examined dataset. This demonstrates that coarse diagnosis on the selected dataset requires a relatively small but varied combination of clinical concepts, and motivates further research to improve concept taxonomy. Results can be translated into clinical benefits by reducing inputs for concept‑based models, improving efficiency for annotation and modeling, and further enhancing interpretability. Code and prompt templates are available at https://github.com/Digital‑Dermatology/SkinLex.
Authors:Zhen Guo, Rongyuan Wu, Qiaosi Yi, Chenxi Xie, Xinyu Wei, Lei Zhang
Abstract:
Recent diffusion‑based image generation backbones have grown substantially in scale, making the network inference cost increase rapidly. While diffusion distillation techniques can reduce the number of inference steps, high‑quality image generation within a single full‑backbone‑forward compute budget remains challenging. Existing one‑step methods typically allocate this budget to a single evaluation of a monolithic student. However, approximating the heterogeneous coarse‑to‑fine transport with a single monolithic mapping is difficult and often leads to over‑smoothed outputs. To address this issue, we propose Phase‑wise Velocity Distillation (PVD), which partitions the generation timeline into a coarse and a fine phase, and models the transition within each phase via the average velocity. A dedicated half‑sized expert is assigned to each phase, decoupling structural composition from detail refinement while keeping the cumulative computation equivalent to one full‑backbone forward pass. We show that the use of two half‑sized phase‑specific experts outperforms a single full‑size monolithic student. On class‑conditional image generation, PVD achieves an FID of 1.48 on ImageNet 256 x 256. On more complex text‑to‑image (T2I) tasks, PVD‑distilled models (Stable Diffusion 3.5‑Medium, FLUX.1‑dev, Qwen‑Image) produce results competitive with their multi‑step teachers, significantly outperforming prior distillation methods. Moreover, across the evaluated T2I backbones, PVD reduces active parameters by 49.10‑50.89% and peak VRAM by 45.76‑48.36% compared to the corresponding teachers. Source code and distilled models are available at https://github.com/PolyU‑VCLab/PVD.
Authors:Anuar Aimoldin, Yankai Chen, Ayana Mussabayeva, Xue Liu
Abstract:
A mean shift between two data sources can be easy to detect but hard to remove without substantially changing their representations. We cast its removal as a statistical decision problem: from noisy differences between paired calibration measurements in \mathbbR^d, learn one linear map, applied to both sources under a hard distortion budget, that leaves as little of the shift as possible on fresh data. We derive the exact finite‑sample minimax risk over all such maps, (d‑k) \mathbbE[1/(d+2J)] with J~\mathrmPois(κ/2), where the budget allows deleting k directions and κ is the calibration signal‑to‑noise ratio. Projecting out the mean calibration difference attains it without knowing κ or the noise scale. This exposes a detection‑repair gap: detecting the shift needs only κ\gg\sqrt d, whereas removing a fixed fraction of it at constant distortion needs κ\asymp d, as for estimating its direction. Standard linear concept erasers (MP, SAL, LEACE) remove the same calibration difference, so the formula gives, before fitting, exactly how much shift they leave on fresh data and how much calibration a target requires. The limit is robust: pairing keeps it exact for non‑Gaussian shared content, the projection keeps its guarantee under anisotropic noise, and selective abstention cannot close the gap. On paired clinical and wearable sleep EEG, where differences between participants act as calibration noise, the formula predicts the device shift left in new participants, and more recordings per person soon stop helping. Together, these results tell whether a correction that falls short needs a better method, more recordings, or more participants.
Authors:Guo Tang, Yongtao Wang
Abstract:
Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual‑‑tactile data remains costly. Visual‑to‑tactile synthesis offers a promising approach to augmenting such data, but learning this mapping is complicated by the gap between visual appearance and contact‑related material properties, as well as spatial misalignment in paired observations. To address these challenges, we present PhysTacGen, a visual‑to‑optical‑tactile image generation framework that integrates material‑aware descriptions with geometric conditioning. First, we introduce Group Tactile Policy Optimization (GTPO), a reinforcement learning strategy that refines a vision‑‑language model to generate structured material descriptions using task‑specific rewards. Second, we combine DINOv2‑based pair curation with monocular relative‑depth estimation to select training pairs and provide geometric priors. Finally, an SDXL ControlNet synthesizes optical tactile images conditioned on RGB, relative depth, and GTPO‑generated text. Experiments on curated SSVTP data demonstrate improved structural similarity over the compared baselines, while a blinded user study shows a preference for GTPO‑generated descriptions. Generated tactile inputs also improve performance on an attribute‑derived force‑coefficient prediction proxy. Together, these results demonstrate the effectiveness of PhysTacGen for optical tactile image synthesis and its utility in the evaluated downstream task.The code will be available at https://github.com/VDIGPKU/PhysTacGen.
Authors:Weiwei Qi, Chongyu Wang, Tianhang Zheng, Zefeng Wu, Zhilin Guo, Xiaojun Jia, Zhongjie Ba, Kui Ren
Abstract:
Supervised fine‑tuning can substantially improve the downstream utility of large language models (LLMs) but may compromise their safety. Existing safety‑preserving methods constrain downstream updates using safety‑related parameters or subspaces, but mainly focus on safety preservation rather than joint safety and utility enhancement, lack a theoretical characterization of the optimal safety‑related subspace and safety‑preserving task update, and typically rely on a static safety subspace that may become outdated during fine‑tuning. To address these limitations, we propose ASCENT, a downstream fine‑tuning framework for safety‑‑utility co‑enhancement through first‑order optimal safety‑aware periodic calibration and task optimization. We model safety as a function of LLM parameters S(θ) and use its first‑order approximation to characterize safety changes under parameter updates. Under a fixed rank and Frobenius‑norm budget, we prove that the update constructed from the top‑r singular components of the safety‑function gradient maximizes the estimated safety change, and use it for periodic calibration to preserve and improve safety. We further derive a unique safety‑preserving task update that stays close to the original task update while penalizing negative effects on the estimated safety change. ASCENT alternates these optimal task and calibration updates to jointly enhance safety and utility. Experiments across multiple LLM families and downstream tasks show that ASCENT improves downstream utility by up to 20.3% and reduces attack success rate by up to 35.5%, achieving state‑of‑the‑art safety and utility across all evaluated settings. Our code is available at https://github.com/ZJU‑LLM‑Safety/ASCENT.
Authors:Antoine Edy, Max Conti, Victor Xing, Marc-Antoine Allard, Nawfal Benhamdane, Gautier Viaud
Abstract:
LLM agents often lack the operational knowledge to act reliably in new environments, as they must discover specific tool behaviors or environment conventions on their own. Without memory of past attempts, they repeat the same mistakes across tasks, leading to more task failures and longer trajectories. To address this, agentic systems typically rely on human‑written guidelines or on procedural memory built from training tasks and an oracle verifier, both of which require prior knowledge of the environment. We present DAEDALUS, a method for bootstrapping reusable agent memory from self‑generated practice without existing tasks or oracle verifiers. DAEDALUS pairs two agents: an explorer that interacts with the environment to generate challenging yet solvable tasks, and a solver that attempts them. A heuristic is derived from each solver failure and accepted only after the solver repeatedly succeeds with that heuristic in context. These outcomes also provide feedback for the explorer to refine the difficulty of future tasks. Accepted heuristics are then consolidated into a memory bank for test‑time use. Across AppWorld, τ^2‑bench, and AutomationBench, DAEDALUS improves mean success rates by up to 15.9 points and pass^5 by up to 2.2x over a no‑memory baseline, and is competitive with methods using training tasks, at a lower inference cost than most. We show that performance gains already emerge with a small exploration budget, and that its heuristics also benefit agents from other model families. Our ablations further reveal that solver traces provide the key information needed to derive effective heuristics, while factorizing early discoveries makes exploration more cost‑efficient. Beyond memory construction, we find that the tasks generated by DAEDALUS can serve as a proxy for benchmark tasks when ranking models by performance. Code and artifacts: www.github.com/illuin‑tech/daedalus.
Authors:Jordy Kieto
Abstract:
World models are usually judged from the inside: by prediction loss, by the return a policy earns in imagination, or by how convincing their frames look. We judge one from the outside. We learn a structured, multi‑agent world model of a complete ten‑hero MOBA (206 units, every hero acting every tick, games of up to 6,000 ticks), train a policy only inside it with 1,400‑tick free‑running imagined episodes, and measure that policy in the real game against the opponent the game ships with. The real game never provides a gradient; it provides the policy's own games as training data for the world model, and an online evaluation that selects and anchors the policy. Run as a continuous asynchronous Dyna loop, the policy wins 70.2% of real games as radiant (421 of 600; 95% CI 66.4‑73.7) on seeds never used for any decision, up from 0% for dream training alone and 33.7% before the loop. It wins none as dire, and neither does the shipped opponent when it plays itself. Four findings explain the result. Model exploitation is invisible from inside the dream: every unanchored run collapsed within a few updates while no in‑dream metric tracked the collapse. A world model that is accurate on its training corpus is badly wrong on the policy's own games, and Dyna repairs it there, which is worth +9.2 points of real win rate with the policy recipe held fixed. Finally, the policy inherits its world model's fidelity profile mechanic by mechanic: the model represents the macro game but not crowd control, cast timing or lethality, and the policy wins by map‑wide pressure with almost no coordinated fighting. We release the world model, the dream‑PPO harness, a world‑model debugger, the evaluation protocol, and every policy and log.
Authors:Jorma Valjakka, Juhani Kivimäki, Juha Mylläri, Jukka K. Nurminen
Abstract:
In recent years, several methods for detecting when large language models (LLMs) hallucinate have been developed. These methods are often benchmarked with open‑domain question answering (QA) datasets containing questions and corresponding short reference answers. First, an LLM is used to generate answers to questions within the QA dataset. Then, some automated labeling strategy is used to label these answers as hallucinated or not by comparing them with the reference answers in the dataset. This evaluation setting creates a methodological ambiguity between two criteria: reference faithfulness (whether the answer is fully supported by the reference) and factual correctness (whether the answer is free from contradictions and factually false specific claims). In practice, automated labelers may apply the former criterion even when the intended target is the latter. We study this potential criterion mismatch using 900 human‑labeled question‑answer pairs spanning three commonly used QA datasets and three generator models, with labels targeting answer‑level factual correctness. We evaluate lexical similarity metrics, a reference‑entailment NLI baseline, and seven LLM judges under controlled prompt variants as automated labelers. Our experiments reveal substantial disagreement both among automated labeling strategies and between these labels and human annotations. Many strategies also exhibit strong directional error biases, and for most judge‑generator pairs, replacing a faithfulness‑oriented prompt with a factual‑correctness prompt improves agreement with human annotations and reduces false‑positive dominance, indicating that automated hallucination labels depend strongly on how the target criterion is specified. Label‑source choice should therefore be considered a fundamental part of benchmark design and made explicit, validated, and matched with the benchmark goal.
Authors:Shiroman Prakash
Abstract:
We study cyclic triorthogonal codes in prime dimension p, on which a transversal third‑level diagonal gate acts as a logical non‑Clifford gate. For odd p, we use a definition of triorthogonality that does not require self‑orthogonality. For cyclic codes of length coprime to p, triorthogonality reduces to a sumset condition on the spectral support, which makes exhaustive search feasible: for two natural constructions, we enumerate all maximal codes of this kind for p=3 up to length 121 and p=5,7 up to length 96. For qubits, we prove that a cyclic code of odd length is triply even if and only if it is classically triorthogonal, giving a complete description of triply‑even cyclic codes, and we determine all such codes up to length 763. For qudits, we find codes such as [[19,1,3]]_3 and [[7,1,3]]_7, which improve on the shortest Reed‑Muller and Reed‑Solomon codes in both overhead exponent and distillation threshold. The latter is the first member of an infinite family of [[p,1,2\lfloor p/6\rfloor+1]]_p codes.
Authors:Yuanshuo Zhang, Wenzhe Zhao, Zixing Lei, Bin Chen, Siheng Chen
Abstract:
Video Action Models (VAMs) couple visual dynamics modeling with action generation for robot manipulation. However, video representations are not naturally suited to action generation, as exposing the action policy to excessive visual detail can impair its generalization ability. Therefore, we introduce IronMan (Information‑constRained videO‑actioN learning for robot MANipulation), a robust video‑action learning framework built on the information bottleneck principle. The core principle of this framework is to impose information constraints that suppress irrelevant visual information while preserving action‑relevant dynamics cues. IronMan employs a dynamics‑aware bottleneck that distills noisy, entangled one‑step video features into compact world representations. Extensive simulation and real‑world experiments demonstrate strong in‑distribution (ID) performance and out‑of‑distribution (OOD) robustness while maintaining efficient inference. IronMan achieves success rates of 99.0% on LIBERO and 79.4% on RoboTwin clean2clean, outperforming all the evaluated baselines. Under OOD shifts, IronMan achieves a success rate of 79.1% on LIBERO‑Plus, exceeding the strongest baseline by 10.4 percentage points. Project page: https://youngsoul0731.github.io/ironman‑project‑page/
Authors:JunGyu Lee, Inhwan Bae, Hae-Gon Jeon
Abstract:
Language‑based trajectory predictors represent coordinates as discrete tokens and learn auxiliary tasks such as destination and group reasoning. This formulation enables the model to capture behavioral intent and social context beyond coordinate dynamics alone. However, token‑level objectives provide only indirect guidance for continuous coordinate‑space dynamics. To address this limitation, we introduce MoRE (Mixture of Reward Experts), a refinement framework that transfers numerical forecasting priors into a pretrained language‑based predictor through reinforcement learning. Five frozen numerical predictors provide complementary coordinate‑level knowledge of motion and interactions. Their predictions are converted into expert rewards and combined through an uncertainty‑weighted consensus that penalizes disagreement. A ground‑truth reward anchors the prediction to the target trajectory. To focus refinement on difficult cases, MoRE refines the policy using the top 1% of training samples ranked by predictive entropy. Expert predictions are computed once and cached before PPO training, so the experts are not run during policy updates or inference. In this way, MoRE combines the contextual modeling of the language‑based predictor with coordinate‑level feedback from numerical experts. On ETH‑UCY, MoRE reduces ADE from 0.22 to 0.20 m and FDE from 0.32 to 0.29 m. Relative to the base policy, ADE decreases by 17.9% on SDD and 12.7% on NBA. On ETH‑UCY, MoRE also reduces collision rates and better matches ground‑truth pedestrian spacing, without increasing measured inference memory or latency. The project page is available at https://jungyu0413.github.io/MoRE/.
Authors:Filip Dorm, Leonora Vesterbacka
Abstract:
Redpine Science gives models and agents a single access point to a wide range of peer‑reviewed literature, queried directly through the Model Context Protocol (MCP) and an API. This report evaluates Redpine Science on two levels: the relevance of the retrieved chunks, and a model's answer when it has access to Redpine Science compared to web search. Both public and expert‑validated benchmarks are used. Public benchmarks are a widely accepted way to test model development and are comparable across labs, but risk saturation and memorization. To address this, we complement them with an expert‑validated question set. In total, this report presents four evaluations. On ScholarQABench SciFact, the public answer‑quality benchmark reported here, an agent with Redpine Science answers 94.4% of claims correctly against 87.6% with no retrieval. On the expert‑validated question set, an agent with Redpine Science states 80.1% of the required claims against 70.2% for an agent restricted to web search. On the 668 queries of a public retrieval benchmark whose gold paper Redpine holds, stripped of any model reasoning, Redpine Science places the correct source paper in its top ten results for 83.1% of queries (Recall@10), against 79.3% for the benchmark's creator. A blinded expert relevance panel places Redpine Science's Precision@5 at 75.2% against 39.8% for the PubMed search tool. We release the expert‑validated question set and instructions to reproduce every headline result above, at https://github.com/redpine‑ai/benchmarks.
Authors:Heng Yu, David D. Yuan, Juze Zhang, Changan Chen, Yao Feng, Michelle Baldonado, Steve Cousins, Li Fei-Fei, Jiajun Wu, Ehsan Adeli
Abstract:
World‑action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world‑action modeling framework built around a common causal robot‑video foundation and configurable video‑action interaction. Starting from Wan2.2‑5B, we perform causal robot‑video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture‑of‑Transformers architecture that supports joint, video‑then‑action, action‑then‑video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real‑world bimanual tasks; robot‑video training with causal adaptation improves VTA success on LIBERO‑Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local‑context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held‑out LIBERO‑90 tasks, compared with 47.0% for a full‑context inverse model and 21.5% for a local‑context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same‑state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.
Authors:Xuekang Zhu, Kaiwen Feng, Ruifeng Wang, Xiwen Wang, Xiaochen Ma, Bo Du, Changjiang Jiang, Chenfan Qu, Songyu Ye, Xia Du, Wentao Feng, Jian Liu, Ji-Zhe Zhou
Abstract:
Image Manipulation Localization (IML) is commonly formulated as a fully supervised learning task that estimates the optimal manipulation mask y for a given image x. In this work, we first reveal the latent nature of artifacts and thus reinterpret IML as a latent‑variable problem, P(y|x)=\int P(y|z)\,P(z|x)\,dz, where z denotes the artifacts. Following this interpretation, we pinpoint the cause for the current IML models' insufficiency as their implicit artifacts modeling strategy, highlighting the necessity of modeling z in an explicit manner. Without direct labels, feature disentanglement is the most appropriate solution for this explicit modeling. Accordingly, we propose a two‑stage learning paradigm with the Pairwise Artifacts Learning (PAL) and Standard Localization (SL) phases to estimate P(z|x) and P(y|z) via edit relations. To support our edit‑relation‑based learning, we further curate EditGroup‑45K, a source‑anchored dataset organized into edit groups for pair construction. Extensive experiments show that our PAL paradigm yields consistent improvements across diverse IML architectures, and empirical analyses further verify that PAL does capture artifacts explicitly through feature disentanglement. Code and dataset are available at https://github.com/venus‑guangjian/PAL
Authors:Shrihari Dumbre, Bikash Santra
Abstract:
Gastric adenocarcinoma (GA) is a leading cause of cancer‑related mortality worldwide, and accurate histopathological subtype classification from whole‑slide images (WSIs) is essential for effective treatment planning. While multimodal approaches that integrate pathology report text with WSIs can improve classification, existing methods often depend on computationally expensive transformer architectures and large language models. We propose a multimodal knowledge distillation (MKD) framework that combines a pretrained WSI image encoder and a clinical text encoder using Low‑Rank Multimodal Fusion (LMF) to efficiently model cross‑modal interactions during training. Each WSI is represented as a bag of patches paired with a slide‑level diagnostic caption. The teacher model learns fused image‑text representations for subtype classification, while the student model distills this knowledge to enable accurate image‑only inference. We evaluate our method on the PatchGastric benchmark dataset and achieve at least 3.35% higher mean accuracy than state‑of‑the‑art approaches, without relying on transformer‑based fusion, multi‑task learning, or large language models. The source code is available at https://github.com/helomelo1/MKD‑LMF.
Authors:Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi, Llion Jones
Abstract:
Recurrent neural networks typically compress information into a single vector‑valued recurrent state, forcing short‑term computation and long‑term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of rapid neuron‑level processing and longer‑term retention found in biology. To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix‑valued short‑ and long‑term memory states serving distinct functional roles. Building on the Continuous Thought Machine (CTM), the CMM's short‑term memory tracks recent neural activity, with uniquely parameterized neuron‑level models learning to use these activity patterns for computation. A persistent long‑term memory stores information for later use, with a Transformer jointly updating both memory stores, providing an expressive bidirectional read‑‑write mechanism such that each store can reorganize its own contents and both read from and write to the other. Across algorithmic, in‑context learning, and recurrent reasoning tasks, the CMM outperforms a broad suite of baselines, exhibiting stronger generalization than prior memory‑augmented networks while preserving the CTM's interpretable attention patterns. Code is available at https://github.com/SakanaAI/continuous‑memory‑machines.
Authors:Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Borui Jiang, Bo Han
Abstract:
Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is insufficient to answer the questions. Existing faithfulness evaluations typically assess each question‑context instance in isolation; however, such instance‑level evaluation fails to capture a fundamental requirement of faithful behavior: the ability to adapt model responses to changes in available contexts. In particular, a model should provide correct answers when sufficient evidence is present and abstain when it is not. In this work, we propose a Pairwise Faithfulness Benchmark (PFaithBench) that evaluates whether a model can switch between answering and abstaining for the same question under supporting versus non‑supporting contexts. Our evaluations across thirty‑nine models with seven model families demonstrate that faithfulness fundamentally involves a trade‑off between answering and abstaining, and that most current models exhibit a strong bias toward answering, with most faithfulness errors arising from over‑answering, i.e., models tend to fabricate a response even when the provided context is insufficient. We further conduct a series of studies on faithfulness training under different data constructions. Our results show that training outcomes are highly sensitive to the specific composition of answering and abstaining data. Constructing answering and abstaining data from mismatched sources can cause models to rely on dataset‑specific shortcuts rather than actual context sufficiency. Moreover, increasing answer‑supervised data improves answering performance but exacerbates over‑answering, while increasing abstaining data reduces hallucination but leads to over‑abstention. The code and data are released at https://github.com/tmlr‑group/PFaithBench.
Authors:Inuk Kang, Seung-Woo Seo
Abstract:
Off‑road navigation exposes a robot to potentially hazardous terrain en route. Although learning‑based navigation uses safety supervision to choose which path to drive, it provides no runtime alarm when the robot following that path is heading into danger. Such an alarm must be learned from field logs, where human intervention preempts the failure and the failure itself is therefore never observed. The human judges driving unsafe early but typically intervenes only once failure is clearly near, so the intervention marks that judgment late. That earlier judgment is what a runtime alarm must detect, yet no prior intervention‑supervised method has targeted it. To address this problem, we introduce CUSP (CUSUM‑governed Survival model of the Perception onset), a model‑agnostic runtime hazard alarm that learns this moment from intervention‑terminated logs. "Cusp" is a word for the point at which one state is about to turn into another, and the moment we target is exactly such a cusp: the point at which safe driving turns unsafe in a human's judgment. We call this point the perception onset and annotate it separately from the intervention. A visual hazard head is trained on the annotated onset with a discrete‑time survival objective so that driving with and without an onset both supervise the head, and a CUSUM accumulates the predicted onset risk into alarms. We evaluate CUSP at five unseen sites, two autonomous and three teleoperated, with 142 events and every method tuned to the same rate of ten false alarms per hour. CUSP detected 85 events compared to 26 for the best of nine adapted baselines, and the margin comes from hazards to which every signal in the navigation model is blind.
Authors:Jaehui Hwang, Dongyoon Han, Sangdoo Yun, Byeongho Heo
Abstract:
On‑policy distillation (OPD) has been widely studied as a post‑training method in which a student model obtains token‑level supervision from a stronger teacher on its own rollouts. Recent studies have improved OPD through alternative distillation reward formulations and teacher configurations, while the objective of distillation remains centered on mimicking the teacher. However, when a stronger teacher has limited distributional overlap with the student, such positive guidance can provide insufficient learning signals. In this work, we introduce Negative‑Policy OPD (NP‑OPD), which complements teacher supervision with rollouts from a lower‑performing, lower‑capability negative policy that serves as a negative reference for the student. Rather than modifying the distillation reward formulation, NP‑OPD introduces the negative policy at the rollout stage, continuously supplying tokens preferred by the negative policy over the teacher so that they remain exposed to teacher supervision throughout training. This provides an explicit negative signal through negative‑policy rollouts while preserving the positive teacher supervision used in OPD. Through extensive experiments, we show that NP‑OPD improves OPD across model scales, generation modes, reasoning domains, and different OPD variants. Furthermore, our analyses show that NP‑OPD effectively suppresses tokens preferred by the negative policy over the teacher and moves the student away from the negative policy. These results support our design of introducing negative signals through negative‑policy rollouts and provide new insight into the role of the rollout policy in OPD. Code will be available at https://github.com/naver‑ai/np‑opd.
Authors:Sunchan Park, Beomkwon Cho, Kyeongbo Kong
Abstract:
Large language model agents have been used to search over symbolic structures such as programs and equations. We propose CueRator, an agentic framework for policy‑aware decision‑rule discovery, which adapts frozen contrastive multimodal encoders by searching for the decision rule that converts their cross‑modal similarities into predictions. We validate it on open‑vocabulary audio‑visual event perception, where existing methods involve a trade‑off between adaptivity and generalization to unseen categories: trained modules adapt at the cost of generalization, and fixed rules the reverse. The framework pairs a symbolic formulation for generalization with a lightweight policy that predicts its parameters per video for adaptivity. A report‑guided multi‑agent loop discovers the formulation offline, evaluating each candidate on its expressive ceiling and on whether a trained policy can realize it. On OV‑AVEBench, CueRator raises the total average from 57.8 to 60.2 and unseen‑category performance from 55.8 to 59.9 over the best existing method, reducing the seen‑unseen gap from 7.1 to 1.2. Ablations attribute the gains to both the formulation and the policy and show that both feedback signals are necessary for effective search. CueRator also improves over the respective baselines on two further audio‑visual event perception tasks, and the discovered rule remains competitive across encoders with only the policy retrained. Code is available at https://github.com/cvsp‑lab/cuerator.
Authors:Taha Berk Terekli
Abstract:
Explicit R‑matrices of quantum groups are routinely transcribed from the literature, yet whether they satisfy an identity depends on basis order, tensor placement, the coproduct and, for superalgebras, the Koszul sign rule. quantum‑group is a Python package that builds such matrices exactly with SymPy and returns every identity as an exact residual matrix, whose nonzero entries locate the convention that disagrees. It covers U_q(sl_2) modules, tensor products and the fundamental R‑matrix, and the graded Yang‑‑Baxter equation for GL_q(2|1). Applied to the thesis it was written for, it exposed an R‑matrix that satisfies the Yang‑‑Baxter equation but intertwines the opposite coproduct.
Authors:Ben Wooding, Simone Garatti, Marco C. Campi, Abolfazl Lavaei
Abstract:
The scenario approach is a well‑established statistical framework for data‑driven decision‑making. In particular, in data‑driven optimization, the scenario approach unveils how the problem structure governs out‑of‑sample generalization, and offers a principled basis for assessing and certifying the reliability of the optimal solution as per constraint satisfaction. Despite its strong theoretical development and wide applicability, no software toolbox has been available to date that enables user‑friendly, data‑driven convex optimization within the scenario‑approach framework. In this paper, we introduce Scen‑Opt, an open‑source software tool that integrates convex programming with data samples while providing statistical guarantees grounded in scenario theory. Scen‑Opt is implemented in Python, supporting data‑driven linear, quadratic, and semidefinite programming, and offers a Python‑based web application with an intuitive and reactive graphical user interface (GUI) built using modern web technologies. Scen‑Opt can be used directly through its online interface or installed locally, accommodating both manual input and data‑file uploads (CSV, JSON, TXT, TSV, MAT, Excel, NPY, NPZ, Parquet). Built on a Python backend with a modern JavaScript frontend, Scen‑Opt offers a highly user‑friendly experience and efficient usability across desktops, laptops, tablets, and mobile devices. In this paper, Scen‑Opt is applied to a set of representative benchmarks, demonstrating its practical effectiveness for data‑driven convex optimization with guaranteed performance.
Authors:Shuhao Li, Fanghua Ye, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen
Abstract:
Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distribution correction and rejects tokens that remain highly plausible under the target model when the draft model assigns excess probability. This conservative verification limits the number of draft tokens retained after each verification forward pass. We introduce Nucleus Speculative Decoding (NSD), a relaxed verification method that incorporates target‑model plausibility into speculative decoding. NSD accepts a draft token if it satisfies the standard acceptance rule or belongs to the target model's nucleus. We theoretically characterize the distributional deviation introduced by our method and show that the single‑step error is exactly determined by the draft model's excess probability within the target nucleus. We further derive sequence‑level fidelity bounds that quantify how local deviations accumulate over autoregressive decoding. Experiments across multiple target models and proposal mechanisms demonstrate that NSD consistently improves speculative decoding efficiency while maintaining competitive task performance. Our method achieves throughput speedups of up to 5.16× over autoregressive decoding and up to 3.15× over standard speculative decoding. These improvements coincide with longer accepted lengths, allowing more output tokens to share the cost of each target verification pass. Analysis shows that plausibility‑aware verification provides an effective approach for relaxed verification and speculative decoding efficiency. Our code is available at https://github.com/EIT‑NLP/Nucleus‑Speculative‑Decoding.
Authors:Myunghoon Kang, Jungseob Lee, Jaehyung Seo, Heuiseok Lim
Abstract:
Small reasoning models (SRMs) have shown strong performance on complex reasoning tasks by generating extended chain‑of‑thought trajectories, but they often fail to recover once their reasoning enters an erroneous path. Existing test‑time fusion methods rely on local fusion signals to determine when to trigger fusion, which can be misled by transient uncertainty fluctuations and may reinforce unstable reasoning trajectories. We propose ThinkFuse, a training‑free test‑time fusion framework that selectively intervenes in unreliable reasoning segments. ThinkFuse compares segment‑level uncertainty shifts with trajectory‑level uncertainty trends to identify unstable reasoning points and fuse auxiliary reasoning paths into the primary model's trajectory. Extensive experiments demonstrate that ThinkFuse outperforms baselines on mathematical and knowledge‑intensive reasoning benchmarks, with consistent gains across model‑family combinations, and remains robust with a smaller primary model. Our analysis shows that ThinkFuse requires fewer fusion triggers and generates fewer tokens, highlighting the efficiency of selective triggering. Our code is available at https://github.com/js‑lee‑AI/ThinkFuse.
Authors:Catherine Ji, Vivek Myers, Sergey Levine, Benjamin Eysenbach
Abstract:
Empowerment captures the capacity for an agent to actively control its environment. While conceptually appealing as an information‑theoretic quantity, the connection between empowerment and structurally central states that provide broad access to future outcomes has remained an open question. In this work, we link empowerment maximization and skill‑learning methods to provide new geometries for interpreting and analyzing empowerment. Our analyses answer longstanding open questions on the connections between empowerment and structural centrality. Our analyses also reveal distinctions between information and reward geometries, highlighting important theoretical implications to build scalable empowerment‑maximization methods. Website and code can be found at https://empowerment‑geometry.github.io/.
Authors:Qi Yang, Shuting Xia, Le Yang, Geert Van Der Auwera, Zhu Li
Abstract:
This paper presents a novel effective Gaussian Splatting (GS) sequence Compression method that utilizes the Video codec (GSCV). Existing video‑based GS sequence compression relies on the Parallel Linear Assignment Sorting (PLAS) and tracked primitive information to convert GS into smooth 2D videos. However, tracked information is not available for most practical applications, and without it, using the vanilla PLAS can generate images exhibiting weak inter‑frame correlation, due to its stochastic nature. GSCV incorporates a simple yet efficient Inter‑PLAS method to produce close images between the I‑ and P‑frames of GS, enhancing the inter‑frame performance of video codec greatly. GSCV also realizes a new pipeline based on the state‑of‑the‑art video codecs with high bit‑depth GS images, achieving higher compressibility while simultaneously providing a higher quality upper bound. Experimental results show that the proposed GSCV exhibits obviously improved performance over MPEG video and point cloud‑based anchors in GS sequence compression. The code is available at https://github.com/Qi‑Yangsjtu/GSCV.
Authors:Yuezhe Zhang, Lei Wei, Jingnan Du, Shuai Wan
Abstract:
Non‑contact three‑dimensional reconstruction of thin, sheet‑like heritage artifacts poses significant geometric and registration challenges. Due to their fragility, these artifacts cannot be suspended or equipped with artificial markers, necessitating independent acquisition of their front and back surfaces. Subsequent registration proves difficult due to the limited number of shared geometric features and the scarcity of explicit physical constraints, which may result in rotational ambiguity, instability, and structural collapse during iterative optimization. To address these challenges, we propose a geometry‑constrained bidirectional point cloud registration method specifically tailored for thin, sheet‑like heritage artifacts. The method integrates semantic‑guided preprocessing, Principal Component Analysis (PCA)‑based geometric normalization, and a thickness‑aware registration strategy. The estimated physical thickness is incorporated as a geometric constraint to preserve structural integrity during registration. Rotational ambiguity is resolved by evaluating a finite set of global rotation hypotheses, each refined using the point‑to‑plane Iterative Closest Point (ICP) algorithm, with the optimal transformation selected via a geometry‑aware fitness criterion consistent with the thickness scale. Experimental results show that the proposed method achieves competitive or improved performance in most cases, particularly in projected area consistency and physically plausible front‑back alignment. In addition, the thickness‑aware constraint and rotation hypothesis evaluation reduce the risk of degenerate configurations in which the two surfaces are incorrectly flipped while still yielding deceptively acceptable numerical scores, supporting reliable non‑contact digitization of delicate and thin heritage artifacts. Implementation details are available at https://zyz‑nwpu.github.io/GCBPCR/.
Authors:Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan, Shuowei Jin, Beidi Chen
Abstract:
Large language model agents are increasingly deployed to perform complex tasks in real‑world environments. However, the knowledge required for correct behavior in these environments is often implicit, undisclosed, and subject to change over time. Recent continual‑learning harnesses seek to address this challenge by enabling agents to improve from serving experience. Yet the effectiveness and limitations of these methods are not yet well characterized. Existing benchmarks provide only partial coverage: some explicitly provide the target knowledge, others assume a static environment, and those that support continual adaptation remain limited in scale and knowledge diversity. To enable systematic evaluation, we formalize an evolving‑environment streaming dataset (EESD), in which agents must infer, apply, and revise latent environment knowledge from interaction and outcome feedback as hidden policies evolve, and introduce ServeLearnBench, spanning retail support, banking, and sales‑pitch generation with 53 environment windows and 7,718 tasks. We evaluate five learning harnesses (RAG, Mem0, SkillOpt, Continual Harness, and Prime) across six models (GPT‑5.6 Terra, Opus 5, Kimi K3, GLM‑5.3, DeepSeek V4.1 Flash, and GLM‑5.3 Flash), covering 28 model‑harness pairs and 252 learning runs. Our evaluation reveals three main findings: a substantial gap remains between task capability and learning from experience; continual adaptation is costly and can degrade already‑correct behavior; and insufficient exploration emerges as a key bottleneck to effective adaptation. Overall, ServeLearnBench provides a controlled testbed for diagnosing these limitations and tracking progress toward agents that continually and reliably improve through serving experience.
Authors:Peihua Mai, Zhuoyan Shao, Xinbao Qiao, Meng Zhang, Xinyue Zhou, Yan Pang
Abstract:
Illusory pattern perception is a well‑documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tendency, often described as "connecting the dots" where none exist, can result in systematic reasoning errors. This paper investigates whether Large Language Models (LLMs) exhibit such perceptual tendencies, which can lead to systematic errors in downstream applications. To our knowledge, this work presents the first systematic study of illusory pattern perception in LLMs, adapting classic psychological paradigms to three tasks with direct empirical comparison to human behaviors. We find that LLMs frequently exhibit stronger illusory pattern perception than humans. In particular, models tend to over‑associate frequent positive attributes with majority groups or large organizations, and show increased tendencies to construct causal narratives from ambiguous events. To uncover the mechanism behind these behaviors, we develop a feature interpretability framework based on Sparse Autoencoders (SAEs) to analyze internal representations. Our results reveal that holistic frequency perception and analytic cognitive orientation are linked to the emergence of illusory perceptions. These findings highlight a previously underexplored cognitive‑like illusion that may affect the reliability of LLM reasoning. Code available at https://github.com/NusIoraPrivacy/illusory.
Authors:Yunhui Liu, Xudong Jin, Kang Zhang, Danshuo An, Yu Xing, Te Song, Jia Liu, Tieke He
Abstract:
Attributed graph clustering aims to discover node groups by jointly exploiting node attributes and graph topology, yet its unsupervised nature makes model selection and adaptation inherently difficult. Existing methods typically train and tune a separate model for each input graph, leading to costly and fragile pipelines that often fail to transfer across graphs with different feature spaces, structural patterns, and attribute‑structure correlations. In this paper, we study a one‑for‑all alternative: can a single model be trained once and directly applied to diverse attributed graphs without graph‑specific training, fine‑tuning, or hyperparameter search? We propose OFAG, a foundation model for attributed graph clustering. Building upon Prior‑data Fitted Networks, OFAG learns a reusable clustering inference strategy from synthetic attributed graphs generated under broad priors over latent clusters, node attributes, and graph structures. To handle incompatible feature spaces across graphs, OFAG adopts a dimension‑agnostic signal‑wise graph encoder that treats each feature channel as a graph signal and models its response to shared graph filters. The model is trained with a hyperspherical clustering objective, producing clustering‑friendly node representations in a single forward pass at inference time. On ten datasets, one frozen OFAG model achieves the best mean performance and average rank across NMI, ACC, ARI, and F1, while completing all ten datasets in 12.43 minutes total‑‑‑over 6 faster than the second‑fastest baseline and nearly 28 faster than the second‑best on clustering quality. Our code and pretrained checkpoint are available at https://github.com/Cloudy1225/OFAG, allowing practitioners to directly apply OFAG to their own attributed graph datasets without additional training or tuning.
Authors:Xianwen Deng, Ruijie Zhao, Mingwei Zhan, Yijun Wang, Zhi Xue
Abstract:
Website fingerprinting infers the websites visited by users from encrypted traffic metadata. However, models trained under fixed collection conditions often degrade as website sets, collection times, network paths, browsers, or defenses change. Existing transferable attacks either rely on handcrafted perturbations of individual traces or adapt language‑oriented architectures to traffic, limiting their ability to capture traffic‑native semantics. To address these limitations, we propose SCSM, a traffic‑native foundation model for transferable website fingerprinting. Specifically, SCSM constructs pairs of pretraining views from the same group of unlabeled traces through Segmentation, Combination, Scaling, and Masking. These operations produce diverse observable patterns while preserving the underlying packet events and local traffic dynamics of real trace fragments. The corresponding windowed traffic counting matrices serve as inputs for contrastive pretraining of a state‑space encoder without website annotations. The pretrained encoder is then fine‑tuned on a small labeled support set, and predictions are aggregated across temporal scales at inference. Experimental results demonstrate that SCSM surpasses the strongest baselines by 16.4% in average top‑3 accuracy over six temporal‑drift tasks on GTT23 and by 13.8% in average macro‑F1 over four cross‑domain datasets. The code and datasets will be made available at https://github.com/SJTU‑dxw/WF‑SCSM.
Authors:Junru Zhu, Yixin Yang, Xiaoqing Ding, Ruoyu Qi
Abstract:
Retrieval‑augmented generation systems can route difficult queries to deeper context, but batch deployments must allocate a shared token budget across calls whose costs vary by query. We formulate selective escalation as finite‑batch allocation for financial document question answering. Each of 150 FinanceBench questions first receives a top‑1 retrieval answer. Predictors estimate the adjudication‑quality gain and token cost of an optional top‑5 call, and the allocator prioritizes calls by predicted gain per token. At the nominal 10% budget, gain‑per‑token allocation improves adjudication quality over gain‑only ranking by 0.034 (95% document‑bootstrap CI [0.001, 0.072]) while using 46.6% fewer total tokens than one‑pass top‑5 retrieval. Additional‑call cost is accurately predictable (R‑squared 0.93), whereas beneficial escalation remains difficult to rank (AUROC 0.60). These results show that heterogeneous cost is actionable under tight constraints, while progress across the full budget frontier depends on stronger query‑specific benefit estimates.
Authors:Hyeongheon Cha, Hyungjun Yoon, Sung-Ju Lee
Abstract:
Training‑free token reduction accelerates vision transformers by removing redundant tokens across layers, recovering most of the original accuracy at a fraction of the compute. These methods, however, are designed and evaluated primarily on clean data, and under real‑world distribution shift their accuracy gap to the uncompressed model widens with the removal rate. We show that this gap is governed by the reduction schedule, the depth profile of removal, usually left fixed as an implementation detail. Concretely, we introduce a one‑parameter late‑concentrated power‑law schedule that consistently improves out‑of‑distribution accuracy over flat at no extra inference cost. On ImageNet‑C with DeiT‑S, the late schedule closes 83% of that gap at a 26% compute reduction (+1.17pp), and 99% of it at a lighter 7% reduction (+0.26pp). The gain cannot be attributed to retaining more tokens or using extra compute: held to flat's compute, the late schedule removes more tokens in total and leaves fewer tokens at the end, yet still wins. Single‑layer probes point to a mechanism: earlier reductions perturb features that pass through more remaining layers, front‑loading reduction error in depth. The effect is broad, holding across five token‑reduction methods (ToMe, EViT, ATS, ATC, PiToMe), nine backbones, all ImageNet‑C corruption types, eight further shift suites, and two further modalities, video and vision‑language QA. It is also specific to shift, still positive on clean and rising monotonically to ~4x that at the highest severity 5. The schedule keeps its gain under six test‑time adaptation methods, and needs no per‑input or per‑domain tuning.
Authors:Shangyuan Yuan, Xinda Qi, Yujiang Pu, Wenliang Guo, Xiaobo Tan
Abstract:
Vision‑language‑action (VLA) models increasingly rely on diffusion‑ or flow‑matching‑based action heads to generate continuous robot actions. These action heads typically process the denoising trajectory in a largely uniform manner. However, we observe that the conditioning focus naturally shifts across denoising stages: early stages combine language instructions and visual observations to establish a coarse action trajectory, whereas later stages place greater emphasis on current visual observations for action alignment. Based on this insight, we introduce StairVLA, a stage‑aware hierarchical action generation framework that uses partially denoised actions as a natural interface between coarse long‑horizon action generation and local refinement. A high‑level VLA performs early denoising to produce a reusable long‑horizon partially denoised action trajectory, while a lightweight refiner operates at a higher frequency to refine local action chunks using the latest observations. This design amortizes expensive high‑level VLA computation while preserving frequent closed‑loop correction. On LIBERO, our GR00T‑style instantiation improves average success from 96.5% to 97.8% while reducing amortized inference latency from 115.0 ms to 44.2 ms per action chunk. More broadly, across two VLA backbones, simulation benchmarks, and real‑robot tasks, StairVLA consistently reduces inference cost while maintaining strong task performance.
Authors:Ching-Hsiang Chang, Tzu-Yu Chuang, Yi-Hsiu Lee, Yi-Ting Chen, Yuan-Fu Yang, Min Sun
Abstract:
Pose error in precision assembly can corrupt not only what a robot observes but also the coordinate frame in which it acts. On the FORGE benchmark, the official state‑based policy succeeds in 97% to 99% of episodes with the true pose but only 32% to 60% at the benchmark's σ=5 mm pose‑noise setting. The same estimated pose enters the observation and anchors the action frame, making the offset unidentifiable from proprioceptive state alone before contact. We supply this missing information during training in two ways. A privileged teacher observes the offset in simulation, while clean demonstrations can instead be relabelled into the displaced frame in closed form. The deployed student is trained with behaviour cloning followed by one DAgger round and receives only noisy state, a raw wrench window, and two RGB cameras at test time. On the unmodified FORGE tasks, the teacher‑route student maintains 92% to 99% success across σ=0 to 5 mm, while six non‑privileged baselines fall to 2% to 80%. A matched behaviour‑cloning experiment isolates the source of this robustness. With the same student architecture, data budget, and training procedure, demonstrations generated without offset access yield only 31.5% success at σ=5 mm, whereas privileged and relabelled demonstrations reach 88.7% and 95.8%. Deployed zero‑shot on a Franka, the student reaches 83.3% pooled success at σ=5 mm against 34.4% for the strongest state‑based policy. Deployable sensing alone is insufficient. Robustness requires supervision that encodes compensation for the latent frame offset.Project page: https://drychang.github.io/displaced‑frame/
Authors:David Pissarra, Jinkun Lin, Haitian Jiang, Aurojit Panda, Jinyang Li
Abstract:
Optimized kernels such as FlashAttention and FlashDecoding are crucial for accelerating today's large models. Most of them are handwritten by experts because existing ML compilers cannot match their efficiency. Producing such kernels requires fusing computations with multiple reductions, which requires both algebraic transformation of the computation graph and operator scheduling of the transformed graph. Unfortunately, searching the two jointly yields a space too large to navigate. We propose Cleave, an ML compiler built on symbolic decoupling: Cleave discovers transformations by performing superoptimization on a graph with symbolic shapes, and then schedules each resulting graph on concrete shapes. Representing shapes as symbols makes equivalence checking cheap and lets a new Split operator, with a symbolic split count, parallelize along a reduction dimension. Cleave's scheduler fuses graphs with multiple reductions through iterative tiling and horizontal fusion. Evaluation on common LLM subgraphs shows that Cleave generates kernels up to 2.8x faster than the best baseline (1.6x on average) and reduces compilation time by 5.9x on average compared to Mirage. For dynamic workloads captured from production serving traces, Cleave compiles each operator once and achieves geometric mean speedups of 1.4x and 1.7x over FlashInfer's handwritten FA2 and FA3 backends. Cleave's code is available at: https://github.com/nyu‑systems/cleave
Authors:Shuyu Gan, Young-Jun Lee, Dongyeop Kang
Abstract:
Typed decision models answer a declared question without generating text: a decision head returns a probability for each of the declared options in a single forward pass. A single pass is fast, intuitive System 1 thinking. We study what lies between one pass and generated reasoning: looping, in which the same layers are recursively applied several times before one typed readout. Each loop lets the model revise its hidden state before it commits to an answer, without generating a token; we call this System 1.5 thinking. We propose SanSi, which turns a pre‑trained looped language model into a typed decision model. The option probabilities are read after every loop, and every loop is trained with a proper scoring rule, so that one model serves every budget from one loop to eight in a single run. On 10,027 test decisions from 59 sources, SanSi reaches 72.0% accuracy: 13.5 points above a non‑looped model of the same shape trained with the same recipe, 5.3 points above a newer non‑looped model of its size, and 1.8 points below one with three times the parameters. On two depth‑controlled tasks, loops extend the solvable depth beyond the depths seen in training, where the larger single‑pass model fails. Used as the judge for policy optimization with reinforcement learning, without gold answers, SanSi raises the generator's F1 by 7.7 points.
Authors:Kai Zhou, Chuanshen Chen, Runhao Zeng, Meng Dai, Yifan Yang, Jinwu Hu, Daiyuan Li, Mingkui Tan, Fei Liu
Abstract:
Videofluoroscopic Swallowing Study (VFSS) is one of the gold standard for diagnosing swallowing disorders, providing dynamic X‑ray imaging of the swallowing process. Automated kinematic analysis in VFSS relies fundamentally on precise anatomical keypoint localization. However, existing studies focus on limited keypoints (e.g., cervical vertebrae or the hyoid) and overlook critical regions such as the soft palate, while annotating only active swallowing segments and ignoring abundant non‑swallowing data, resulting in poor data efficiency. Moreover, leveraging this unlabeled data via standard semi‑supervised learning is suboptimal, as generic methods are prone to spatial bias. In medical X‑rays with fixed layouts, models tend to memorize absolute coordinates rather than understanding anatomical structures. To tackle these challenges, we introduce VFSSKep, a novel dataset that extends annotations to the soft palate and incorporates large‑scale unlabeled data. We further propose S^3KL, a Structure‑aware Semi‑Supervised Keypoint Localization framework designed to overcome spatial bias. It integrates a Structure‑Aware Learning strategy to extract high‑resolution structural cues for structure‑aware representation learning, and a Structural Representation Consistency Learning strategy with block shuffling to enforce invariant structural recognition. Experiments show our method achieves state‑of‑the‑art semi‑supervised performance, even with unlabeled and 25% labeled data surpassing fully supervised learning with 100% labeled data. Code and data will be made publicly available at: https://github.com/kaai520/S3KL.
Authors:Ran Li, Lei Chen
Abstract:
Prefill‑only decision models inspired by the Jev model score every candidate in a menu during a single forward pass and never decode, which makes one call one to two orders of magnitude cheaper than a same‑scale generative language model. We show that this read‑out structure comes with a testable property. When an intervention changes only the candidate menu and leaves the input text fixed, the post‑intervention accuracy is already determined by the cached first‑pass distribution. The estimator restricts the pass‑1 probabilities to the menu, renormalizes, and reads off the argmax; it uses no labels and no second forward pass. Across seven model families, ten datasets and two task types, menu‑only interventions are predicted to within 4.2 points, and for one family the prediction is exact. A probability‑level variant of the same estimator errs by 21.0 points, so the property lives in the ranking rather than in the probabilities and is not recovered by calibration. Same‑scale generative language models do not share the property. On those models the same estimator errs by 1.6 to 15.8 points and degrades as the model grows. The property turns inference‑time compute into a decision that can be made before deployment. Uniform extra passes buy calibration but almost no accuracy; at matched cost a confidence cascade outperforms every scheme that re‑asks the same model, and curating the menu beats enlarging the model, with a 0.8B model on a curated 5‑candidate menu reaching 95.4% on CLINC150 against 80.0% for a 4B model on the full 150‑label menu.Code and data are available at https://github.com/rlisml/jev‑cascade.
Authors:Byeonghu Na, Donghyeok Shin, Yeongmin Kim, Mina Kang, Il-Chul Moon
Abstract:
Autoregressive large language models (LLMs) have rapidly advanced in capability, but their increasing scale comes with substantial computational and memory costs at inference time. Knowledge distillation (KD) offers a practical solution by transferring knowledge from a large teacher model to a smaller student model via alignment of discrete probability distributions. However, existing KD methods for LLMs primarily rely on divergences that evaluate discrepancies through probability values at each vocabulary index, without explicitly leveraging token‑level semantic information. We propose Wasserstein‑based knowledge distillation (WASD) for LLMs, which incorporates token‑level semantic information via the Wasserstein‑based distance with a cost matrix derived from token embeddings. To ensure computational tractability, we adopt the Sinkhorn divergence and derive a gradient‑equivalent objective that can be efficiently optimized without introducing additional networks. Experiments across multiple LLM families and scales show that WASD consistently improves distillation performance on diverse tasks, including instruction following, mathematical reasoning, and code generation. Our results highlight the importance of semantic information encoded in the token space for effective distribution alignment in LLM distillation. The implementation is publicly available at https://github.com/aailab‑kaist/WASD .
Authors:Tianlun Zheng
Abstract:
Strategy research conflates two problems: finding a profitable rule, and establishing that the finding is not search luck. The latter calls for admission gates ‑‑ statistical criteria that must be satisfied before a conclusion is adopted ‑‑ yet whether gates work, and at what cost, remains untested. We introduce an injected‑truth protocol with a random‑admission control that adopts at the same rate as the gate; only if the gate beats this control does it carry information rather than merely raise a threshold. Across synthetic and real‑calibrated panels, gates eliminate false discoveries in the weak‑signal regime but cut adoption to 1‑‑7%, and add nothing when signals are strong. Most importantly, criteria computed on absolute rather than excess returns silently reject every candidate, including true signals. Keywords: multiple testing, backtest overfitting, strategy admission, injected‑truth validation, excess returns, false discovery rate
Authors:Tao Zhou, Ying Hu, Huazhu Fu, Yi Zhou, Xiao-Jun Wu, Haibin Ling
Abstract:
The integrative analysis of histopathological Whole‑Slide Images (WSIs) and transcriptomic profiles holds significant promise for cancer survival prediction. However, existing methods typically project multi‑modal features directly into a shared latent space without explicit alignment, leading to the entanglement of mismatched morphological cues and molecular signals. Furthermore, current fusion strategies often treat the extreme spatial heterogeneity of WSIs uniformly, lacking mechanisms to adaptively prioritize clinically relevant tissue scales for individual patients. To address these limitations, we propose an Anchor‑driven Multi‑modal Multi‑scale Expert Selection (AM^2ES) framework for survival prediction. Specifically, we present an Anchor‑driven Multi‑modal Fusion (AMF) module, which introduces learnable semantic anchors as cross‑modal mediators to bridge the semantic gap by enforcing a structurally regularized alignment between transcriptomic features and multi‑scale pathology representations. Built upon this aligned semantic space, we further design a Hierarchical Mixture‑of‑Experts (H‑MoE) selection module to decouple the hierarchical prognostic selection process. Mimicking the pathologist's diagnostic workflow, H‑MoE performs (i) Intra‑scale Expert Filtering to discriminatively identify salient tumor regions within each magnification, and (ii) Inter‑scale Hierarchy Routing to dynamically weight and select the most informative resolution levels. Extensive experiments on multiple TCGA cancer cohorts demonstrate that our AM^2ES achieves state‑of‑the‑art performance while offering fine‑grained interpretability by visualizing how specific molecular pathways drive the expert routing decisions across tissue scales. The code will be released at https://github.com/taozh2017/AM2ES.
Authors:Juntong Li, Lingwei Dang, Haomin Wu, Ziyan Qiu, Qingxin Xiao, Qingyao Wu
Abstract:
Vision‑Language Models (VLMs) such as CLIP excel in global semantic alignment but often lack fine‑grained perceptual capabilities. This hinders dense prediction tasks and bottlenecks the visual potential of Multimodal Large Language Models (MLLMs). Existing research has attempted to enhance CLIP's visual representations by incorporating geometric priors from vision‑centric models. However, these strategies often struggle to achieve deep alignment for both local spatial structures and global semantics, potentially even distorting the original image‑text space. To address these limitations, we propose SALM, an unsupervised embedding alignment framework based on structurally‑aware latent mask modeling. SALM effectively synergizes local and global alignment via a dual‑path design combining explicit and implicit mechanisms, without requiring any image‑text pairs. First, we introduce a dual‑matrix alignment strategy that explicitly calibrates intra‑sample spatial correlations and activation intensities, thereby effectively injecting local geometric priors. Based on this, we further design a latent mask modeling mechanism to guide CLIP to restore the missing semantic details of the target model, thereby implicitly aggregating fine‑grained structures into the global semantic space. Furthermore, driven by the empirical observations that CLIP's shallow features inherently possess strong spatial observational capabilities, we naturally extend SALM to a highly efficient self‑distillation paradigm, SALM‑Self. This unlocks CLIP's intrinsic fine‑grained potential without relying on any external models. Extensive experiments demonstrate that SALM not only significantly improves performance in dense prediction tasks but also boosts CLIP's zero‑shot accuracy, effectively enhancing the fine‑grained understanding capabilities of MLLMs. Project page at https://qzfm.github.io/salm_project_page/.
Authors:Haipeng Liu, Yang Wang, Meng Wang
Abstract:
Personalized image generation aims to synthesize text‑driven images conditioned on reference images, while mainly casting the generation as image customization for foreground and style transfer for background. Previous arts of diffusion models suffers from the text misalignment with background for image customization and foreground for style transfer during the denoising process. Such facts, as we observed, rooted from the entanglement among hybrid frequency bands during the denoising process. To address such salient limitation, in this paper, we study personalized generation based on dual references ‑ customization and color and style reference ‑ and propose a paradigm to disentangle these Dual image references within Frequency‑aware Diffusion Models, dubbed Dual‑FDM, to simultaneously tackle two crucial personalized image generation tasks: customization style transfer and color style transfer, by disentangling different frequency bands via mask strategy within frequency domain. For customization style transfer, we replace the mid‑frequency band of the background in the style reference with that from the foreground of the customized reference. For color style transfer, we substitute the low‑frequency band of the background in the style reference with that from both the foreground and background of the color reference. Both the substituted frequency bands are used as the key and value to reconstruct the query foreground and background of the denoised personalized image.Extensive experiments validate the superiority of Dual‑FDM over the state‑of‑the‑art diffusion models for personalized image generation. Our code can be accessed from https://github.com/htyjers/Dual‑FDM.
Authors:Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han
Abstract:
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting stage. A verification nevertheless follows each drafting stage, resulting in unnecessary target‑model forward passes even when drafting could have continued. Adaptive draft length methods decide during decoding how many draft tokens precede a verification, but they raise the speedup only for autoregressive draft models. For a parallel draft model, drafting further requires target‑model hidden states for draft tokens that have not been verified. We propose DLoop, a looped form of speculative decoding that adaptively performs multiple drafting stages before verification. DLoop continues drafting while the draft model remains confident and verifies all accumulated draft tokens together. Loop‑aware training keeps the draft model reliable in the additional drafting stages by exposing it to its own hidden states for unverified draft tokens. By spending additional draft‑model forward passes, DLoop reduces the number of target‑model forward passes required for verification. Across diverse speculative decoding methods including EAGLE‑3, DFlash, Domino, DSpark, and multi‑token prediction modules, DLoop improves the wall‑clock speedup by 5 to 41 percent while preserving lossless decoding. Code will be available at https://github.com/naver‑ai/DLoop.
Authors:Hyeongwoo Nam, Woongje Cho, Juwon Kim, Jongeun Choi
Abstract:
Large language model (LLM)‑based robot task planning is promising for open‑ended instruction following, but degrades on long‑horizon tasks in large environments. When spatial information is conveyed to the LLM through text, the model can fail to capture spatial context, and token cost grows with environment size. Generating action sequences directly with an LLM also makes it difficult to satisfy the current world state and action preconditions. We address this with an ontology‑grounded scene representation that aligns objects, spaces, relations, and states in a shared symbolic vocabulary for spatial reasoning and task planning, and with OntoPlan, an agentic framework that interprets instructions, selectively retrieves task‑relevant information, formalizes goals and constraints, and produces executable plans. Across 150 general tasks spanning five indoor environments and three scene scales, OntoPlan achieves 0.89 average task success, compared with 0.27 for the strongest baseline, while using 18.1k total tokens per task on average, about 5.6× fewer than the most efficient baseline. These advantages persist as scene scale increases, whereas prior methods degrade more sharply in success and remain far more costly in tokens. OntoPlan also responds appropriately to ambiguous or infeasible instructions by asking follow‑up questions or reporting insufficient information rather than committing to invalid plans. Code available at https://github.com/namhyeongwoo/OntoPlan.
Authors:Lizhi Zhang, Xin He, Dianxuan Fu, Yuyuan Feng, Jiatong Li, Qi Wang, Xin Wang, Qinggang Zhang
Abstract:
Self‑improving LLM agents increasingly distill successful experiences into persistent, reusable skills. Existing skill attack methods corrupt this learning pipeline by injecting malicious triggers, behaviors, or false facts into individual experiences or extracted skills. However, such attacks are easily detected, and the injected malicious behaviors often fail to accumulate as persistent skills. In this paper, we show that skill poisoning can arise even from verified successful experiences, without making any individual trajectory malicious. Based on this insight, we propose SkillPoison, a novel framework that progressively poisons skill via successful experiences. SkillPoison first constructs a set of successful experiences that reinforce a target behavior, and then removes the contextual conditions that constrain when the behavior applies. Rather than injecting malicious content, SkillPoison shapes how the skill extractor generalizes, allowing useful behavior to support task success while inducing harmful behavior when they are misapplied. Extensive experiments on three benchmarks show that SkillPoison achieves 95.71% attack success rates, while all injected experiences remain task‑correct and pass verification and lexical inspection. Our code, data and implementation details are available for the community at https://github.com/DEEP‑JLU/SkillPoison.
Authors:Zhengyang Zhu, Liming Huang, Runmin Ji, Mingxi Ye, Zihan Zhou, Hanyang Guo, Jingwen Wu, Yuhan Ye, Yuming Feng, Hong-Ning Dai, Zibin Zheng
Abstract:
Coding agent harnesses mediate tool use and authorize actions, yet their security mechanisms and runtime effects remain incompletely characterized. We present HarnessSecurity, the first systematic empirical study and benchmark of open‑ and closed‑source coding agent harnesses. First, we derive a ten‑mechanism taxonomy and then assess 400 harness‑mechanism cells using independent ratings by researchers and large language model (LLM) judges. We find that about half of confirmed mechanism implementations are opt‑in, while closed‑source harnesses exhibit substantial evidence gaps. Second, we introduce HarnessSecurity‑Bench, a benchmark of 23 tasks across five attack surfaces without sacrificing legitimate task requirements. Using separate deterministic oracles to measure task utility and attack effects with security setting comparisons, we evaluate nine mechanisms across six leading harnesses: Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code, and GitHub Copilot. Under a controlled LLM baseline GLM‑5.2, we conduct 2,500 trials, recording 81,155 tool calls and over 2.2 billion tokens. Enabling auto‑approve increases utility and raises attack success from 29.2% to 95.6%. Network isolation and read‑only mode reduce attack effects with substantial utility losses, while command allowlisting and command denylisting reduce attack effects with a small utility loss and a utility gain, respectively. Task‑level cases show that restrictions on a shared capability can obstruct both legitimate and malicious operations, and that allowed tools or commands can leave unauthorized operations reachable through alternative execution paths. Harness providers should make security settings verifiable, test alternative execution paths to protected operations, and assess attack effects alongside task utility and execution costs.
Authors:Zexi Zhang, Zecheng Zhu, Zidong Chen, Zulkhuu Tuya, Stephen James
Abstract:
Humanoid household manipulation requires the arms to act while the body balances, steps and changes posture. We present BiGym 2.0, an adaptation of BiGym for the Unitree G1 across 20 household tasks using a unified whole‑body controller for demonstration and evaluation. The suite provides 60 native human virtual‑reality demonstrations per task with synchronised multi‑camera views and full‑body execution records. We benchmark vision‑language‑action fine‑tuning, imitation learning, demo‑driven reinforcement learning, and cold‑start coding agents given the interaction budget of online reinforcement learning. With the same onboard views, proprioception and whole‑body controller for every method, vision‑language‑action fine‑tuning has the highest nine‑task mean, and agent‑developed programs outperform every demo‑driven reinforcement learning baseline on this mean and lead on bimanual reaching. Cross‑workspace stacking remains open, π_0.5 stays low on pick‑box, and multi‑object transport is hard for imitation learning, demo‑driven reinforcement learning and coding agents. All environments, human demonstrations, and evaluation traces are open‑sourced at https://github.com/swirl‑uk/BiGym2.
Authors:Yifan Zhang, Jichen Feng, Shihan Qin
Abstract:
State tracking requires an update at every input, but the depth a Transformer applies to each token is fixed regardless of sequence length. We introduce the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder. At each token, the decoder merges the encoder output with the previous token's final decoder state, so the computation path grows with sequence length at a fixed per‑token cost. On six algorithmic tasks, we compare five splits of eight layers with an eight‑layer Transformer over three seeds. Trained on at most 40 bits, two RLT splits generalize parity to 256 bits with 100% accuracy in every seed, while the Transformer stays at chance. On swap‑based S_5 permutation tracking at eight times the training length, RLT reaches 97% final‑state accuracy versus under 1% for the Transformer, and accuracy increases with decoder depth. On modular arithmetic beyond the training lengths, RLT reaches up to 93% versus 33% for the Transformer. Ablations show that these gains depend on the feedback: removing it drops parity and swap‑based S_5 to chance at every split. Updating the feedback once per four‑token chunk lets known tokens in a chunk run in parallel and keeps 64‑bit parity at 99%, while permutation tracking depends on per‑token feedback: chunking lowers length‑64 swap‑based S_5 from 100% to 20%.
Authors:Sheir A. Zaheer, Jihwan Moon, Chan Y. Park
Abstract:
We propose a scalable roto‑reflection‑group‑equivariant vision transformer based on windowed group‑convolutional self‑attention and a hierarchical feature architecture. We demonstrate that our approach can be scaled to group‑equivariant vision transformers (ViTs) with millions of parameters and large datasets with practically sized images, i.e., ImageNet. The code and pretrained weights for the proposed Hierarchical Windowed Roto‑reflection Equivariant ViTs (REViT‑v2) are available at https://github.com/kc‑ml2/revit.
Authors:Kevin Lee
Abstract:
Cassini synthetic aperture radar (SAR) images reveal the dunes, plains, and lake basins of Titan, providing an instance of representations learned from Earth imagery for planetary terrain classification. Cross‑domain Evaluation of Earth‑to‑Titan Transfer Using SAR (CETUS) compares features from DINOv2, DOFA and CROMA with classical image measurements and features from an untrained vision transformer on the U.S. Geological Survey's Cassini SAR mosaic. The classifiers learn terrain labels from an expert geomorphological map and predict those labels in geographically separate Titan regions. Under logistic regression settings, pretrained encoders achieve higher mean macro F1 than the combined classical features. Encoder rankings change when feature scaling, optimization, and regularization change together. Further training on Titan improves DINOv2 performance, degrades DOFA performance, and leads to mixed results for CROMA under the tested settings. Architectural and input processing differences prevent these comparisons from isolating the effect of pretraining. Classifier fitting and performance on individual terrain classes matter when assessing representation transfer for planetary mapping. Since the map draws partly on the same radar observations, the scores measure agreement with expert interpretation.
Authors:Binh Long Nguyen, Kien Nguyen, Clinton Fookes, Peyman Moghadam
Abstract:
Dense 3D mapping with semantic understanding is essential for robotic perception in complex environments. Recent 3D Gaussian Splatting‑based mapping approaches enable high‑fidelity geometry and efficient open‑vocabulary perception, but typically represent semantics as unstructured feature fields that limit object‑centric reasoning. In contrast, 3D scene graphs explicitly model objects and their relationships for structured reasoning, but are commonly constructed from sparse geometric representations that do not fully exploit dense semantic maps. In this work, we present OpenSplatGraph, a unified framework that constructs persistent 3D scene graphs directly from an online Gaussian‑based open‑vocabulary semantic map. The proposed framework augments the dense semantic map with a reliability‑aware semantic field that maintains lightweight observation statistics for confidence‑aware, query‑conditioned object extraction. Extracted object instances are associated with persistent graph nodes, allowing object attributes and relationships to be incrementally updated across observations and queries. By tightly coupling dense semantic mapping with persistent object‑centric representations, our framework supports both language‑guided object grounding and structured relational reasoning while preserving the geometric fidelity of Gaussian‑based mapping. Comprehensive evaluations on standard 3D scene understanding benchmarks and real‑world robotic experiments demonstrate that OpenSplatGraph achieves competitive performance for online open‑vocabulary perception and downstream robotic tasks. Project page: https://csiro‑robotics.github.io/OpenSplatGraph.
Authors:Jin Huang, Diego Ferreras Garrucho, Yutong Xie, Walter M. Yuan, Qiaozhu Mei, Chen Lian, Jonathon Hazell
Abstract:
Large language models (LLMs) have the potential to meet a key goal in economics: a quantitative model of household decision making, across a variety of settings. Yet existing evaluations cover few surveys and outcomes, and do not study how households adjust to changing economic conditions. We introduce a new evaluation, HouseholdBench, which unites 6 U.S. household surveys and 32 prediction tasks spanning numeric, categorical and probabilistic outcomes, related to consumption, income, labor, expectations, and housing. Using past behavior, demographics and macroeconomic conditions, the tasks test whether LLMs predict behavior, including how households adjust to changes in various policies. We evaluate 13 proprietary and open‑weight LLMs against a no‑change baseline and a gradient‑boosted tree model. Most LLMs outperform the no‑change baseline, including for policy response tasks ‑‑ with the best model lowering error for numeric outcomes by 12.2%. Across most tasks, gradient‑boosted trees rank first; leading proprietary LLMs approach their performance, but open‑weight models lag. LLMs exhibit systematic over‑ and underprediction across different tasks. We identify methods that enable a 4 billion parameter open‑weight model to match proprietary models' performance: fine‑tuning and aggregating 16 predictions per observation. Improvements generalize to policy‑response tasks, which are excluded from fine‑tuning. We release our datasets, code, and leaderboard on our website: https://jn‑huang.github.io/householdbench
Authors:Zubair Ajmal Farooq, Diptesh Kanojia
Abstract:
We present a fully offline speech‑to‑speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper‑tiny ASR feeds an Opus‑MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary‑pass correction when confidence falls below a pre‑defined threshold τ. We compare three correction methods: QE reranking (M1), Minimum Bayes‑Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES‑200 sentences (English‑Spanish), M2 at τ=0.90 produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1\toM2) does not hurt quality and frees 680 MB from the critical path. Using a gain‑to‑edit ratio adapted from the post‑editing‑effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six language pairs.
Authors:Wonjun Lee, Kyungsik Yang, Gaeun Ji, Vaidehi Patil, Haon Park, Bumsub Ham, Mohit Bansal, Suhyun Kim
Abstract:
LLMs have advanced rapidly, raising growing concerns about their safety. Recent work has proposed approaches to detect and defend against attacks including defenses at decoding stage that leverage models' hidden states. However, existing decoding‑stage defenses suffer from two limitations. First, they introduce a trade‑off between safety and over‑refusal, where strengthening safety degrades the model's helpfulness on benign queries. Second, many of these methods rely on internal hidden states and are thus restricted to specific architectures, incurring substantial overhead and limited generalization across models. To address these limitations, we introduce LADE (Latent Safety Signals for Defense), which leverages latent safety signals extracted by contrasting harmful and benign queries from dark knowledge (i.e., information carried by the output probability distribution beyond its argmax) in the first‑token output probability distribution. Our key insight is that, beyond surface‑level refusal tokens, the dark knowledge in the first‑token distribution contains latent safety signals, defined as tokens whose probabilities differ sharply between harmful and benign queries. We show that these signals consistently align across LLMs, forming a model‑agnostic direction that emerges from safety alignment. LADE consists of three components: (1) Extracting Latent Safety Signals from Dark Knowledge, which selects top‑k safety‑discriminative tokens from the first‑token probability distribution; (2) Tokenizer Mapping, which maps these tokens across different tokenizers to enable model‑agnostic application; and (3) kNN‑based Discrimination, which classifies queries via a k‑Nearest Neighbors search over the mapped tokens. Across diverse LLMs and benchmarks, LADE is robust against a wide range of jailbreak attacks and lowers attack success rates while maintaining a competitive safety‑utility trade‑off.
Authors:Qiusi Zhan, Nian Lyu, Stephanie Ding, Arnav Mehta, Chris Canal, Connor Tabarrok, Xander Davies, Daniel Kang
Abstract:
Developers can build LLM agents by adapting third‑party models through benign post‑training. We study a supply‑chain threat in which an attacker supplies a model with a backdoor: hidden behavior that produces malicious outputs when a particular input pattern appears. Focusing on software‑engineering agents, we ask whether such backdoors survive the developer's supervised fine‑tuning (SFT) and subsequent task‑level reinforcement learning (RL). We observe that benign SFT substantially reduces attack success, but subsequent RL often preserves the residual behavior and sometimes even increases attack success. Our analysis of backdoor erosion during SFT identifies two factors that may favor survival: initial backdoor strength and gradient compatibility with benign training. These factors motivate PersistBD, which refines an already‑backdoored model before release to improve its persistency through the benign post‑training process. On Qwen2.5‑Coder‑7B, PersistBD raises attack success from 20% to 74% after SFT and from 20% to 76% after SFT‑RL, while maintaining comparable benign task performance. Together, our results show that backdoors can remain active through benign post‑training and that adversaries can deliberately increase their persistence. This highlights a supply‑chain risk for AI developers and motivates stronger techniques for detecting and mitigating inherited backdoors when adapting third‑party models into agents. Our code is available at https://github.com/uiuc‑kang‑lab/PersistBD.
Authors:Victor De Lima, Grace Hui Yang
Abstract:
Information elicitation is an open‑ended information‑seeking problem in which an interaction can unfold in many potentially valuable directions, requiring an elicitor to continually determine which information to pursue as new information emerges. In agentic elicitation, these decisions may be delegated to a foundation model, yet how model choice shapes the resulting information‑seeking behavior remains understudied. We study how judgments about information value vary across LLMs and how these differences shape sequential information seeking. We first examine these judgments across 11 LLMs spanning multiple model families and parameter scales, using a shared set of information and elicitation objectives. We then develop a controlled elicitation simulation in which different models encounter the same information space and use the same selection rule, isolating these judgments from question generation and respondent behavior. Using this setting, we characterize the breadth‑depth behavior that emerges from model‑specific information‑seeking preferences over the course of elicitation. We further examine how interaction history changes the evaluation and subsequent selection of prospective information. We test the robustness and boundaries of these findings through sensitivity analyses and ablations over the opportunities available to the elicitor, the response labels used to operationalize information‑seeking preferences, the presence of interaction history, and whether redundancy is explicitly relevant to the assessment. The project code, data, and trajectory files are available at https://github.com/infosenselab/open‑elicitation.
Authors:Paweł Lorek, Rafał Nowak, Rafał Topolnicki, Tomasz Trzciński, Maciej Zięba
Abstract:
We study estimating rare‑event probabilities I = \mathbbP(g(\mathbfX) > γ) with \mathbfX ~ \mathcalN(\boldsymbolμ, \boldsymbolΣ) and general g : \mathbbR^d \to \mathbbR. We address this problem through importance sampling, and propose a framework that substantially improves efficiency and robustness over baselines such as crude Monte Carlo, adaptive cross‑entropy, variational‑inference‑based methods (including reverse‑ and forward‑KL approaches), as well as Safe‑ICE, Subset Simulation, and Sequential Monte Carlo, drawing on ideas from both rare‑event estimation and cross‑entropy optimization. The key contribution has two parts: first, we separate the problem into coverage, to overcome the cold‑start barrier, and fitting, to refine proposals once a meaningful signal is available; second, we constrain the final GMM proposal so that it has finite importance‑sampling variance (since coverage alone is not sufficient ‑‑ without safeguards, importance sampling may still suffer from infinite variance). Together, these ingredients yield expressive proposals; finite variance does not by itself guarantee practical stability at a fixed sampling budget. Extensive experiments demonstrate substantial variance reduction, strong robustness across diverse benchmarks, and favorable cost‑‑efficiency trade‑offs, with the proposed approach often outperforming these baselines, particularly in high‑dimensional and multimodal settings where competing methods frequently become unstable or fail. Our code is available at https://github.com/lorek/robust‑cfi‑is.
Authors:Tzu-Hsin Hsieh, Ricardo Marroquim
Abstract:
Inserting objects into existing 3D scenes requires more than selecting a plausible location: the inserted object must also fit local geometry while preserving semantic intent and physical plausibility. Although recent Vision‑Language Models (VLMs) and generative models enable semantic reasoning and visual content creation, they offer limited 3D grounding and geometric control when an inserted object must fit into constrained local spaces. We introduce ElasticFit, a VLM‑guided framework for fit‑aware object insertion centered on a novel scene‑grounded representation. Given a language instruction and rendered scene observations, ElasticFit infers structured fitting cues that specify where the object should be grounded, what volume it should occupy, how it should be oriented, and its adaptation mode (rigid placement, uniform scaling, or elastic fitting). These cues convert high‑level VLM reasoning into explicit 3D constraints that condition object generation and guide downstream geometric fitting. ElasticFit then generates a scene‑conditioned object prior, reconstructs it in 3D, and refines the mesh through mode‑specific fitting while enforcing collision avoidance, contact consistency, and physical grounding. In fixed‑asset baseline comparisons, ElasticFit improves spatial relation success from 50.8% to 69.7% and support success from 48.3% to 91.7% over the strongest baseline, while providing novel support for generative "make‑it‑fit" insertions in complex scenarios.
Authors:Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng
Abstract:
Fine‑grained mixed‑precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision‑boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post‑training quantization method that uses GPU‑compatible two‑dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. This shared partition lets precision follow sensitivity within output channels. Joint prefill/decode calibration scores precision reductions using projection‑output perturbations weighted by language‑model loss gradients under quantized activations. Phase‑normalized scores prioritize higher precision for tiles important to either phase under a model‑wide weight‑storage budget. Each tile stores one selected representation, while phase‑specialized kernels reuse the packed model and expand lower‑bit weights for INT8 computation with 8‑bit activations. Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to 2.50× generation speedup over BF16 while preserving model quality. Evaluations further cover three GPUs and contexts up to 64K tokens. These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. The implementation is available at https://github.com/HanzhiZhang‑Ulrica/AlignQuant.
Authors:Aadi Chauhan, Arthur Ilyasov
Abstract:
A GUI agent decides which action to take and where to take it; we ask how a small grounding model should receive the action type. Fine‑tuning Qwen2‑VL‑2B with LoRA on Android in the Wild, we compare a flat baseline with five ways of supplying the type under matched data, compute, and decoding: an auxiliary loss, a hard‑routed action word, an additive learned embedding, a prepended learned token, and the type written into the prompt. With five seeds, an episode‑clustered bootstrap, and seed‑level paired tests, the ranking on a mixed stream is clear: the auxiliary loss, the additive embedding, and the prompt word each gain five to seven hit@0.10 points over the baseline, while hard routing and the prepended token are not distinguishable from it. Much of that gain is protection from a preprocessing choice of ours rather than a spatial prior. Our serializer clamps the off‑screen touch point AITW records for type events to the origin; that class degrades the baseline's click grounding, and removing it lifts the baseline by nearly seven points, after which no mechanism's hit rate beats it and the intervals exclude a two‑point effect, though the auxiliary loss still shortens the average miss; on a stream of taps and swipes none helps. Whether this generalizes beyond one serialization is open. For deployment, the pipeline's margin over the baseline with predicted rather than gold types is not established (+0.016, 95% interval [‑0.017, +0.052]), and a wrong type collapses every model conditioned at inference. The prepended token does not help at the shared learning rate, where its rows barely move from initialization; trained ten times faster it reaches the level of the other three, with a margin three seeds do not establish. We also document a silent failure: injecting conditioning through inputs_embeds makes Qwen2‑VL fall back to 1‑D positions for image tokens, costing nine points.
Authors:Haochen Chai, Qixu Zhu, Siyao Li, Fangfang Jiang
Abstract:
Artifact removal routinely precedes the classification of electrodermal activity (EDA), on the assumption that a cleaner signal supports a better decision. We tested this assumption in a virtual‑reality (VR) balance‑disturbance task. A residual gating network was trained on a benchmark with expert‑corrected EDA, frozen, and applied to VR recordings, where raw and gated signals were classified by five published time‑series methods under identical leave‑one‑participant‑out evaluation. On the benchmark the gate detected artifacts well (median record AUROC 0.94) and reduced error inside artifact regions by 17.8%. In the VR task it did not improve classification. Changes in balanced accuracy ranged from ‑1.35 to +0.93 percentage points, no classifier improved and two lost accuracy, and all five were equivalent to raw input within +/‑ 3.32 points. The benefit was lost between waveform and decision. The correction that lowered waveform error also reduced skin conductance response detection in all 43 benchmark records. Processing left 92.8% of predictions unchanged, and the predictions it did change were corrected and corrupted at similar rates. The VR recordings also carried little contamination (an estimated 4.6% of samples), and even perfect localization of deliberately injected artifacts recovered only 3.3 points in the most sensitive classifier. A pooled association between artifact level and accuracy (11.3 points) disappeared within participants (0.1 points), showing how differences between people can make cleaning look useful. Preprocessing should be judged by the decision it supports, against an unprocessed arm.
Authors:Yuqing Liu, Huiyuan Chen, Yibo Wang, Wooseong Yang, Philip S. Yu
Abstract:
Semantic ID‑based generative recommendation represents each item as a sequence of discrete tokens, enabling structured modeling of item semantics. A critical challenge is constructing semantic IDs that are both semantically expressive and computationally efficient. While recent approaches favor complex learned quantization, simple hashing‑based methods such as SimHash are widely regarded as fundamentally inferior. In this work, we challenge this consensus by showing that the apparent performance gap does not stem from inherent limitations of hashing, but rather from a structural mismatch with autoregressive decoding, coupled with the inevitable information loss during rigid discretization. Based on this insight, we propose FLASH, a two‑stage framework that revitalizes training‑free SimHash tokenization through parallel decoding and explicit semantic alignment. Despite its simplicity, FLASH achieves state‑of‑the‑art performance across multiple datasets without requiring any tokenizer training, while exhibiting stronger generalization in cold‑start scenarios. Notably, we demonstrate that semantic alignment acts as a universally effective mechanism across diverse paradigms. Our findings suggest that, with compatible decoding and semantic grounding, simple and efficient tokenizers can achieve performance comparable to complex learned counterparts in generative recommendation. Our code is available at https://github.com/KevinC2015/Flash.
Authors:Hadi Vafaii, Tejas Rao, David Chanin, Thomas Fel, Jacob L. Yates, Bruno Olshausen, David Klindt, Dileep George, Miguel Lázaro-Gredilla
Abstract:
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. These challenges involve both inferring which features explain an input and learning the dictionary that represents them. Here, we unify inference and dictionary learning as natural‑gradient flows on a shared variational free energy. We instantiate this framework as BeFOND, an encoder‑free sparse coding model with closed‑form inference and learning dynamics. We show how recurrent explaining away reduces interference between overlapping features, while Fisher preconditioning can compensate for the slow learning of rare features. On synthetic data, BeFOND improves dictionary recovery and rare‑feature detection, with a growing advantage over amortized baselines as superposition increases. On language‑model activations, it improves single‑feature concept detection and selective intervention, outperforming pretrained reference SAEs with substantially less training data. Its feature quality continues to improve with dictionary width, whereas the evaluated baselines largely plateau. Together, these results show how improving inference and learning within a unified probabilistic framework can make better use of data and dictionary capacity to interpret and intervene on neural representations.
Authors:Xinting Liao, Siyan Liu, Rabab K. Ward, Holger R. Roth, Xiaoxiao Li
Abstract:
Large language model (LLM) agents increasingly operate in interactive environments, where they need to make sequential decisions through observation, action, and feedback. Although memory can help agents reuse experience, existing work designs memory in isolation, where collecting enough trajectories to populate it is expensive. Existing shared‑memory approaches mitigate isolated experience by pooling episodic memories across tasks and environments. However, retrieving shared memory is challenged by the granularity, where retrieved memories can be either too specific to preserve current grounding or too coarse to support the next action. In this work, we propose MemCo, a memory‑centric collaboration framework for generalizing LLM agents to unseen interactive environments. It maintains complementary local and global memory spaces, preserving environment‑specific details locally while promoting transferable workflows induced from local trajectories to global memory. During online interaction, MemCo routes relevant local and global memories in terms of the agent's current state and decision phase, enabling agents to reuse the experience of other agents without blindly transferring environment‑specific details. Experiments on interactive decision‑making benchmarks show that MemCo improves task success and reduces redundant exploration compared with isolate‑memory and shared‑memory baselines. Our code is available at https://github.com/SYannL/nvdamas.
Authors:Alexander Coker, Leilei Cui
Abstract:
Model predictive path integral (MPPI) control plans actions by sampling and evaluating trajectories, but standard MPPI does not guarantee safety. Incorporating control barrier function (CBF) filters into each rollout allows the planner to account for safety corrections. However, combining multiple CBFs through a soft‑minimum approximation can exclude states that satisfy the original safety constraints, restricting the trajectories available to the planner. This paper proposes Exact‑Safe MPPI, which uses the exact minimum of multiple barrier functions and applies a CBF‑based quadratic program during both planning and control execution. To handle nondifferentiability where multiple barriers attain the minimum, the filter simultaneously enforces the CBF conditions for all δ‑active barriers, whose values lie within a prescribed tolerance δ>0 of the minimum. Under suitable feasibility and regularity assumptions, we establish forward invariance of the certified safe set without the shrinkage introduced by soft‑minimum smoothing. We also characterize the state‑dependent approximation gap and explain how multiple barriers with values at or near the minimum increase this shrinkage. Furthermore, we show that increasing the number of MPPI samples cannot recover trajectories excluded by the soft‑minimum filter. We demonstrate the effectiveness of the proposed approach in a navigation task requiring passage through a narrow gap between two obstacles. Video demonstrations are available at https://github.com/lc‑lab25/Exact‑Safe‑MPPI .
Authors:Chenglin Yang
Abstract:
Runtime gates for agent tool calls are stacked on the assumption that their errors multiply. We test it on 1,119 labelled agent actions from three corpora, without an adaptive adversary. The stack has one deterministic rule layer and four LLM judges, three of them re‑collected with the served model recorded on every call. We read each stack as a number of multiplication‑equivalent layers, n_mult, with its floor under perfect coupling. Under the STRICT miss definition (escalation to a human scored as not stopped), any two judges compose to about 1.2 to 1.4 layers (ϕ median +0.430, 6 of 6 pairs significant, floors 1.02 to 1.17). The rule layer plus one judge composes to 1.86 to 2.09 layers (ϕ median +0.014, 0 of 4 significant, floors 1.01 to 1.09). Under PRIMARY (escalation scored as caught) the bands are 1.21 to 1.57 and 1.80 to 2.13. Intervals separate on the pooled data, point estimates split on each corpus, and a third‑vendor judge lands in the judge band. Solo accuracy does not predict what a layer adds: a cloud rule pack lowers the rule layer's solo miss rate by 20% and adds no new joint coverage. The difficulty share of judge coupling is not identifiable: 31.8% to 61.8% depending on the probe and the miss definition. One judge tier was served by an unrequested model version in 50 of 112 batches, concentrated on the external corpus. That event overturned a pre‑declared analysis rule, and the scoring of review verdicts reversed five conclusions. We report both.
Authors:Zhisheng Qi, Li Zhu, Utkarsh Sahu, Douglas Tommey, Josh Roering, Yu Wang
Abstract:
Post‑wildfire debris flows (PFDFs) are destructive sediment‑laden hazards triggered when intense rainfall strikes recently burned terrain, destabilizing hillslopes and threatening infrastructure, local economies, and community safety. Data‑driven methods have been proposed to learn predictive patterns directly from historical PFDF observations. However, the current research landscape of data‑driven PFDF prediction remains highly fragmented across feature spaces, model architectures, and evaluation protocols, making rigorous comparison and the derivation of scientific insights difficult. Moreover, existing studies lack a systematic investigation into the relative importance of heterogeneous factors (e.g., meteorological conditions, terrain characteristics, soil properties, and burn severity) in triggering PFDF. To address these limitations, we present a unified benchmark for data‑driven PFDF prediction, enabling fair and comprehensive evaluation across diverse models and feature configurations. Furthermore, to better understand the underlying drivers of PFDF formation, we propose a reinforcement learning‑based feature selection framework that identifies factors whose perturbations render positive and negative events indistinguishable, thereby discovering the regional underlying mechanisms of PFDF occurrence across regions. Our code and benchmark are publicly available at https://github.com/KINDLab‑Fly/PFDF‑Benchmark.
Authors:Loka Li, Jin Tian, Kun Zhang
Abstract:
Protein‑molecule virtual screening is increasingly cast as a problem of representation learning in a shared embedding space. Existing methods rely on dense holistic alignment, entangling invariant binding determinants with nuisance correlations and limiting transfer to new targets. It has been noted that binding in protein‑molecule systems involves sparse cross‑modality interactions: binding is governed by a small contact interface and a few decisive local interactions (e.g., hydrogen bonds, hydrophobic contacts, and salt bridges) rather than the global structures of the protein and molecule. We hypothesize that uncovering and leveraging sparse interaction patterns is critical for generalization beyond the training data, as these patterns are reusable and expected to improve performance across different scenarios. In this paper, we aim to identify and leverage sparse interaction patterns, and verify our hypothesis. Since the training data contain only observed binding pairs, we formalize this prior via a V‑structure causal model under Heckman‑style selection, and establish three theoretical results: (i) the latent concepts of interacting proteins and molecules are not identifiable without appropriate sparsity constraints; (ii) these concepts and their sparse interactions are component‑wise identifiable under structural sparsity conditions; and (iii) a low‑rank relaxation of these conditions yields subspace identifiability of the concepts and interactions. Inspired by these principles, we propose CausalBind with three implementation variants. Extensive experiments on DUD‑E and LIT‑PCBA benchmarks show that all variants consistently outperform strong retrieval baselines, with the largest gains on LIT‑PCBA early enrichment, and further generalize to target‑ and scaffold‑level out‑of‑distribution splits. Code is available at https://github.com/lokali/CausalBind.
Authors:Junseob Kim, Jade Chng, Ayman Ali, Victor Moas, Yichun Lee, Po-Chun Chin, Sunil Hwang, Rishikesan Kamaleswaran
Abstract:
Tactical Combat Casualty Care (TC3) requires responders to connect visual observations of injuries and interventions with established clinical guidance. Developing vision‑language models to support this process requires supervision that links visible evidence to traceable doctrine. We present TC3‑VQA, a dataset constructed from public instructional and field TC3 videos and authoritative TC3 documents. It contains 581 items spanning 11 concepts, with 1,860 questions covering intervention recognition, doctrine, clinical reasoning, procedural guidance, and refusal when visual information is insufficient. Doctrine‑based answers preserve verbatim source passages and character offsets. Construction combines visual annotation, passage retrieval, entailment checks, and verification across model families. Equipment boxes, anatomical labels, temporal segments, and source metadata accompany the question‑answer pairs. Automated audits and ratings by two physicians and two medical students characterize annotation quality, with human ratings available for 88 retained items. The dataset provides a resource for adapting vision‑language models to TC3, studying the connection between visual evidence and clinical knowledge, and evaluating recognition, doctrine recall, and abstention.
Authors:Kwanghee Choi, Suwon Shon, Dmitriy Serdyuk, Guitang Lan, Chao-Wei Huang, Mohammad Sadegh Rasooli, Sangeeta Srivastava, Zhaojiang Lin, Saurabh Adya, Ming Sun
Abstract:
Audio benchmarks are built around short, pre‑segmented clips, limiting model design to brief inputs or fixed vocabularies. To close this gap, we introduce Logbook, a benchmark for hour‑scale audio understanding, with recordings ranging from ten minutes to six days. Given a continuous audio recording and an event label vocabulary, a system must predict a gap‑free segmentation with an event label and a description per segment. We compare 52 systems, end‑to‑end and cascaded, and ablate fine‑tuning, context length, and reasoning budget. We find the task tractable, though the best systems remain below the human reference. Also, over‑segmentation is pervasive, and fine‑tuning partially mitigates it. Finally, end‑to‑end are often better than cascaded systems, but degrades with longer context.
Authors:Zi Wang, Xingqiao Wang, Emmanuel Addai, Devika Ambekar, Xiaowei Xu
Abstract:
Long‑term‑memory agents can retrieve relevant information that is inadmissible for the current request because it belongs to another principal, violates policy, or reflects an incompatible lifecycle state. Recall and final‑answer accuracy do not reveal this: a route can appear safe by missing required evidence, while a correct answer may follow inadmissible prompt exposure. We introduce a retrieval‑admissibility verification framework that assigns each memory‑query pair one of three statuses (admissible, inadmissible, or unresolved), compares routes at matched required‑evidence recall with bounds for unresolved cases, and tracks memory IDs through prompt exposure while linking exposure to target‑level disclosure. We evaluate its stages on separate, non‑pooled populations. A post‑hoc top‑20 reanalysis of frozen rankings from two public long‑term‑memory benchmarks, RHELM and MemOps, covers 3,767 queries. All released anchors lie within trusted query namespaces; with within‑namespace scores unchanged, off‑namespace filtering cannot lower their ranks. Top‑20 anchor recall increases from 0.432 to 0.533, 80% recall feasibility from 0.237 to 0.311, and exact similarity evaluations decrease by 98.3%. In a frozen 72‑case development diagnostic, a released‑metadata reference preserves required evidence, whereas neither text‑only verifier detects violations under the 1% required‑anchor false‑denial limit. Across 1,523 paired benchmark‑native cases, namespace routing is associated with judged‑accuracy gains of 0.053‑0.068 across three readers; recall also changes, so this comparison is observational. In 16 controlled exposure scenarios, only one of four reader‑specific 95% confidence intervals excludes zero for relevant‑inadmissible literal disclosure (+0.156, 95% CI [0.031, 0.312]). Results motivate separate verification of candidate support, admissibility, prompt exposure, and answer disclosure.
Authors:Ayesh Abu Lehyeh, Jay Hwasung Jung, Safwan Wshah
Abstract:
Cross‑view geo‑localization is commonly solved as an image retrieval problem, matching a ground‑level image against a database of satellite tiles through a jointly trained embedding. Such models are accurate, but they need large paired supervision and cannot show what evidence supports a match. In this paper, we study a different question: how much of this task can be solved through language alone? We prompt a multimodal large language model (MLLM) to describe each ground panorama and each satellite tile as structured text, and localize by comparing these descriptions. No component is trained. We evaluate on 9,826 VIGOR pairs from four U.S. cities, in three settings. First, the descriptions are faithful but not discriminative. They agree closely across the two views, yet ranking the full pool by description similarity almost never returns the correct tile (0.39% Recall@1). Second, we narrow the pool to ten neighboring tiles, as a coarse prior would do. The same descriptions now become useful: an MLLM judge that scores structural consistency doubles random ranking and matches a strong lexical baseline. It also states which fields of the two descriptions agree and which conflict, which an embedding distance cannot do, and which we see as a step toward interpretable localization. Third, we place the judge on a trained visual retriever. On the queries it ranks wrongly, reranking from images works, while reranking from our descriptions does not (23.5% against 10.7% Recall@1). Scene structure survives the conversion into language, while the fine appearance detail needed to separate nearby places does not. Code and prompts are publicly available at https://github.com/AyeshAbuLehyeh/GeoLingual.
Authors:Elyas Irankhah, Muhammad Arif
Abstract:
Retriever upgrades are typically evaluated using aggregate metrics, which can hide regressions on queries the previous retriever already served correctly. We study these regressions as negative flips: queries for which BM25 retrieves a judged relevant passage and the replacement does not. We evaluate BGE‑large, E5‑large‑v2, and SPLADE on three BEIR collections: Natural Questions, HotpotQA, and FiQA, across five retrieval depths. All replacements improve overall retrieval coverage. Negative flips occur in every setting and vary substantially by corpus, retriever, and depth. At k=1, 8.6‑37.5% of BM25 successes are lost across the evaluated settings. Negative‑flip rates are lower in the larger‑depth settings, where the BM25‑supported cohort is defined separately at each depth. These rates use the any‑relevant support label. On HotpotQA at k=10, requiring every positive qrel passage raises the negative‑flip rate to 12.3‑17.3%. Simple fixed‑budget combinations with BM25 reduce these regressions, and a HotpotQA reader experiment provides a limited downstream check in which some retrieval flips are accompanied by answer regressions. These results motivate evaluating retriever updates using query‑level compatibility alongside aggregate retrieval quality.
Authors:T. Y. Tsui, Zihao Ye, Pengxiang Cai, Yanchao Li, Yuqiang Li, Zhehong Ai
Abstract:
``What are the irreducible conditions that are sufficient to produce an outcome?'' is one of the most common questions that recur across computation and science. Its answers, the minimal sufficient witnesses, are what we mean by explanations, mechanisms and reasons. These problems usually ask for multiple minimal witnesses, yet standard RL methods may reveal only one solution or redundant ones. We formalize this problem as minimal‑witness identification and introduce Minimal‑Witness Reinforcement Learning (MWRL). MWRL takes the union of the sets certified by successful proposals sampled from the policy and credits each proposal for the coverage the group union would lose without that proposal. This credit assignment, derived directly from the problem definition, unifies the demands for minimality and recovery of alternatives from a single black‑box verifier bit. Under this principle, we derive a value iteration planner that recovers the entire family of witnesses and a policy gradient method that can scale to large language models. Across different experimental settings, MWRL recovers most minimal witnesses, while other methods return redundant supersets or a single witness. By making witness families learnable from verifier feedback, MWRL expands the scope of reinforcement learning beyond single‑solution optimization. Our code is available at https://github.com/TSUITUENYUE/MWRL.
Authors:Xin Teng, Muxiao Li, Hongyi Wen
Abstract:
Mixture‑of‑Experts (MoE) transformers scale capacity by activating only a few experts per token, but this sparsity creates a hidden reliability problem: when routing is imperfect, load‑balanced models may send tokens to experts that are insufficiently trained for the assigned inputs. We propose Distributionally Robust MoE Training (DRMoET), a drop‑in objective that treats layer‑wise experts as endogenous robustness groups and optimizes high‑loss routing outcomes rather than merely equalizing traffic. DRMoET updates a per‑layer expert distribution by an entropy‑regularized softmax rule on EMA‑smoothed, activation‑weighted expert losses, strengthening plausible non‑top routing paths while preserving standard MoE computation. Under the FLAME‑MoE recipe at 746M‑total and 10.3B‑total scales, DRMoET improves downstream averages over both standard FLAME‑MoE and auxiliary‑loss‑free balancing. At 10.3B total parameters and 67B training tokens, DRMoET improves the seven‑task average from 0.6625 to 0.6767, while the auxiliary‑loss‑free baseline achieves 0.6431. Mechanistic analyses show lower expert‑loss variance with nearly unchanged mean loss, 4.3% lower excess loss under forced mid‑k misrouting, and improved domain‑expert specialization. These results position routing robustness‑not only utilization balance‑as a practical objective for reliable sparse MoE scaling. Project page and code are available at: https://drmoet.github.io/.
Authors:Berke Arda, Ahmetcan Yavuz, Paul Gerry, Sebastian Lobentanzer, Nobin Sarwar, Joan Giner-Miguelez, Kongtao Chen, Luyao Zhang, Mrinmaya Sachan, Mubashara Akhtar
Abstract:
Croissant has emerged as a standard for machine‑readable dataset metadata, yet populating its fields remains labor‑intensive and requires careful reading of accompanying dataset documentation. We present the first benchmark enabling end‑to‑end evaluation of metadata extraction aligned with a community‑standard schema. The benchmark comprises 602 papers, including 102 with human‑validated gold annotations and 500 with LLM‑generated silver annotations, covering the full Croissant schema with both core and Responsible AI (RAI) fields. Using this benchmark, we evaluate a range of extraction systems spanning frontier models, open‑weight models, and agentic architectures, under a two‑tier evaluation framework that combines rule‑based scoring with an LLM judge selected via human audit. We find that single‑pass extraction consistently outperforms the four agentic architectures we evaluate: across backbones, these decomposed variants achieve lower accuracy than a single full‑context pass. The largest gap appears on long‑form RAI fields, which require synthesizing and interpreting information scattered across a paper rather than copying it from a single location, a setting where current systems remain far from reliable. We release the benchmark, evaluation code, judge audit, a live demo, and a leaderboard open to new systems.
Authors:Adji Marieme Sita Cissé, Malek Mouhoub
Abstract:
Aggregating the preferences of diverse user groups into a collective outcome raises fundamental challenges of equity, diversity, and inclusion (EDI): classical aggregation rules such as Borda and Condorcet have no mechanism to prevent results from systematically favoring majority groups, collapsing onto homogeneous items, or under‑representing minorities. We address this problem through EDI‑constrained graph summarization. User preferences are modeled as a weighted attributed bipartite graph, and a greedy coarsening algorithm iteratively merges user nodes while enforcing three structural EDI criteria: an equity gap constraint (ΔE), an intra‑list diversity constraint (ILD), and a group inclusion constraint. Rather than correcting fairness after aggregation, our method embeds EDI preservation directly into the graph structure. We evaluate across five datasets spanning four domains: MovieLens 100k and 1M, libimseti.cz, Rate My Professors, and OpenAlex (2018‑2023). Our method, AURORA, achieves the largest and most consistent diversity gains over classical voting rules, and on MovieLens 100k at k = 20 it simultaneously improves all three EDI criteria over both Borda and Condorcet. On Rate My Professors, it combines high diversity (ILD = 0.808) with the highest female item representation (60%), at a moderate equity cost, and it achieves the lowest equity gap (ΔE = 0.031) on OpenAlex, where Borda‑based methods recommend zero female authors. On libimseti.cz, the only dataset where the sensitive attribute is present on both sides of the bipartite graph, our method does not reduce the equity gap, a limitation we connect to prior findings that demographic parity is not always an appropriate target. These results demonstrate that embedding EDI constraints into aggregation structure yields more robust fairness‑diversity trade‑offs than post‑hoc approaches.
Authors:Abhinav Sudhakar Dubey, Scott Sirri, Vaggos Chatziafratis, C. Seshadhri
Abstract:
While open‑weight models have enjoyed steady progress in capabilities and wide adoption across multiple domains, their safety remains an important concern. One key feature is the ability to refuse or deflect harmful, malicious, or insensitive prompts. In this paper, we expose safety vulnerabilities across six common open‑weight LLMs of various sizes that consistently lead to harmful or unsafe responses on the JailbreakBench benchmark dataset. Our proposed attack, Perturbed Embedding Vector (PEV), is a simple and fast "jailbreaking" technique that is cheaper than prior approaches, which typically require gradient computations, per‑prompt optimizations, or altering internal weights of the models. PEV just adds independent Gaussian noise in the embedding vector representations of the prompt, with no need for further manipulations. To generate unsafe responses, we repeatedly sample additive noise from this distribution. In our experiments, we observe that the average compute cost to get the first successful attack is up to an order of magnitude less than previous attacks. The first successful jailbreak on a new prompt typically arrives within one minute on every tested model, and PEV generates unsafe responses across all models for all prompts in JailbreakBench. No other tested method achieves such results, despite them taking longer to run. More broadly, we believe that understanding the behavior of LLMs under perturbations in the embedding vectors is an important research direction: while perturbations constitute a major security risk, they can also serve as a valuable tool for exploring the dynamical behavior of such models.
Authors:Gabriele Magrini, Riccardo Catalini, Federico Becattini, Guido Borghi, Pietro Pala, Roberto Vezzani, Lorenzo Seidenari
Abstract:
Understanding and modeling interactions between autonomous agents is a fundamental challenge in robotics, with broad implications for collaborative systems, social robotics, and human‑robot coexistence. Although the study of robot interactions has emerged as a compelling research direction, progress has been severely hampered by the absence of large‑scale benchmarks. In this paper, we introduce Robot‑to‑Robot Interaction (R2RI), the first dataset specifically designed to address the Robot‑Robot Interaction (RRI) task. R2RI consists of different humanoid robots and realistic interactions modeled on real human social behaviors. Complementary viewpoints are available, i.e., an egocentric perspective from each robot's onboard sensors, and an exocentric perspective from external fixed cameras, thus enabling rich spatial and contextual understanding of the interaction dynamics. The dataset comprises more than 6.5M frames and \approx5000 videos at 120 fps, including Event and RGB domains. We investigate pros and cons of each domain, comparing state‑of‑the‑art approaches for a number of key sensing and interaction based tasks. We publicly release the dataset and its annotations for all tasks and modalities at https://github.com/MagriniGabriele/R2RI.
Authors:Ziyun Chen, Jerry Li, Kevin Tian, Yusong Zhu
Abstract:
The Fréchet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug‑in estimator suffers from finite‑sample bias [BSAG18, CF20]. We study the sample complexity n of estimating FID to error ε between d‑dimensional Gaussians with bounded mean distance and covariances, when one distribution is known. Our contributions are threefold. (1) We establish tight finite‑sample Θ(\fracd^2n) bias and Θ(\fracdn + \frac d^2 n^2) variance bounds for the empirical plug‑in estimator, establishing a \gtrsim d^2 sample complexity. (2) To debias the empirical plug‑in estimator, we generalize the \rm FID_\infty estimator of [CF20] to extrapolation methods of arbitrary order k. We further prove tight bias and variance bounds of Θ(\fracd^k + 2n^k + 1) and Θ(\frac d n + \fracd^2n^2) for any order‑k extrapolation under our framework. (3) We introduce Relative Taylor Debiasing (RTD), a new, computationally efficient FID estimation algorithm using debiasing techniques inspired by U‑statistics. We show that RTD achieves an O(\frac d ε^2) sample complexity, and prove that this is optimal. We provide a complementary empirical evaluation of our new estimators. Our experiments on synthetic Gaussians validate the predicted residual bias and support the tightness of our bounds. On ImageNet with Inception embeddings, RTD achieves the lowest mean estimation error at the standard 50K sample budget, while our second‑order variance‑aware extrapolation estimator (VALE_2) uses only 10K samples to achieve accuracy comparable to FID_\infty at 50K samples.
Authors:Yun Jiang, Bo Zheng, Yingying Zhang, Xueming Xiao, Tao Hu, Hutao Cui, Zhiguo Meng, Ke Gao, Yang Gao, Meibao Yao
Abstract:
High‑quality 3D reconstruction of lunar terrain from sparse rover images is indispensable for autonomous lunar exploration, but remains challenging because viewpoint overlap is insufficient, surface textures are weak, and data volume is limited. We propose MoonGS, the first feed‑forward 3D Gaussian Splatting framework tailored to lunar scenes. Given only two input images, MoonGS predicts pixel‑aligned Gaussian primitives in a single forward pass and renders photorealistic novel views without any per‑scene optimization. MoonGS (i) adopts an adaptable backbone design that seamlessly integrates advanced vision foundation models to extract robust depth features; (ii) integrates semantic priors in two manners: merging semantic cues with visual features to refine Gaussian parameter estimation, and adopting a semantic ranking loss that regularizes background depth; and (iii) employs an entropy‑guided heuristic resampling strategy to augment sparse observations by selecting the most informative distant viewpoints with negligible overhead. Experiments on the LuSNAR benchmark and our synthetic weak‑texture MoonBlender dataset show that MoonGS surpasses state‑of‑the‑art feed‑forward NeRF/3DGS baselines by +4.9 dB PSNR, +0.29 SSIM, and 40% lower LPIPS while maintaining sub‑second inference. Furthermore, we validate the broad applicability of our framework by demonstrating that it effectively leverages state‑of‑the‑art backbones, including VGGT, to significantly boost performance. Qualitative evaluations on Chang'e mission imagery also show the best visual quality among compared methods, indicating robustness on real lunar data. The source code and dataset are publicly available at https://github.com/InRobots/MoonBlender.
Authors:Jinfeng Xu, Zheyu Chen, Ziyue Peng, Zheng Lin, Wenhao Yuan, Jian Chen, Shujie Li, Edith Ngai
Abstract:
Recommendation recursive self‑improvement (Rec‑RSI) feeds recommender outputs into subsequent training. Evaluating each round solely through its latest model assumes that the successor consolidates the update, although pre‑ and post‑update models may retain complementary ranking decisions. We term this \emphdistributed progress and quantify it using cross‑generation advantage (CGA), a marginally matched contrast between cross‑ and within‑generation model pairs. A rank‑separation statistic, label‑free at selection time, predicts which family to retain. Across four datasets and three sequential recommendation encoders, the preferred retention regime varies by architecture: cross‑generation pairing benefits GRU4Rec and SASRec, whereas FMLP initially favors within‑generation pairing and shifts toward cross‑generation pairing after a second update. Rank separation selects the stronger family in 12/12 first‑update and 5/6 second‑update dataset‑encoder settings; on held‑out tests, the selected family outperforms the direct successor in 34/36 trajectories. Five transfer mechanisms do not consistently reproduce these gains in one model. These findings establish state retention as a distinct Rec‑RSI problem: progress may reside in relations between generations as well as in the latest model. Code is available at \hrefhttps://github.com/Jinfeng‑Xu/RecRSIhttps://github.com/Jinfeng‑Xu/RecRSI.
Authors:Thibaut Boissin, Thomas Massena, Mathieu Serrurier, Franck Mamalet
Abstract:
Muon, an optimizer known for its efficiency, has a clear interpretation for matrix‑valued updates, but convolutional kernels are stored as four‑dimensional tensors. Standard implementations reshape these tensors into matrices, a shortcut which breaks the theoretical understanding behind Muon. To investigate this, we formalize the corresponding optimization objective directly in convolutional operator geometry and introduce Convolutional Newton‑Schulz (Conv‑NS), which approximates the polar factor in this geometry while preserving kernel support. When applied in fast training experiments, Conv‑NS and reshape‑based Muon are both computationally efficient and achieve comparable accuracy on CIFAR‑10 and ImageNet classification tasks. However, as one could expect a theoretically aligned Conv‑NS to outperform reshape‑based Muon, we investigate this mismatch between practice and theoretical understanding, with the hypothesis that exact convolutional orthogonalization may overconstrain updates. These findings highlight Muon's strong practical performance while opening directions for its further development on convolutions. Our code is publicly available at \hrefhttps://github.com/thib‑s/muonconv‑cifar10‑airbenchgithub conv‑muon.
Authors:Aniruddh Pramod, James Oldfield, Adel Bibi
Abstract:
LLM agents increasingly act in multi‑actor environments, exposing them to misuse from multiple sources: decomposition attacks, where a harmful request is split into innocuous sub‑requests, and prompt injection attacks, where a compromised tool delivers a malicious instruction. Existing evaluations treat these threats separately and ask whether a trajectory is harmful, rather than when it becomes harmful. We propose monitoring the agent's responses, where its actions are externalised, and ask whether the first point where monitors identify harm lands within a harm window (from the agent's first harmful commitment to goal execution). We develop a unified formalism for trace‑level misuse monitoring and use it to construct a benchmark of ~6,200 conversation transcripts between a user, an LLM agent, and the external environment, spanning both threats in a shared schema, with a labelled harm window, corresponding benign controls, and matched instances of refusals to these requests. Across 17 monitor configurations, we find that our proposed action‑framed monitors perform well on both threats under classical metrics (AUC: 0.95 and 0.99 respectively), while content‑framed monitors collapse on injection attacks (AUC: 0.52). We also show that classical position‑blind metrics paint an optimistic picture of monitor performance, since all monitors localise decomposition attacks poorly under the interval metric, which measures the ability to localise harm. Broadly, we illustrate the need for a unified study of misuse monitoring.
Authors:Zhi-Kai Chen, Song-Yan Li, De-Chuan Zhan, Han-Jia Ye
Abstract:
LLM agents interact with external systems by generating structured tool calls. Given a user request, conversational context, and a catalog of tool schemas, a tool‑calling model must select tools and generate their arguments, potentially producing multiple calls in a single response. Standard autoregressive decoding generates these calls token by token, incurring substantial latency for requests involving multiple calls or many argument fields. The explicit argument structure offers opportunities for parallel generation, but later argument values may depend on preceding fields and calls, so independently generated values can differ from the target model's output. We present SchemaFill, a framework for efficient LLM tool calling through slot‑parallel speculative decoding. SchemaFill generates future slot values concurrently as candidates, without requiring advance knowledge of the actual call sequence or argument values. Candidates spanning multiple fields and calls are concatenated for verification by the target model under the actual output prefix. Only verified tokens are committed, and the target supplies corrections when candidates disagree. This applies target verification while exploiting parallelism across slots and calls. On Glaive and BFCL, SchemaFill achieves up to a 4.05× improvement in end‑to‑end throughput over autoregressive decoding. Code is available at https://github.com/Czzzk/SchemaFill.
Authors:Julian D. Santamaria, Kai Wang, Jesús Malo, Javier Vazquez-Corral, Alexandra Gómez-Villa
Abstract:
Variational autoencoders (VAEs) are a key part of modern text‑to‑image models, which generate images within their latent space. VAEs are known to disentangle the main factors of variation in the data, and color is known to be one of the most structured of these in natural images: decorrelating it yields one luminance axis and two opponent‑color axes. Color should therefore be expected to emerge as a distinct factor in the VAE latent space. Yet how these latent spaces represent color remains largely unexplored. In this work, we show that the VAEs of text‑to‑image models share a color subspace aligned with brightness and opponent‑colors. Through a linear approximation of the encoder and targeted latent steering, we find this subspace consistently across a broad range of VAEs, from SD1.5 to FLUX.2 and Z‑Image. Building on this characterization, we propose three applications: ColorTuning, which achieves state‑of‑the‑art in precise numerical color generation on the fine‑grained CSS3/X11 system of GenColorBench, saturation control, to adjust the global chromatic intensity, and color transfer, to change the palette to match a reference. The code and models are publicly available at https://julian075.github.io/Color_Subspace/
Authors:Hyebin Cho, Bumsoo Kim, Joon Son Chung
Abstract:
Impact sound rendering synthesizes the sound produced when a 3D object is struck, but practical renderers often rely on fixed material presets such as wood, plastic, or steel. These presets limit the range of impact sounds a renderer can express, while manually adjusting the underlying material parameters remains difficult without expertise in material acoustics. We therefore study inverse impact sound rendering: predicting material parameters from a reference impact sound so that a simulator can recreate a similar material response. To support this task, we introduce ImpactMat, a dataset and benchmark of single and blended material impact sounds paired with ground‑truth material parameters. We further propose a feed‑forward model that predicts these parameters from one or more recordings, using blended materials to learn smooth transitions between material types. Experiments show that our method outperforms competitive baselines and enables re‑rendering from real recordings without manual parameter tuning. The project page is available https://material‑from‑impact.github.io/material‑from‑impact/.
Authors:Sitian Shen, Jiuming Liu, Mengmeng Liu, Yian Wang, Michael Ying Yang, Francesco Nex, Hao Cheng, Daniele De Martini, Ayush Tewari, Per Ola Kristensson
Abstract:
Recent video world models have witnessed the paradigm shift from single‑agent to multi‑agent involvements, which can reveal more complicated dynamics and cross‑agent interaction in the real world. However, existing approaches commonly adopt implicit inter‑agent communications via cross attention, which lack explicit geometry constraints and unified 3D state, thereby leading to poor multi‑view consistency and struggling with recovering out‑of‑sight agents. In addition, most of them assume a static background, failing to represent uncontrolled background dynamics. To address these problems, we propose Artemis: a geometry‑grounded multi‑agent world model with explicit memory sharing. An explicit 3D world map is reconstructed from multi‑agent observations to enforce a unified 3D state across agents, offering high cross‑view consistency. Specifically, an action‑guided geometric injection module is developed to simultaneously render decomposed foreground‑background control maps, which are then injected into a diffusion transformer through a designed GeoAdapter block. Compared to previous methods assuming static‑only background, our GeoAdapter can also distinguish uncontrolled non‑agent dynamics, which are conditioned on their own multi‑frame history positions to provide consistent motion cues. Keyframes selected from progressive video rollouts are used to progressively update the reconstructed 3D world maps. To effectively capture complex dynamic patterns, we curate a novel dataset sampled from the CARLA simulator called MA‑CARLA. Extensive experiments demonstrate the superiority of our proposed method in terms of visual fidelity and cross‑view consistency in the generated videos. In addition, our Artemis can support simultaneous multi‑modal rollouts with both 2D video and 3D point map maintenance, scale to scenarios beyond two agents and multi‑camera setting.
Authors:R. Zhang, J. C. F. de Winter, T. Dicke, D. Dodou, Y. B. Eisma
Abstract:
As large language models (LLMs) are increasingly integrated into engineering workflows, students require hands‑on experience to learn how to collaborate with them critically. This paper presents a scalable gamified workshop designed for engineering Master's students to practice human‑AI collaboration in navigation planning. Using a mobile web interface across 10 workshop sessions, a total of 226 students wrote prompts for a non‑reasoning and a reasoning LLM to solve grid‑based navigation tasks of increasing complexity. The system returned robot‑executable plans, trajectory visualizations, and automated scoring, culminating in a live demonstration on a Boston Dynamics Spot robot. In a post‑workshop questionnaire, 81.5% reported substantial learning and 91.0% reported high engagement. Analysis of the submitted prompts revealed that students changed their strategies from step‑by‑step instructions for the non‑reasoning LLM toward providing higher‑level guidance for complex problem‑solving tasks. We conclude that such interactive simulation‑to‑reality environments are viable for teaching the verification and collaboration skills necessary for responsible LLM use in engineering. Code is available at: https://github.com/renchizhhhh/LLM‑robotics‑workshop
Authors:Johann Emmanuel Li
Abstract:
Fiorillo v0.5 is an open model that answers typed questions with a probability for each answer. Its main specialist reads a randomized trial's article, cut to 6,144 tokens, and answers whether an intervention significantly increased, significantly decreased or did not significantly change an outcome against a comparator (Evidence Inference 2.0, EI). It is Qwen3‑4B‑Base with low‑rank adapters and a decision head, fine‑tuned for EI only on the 1,431 of 2,657 training articles whose own license allows reuse. Four criteria registered on the Open Science Framework before this version's test predictions decided its release, the second bar judged on EI's test split, whose labels are public. On that split (1,218 prompts in 333 articles), the expected calibration error was 0.0168 against a limit of 0.05; log loss was below the prior's by 0.8603 (95 percent interval 0.8104 to 0.9078) and below that of Gemma 4 31B‑it, reading the same input, by 0.1829 (0.1164 to 0.2598); and macro‑F1 was 0.9248 against 0.8668, so all four criteria passed. Training the same recipe on clean articles alone cost 0.0123 in accuracy (0.0034 to 0.0207; descriptive). With no article, macro‑F1 fell to 0.4384; the title alone raised it by 0.0939 (0.0655 to 0.1234), which a title stating the result or recall of the trial could explain; exchanging intervention and comparator reversed 0.6652 of its direction answers. Run as released, the files matched the evaluated predictions within limits set in advance. The release is under the Apache License 2.0 (digital object identifier 10.57967/hf/10722).
Authors:Junhui Yin, Yuchen Yang, Yilin Yin, Shuai Na, Haoran Xi, Jianhua Yang, Muyi Sun, Man Zhang, Shengfeng He
Abstract:
Few‑shot class‑incremental learning (FSCIL) aims to learn novel classes from limited annotations while preserving prior knowledge. Existing methods typically assume a sufficiently large base session, but this assumption fails when both base and incremental data are scarce, leading to weak initial representations, semantic drift, and unstable boundaries. We study this underexplored yet realistic setting, termed Generalizable FSCIL (G‑FSCIL), where the base session itself contains only a few classes. Although synthetic data can alleviate supervision scarcity, naively mixing generated samples often introduces semantic noise and exacerbates old‑new boundary conflicts. To address this, we propose a framework that curates trustworthy synthetic knowledge for stable G‑FSCIL. Specifically, we first construct class‑specific synthetic candidate pools using a frozen latent diffusion model, where class inversion is performed at the first observation and the resulting condition embeddings are reused for on‑demand generation. Building on these candidates, we learn a knowledge curation strategy that selects samples with both semantic consistency and visual diversity, and distill this process into a transferable selection policy during the base session, which is then reused without further optimization. Leveraging the curated synthetic data, we further design a boundary‑stable incremental adaptation scheme, including synthetic‑informed prototype initialization and bidirectional boundary calibration to mitigate old‑new conflicts. Extensive experiments demonstrate that our method consistently outperforms existing FSCIL baselines, with reduced forgetting and improved balance between old and new classes. Code is available at https://github.com/NiHaoWoJiaoYYC/G‑FSCIL.
Authors:Xiaojun Jia, Simeng Qin, Yiming Li, Jie Liao, Sensen Gao, Ke Ma, Yang Liu, Xiaochun Cao
Abstract:
Multimodal large language models (MLLMs) remain vulnerable to transferable adversarial examples, especially in black‑box settings where only open‑source surrogate models are accessible. Existing targeted transfer attacks mainly align adversarial and target samples using global image‑level features, such as encoder [CLS] embeddings. However, such coarse alignment insufficiently exploits patch‑level visual structures, limiting transferability across heterogeneous closed‑source MLLMs. We propose IAU‑FOA, a visual‑invariance‑augmented feature optimal alignment attack with adaptive unbalanced transport, to improve targeted transferability against closed‑source MLLMs. IAU‑FOA aligns adversarial and target samples at both global and local levels: a cosine‑based objective narrows their global semantic gap, while patch tokens are clustered into compact local patterns and matched through optimal transport for fine‑grained feature alignment. Balanced optimal transport enforces fixed marginal masses even for local clusters without reliable counterparts, potentially introducing misleading alignment gradients. We therefore introduce confidence‑adaptive unbalanced transport to relax these constraints for weakly matched clusters, aiming to reduce unreliable local alignment and improve adversarial transferability. We further study the effect of input transformations and propose visual‑invariance augmentation, which applies bidirectional pixel‑intensity rescaling and per‑channel white‑balance adjustment to simulate exposure, contrast, illumination, and color‑temperature variations. This strategy encourages adversarial perturbations to generalize across different visual encoders. Extensive experiments on open‑source and closed‑source MLLMs show that IAU‑FOA consistently outperforms state‑of‑the‑art transferable attack methods. Code is available at https://github.com/jiaxiaojunQAQ/IAU‑FOA.
Authors:Jiangtao Liu, Zishang Xiang, Yage He, Lingguo Cui, Baihai Zhang, Runqi Chai, Senchun Chai
Abstract:
World action models jointly learn visual predictionand robot actions, providing a way to use observations ofscene evolution for policy learning. Their video and actionlosses, however, provide no explicit target for the geometricconsequences of a demonstrated action sequence. Moreover,visual features taken after temporal attention can contain futureobservations, making them unsuitable as the sole current visualinput to an auxiliary predictor. We introduce ACG‑WAMand its auxiliary objective, the Action‑Conditioned GeometricJoint‑Embedding Predictive Architecture (ACG‑JEPA), whichpredicts geometric features at several horizons from the currentobservation and intervening actions, using the future slot of afrozen VGGT encoding of each current and future image pairas the target. We apply this supervision from the head and wristcameras to a shared visual embedding before temporal mixing,and remove the teacher and auxiliary modules at inference.On 50 RoboTwin 2.0 tasks, ACG‑WAM achieves 93.46%success in clean scenes, with the best randomized success(92.68%) and mean across both settings (93.07%) among thecompared methods; across three tasks on a real robot, itachieves 85.00% success and 91.67% partial completion score,exceeding Motus by 10.00 and 9.17 percentage points, respec‑tively. Code:https://github.com/RoboOpus/ACG‑WAM.Website:https://RoboOpus.github.io/ACG‑WAM.
Authors:Kunming Luo, Hongyu Yan, Ken Deng, Chengcheng Zhou, Tianyu Liu, Haipeng Li, Haibin Huang, Xuelong Li, Ping Tan
Abstract:
We present DistScene, a framework for single‑image compositional 3D scene generation by jointly modeling the environment and individual objects. Unlike existing methods that represent scenes primarily as collections of objects, we model the environment as an explicit scene component to provide geometric context for object placement. Specifically, we introduce Scene‑Frame Generation, which jointly generates separate environment and object components in a shared coordinate frame, allowing their geometry and relative placement to be learned together. Then we introduce Object‑Centric Refinement to refine each object in a local frame with scene context. Finally, we develop Object‑to‑Scene Distillation to transfer pretrained object‑generation priors to scene generation through automatically composed and rendered synthetic scenes. Evaluations on indoor and outdoor benchmarks demonstrate improved scene‑level spatial coherence over the evaluated baselines. Project page: https://coolbeam.github.io/DistScene/
Authors:Jing Ding, Ziqiao Ma, Jiayuan Mao, Joyce Chai, Freda Shi
Abstract:
We study how Vision Transformers ground abstract concepts (e.g., angry) when training data provide limited direct referential evidence. We hypothesize a metonymic grounding mechanism in which abstract predictions are driven by concrete, interpretable anchor concepts (e.g., fire) that bridge visual signals to abstract semantics. By applying Transcoders on CLIP and DINO vision encoders, we recover intermediate features that can be associated with semantic labels for more concrete concepts, and trace their contributions in circuits underlying abstract concept recognition. Experiments on a carefully curated icon dataset reveal structured metonymic circuits, in which perceptual primitives dominate early layers and object‑like anchors precede abstract targets. Images containing rendered text instead recruit a distinct perceptual‑to‑textual route. Causal interventions further validate that metonymic intermediates are functionally involved in grounding abstract concepts.
Authors:Akash Kannan, Kiho Park, Victor Veitch
Abstract:
This paper concerns how semantic context determines geometry in learned vector representations. Similarity is typically measured using cosine similarity, which provides a single fixed geometry. Semantic similarity, however, is inherently context dependent: two images may be similar because they depict the same object, share a visual style, or are relevant to the same clinical finding. We show that contrastive representations naturally encompass a family of geometries that can be specialized to particular semantic structure. The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context‑specific semantic geometries on fixed representations. Under this correspondence, modeling the anchor distribution models the geometry itself. Experiments on retrieval show that anchor divergences provide an effective and efficient way to specify context‑specific semantic similarity.
Authors:Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich, Yizhou Sun, Dawn Song, Chenguang Wang
Abstract:
Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component‑level interventions across four open‑weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect. We find that refusal directions concentrate in sparse component mechanisms comprising 28‑‑48% of upstream components, retaining 88‑‑101% of steering effectiveness. Within these mechanisms, effective steering further concentrates in approximately 50% of residual stream dimensions, retaining 85‑‑98% of the component‑mechanism baseline, consistent with a privileged basis structure. Sparsity thus operates at two levels: which components are steered, and which dimensions within those components carry the signal. Together these findings show that refusal is not diffusely encoded across a transformer but assembled by a structured, identifiable mechanism, providing a foundation for mechanistic understanding of how refusal behaviors are represented and steered. To facilitate reproducibility, we release all code and raw experimental results in https://github.com/wang‑research‑lab/Refusal_Mechanisms.
Authors:Tian Zijian, Zhang He, Chen Xinjie, Wang Wenhai, Liu Xinggao
Abstract:
Moving target defense (MTD) against advanced persistent threats (APTs) in industrial control systems (ICS) has well‑established game‑theoretic formulations, but their practical value hinges on equilibrium computation, which faces two gaps: full‑rank value iteration is prohibitively expensive at industrial scale, and the resulting defense strategies admit no certified robustness against adversarial perturbations. We first reveal that the attack and defense influence matrices of ICS dynamics are intrinsically low‑rank: APTs infiltrate through a handful of entry points and MTD reconfigures only a few components per cycle. Our theory makes four contributions. First, an augmented gradient matrix certifies that the low‑rank structure propagates through the non‑smooth Bellman operator of the zero‑sum stochastic game, so that every Bellman target lies near a low‑dimensional subspace and low‑rank truncation incurs an explicit error bound (Lemma 1, Theorem 1). Second, we propose the Dynamical Low‑Rank Nash Equilibrium algorithm, named DLR‑NE, which augments the rank‑r search space each iteration, regularizes the core matrix spectrum, and retracts via truncated SVD, and prove that it converges geometrically to a neighborhood whose error decomposes into five physically interpretable sources (Theorem 2). Third, its per‑step cost is O(nr^2), a Theta(n/r^2) speedup over full‑rank value iteration (Theorem 3). Fourth, a single weight trades accuracy against a certified sensitivity bound of the induced defense strategy under core‑matrix perturbations (Corollary 1). Six experiments on a nonlinear power‑system testbed confirm each prediction, with 94% parameter compression at 2.3% utility loss. All experimental data and code are publicly available.
Authors:Prajwal Vijay
Abstract:
We study assignment quality in 3D heterogeneous multi‑agent reach‑avoid games and identify a recurring failure mode of cardinality‑only matching in geometrically structured scenarios, which we term \emphGeometric Sprawl. In these cases, multiple maximum‑cardinality assignments are available, but some induce spatially incoherent pairings and inefficient pursuit trajectories. Building on the evasion‑space framework of Yan et al.~\citeyan2022, we introduce a cardinality‑first weighted sequential matching method in which the Hamilton‑‑Jacobi‑‑Isaacs interception value z_I(s,j) is used as a secondary assignment weight. Each sequential stage is solved with a min‑cost max‑flow backend, while the unweighted baseline uses the same solver with the weight term removed. We evaluate both methods on a deterministic 35‑scenario benchmark (7 families & 5 initialization variants) under two regimes: a diagnostic stationary‑unmatched setting and a hybrid saddle‑point setting with goal‑directed unmatched evaders. On this benchmark, the weighted method resolves 13 of 15 geometric stress‑test instances that cause repeated timeouts for the unweighted baseline in the diagnostic regime, and under the hybrid regime improves mean captures from 3.20 to 3.91 (22% increase) and mean interception height from 3.87 to 5.36 (40% increase). We also report lower path tortuosity and lower angular‑effort proxy values, suggesting smoother pursuit trajectories in this first‑order simulation model. We release the simulator and benchmark suite at \hrefhttps://github.com/Prajwal‑Vijay/geometric‑coherence‑weighted‑matchinggithub.com/Prajwal‑Vijay/geometric‑coherence‑weighted‑matching to support reproducible evaluation of assignment strategies for 3D reach‑avoid games.
Authors:Jiarui Fang
Abstract:
For approval‑based committee elections, the sizewise least‑core value Γ(P) minimizes the maximum excess support of a deviation over size‑k committees. Deviations are nonempty and contain at most k candidates, with support measured relative to their Hare quota. A negative value certifies core nonemptiness, while ‑Γ(P) is the largest worst‑case quota shortfall secured by any committee. We quantify this margin on candidate‑disjoint approval chains, a domain whose qualitative core nonemptiness is already known. We first factor each committee's deviation score over disjoint components. For chains, a closed local table yields the exact value and an attaining committee in polynomial time for rational weights. For nonnegative normalized real weights, let q_\mathrmact\ge1 count the connected components of the positive‑support incidence graph. Then \[ Γ(P)\le‑\frac1k\min\q_\mathrmact,k+1\. \] Explicit extremal profiles show that this dependence on the active‑component count is sharp. The margin is 1‑Lipschitz in total variation on the full ballot space. Each negative margin therefore certifies a neighborhood with nonempty core, even when perturbations leave the chain domain.
Authors:Anita Graser, Melitta Dragschnig
Abstract:
This demo paper presents the conceptual foundations and the first steps towards implementation of a novel no‑code solution for movement data analytics based on the open‑source Python library MovingPandas and the open‑source geographic information system QGIS. The resulting Trajectools plugin is available open‑source at https://github.com/movingpandas/qgis‑processing‑trajectory.
Authors:Shaokun Zhang, Yifan Zhang, Jian Hu, Yueying Li, Hao Zhang, Binfeng Xu, Jan Kautz, Yi Dong
Abstract:
Reinforcement learning has greatly advanced the capabilities of large language models, but its memory demands remain a barrier to broader adoption. We introduce LoGRA, an approach to RL post‑training that reduces memory by retaining useful learning signals in low‑rank gradient sketches. These compact representations support both model updates and efficient policy synchronization. To prevent overly large updates from disrupting learning, we complement gradient compression with predicted‑KL step control, which estimates policy changes before applying each update and adjusts its magnitude accordingly. With all techniques combined, LoGRA reduces average training memory usage by up to 45.7% across reasoning tasks without compromising performance. It also enables stable training of a 27B‑parameter model for over 1,100 steps on a single eight‑GPU node, where dense Adam runs out of memory, making previously memory‑infeasible RL training practical. Code is available in the \hrefhttps://github.com/skzhang1/labs‑molt/tree/logra/examples/scripts/lograMolt library.
Authors:Chengtao Lv, Jinyang Du, Shuyi Feng, Yang Yong, Shiqiao Gu, Shunzi Yang, Ruihao Gong, Shen Ren, Tianwei Zhang, Wenya Wang
Abstract:
World Action Models (WAMs) incorporate visual representations from video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture‑of‑Transformers (MoT) architectures and compute video representations once for reuse by the action expert. However, intra‑expert iteration (\ie, multi‑step action denoising) and inter‑expert waiting (\ie, sequential execution of the video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one‑step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra‑expert iteration, we propose Teacher‑Anchored Consistency Distillation (TACD) to address a local‑global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi‑step rollout endpoint, enabling accurate one‑step action generation. Additionally, we propose Cross‑Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert‑level waiting. It overlaps the two experts through block‑wise sharing of the video KV cache, synchronizing only immediately before the corresponding action attention consumes it. Extensive experiments across diverse benchmarks (\eg, LIBERO, LIBERO‑Plus and RoboTwin) and model variants (\eg, Fast‑WAM and Faster‑WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near‑lossless performance (\ie, <1% drop) across these benchmarks while delivering significant end‑to‑end speedup (\eg, ~25× on H100). Our code and checkpoints are available via this \hrefhttps://github.com/ModelTC/LightX2V/tree/main/scripts/realtimewamlink.
Authors:Sungjae Choi, Hanna Bae, Sunghyun Baek, Junmo Kim
Abstract:
Feed‑forward visual geometry transformers such as VGGT reconstruct dense 3D structure from images in a single forward pass, simplifying multi‑view 3D reconstruction. However, their quadratic attention complexity makes them difficult to scale to long sequences with thousands of frames. Chunk‑and‑align frameworks address this by splitting a long sequence into overlapping chunks and stitching their local reconstructions into a pose graph. Yet existing methods connect only sequentially adjacent chunks, so small per‑frame errors accumulate along the chain into large‑scale drift. To move beyond sequential edges, we propose VGGT‑Bridge, which adds long‑range skip edges that directly constrain non‑adjacent chunks without retraining. By running VGGT on sparsely sampled coarse chunks, each coarse chunk bridges distant fine chunks into a single direct constraint. We further turn VGGT's first‑frame scale bias into a drift correction by feeding selected coarse chunks in reverse, and a loop‑aware policy keeps this reversal compatible with existing loop closures. VGGT‑Bridge reduces ATE by 28.3% on KITTI Odometry, 18.8% on Virtual KITTI, and 10.0% on Waymo Open over the SwiftVGGT baseline, achieving the best performance among all chunk‑and‑align methods. Our code is available at https://github.com/BeCow5X5/VGGT‑Bridge.
Authors:Ziying Song, Yuchen Liu, Zhuoran Xu, Ziyang Liu, Jian Jin, Jiangtao Su, Haibao Yu, Lei Yang, Yuanpei Chen
Abstract:
Embodied world models predict the visual consequences of candidate actions before execution. However, existing action‑conditioned world models often adopt uniformly weighted visual generation objectives that can be misaligned with embodied prediction needs. Even with explicit motion conditioning, these objectives can underemphasize spatially sparse changes that are critical to interaction. We propose KineWorld, a transport‑aware world‑modeling framework that extends robot kinematics from motion conditioning to the spatial allocation of generative supervision. Kinematic Transport Lifting (KTL) constructs renderer‑derived, camera‑aligned transport fields from commanded robot motion. Transport‑Aware World Diffusion (TAWD) calibrates their motion support on the video‑latent grid and reweights future‑RGB flow matching through a normalized mixture of uniform and transport‑focused distributions. We train KineWorld using ALOHA‑AgileX bimanual manipulation data from RoboTwin 2.0. KineWorld achieves an EWMScore‑P of 68.95 in single‑view evaluation and a TWB‑Score of 54.82 in multi‑view evaluation. These results support a shift from appearance fitting toward action‑consequence modeling for embodied decision‑making.
Authors:Haoyu Wu, Ling Lin, Pascal Lefèvre, Ruizhe Li, Xiaowu Sun
Abstract:
Three‑dimensional (3D) left ventricular (LV) reconstruction from sparse cardiac magnetic resonance (CMR) imaging remains challenging due to inter‑slice misalignment and insufficient local spatial information between slices. Global aggregation of contour features may obscure local contour‑to‑surface relationships. We propose Local2Mesh, a spatially localized contour‑to‑mesh framework that deforms a template mesh to reconstruct 3D LV geometry from sparse 2D contours without 3D mesh annotations. The framework introduces geometry‑aware alignment to correct inter‑slice misalignment and a plane‑aware Local Router that routes contour features to template vertices using vertex‑to‑plane distances. Local and global contour features then jointly guide graph‑based template deformation for 3D LV reconstruction. Experiments on two public datasets, M\&Ms‑2 and ACDC, demonstrate superior geometric reconstruction and functional estimation over existing methods. Zero‑shot transfer from M\&Ms‑2 to ACDC demonstrates strong cross‑dataset generalization. Reconstructed meshes also improve disease classification over sparse contours, supporting their utility for downstream cardiac analysis. These results demonstrate that combining geometry‑aware alignment with local contour‑to‑vertex modeling improves LV reconstruction from sparse 2D contours and supports downstream cardiac analysis. The code is available at https://github.com/hwu918945‑alt/loca2mesh.
Authors:Youngmin Lee, Byungha Ko, Guhnoo Yun, Dong Hwan Kim
Abstract:
Multi‑view panoptic segmentation assigns a semantic class and a scene‑level instance ID to every pixel of an unordered set of images, and recent feed‑forward 3D models predict these labels for the input views in a single forward pass. Their predictions, however, have been evaluated with the scene‑level PQ (PQ^scene) borrowed from per‑scene optimization methods, typically on rendered held‑out views. PQ^scene tiles all views of a scene into a single image, so that a missed appearance or a change of ID lowers the score of the matched pair only in proportion to its area. We propose View‑Consistent Panoptic Quality (VC‑PQ), which extends PQ from a single image to a set of input views, counts equally every view in which an instance is visible, and penalizes a prediction that is not visible in the same views as its ground truth. A decomposition of VC‑PQ attributes the score a method loses to mask accuracy, view consistency, and the matching threshold. A single additional parameter recovers the area weighting of tiling for comparison. Under a fixed evaluation protocol on ScanNet++ and ScanNetv2, recent feed‑forward methods are evaluated with VC‑PQ and PQ^scene, and the decomposition shows where each of them loses its score. Controlled perturbations of the ground truth show that VC‑PQ responds to the number of views in which an instance is missed or changes ID, whereas PQ^scene responds to their area. The aim of this work is to make view consistency part of the evaluation of multi‑view panoptic segmentation, with VC‑PQ reported alongside PQ^scene. The evaluation code and the input sets of the protocol are available at https://github.com/Renem24/vcpq.
Authors:Chukwunonso Henry Nwokoye, Khalil El-Khatib, Li Yang
Abstract:
Machine learning (ML)‑powered poisoning attack vectors are adversarial maneuvers whereby an attacker intentionally inserts, corrupts, or alters training data to distort an ML model's learning process. The objective is to diminish model efficacy, instill biases, induce misclassifications, or include concealed backdoors that may be attacked during implementation. In streaming contexts, poisoning attacks pose significant risks since models perpetually update based on incoming streams of data. An assailant may incrementally introduce harmful samples into this data stream, leading the model to assimilate erroneous features over time without timely identification. Therefore, this study is aimed at evaluating the efficacy of the adversarial training (AT) defense approach against poisoning attacks (label flip and noise injection) using an online AutoML pipeline for Internet of Things (IoT) networks. Specifically, poisoning attacks (label flip and noise injection) were applied to streaming‑capable AutoML learners (Hoeffding Tree (HT), Leveraging Bagging (LB), Adaptive Random Forest (ARF), Hoeffding Adaptive Tree (HAT), and Streaming Random Patches (SRP)). Under the strongest poisoning rate (PR = 1.0), AT‑SRP achieved the highest F1‑score against label flip poisoning (0.904), while AT‑LB achieved the highest F1‑score against noise‑injection poisoning (0.933). Finally, several drift detection methods were used for rolling accuracy and prequential evaluation.
Authors:Shen Ruan, Wenchang Gao, Jin Wang, Siao Liu, Zhaxizhuoma, Dongchun Ren, Xin Zheng
Abstract:
Post‑training quantization has been shown to preserve VLA performance under standard evaluation conditions, but whether it preserves the full‑precision model's robustness and generalization remains underexplored. In this study, we systematically study the robustness and generalization of post‑quantized VLA policies under environmental disturbances. Empirical results show that quantized policies can become fragile to subtle environmental variations despite retaining comparable in‑distribution performance. We further observe that action discrepancies are concentrated in a small subset of rollout states, while teacher guidance has opposite effects depending on discrepancy: it improves generalization at high‑discrepancy states but can degrade it at low‑discrepancy states. These findings reveal that effective post‑quantization recovery requires selectively intervening on vulnerable states rather than globally distilling the student. We therefore propose Policy‑Induced Vulnerability‑Oriented Tuning (PIVOT‑Q), a vulnerability‑aware On‑Policy Distillation (OPD) framework that selectively corrects vulnerable states encountered during quantized‑student rollouts using the frozen full‑precision policy as a teacher. PIVOT‑Q identifies vulnerable states using discounted accumulated discrepancies over a short horizon, applies phase‑balanced sparse supervision, and uses a Behavioral Anchor to prevent unnecessary changes. Experiments under seven LIBERO‑Plus environmental variations demonstrate consistent recovery across multiple VLA backbones and quantization methods. Notably, PIVOT‑Q consistently outperforms full‑state distillation across all settings while using only 7.4% of its state‑level distillation budget. Our code is available at https://github.com/ruanruan‑andy/PIVOT‑Q.
Authors:Yunghee Lee, Jaeyeon Kim
Abstract:
The Fréchet distance is a de facto standard for evaluating generative models across domains, appearing as FID for images and FVD for videos. It summarizes the discrepancy between generated and reference distributions in a single scalar, with lower values typically interpreted as better generation quality. However, this scalar view can obscure what drives the comparison. For example, in COCO dataset, increasing the number of diffusion sampling steps improves ImageReward scores yet worsens (increases) FID. Motivated by this mismatch, we seek to make the Fréchet distance more interpretable by uncovering where the discrepancy lies. To this end, we introduce directional Fréchet distance, the expected squared projection of the optimal transport displacement onto a given direction. Across our image, video, and protein case studies, we find that a small number of interpretable directions account for much of the distance. We use these directions to explain the FID increase in terms of semantic concepts represented by CLIP embeddings, quantify FVD's bias toward per‑frame appearance, and revisit the interpretation of Protein FID. We open‑source our codebase at https://github.com/yhlee‑add/directional‑fd.
Authors:Yuqun Wu, Yao Xiao, Chuhang Zou, Shenlong Wang, Derek Hoiem
Abstract:
Recent works augment Vision‑Language Models with geometry features from pretrained 3D models, expecting that the geometric signal will boost spatial reasoning. However, we find that simply fusing geometry features and training on standard spatial QA yields only marginal improvements on high‑level multi‑hop tasks. We attribute this gap to a training‑signal problem: standard spatial QA can be largely answered from visual features and language priors, so the geometry pathway receives weak gradients and fails to integrate with the visual features. To provide a training signal that requires geometry, we propose novel‑view semantic rendering as an auxiliary training task that requires the model to predict the semantic layout of an unobserved viewpoint, inspired by humans' ability to mentally simulate novel viewpoints during spatial reasoning. This task encourages joint use of both pathways: geometry provides pose‑dependent visibility, while vision provides semantic content. Our auxiliary task yields consistent improvements over the geometry‑augmented baseline across all three benchmarks (up to +1.6 on VSI‑Bench, +2.2 on ReVSI, +2.9 on our 3D‑Point‑QA dataset) and our full model surpasses prior open‑source methods on VSI‑Bench and on ReVSI. Project page: https://yuqunw.github.io/Render2Reason/.
Authors:Huapeng Li, Fuxiang Feng, Jinqiu Fan, Shuo Yang, Fengjiao Chen, Xuezhi Cao, Ran Song, Wei Zhang
Abstract:
Although the advancement of vision‑language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensive evaluation methodology is important to advance the integration of VLMs in robotic navigation and manipulation. However, current benchmarks lack a comprehensive method to evaluate diverse robotic tasks, and evaluation metrics remain relatively constrained, making it difficult to assess the embodied capabilities of VLMs in a thorough and fine‑grained manner. To address this issue, we propose RMMBench, an evaluation benchmark that requires robots to understand language instructions and perform long‑horizon tasks in continuous spaces. RMMBench seamlessly integrates high‑ and low‑level embodied tasks into a unified framework, constructing a "navigation‑manipulation" task suite comprising 70 canonical task scenarios that range from localized manipulation to long‑horizon composite navigation. The results reveal that leading VLMs still face major challenges in spatial localization when performing mobile manipulation tasks, and also highlight the necessity of enhancing the spatial perception capability of robots during long‑horizon interactions. RMMBench can be accessed at https://mxxq‑stack.github.io/rmmbench‑project/
Authors:Kuan-Hsun Tu, Hsuan-Chi Liu, Jia-Wei Liao, Chien-Sheng Chiang, Cheng-Fu Chou, Tsung-Wei Ke
Abstract:
We introduce CNP‑Flow, a flow matching framework for temporal generation that learns conditional source distributions through flow reversal. Whereas standard conditional flow matching (FM) incorporates conditioning through the vector field and draws source samples from a standard Gaussian, CNP‑Flow uses a conditional noise predictor (CNP) to produce an isotropic Gaussian source for each temporal condition. The CNP is supervised by source samples obtained through flow reversal, which maps observed targets backward through a pretrained FM model. A three‑stage pipeline pretrains the FM model, trains the CNP, and fine‑tunes the FM model using the learned source distribution, while preserving the FM backbone architecture. Across video prediction, video interpolation, and 7‑DoF Franka robot motion planning, CNP‑Flow consistently improves generation quality. It also matches baseline performance with fewer function evaluations. Project page: https://embodiedai‑ntu.github.io/cnpflow
Authors:Xiaobiao Du, Zhen Fang, Tianqing Zhu, Richard Hartley, Xin Yu
Abstract:
Recent advances in 3D Gaussian Splatting (3DGS) have achieved remarkable performance in novel view synthesis, yet deploying both static and dynamic Gaussian representations on resource‑constrained mobile devices remains challenging due to heavy storage, redundant primitives, and costly per‑frame computation. We present Mobile‑4DGS, a unified lightweight framework for high‑fidelity real‑time static and dynamic Gaussian rendering on mobile platforms. For compact appearance modeling, we introduce a Monte Carlo Specular Energy Aggregator that compresses high‑order radiance residuals into the first‑order Spherical Harmonics (SH), together with an Attribute‑Conditioned SH Enhancement module whose predicted offsets are pre‑baked before inference. We further propose a Multi‑View Alpha‑Based Densification and Pruning strategy to suppress redundant primitives while maintaining multi‑view consistency. For dynamic scenes, we develop a compact explicit 4D representation by constructing second‑order Gaussian motion, learnable temporal support, and a binary static‑dynamic partition, enabling continuous‑time modeling without runtime deformation networks. Based on this partition, a Depth‑Order Certificate selectively reuses previously committed depth orders to reduce re‑projection, sorting, merging, and index‑buffer updates during playback. Extensive experiments on static and dynamic scenes demonstrate that Mobile‑4DGS substantially reduces storage and rendering overhead while maintaining competitive visual quality, enabling real‑time 3D and 4D Gaussian Splatting on mobile devices. \textcolormagenta\hrefhttps://xiaobiaodu.github.io/mobile‑4dgs‑project/Code has been released: https://xiaobiaodu.github.io/mobile‑4dgs‑project/.
Authors:Runwei Guan, Rongsheng Hu, Shangshu Chen, Ningwei Ouyang, Shaofeng Liang, Heyi Lin, Jinjing Zhu, Yang Shi, Dongming Wu, Jianfeng Dong, Daizong Liu, Henghui Ding, Hui Xiong
Abstract:
Roadside traffic reasoning requires every free‑form textual claim to be backed by visual evidence. Existing grounded multimodal large language models (MLLMs) frequently exhibit say‑point mismatch, in which the textual answer contradicts the bounding boxes the model localizes. Evaluation metrics that score answers and boxes separately leave this failure unpenalized. We trace the mismatch to the conventional answer‑then‑ground factorization, which commits to a numerical claim before any object is enumerated. To measure it, we build RoadSceneVQA‑G, a benchmark of 34.7K question‑answer pairs in which every free‑form answer is linked to the set of boxes that witnesses it, and we propose the Answer‑Grounding Consistency (AGC) evaluation suite. To address it, we introduce Enumerate‑then‑Answer (EtA), which reverses the generation order so that answer‑evidence agreement becomes a property of the output structure, and Enumeration‑Consistent Policy Optimization (ECPO), a reinforcement learning stage that uses the union of multiple rollouts as a recall teacher without ground‑truth boxes. EtA raises say‑point consistency from 26.6% to 93.7% and grounding F1 from 52.2% to 73.0%, and ECPO further increases F1 to 75.6% without per‑box supervision. On gRefCOCO, the same framework outperforms the strongest compared method, indicating that it transfers beyond traffic scenes. The project is available at https://github.com/GuanRunwei/RoadSceneVQA‑G.
Authors:Zeyu Li, Yipu Zhang, Jintao Cheng, Xin Li, Wei Zhnag
Abstract:
Looped language models provide a parameter‑efficient way to scale iterative test‑time computation by repeatedly executing a shared recurrent core. Post‑training quantization (PTQ) can reduce the memory footprint and inference cost of looped language models, but errors introduced by a quantized shared core affect subsequent cores. Among PTQ methods, channel scaling and orthogonal rotations preserve the floating‑point computation while producing representations with different quantization quality. We find that quantization configuration candidate rankings can change with recurrent depth, motivating configuration selection at the target deployment depth. However, evaluating every candidate over the full calibration set at this depth is costly. We therefore propose Loopy, a PTQ framework that formulates shared‑core quantization through a recurrent‑depth‑aware objective, selecting shared low‑bit representations by their final prediction loss at the target deployment depth. Channel scaling and orthogonal rotations parameterize the candidate representations. To approximately solve this selection problem efficiently, Loopy progressively allocates calibration windows to promising candidates while preserving complete target‑depth execution, using only forward evaluations. Across eight settings, Loopy achieves the state‑of‑the‑art results among different baselines. On Ouro‑1.4B under W4A4, Loopy reduces LAMBADA perplexity by 36.5% relative to SpinQuant. Our code is available at https://github.com/Shameless0817/Loopy‑review.git.
Authors:On Tai Tang, Huanyao Zhang, Jiepeng Zhou, Wentao Zhang
Abstract:
The next generation of multimodal research agents must reason over long‑lived research histories rather than short model completions. During a single task, an agent may repeatedly search the web, inspect visual evidence, revisit earlier hypotheses, and accumulate tens of thousands of tokens of multimodal context. Despite this trend, online RL for multimodal research agents remains largely confined to shorter contexts and interaction horizons. We push online RL training to 128k context and 75+ tool‑interaction turns. To our knowledge, this is the first online multimodal deep‑research RL study trained at 128k context, and the first trained with a 75 tool‑turn horizon. Scaling to this regime exposes several practical limitations of conventional RL training. Early in training, weak policies make poor use of large interaction budgets, causing expensive rollouts with little reward improvement. Later, policy entropy can collapse before performance has saturated, prematurely ending useful learning. We introduce Long‑MDR, a three‑component training recipe designed specifically for this setting: On‑Policy Distillation Warmup, Progressive Horizon Expansion, and Entropy‑Triggered Rescue. Together, these techniques improve both the learning efficiency and stability of long‑horizon RL, enabling continued gains in a regime where direct training is slow and costly. Code will be available at https://github.com/navileon/Long‑MDR
Authors:Shengjian Wu, Li Sun, Yu Shangguan, Qingli Li
Abstract:
DETR‑style detectors use one‑to‑one bipartite matching during training to assign object queries to ground‑truth objects, enabling end‑to‑end set prediction without non‑maximum suppression (NMS). However, without an explicit de‑duplication procedure, multiple queries can still produce highly similar hypotheses for the same object, making training unstable and predictions less decisive. Inspired by the sequential ordering of NMS, we propose DETRNN, a plug‑and‑play module that turns unordered object queries into a competition‑aware sequence for recurrent refinement. DETRNN builds an explicit confidence‑and‑similarity based order from prior predictions, then refines queries with an RNN along this order to model competition inside the decoder. This ordered recurrent refinement reduces redundant predictions, stabilizes optimization, and improves final detection accuracy. Experiments on multiple DETR‑style detectors show consistent gains with comparable efficiency. Code repository: https://github.com/nigelwoo2008‑coder/DETRNN.
Authors:Youbeen Jeon, Uiseong Shin, Hwanchul La, Jaeseong Kang, Hyelim Choi, Yongseok Lee
Abstract:
Learning dexterous manipulation benefits from human demonstration datasets that capture diverse and natural hand‑object interactions. In particular, contact points and forces provide supervision on where and how strongly to interact, which cannot be fully captured by motion trajectories alone. However, methods for jointly capturing hand and object motion, contact points, and forces remain limited. Moreover, severe occlusion from hand‑object interaction challenges accurate tracking of both hands and objects. To address these limitations, we present a Visual‑Inertial‑CONtact based hand‑object tracking (VICON) framework. It holistically captures both hand and object motion along with contact information during manipulation, even under severe occlusion. First, we adopt a visual‑inertial glove and an RGB‑D camera for accurate hand tracking, and redesign the glove to incorporate contact sensing. Specifically, force‑sensitive resistors (FSRs) are placed on the glove based on human grasp frequency to synchronously record contact states and calibrated normal forces. Second, without requiring pre‑existing CAD models, we estimate object poses using RGB‑D images and a mesh reconstructed from a monocular video. We propose factor‑graph‑based object trajectory estimation that fuses object‑pose estimates weighted by visibility under hand‑object occlusion, FSR measurements, and a hand‑motion prior. Across 40 motion‑capture sessions with five objects, VICON achieves a 2.5% failed‑frame rate compared with 50.9‑64.6% for the baselines, with median errors of 3.9 mm and 3.0 degrees under occlusion. Using VICON, we construct a dataset containing synchronized hand‑object motion, contact points, and normal forces, and will publicly release an expanded dataset covering 10 object categories at https://github.com/VICON‑dataset/dataset.
Authors:Jiahao Ying, Wei Tang, Boxian Ai, Yaoning Wang, Haotian Chen, Wenhe Sun, Caijun Xu, Haozhan Cai, Changyi Xiao, Yixin Cao
Abstract:
Large language models (LLMs) progressively revise their predictions across Transformer layers, yet we typically observe only the final output, discarding the trajectory through which it is formed. We introduce Belief‑Trajectory Energy(BTE), a model‑grounded measure that characterizes an input through the layerwise predictive revisions it induces in a model. By mapping intermediate states into a shared predictive space, BTE provides a principled measure of belief change that can be summarized as either a scalar or a structured depth profile. Theoretically, we show that local BTE corresponds to predictive revision under the Fisher‑Rao geometry, while the sequence of revisions captures information beyond the initial‑to‑final belief change. Empirically, scalar BTE provides a model‑relative signal of difficulty across diverse reasoning tasks, while richer BTE representations support human‑LLM review detection and fine‑grained generator attribution, reaching up to 0.998 macro‑AUROC and 95.6% eight‑way attribution accuracy. Further analysis shows that BTE develops throughout pretraining and is selectively reshaped by targeted training, demonstrating that the resulting measurement reflects what the scoring model has learned. Together, our results establish belief trajectories as a principled model‑grounded signal and suggest a broader perspective in which learned models can themselves serve as instruments for characterizing the data they process. More demonstrations can be found at https://yingjiahao14.github.io/BTE‑web/.
Authors:Richard Lee Kim, Yeongmin Kim, Gyuwon Sim, Taekyu Kim, Minsang Park, Il-chul Moon
Abstract:
Diffusion models achieve strong generative performance but often struggle to align generated samples with human preferences measured by a reward model. A simple yet effective algorithm for test‑time alignment is Best‑of‑N (BoN) sampling, which draws N i.i.d. samples from a pre‑trained diffusion model and outputs the single highest‑reward sample. Despite its empirical success, BoN makes limited use of reward information, as it is incorporated only at the final selection stage without influencing the reverse diffusion trajectory during sampling. Consequently, BoN sampling does not improve the average alignment of generated samples and is primarily suited to single‑output settings. We propose Best‑of‑N Guidance (BoNG), a novel method that integrates the principle of BoN sampling directly into the reverse diffusion process. BoNG performs online BoN selection over denoising particles and adjusts the reverse diffusion process to steer the particle population toward higher‑reward regions during generation. Specifically, by introducing an asymmetric guidance interaction among denoising particles, BoNG uses the current BoN particle as a guidance signal to the rest of the particle population. This particle‑level interaction reshapes the sampling process toward higher‑reward regions, enabling BoNG to improve not only the final best sample beyond Vanilla BoN sampling, but also the average quality of generated samples. Over 36 empirical comparisons, BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling. BoNG also supports multi‑output capability, achieving 1.3× ImageReward score of the latest sample‑based guidance method with a 1.6× speedup. We release the code at https://github.com/aailab‑kaist/BoNG.
Authors:Xincheng Li, Xiaoqi Sheng, Xinyu Zhang
Abstract:
Accurate retinal vessel segmentation requires features that capture vascular geometry while preserving information for fine‑scale reconstruction. We propose CGDD‑Net, a context‑guided dynamic detail modeling network that connects adaptive feature extraction to a shared decoder pathway. Context‑Guided Scale‑Adaptive Deformable Encoding (CSDE) combines fixed‑grid convolution with deformable local attention to capture vascular patterns at multiple spatial extents. Spatially Adaptive Multi‑Kernel Gating (SAMG) selects receptive‑field responses at each location. Dynamic Cross‑Scale Detail Fusion (DCDF) aligns the gated intermediate features and compresses them into an eight‑channel representation, which is reused at three decoder resolutions together with selected encoder skips. This design consolidates intermediate information before decoding instead of transferring each middle‑stage feature through a separate direct skip. On DRIVE, CHASE\_DB1, STARE, and HRF, CGDD‑Net achieves AUC values of 0.9824, 0.9938, 0.9895, and 0.9874, with F1 scores of 0.8323, 0.8102, 0.8510, and 0.8157, respectively. The complete model contains 1.96 million trainable parameters. In cumulative ablations, the full model improves F1 over the internal baseline by 2.48, 0.61, and 3.49 percentage points on DRIVE, CHASE\_DB1, and STARE. Twelve directed cross‑dataset experiments further characterize transfer without target‑domain adaptation. The results support shared intermediate detail delivery as an effective, parameter‑compact architecture for retinal vessel segmentation. Code is available at \urlhttps://github.com/lixincheng‑xcl/CGDD‑Net.
Authors:Yucheng Song, Jincan Wang, Haokang Ding, Zhiqiang Tian, Kangxu Fan, Zhifang Liao
Abstract:
Domain Generalization (DG) for medical image segmentation is both highly challenging and critically important. However, existing medical DG methods largely overlook the issue of Catastrophic Forgetting (CF): Models often sacrifice their ability to retain source‑domain knowledge while pursuing cross‑domain robustness. This can directly threaten diagnostic safety in already‑deployed clinical scenarios. To address this, we investigate data augmentation strategies and catastrophic forgetting for medical image DG segmentation. First, we propose a structure‑guided style diffusion augmentation method. Constrained by anatomical structure consistency in the frequency domain, this method performs cross‑domain diffusion on the amplitude spectrum, generating samples with more diverse and broader style coverage to better support domain generalization. Then, we design a collaborative learning network with a dual‑branch interactive architecture (CoDG‑Net), together with a novel learning bias‑guided strategy that adaptively regulates knowledge transfer at both the layer level and the task level, thereby effectively mitigating catastrophic forgetting on the source domain. Experiments and ablation studies on single‑source and multi‑source medical DG benchmark datasets demonstrate that CoDG‑Net not only outperforms existing state‑of‑the‑art methods in target‑domain segmentation performance, but also achieves a lower forgetting rate on the source‑domain data. The code is available at: https://github.com/wangprocess/CoDG‑Net.
Authors:Ruida Hu, Yuanhao Wang, Chao Peng, Yakun Zhang, Cuiyun Gao
Abstract:
Large language models (LLMs) are increasingly used through terminal user interfaces (TUIs), yet task completion alone does not capture how difficult an interface is to understand and operate. Existing human assessments and LLM‑generated ratings or reports do not provide repeatable measurements of interaction effort grounded in verified task execution. We propose Agent‑as‑a‑User, an evaluation paradigm that places an LLM agent in the user role. Agent‑KLM separates agent‑side interpretation and operation costs and relates them structurally to human interaction. TUINaut operationalizes the paradigm by recording interaction trajectories and verifying outcomes with actor‑independent oracles. Using TUINaut‑Bench, we study 84 tasks across 15 real‑world TUIs with six human participants and five observation‑evaluator configurations, and evaluate 45 TUIs generated by three leading stacks. Similar aggregate success rates mask differences in which tasks humans and agents complete, whereas interpretation and operation costs follow correlated task rankings, especially for operation. This pattern persists across configurations. Generated TUIs can implement correct functionality while still requiring usability improvements. These results position Agent‑as‑a‑User as a repeatable, execution‑grounded paradigm for measuring task‑based usability.
Authors:Wei Wu, Ziyang Xu, Zeyu Zhang, Yang Zhao, Hao Tang
Abstract:
Generating a high‑quality gaming world from a natural‑language game intent requires joint reasoning about scene structure, spatial layout, gameplay objectives, interactive entities, and executable gameplay logic. Existing coding agents can generate individual assets, scenes, or scripts, but often struggle to maintain consistency across these components. We propose Code2Games, an agentic framework that builds a structured gaming world upon a base Blender world generated from the same game intent. Code2Games coordinates scene analysis, gameplay planning, constrained gaming‑world generation, and gaming‑engine customization through a shared scene‑gameplay representation with persistent element correspondence. After world generation, Code2Games adapts the generated world to Unreal Engine 5 and employs an execution‑guided reconstruction process that uses compilation diagnostics, runtime feedback, and gameplay test results to resolve inconsistencies arising during engine adaptation. To systematically evaluate gaming‑world generation, we introduce the GameCode4D benchmark, which comprises ten fixed game prompts spanning different levels of scene and gameplay complexity. We evaluate the generated results across four dimensions: visual quality, interactive fidelity, multimodal artifact quality, and playable‑game quality. Experiments demonstrate that, compared with direct gaming‑world generation by coding agents and existing baseline methods, Code2Games consistently improves the visual quality and interactive fidelity of generated gaming worlds, as well as the quality of the resulting games after engine adaptation.
Authors:Xueyi Chen, Shiyu Liu, Xin Jin, Yuhua Zheng, Xin Li, Haolei Bai, Junhan Zhu, Huan Wang
Abstract:
Recent GPU kernel optimization agents retain what they learn in knowledge bases or as distilled skills. Kernel benchmarks score each attempt's implementation for correctness and speed but leave the reuse value of retained experience unmeasured. We introduce MetaKernelBench, which measures whether experience distilled from an attempt in one kernel domain‑specific language (DSL) improves a fresh attempt at the same problem in another. Its 74 problems are fused subgraphs in six families, each posed as a pair of CuTe DSL and TIRx variants that differ only in the DSL. The agent first attempts each variant solo and is instructed to distill what it learns into a natural‑language skill, which is transferred whether or not the source attempt passes verification. The skill is the only extra input to a skill‑conditioned attempt by the same model in the other DSL. We compare each skill‑conditioned attempt with the solo attempt on the same variant under matched per‑attempt budgets, scoring correctness and end‑to‑end runtime. Across six models and both directions, paired lift over solo attempts ranges from ‑19% to +29%. Four models gain in both directions, yet regressions occur on 16% to 45% of problems in every model and direction. Outcomes follow the source attempt's result relative to the target's solo attempt rather than source success alone, improving in 71% of comparisons when the source stands above and regressing in 54% when it stands below. MetaKernelBench complements implementation‑quality metrics by measuring same‑problem cross‑DSL kernel knowledge transfer.
Authors:Aoyang Cai, Boning Zhao, Shaoxuan Xie, Dahui Gao, Huan Yang, Zhongyuan Wang, Zhiwei Yu, Guocai Yao
Abstract:
Lifelong physical agents must reason over extended interactions where past events continue to shape the world long after they disappear from view. Beyond recalling what happened, agents must infer how history changes the current state and constrains future actions. Yet existing embodied and video‑memory benchmarks largely focus on historical retrieval and summary, leaving such history‑dependent causal reasoning underexplored. We introduce EMBER‑Bench, an egocentric benchmark for cross‑event causal reasoning in long‑horizon embodied tasks, for which we newly created the task design, video recording, and data annotation. It contains 189 household tasks and 699 QA pairs, spanning task progress, failure recovery, external interventions, and compound long‑horizon tasks with distant dependencies and prerequisites, with fine‑grained event and causal‑chain annotations. EMBER‑Bench evaluates reasoning in both directions: next‑action prediction selects the next action from history, and causal traceback, given that action, identifies the historical event that makes it necessary. Input ablations that add action logs or privileged cause‑and‑consequence annotations to the video indicate which kind of historical information models fail to use. Among the 16 evaluated models, the highest overall accuracy is 61.2%, compared with a mean of 98.3% across two human evaluators. At paired decision points, correct traceback is not associated with correct next‑action prediction. Adding action logs yields a gain of 1.6 points, whereas cause‑and‑consequence annotations yield an additional gain of 13.0 points on top of that. These results suggest that extracting causal information from past events and converting it into constraints on current actions remains a key difficulty for long‑horizon embodied agents. Project Page: https://zhaoalexgoat.github.io/EMBER‑Bench/
Authors:Haocheng Peng, Boyang Zhou, Jiarui Hu, Xiyue Guo, Ziyang Zhang, Boming Zhao, Yifan Gao, Xiao Li, Hujun Bao, Zhaopeng Cui
Abstract:
Floorplans provide compact and widely available geometric maps for indoor localization, but existing high‑performing floorplan‑based methods still convert them into dense scene‑specific offline databases, tying accuracy, storage, and runtime to the sampling resolution of the discretized pose space. We present FreeLoc, an online RGB‑based floorplan localization framework that treats the floorplan as a directly queryable geometric map. FreeLoc introduces an efficient online geometric querying and diffusion‑aided refinement scheme, which retrieves plausible pose anchors through on‑the‑fly floorplan ray querying and refines them into accurate continuous pose estimates. For sequential localization, FreeLoc develops an online likelihood construction strategy that bridges single‑frame localization and probabilistic temporal fusion by constructing likelihoods from coarse‑sampled candidates and refined pose hypotheses, enabling histogram‑filter‑based temporal fusion without offline databases. Experiments demonstrate real‑time online inference and state‑of‑the‑art performance in both single‑frame and sequential localization, while real‑world results validate practical deployability in indoor robotic localization scenarios.
Authors:Avi Epstein, Snir Nehemia, Haim Suchowski, Lior Wolf
Abstract:
Full‑wave electromagnetic (EM) simulation enables accurate patch‑antenna analysis but is computationally expensive for large‑scale forward prediction and inverse design. We present a mesh‑native, physics‑augmented graph‑learning framework that treats radiation‑pattern prediction as signal reconstruction on an irregular surface mesh. For the forward problem, a GPS graph transformer is trained with Physics‑Augmented Intermediate Supervision (PAIS), an auxiliary node‑level objective that predicts complex surface currents, the physical intermediate linking geometry to radiation. PAIS improves multiple GNN backbones at no inference‑time cost, while shuffled‑current and non‑physical controls show the gain comes from physical correspondence. Direction‑conditioned decoding and a differentiable radiation‑integral consistency loss further exploit this structure. On an 80,000‑sample CST benchmark, GPS+PAIS reaches MSE 0.17 / PSNR 19.67, generalizes to a PCA split, and transfers zero‑shot to canonical patches. For inverse design, surrogate‑filtered diffusion beats nearest‑neighbor retrieval by 32% relative MSE.
Authors:Jiaxi Liu, Hang Zhou, Hangyu Li, Yifan Wang, Keke Long, Chengyuan Ma, Bin Ran, Xiaopeng Li
Abstract:
Generating corner‑case scenarios with appropriate adversity in a simulation environment is critical for testing an autonomous vehicle (AV) software stack's safety performance before deployment. Existing autonomous‑driving scenario generators can enforce specific behavior, adversity, or feasibility conditions, but they provide limited control over how extreme a generated scenario is relative to plausible futures in the same traffic context. This study represents the adversity of a generated scenario as its percentile in the conditional distribution of future risk given the observed history. This view supports calibrated answers to two questions: how "corner" a generated corner‑case scenario is and how its "cornerness" can be fine‑tuned. To this end, we formulate history‑conditioned risk‑percentile requests and learn a reference risk distribution that maps each requested percentile to a physical risk target. We then use a percentile‑conditioned joint diffusion model with sampling‑time risk guidance to generate multi‑agent futures, together with a reference‑based criterion for evaluating percentile realization. Experiments use the minimum post‑encroachment time (PET) between the ego and its surrounding vehicles as the risk surrogate on highD. On the primary evaluation set, our method realizes 1,422 of 1,440 requests within a 0.05 percentile tolerance (98.75%), with mean percentile error 0.00673 and PET‑target error 0.00991 seconds. The resulting interface connects context‑relative risk specification, physical realization, and evaluation through a common risk scale. Project website and videos of generated scenarios are available at https://hhj233.github.io/CornerPercentile/.
Authors:Qianlong Xiang, Miao Zhang, Kun Wang, Yupeng Hu, Junhui Hou, Liqiang Nie
Abstract:
Concept erasure is essential for the safe deployment of text‑to‑image diffusion models, as they may reproduce harmful, copyrighted, or privacy‑sensitive content learned from unconstrained large‑scale data. Existing methods typically erase unwanted concepts while preserving general generation capability by redirecting target‑related text‑to‑image mappings. However, recent studies show that erased models may still retain visual generative trajectories of target concepts, leaving them vulnerable to adversarial recovery attacks and revealing a fundamental gap between redirecting text‑to‑image mappings and truly removing visual knowledge. To bridge this gap, we propose VisualErase, a new paradigm that redirects concept‑bearing visual generative trajectories toward explicitly defined concept‑removed outcomes. To enable this redirection, we use structure‑preserving image editing to construct content‑aligned, concept‑removed counterparts for source images, providing explicit visual endpoints that retain non‑target content. We then derive a denoising target from each source‑to‑counterpart pair and use a dual‑branch redirection loss to align both text‑conditioned and unconditional predictions with this target, since conditional supervision alone does not explicitly constrain generation without textual guidance. To mitigate the adverse effects of concept erasure on non‑target generation, we jointly optimize the redirection loss with a counterpart retention loss that matches denoising predictions from the frozen pretrained model. Across style, celebrity, and nudity erasure, VisualErase limits the maximum attack success rate over seven attacks to 0%, 8%, and 0.1%, respectively, while retaining general generation quality. These results highlight the importance of visual trajectory redirection for robust concept erasure beyond text‑to‑image mappings alone.
Authors:Shang Wang, Tianqing Zhu, Huajie Chen, Jiayang Li, Meng Yang, Bo Liu
Abstract:
Jev turns natural‑language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implications of this emerging use remain largely unexplored. To address this gap, we conduct the first systematic study of these implications using the official Jev API and NanoJev, a local model with controllable training data and updates, focusing on three research questions: (1) What security threats arise when Jev is deployed as an application decision layer? (2) What private information can Jev reveal despite returning constrained typed outputs? (3) How can Jev's general‑purpose decision capability be used for beneficial purposes or misused? Jev's decisions depend on application state and may be influenced by user‑provided inputs. We therefore adapt prompt injection and adversarial suffixes to manipulate its decisions. Open‑source Jev distribution and updates introduce supply‑chain risks, which we examine by implanting backdoors in NanoJev through training data poisoning. Since Jev's outputs reflect both application state and information learned during training, we further adapt membership, private attribute, and internal knowledge inference attacks to recover sensitive information despite its constrained output format. Finally, Jev can serve as a general‑purpose decision oracle for defensive and malicious workflows. We examine this dual use through four detection tasks covering prompt injection, jailbreak inputs, harmful content, and AI‑generated text, alongside misuse scenarios involving jailbreak and model extraction. Our empirical evaluation shows that Jev remains vulnerable to the examined security and privacy threats, while its decision capability can support beneficial and malicious uses.
Authors:Kaiyue Wen, Tengyu Ma, Percy Liang
Abstract:
We evaluate language models' capability to generate novel research ideas through the lens of compression. We aim to lower‑bound the potentially tiny probability that a language model generates the essence of a future research idea without any hints. Rather than estimate this probability through expensive repeated sampling, our Priced Guidance framework measures the compression cost: how many additional bits of information are needed to guide the model to recover the target idea. We prove that if the model can recover the target idea with at most K bits of guidance in expectation, then it can generate the idea without any guidance with probability at least 2^‑K. In our framework, the language model, called the generator, can pose a sequence of multiple‑choice questions and specify a probability distribution over possible answers. A guide, which is a language model with access to the target idea, selects answers. If a selected answer has prior probability p, the generator pays ‑\log_2 p bits. The generator aims to produce an idea that matches the essence of the target idea with minimal cumulative cost. This cumulative cost equals, up to an additive constant, the number of bits of information sent by the guide. Using this methodology, we evaluate five generator LLMs (Opus 5, Fable 5.1, GPT‑6 Astra, GPT‑5.6 Sol, and GLM 5.3) on the core ideas in 87 recent high‑quality deep learning papers and we use LLM as judge to determine whether the generated idea matches the target in terms of the central research object and defining mechanism. Fable 5.1 achieves the lowest median compression cost at 69.9 bits, substantially lower than gzip's median of 5,712 bits for losslessly compressing the summary of the target idea. A uniform ensemble of Fable 5.1, Opus 5, and Astra further reduces the median compression cost to 55.8 bits and improves the generation probability lower bound by 18,000 times.
Authors:Shuaitong Li, Baochen Xiong, Xiaoshan Yang, Xizhe Zheng, Yifan Xu, Jianhao Huang, Changsheng Xu
Abstract:
Model merging offers a way to integrate the capabilities of specialized experts, but existing agent merging methods typically require all of them to be available at once. We study continual agent merging, which integrates incoming experts sequentially without retaining previously merged experts. Yet merging in parameter space or feature subspaces does not ensure that the merged model acquires an incoming expert's behavior on interaction trajectories. Moreover, updates toward a new expert can disrupt the merged model's previously integrated interactive behavior. Therefore, we propose Behavior‑Aware Continual Agent Merging (BACAM), which learns parameter‑wise merging gates from candidate‑generated trajectories using expert‑guided behavioral supervision. Task‑level stability‑plasticity control and tensor‑level conflict‑aware update budgets limit interference with existing capabilities while allowing new ones to be acquired. The learned gates are folded into the model weights without additional inference‑time parameters. Across four interactive tasks ‑ web shopping, tool use, information retrieval, and embodied interaction ‑ BACAM achieves an average success rate of 62.82%, exceeding the strongest evaluated merging baseline by 21.69 percentage points. Our code is publicly available at https://github.com/shuaitongli/BACAM.
Authors:Shih-Ying Yeh, Tzu-Sian Wang, Xuehai Wang, Jia-Hua Lee, Daniel Z. Kaplan, Ming-Qi Xu, Wuqian Tang, Chun-Yao Wang, Shang-Hong Lai, Chun-Yi Lee
Abstract:
Floorplanning arranges the blocks of a chip and decides their shapes under objectives that press blocks together, short wirelength and a small outline, and constraints that hold them apart, non‑overlap, clusters, MIB shapes and boundary blocks. Recent diffusion placers train on reference layouts alone and leave this coupled system to guidance, post‑hoc loops and a legalizer, reporting only the endpoint, which hides what the generator contributes. We re‑implement four of them under one recipe on FloorSet, score raw, refined and legalized layouts on one scale, and propose Trinity, a flow‑matching floorplanner whose six differentiable functions for the constraints and objectives are its training loss term, the energy of a closed‑form refiner after sampling and the base of a soft cost for every stage. The network thus learns the correction prior placers apply in their samplers, and sampling needs no guidance. Stage by stage, the training term lowers a plain transformer's raw soft cost by 26% and matters most at short budgets, the shared refiner decides more of the final cost than the generator and matches a ported placer's loop in 16 to 660 times fewer steps, Trinity's refined soft cost is 36% below the best ported pipeline, the soft cost ranks settings as the contest's hard cost does, and on the FloorSet val set the pipeline reaches a mean hard cost of 1.014 in 1.63 s per case.
Authors:Ruogu Chen, Jie Han
Abstract:
Euclidean combinatorial optimization problems (ECOPs), such as the Traveling Salesman Problem (TSP) and Capacitated Vehicle Routing Problem (CVRP), possess inherent symmetries under the two‑dimensional Euclidean group E(2), including rotations, reflections, and translations. Existing learning‑based methods, including recent diffusion‑based methods, rely on data augmentation or regularization to approximate E(2)‑equivariance. This paper presents EDISCO, the first discrete diffusion model for ECOPs with exact E(2)‑invariant generative distributions over node‑index solutions. EDISCO introduces an E(2)‑equivariant edge‑score network coupled with a categorical continuous‑time Markov chain over discrete edge variables, and exact posterior sampling provides efficient multi‑step inference. This design gives EDISCO a local geometric inductive bias: edge neighborhoods with the same relative geometry and combinatorial context are represented consistently regardless of absolute position or orientation, making learning more efficient and inference more robust than non‑equivariant methods. EDISCO outperforms previous learning‑based state‑of‑the‑art solvers on synthetic TSP from 100 to 10000 nodes and CVRP from 50 to 2000 customers, while using only 33‑50% of the training instances. Trained only on uniform synthetic data, EDISCO also outperforms competing learning‑based baselines under spatial distribution shift and CVRP constraint‑tightness shift. Code is available at https://github.com/ValleyC/EDISCO.
Authors:Qingyang Zhang
Abstract:
Cross‑system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, and healthcare workflows differ substantially across institutions and countries. We introduce Asymmetric Supervised Contrastive Learning (Asymmetric SupCon), a task‑specific pre‑training objective motivated by the heterogeneity of negative clinical outcomes. The objective clusters patients sharing a target positive outcome without explicitly attracting negative trajectories toward one another. We pre‑train temporal Transformer encoders on longitudinal records from 3.98 million patients in the Taiwanese National Health Insurance Research Database (NHIRD) and transfer them to two U.S. EHR datasets, MIMIC‑IV and EHRSHOT. A hybrid semantic mapping pipeline combining direct mappings with embedding‑based retrieval enables transfer across heterogeneous clinical vocabularies. On MIMIC‑IV, NHIRD pre‑training consistently improves over random initialization while substantially narrowing the performance gap to task‑specific in‑domain pre‑training. On EHRSHOT, the transferred models show particularly strong few‑shot performance for incident disease prediction. A controlled objective ablation under a matched pre‑training scale shows that Asymmetric SupCon achieves higher mean AUPRC than direct supervised BCE transfer on all four evaluated tasks and Standard SupCon on three of four, with a 0.003 AUPRC deficit on readmission. These results support asymmetric contrastive pre‑training as an effective approach for task‑specific cross‑national EHR representation transfer. Code is available at https://github.com/qingYzhang/Asymmetric_SupCon.
Authors:Junhoo Lee, Injun Baek, Seungyeon Kim, Suhyun Jeon, Minkyu Kim, Baekseung Kim, Nojun Kwak
Abstract:
Robot agents must connect their intended actions to observed outcomes while retaining the context needed to revise their choices over repeated attempts. Existing interfaces often leave these choices inside predefined tools or require agents to manage detailed execution code and its growing history. We introduce RobotUse, a robot agent harness that organizes computation, context, and decisions around specifying and revising physical actions. Agents visually select targets and poses, while the backend handles geometry, motion planning, and control. Subagents retain detailed interactions within each subgoal and return the information needed for subsequent decisions. Continual harnessing lets agents learn from execution by updating a persistent playbook. On RoboLab, RobotUse achieves 45% task success, outperforming CaP‑X by 6.7 percentage points while maintaining compact decision contexts and reducing reliance on predefined action abstractions. Furthermore, we show that RobotUse learns from real‑world execution despite imperfect feedback and transfers what it learns to subsequent tasks. Project page is available at https://robotuse‑team.github.io/.
Authors:Yi Hou
Abstract:
This paper studies GRPO at 1B scale from both directions: what estimator choices do to the learning signal, and what a degraded reward signal does to what is learned. We train OLMo‑2‑0425‑1B on GSM8K with a from‑scratch implementation and measure both sides in controlled sweeps, including a verifier‑quality experiment that degrades the training reward and the test‑time selector identically. Four results stand out. The prompt is the first‑order decision: the zero‑shot prompt leaves the base model at 0.08% (its outputs are degenerate continuations, not wrong answers), so almost no group carries a gradient, and training succeeds because the 3‑shot prompt reaches 18.3%. At this scale the estimator variants sit within seed noise, with Dr. GRPO ahead on both seeds. In the off‑policy regime, clipping is the whole story: training on data without a clipped ratio loses 4‑6 points relative to the on‑policy reference, while GRPO‑style clipping and GSPO recover the loss entirely. Finally, the same weak verifier is far cheaper in RL than in test‑time selection: a 10%‑flip verifier leaves RL's attainable gain intact (91% and 106% retained across two seeds) where selection retains 57%, and a format‑only verifier leaves RL with 16‑30% of its gain and selection with essentially nothing. Flip noise acts as an affine transform on the expected reward, and the group‑normalized advantage with Adam's rescaling removes it exactly; the residual is a second‑order variance effect that the matched‑step comparison at a 30% flip rate tests.
Authors:Duc Khoan Le, Kim Ngoc Tran, Minh Nhat Le, Thanh An Tran, Viet Toan Nguyen, Khanh An Lay, Tran Thai Son, Hoang Pham Minh
Abstract:
Reference‑guided style transfer aims to preserve the semantic structure of a content image while transferring the visual appearance of a style reference. Recent diffusion‑based methods achieve impressive stylization quality by exploiting strong pretrained generative priors. However, training‑free approaches still face a difficult trade‑off among style fidelity, content preservation, and content leakage. Direct style injection may unintentionally transfer semantic content from the style image, while fixed guidance schedules often ignore the time‑ and state‑dependent nature of diffusion sampling. To address these limitations, we propose TRACE, a training‑free diffusion style transfer framework with Time‑adaptive Residual Attention Control and Content‑Style Decomposition. TRACE first performs offline CLIP‑based subspace analysis to separate content and style directions from paired data. During inference, it removes content‑related components from the style reference and style‑related components from the content reference to reduce leakage. It then injects style information through residual cross‑attention and applies uncertainty‑aware guidance to adapt the guidance signal at each denoising step. Experiments show that TRACE achieves a favorable trade‑off between stylization and preservation. Compared with optimal‑control‑based baselines, TRACE substantially improves style fidelity (+17.28 CSD and +34.10 SRA). While, compared with stylization methods, it better preserves content structure (+12.80 DINO, +5.52 CLIP‑I, and ‑8.19 LPIPS) and reduces directional semantic leakage by 29.5% in DCL. Our code is publicly available at https://github.com/pixelchemy‑research/TRACE.
Authors:Zhexin Lou, Guancheng Lu, Zeyu Zhang, Yi Zhang, Yang Zhao, Hao Tang
Abstract:
Pretrained world models can generate diverse environments, yet users often want to explore a particular scene specified by their own video. This requires learning the scene's visual identity while retaining the quality of action‑conditioned generation. We introduce Personalized World Models (PWM), a framework for customizing interactive world models from short scene videos through online reinforcement learning. In PWM, the support trajectory and its associated controls provide reward feedback on continuations sampled from the current policy. In the GRPO instantiation, group‑relative optimization updates a compact LoRA adapter using a unified reward for scene appearance, visual continuity, and motion, while base‑policy anchoring regularizes changes to the pretrained generation prior of a frozen Yume‑5B backbone. The same adaptation procedure is applied across real and rendered environments. We also instantiate PWM with DiffusionNFT as an alternative reward‑guided optimization method for learning the scene‑specific adapter. We also introduce PWM‑Bench, comprising 150 customization tasks across Indoor, Outdoor, and Gaming, with paired evaluation on held‑out continuations. The GRPO and DiffusionNFT instantiations of PWM improve customization over native Yume in 71.3% and 65.3% of the evaluated scenes, respectively, with positive mean gains across all three domains. For the GRPO instantiation, matched SFT comparisons further demonstrate higher mean customization gains and better mean image‑quality scores in every domain, while retaining frame‑level visual quality close to the pretrained model.
Authors:Ruiyan Xu, Haisheng Su, Sixu Lin, Zhaokun Yue, Chengming Hu, Xin Jin, Guiliang Liu
Abstract:
World Action Models (WAMs) emerge as a promising foundation for policy refinement by predicting the consequences of sampled actions. However, visually plausible predictions can mislead policy refinement if they fail to reflect the input actions. To address this mismatch, we introduce R^2‑WAM, a two‑stage repair‑and‑reject post‑training framework that first improves the consistency of predicted futures with input actions, then uses these futures to select inferior action samples for negative fine‑tuning. The repair stage grounds imagination in observed robot behavior through a kinematic alignment score that measures agreement between predicted and demonstrated motion, enabling the predicted video to faithfully reflect its input actions. Using the repaired video model, the rejection stage compares imagined outcomes of sampled and demonstrated actions, selectively applying negative fine‑tuning to samples whose predicted task progress falls below the demonstrated reference by a prescribed margin. Together, the two stages extend video prediction from representation learning to consequence‑based policy refinement without additional environment interaction or changes to the inference procedure. R^2‑WAM achieves 93.8% average success on RoboTwin 2.0 across clean and randomized settings. On the long‑horizon real‑world Fold Shirt task, it achieves 87.5% average success, compared with 0% for Fast‑WAM. Our project page is available at https://r2‑wam.github.io/.
Authors:Luoyu Chen, Xiaoyu Ding, Weiqi Wang, Chenhan Zhang, Zhiyi Tian, Jianhuan Huang, Shui Yu
Abstract:
Subliminal learning lets a student inherit a teacher's behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual‑stream vector Δ_T, so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token‑level (hard) NLL supervision with full‑distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with Δ_T. Under iterative optimization, however, they diverge: soft supervision recovers Δ_T almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering‑vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix F, so gradients point toward FΔ_T rather than Δ_T. Gradient descent then acts as a progressively less‑damped inverse of F. With soft targets, this inverse restores low‑curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5‑7B and Gemma‑2‑9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at \urlhttps://github.com/luoyuchenmlcv/subliminal‑data.
Authors:Amr M. Zaki, Farhoud Jafari, Honggeun Ji, Komal Sarda, Marin Litoiu
Abstract:
Industrial Internet of Things (IIoT) systems generate high‑dimensional sensor telemetry from interconnected components, where faults can propagate across the system. To address these challenges, we propose SFlexRCA (Scalable and Flexible Root Cause Analysis), a topology‑free RCA framework designed for resource‑constrained IIoT environments. SFlexRCA transforms multivariate telemetry into compact orthogonal representations and applies shared lightweight linear modeling, avoiding explicit graph construction, message passing, and per‑variable or lag‑specific parameter growth. We evaluate SFlexRCA on three publicly available IIoT datasets, BATADAL, SWaT, and WADI, spanning different numbers of monitored variables, temporal characteristics, and training‑data regimes. SFlexRCA is compared with 10 statistical, causal, and non‑causal baselines in terms of RCA accuracy, training efficiency, inference latency, and memory consumption. In addition, inference efficiency and memory consumption are evaluated on Raspberry Pi 3 and Raspberry Pi 5, while energy consumption is additionally measured on Raspberry Pi 5. We further investigate temporalcontext sensitivity, architectural and loss components, and alternative representations. Notably, SFlexRCA maintains strong localization performance on BATADAL despite its limited normal‑operation training data, while its compact shared architecture avoids the parameter growth associated with causal and graph‑based approaches. Its lightweight shared architecture further enables efficient deployment on resource‑constrained IIoT edge devices. The SFlexRCA code is available at https://github.com/theamrzaki/RootCause‑ Analysis‑Correlation‑Attentive‑Modeling.
Authors:Ji Zhang, Zixin Liu, Yiran Ding, Jiayi Wang, Yilu Du, Weijia Xuan
Abstract:
An interaction has two parts: someone decides to act, and then chooses whom to act on. Temporal link prediction has concentrated on the second, and we show that it is blind to the first by construction: a standard negative keeps the real source and swaps the destination, and we prove that this cancels the source's activity exactly from the optimal score, so no model trained and evaluated this way is ever rewarded for learning it. Under the harder historical and inductive negatives, whose sources differ, the same factor becomes the dominant signal. We model it with Source Node Activity Modeling (SNAM), a self‑exciting event intensity fitted by an exact point‑process likelihood to decayed interaction counts the history states already contain; it has fewer than 20 parameters. On their own, never looking at the destination, these parameters beat DyGFormer and TPNet on four of five datasets under historical negatives. Added to the frozen scores of TPNet, TGN, DyGFormer and DSRD, four models of different design, without retraining anything, they raise AP on almost every backbone‑dataset pair in both settings, by up to 25 points. Our full model ranks first overall against eleven baselines on 13 datasets and three protocols, and on million‑event streams trains an epoch 9‑100x faster than TPNet and DyGFormer. We conclude that source activity is a blind spot of temporal link prediction, and a cheap, transferable one to close. Code is available at https://github.com/Erutaner/Your‑Temporal‑Link‑Predictor‑Is‑Blind‑to‑Who‑Is‑Active.
Authors:Hongming Xu, Le Zhou, ZhongHe Jin, Xiang Zhang, Bo Tang, Zhiyu Li, Xuanhe Zhou, Juncheng Zhang
Abstract:
As coding agents take on long‑horizon software evolution tasks spanning multiple files and stages, longer execution trajectories introduce two coupled challenges: (1) accumulated histories strain context budgets, and (2) repository changes can invalidate earlier execution evidence. Existing approaches address these challenges through techniques like larger context windows, compression, retrieval, or repository representations, but often fail to reconstruct a consistent task state after a context refresh or verify whether recalled evidence remains valid. Thus, we introduce MemTrace, a provenance‑aware memory system that preserves execution history and aligns its reuse with the evolving task (e.g., iterative cross‑file repair) and repository state. MemTrace stores history as immutable Memory Traces anchored to key information (e.g., files, symbols, tests), and organizes their execution order and dependencies in a Memory Trace Graph. When context is constrained, working memory retains only compact Memory Anchors, from which the agent can reconstruct the latest execution state and locate evidence relevant to its next action. Before restoring historical evidence, MemTrace checks its validity against the current repository state and retrieves only what the next action requires. Across three complementary long‑horizon coding benchmarks, MemTrace consistently outperforms all fully evaluated baselines under the same backbone and harness, improving DeepSWE pass@1 by 21.2 points, SWE‑EVO Resolved Rate by 4.4 points, and SWE‑Milestone Score by 17.8 points under Codex CLI.
Authors:Fuyuan Liu, Nayu Liu, Wenhao Yu, Peijin Wang, Yingchao Feng, Fanglong Yao, Liang Wan, Wei Feng
Abstract:
Remote sensing agents rely on perception, measurement, and raster analysis tools to solve Earth observation tasks. We refer to their judgments and quantitative results about ground objects as tool observations. However, these observations are subject to substantial uncertainty and may be incorrect even when the tools execute successfully. When agents accept incorrect observations, the errors can propagate through subsequent reasoning and cause task failure. We analyze 1,229 execution trajectories across three remote sensing agent benchmarks. On each benchmark, at least 88.1% of tasks depend on tool observations. Among these tasks, at least 22.7% contain incorrect observations despite successful tool execution. These errors propagate to the final answer in at least 82.0% of affected tasks on each benchmark. To address this problem, we propose RSure‑Agent, a framework for verifying tool observations and limiting error propagation. We introduce a verifiable observation protocol that requires tools to return process evidence for the agent to verify their observations. We also construct a task‑tool reliability prior from offline task feedback. The prior summarizes each tool configuration's past performance across task types and provides a task‑specific reference for verification. Using process evidence and this prior, RSure‑Agent decides whether to accept an observation, request additional evidence, or reject it. We evaluate RSure‑Agent on EarthBench, ThinkGeo, TerraLogic, and CHOICE‑420. Across the three agent benchmarks, RSure‑Agent reduces the error propagation rate by 21.3 to 25.9 percentage points relative to the base configuration with verification and the prior disabled. On CHOICE‑420, it improves overall accuracy over direct answering by 5.71 percentage points on average across 11 backbone models. On EarthBench, it reduces the tool‑call ratio by 25.9% relative to Earth‑Agent.
Authors:Dingdong Wang, Shujie Liu, Yayue Deng, Yuxuan Hu, Yunrui Cai, Jincenzi Wu, Jianwei Yu, Jinyu Li, Helen Meng
Abstract:
Empathetic spoken dialogue is a sophisticated cognitive process that requires not only recognizing emotions but also inferring a user's latent mental states to provide appropriate support. However, current SpeechLLMs often treat empathy as a direct input‑to‑response mapping, leading to "superficially warm" but emotionally hollow interactions. In addition, since empathy relies on a multi‑stage process with strong inter‑step dependency, errors at any intermediate step can cascade through subsequent steps and lead to inappropriate responses, while existing training paradigms lack mechanisms to precisely localize and improve such errors. In this work, we propose EchoChat, a unified framework that reformulates empathetic spoken dialogue as a structured cognitive reasoning process integrating perception, mental‑state reasoning, and response generation. To support this paradigm, we first construct EchoDialogue‑400K, an acoustically rich dataset for multi‑stage empathetic supervision. During the SFT stage, we strengthen acoustic grounding through proposed Acoustic‑Anchored Attention (AAA). During the RL stage, we further introduce a novel stage‑aware optimization objective with Step‑Decomposed Credit Assignment (SDCA) to localize reasoning errors and mitigate cascaded error propagation. In addition, we introduce EchoEval, an expert‑annotated benchmark for multi‑dimensional empathy evaluation. Extensive experiments demonstrate that EchoChat achieves state‑of‑the‑art performance in perception, reasoning, and response alignment. Project page: https://github.com/dingdongwang/EchoChat
Authors:I-Chun Arthur Liu, Jason Chen, Gaurav S. Sukhatme, Daniel Seita
Abstract:
Three‑dimensional perception is critical for robotic manipulation, particularly for high‑precision tasks, as recovering metric depth and precise 3D object positions from monocular RGB observations is inherently ill‑posed. However, many Vision‑Language‑Action (VLA) models rely solely on RGB observations for perception. Leveraging recent advances in foundation models for stereo matching, we introduce ExStereo, a stereo module that augments pre‑trained 2D VLAs with 3D perception. ExStereo reconstructs scene geometry from stereo image pairs and renders multi‑view observations as an explicit stereo representation for stereo feature extraction. The action tokens from the action expert selectively attend to the resulting stereo tokens through our proposed action‑stereo cross‑attention mechanism, enabling the policy to generate robot actions conditioned on 3D scene information. To learn robust 3D representations, we introduce a mid‑training stage before task‑specific post‑training, using a self‑supervised learning objective on large‑scale stereo data. We validate our approach by fine‑tuning two publicly available VLAs, π_0.5 and SmolVLA, and evaluate them in simulation and on a real‑world bimanual PiPER platform. Across both settings, VLAs fine‑tuned with ExStereo consistently outperform baselines, demonstrating the effectiveness of stereo perception for robotic manipulation. Our project website is at: https://exstereo‑vla.github.io/ExStereo/.
Authors:Uma Sushmitha Gunturi, Jimin Mun, Maarten Sap, Maria Antoniak
Abstract:
Counter‑storytelling is a powerful mechanism people use to challenge dominant narratives. Unlike other forms of counterspeech that have been widely studied in computational social science, counter‑storytelling has largely been overlooked. Counter‑stories are difficult to detect automatically; they are relational (defined with respect to expressions of racial stereotypes) and structurally diverse (drawing on stories that describe lived experiences, witnessed events, exemplars, and hypotheticals). We introduce a first framework for detecting and characterizing counter‑storytelling against racial stereotypes at scale. This includes (1) a three‑dimensional taxonomy grounded in narratology and Critical Race Theory and (2) a multi‑stage pipeline that identifies relational pairs of stereotypes and counter‑stories in noisy Reddit discourse. Using this pipeline, we annotate 25,549 Reddit posts across 615 communities and identify 1,312 counter‑stories. Our analysis shows that speaker identity and post context shape how counter‑stories are told. For example, in‑group writers favor first‑person testimony, often adopting the role of self‑reflective insiders. Our work shows how computational methods can scale qualitative approaches to identify and characterize counter‑storytelling as a contextual narrative practice, with implications for content moderation, narratology, and racial discourse analysis.
Authors:Xinyu Guan, Zhirong Zhang, Hongyuan Liu, Pengcheng Xu, Yu Sun, Chen Song, Qianyang Zhao
Abstract:
Event‑informed forecasting requires translating reports and historical responses into changes to a numerical forecast. We propose TIMBRE (Temporal Integration of Memory‑Based Responses and Evidence), which combines source‑aware representation, state‑conditioned response transfer, and reliability‑guided fusion before a frozen forecast head. A separate readout adjusts interval widths while preserving the median. In a single‑seed, one‑epoch development study of 13 tasks, TIMBRE improves MAE over ordinary fusion on eight tasks but over native Chronos‑2 on only two. Disabling response transfer in the trained model reduces BTC and AULF MAE by 47.04% and 6.81%, respectively. These findings identify sensitivity to learned response transfer rather than a general forecasting advantage. Missing development‑set scores and the absence of retrained ablations limit attribution to individual evidence mechanisms.
Authors:Ansuman Mullick, Eray Tüzün
Abstract:
An agent with long‑term memory can answer from a record it should no longer use, such as a plan the user later cancelled. We ask what a memory store must expose for an agent to know this before retrieving anything, and we score that judgment on its own, as metamemory monitoring. Readers see only a value‑free summary: record counts by lifecycle state and a list of attribute names. Each of 148 questions is asked against three versions of one store that differ in one attribute, so wording cannot give the answer away. What a model adds depends on the length of that list. On the short lists the benchmark's ground truth produces (2.6 names on average), none of five language models reliably beats a cosine‑similarity lookup over the names, and the three stronger ones are equivalent to it within 0.05. Under a control that fixes counts and list length, none is reliably above it and none is shown equivalent. On lists longer than the stores we tested write, padded to 60 names, the lookup loses 0.18; GPT‑5.6 Luna and Sol lose 0.06 to 0.07 and lead it by 0.13 to 0.14, while the other three fall with it. The lead holds for Sol against a sibling padded to the same length and counts. Reworded to share no word with the names, the lookup loses 0.02 and one stronger reader edges 0.04 ahead. On the short lists the three stronger readers lead only where the summary counts a state without naming it, and one count feature closes that lead within a question, though not when items are pooled. The backends we tested omit or misstate this information: under FR‑Bank's own metadata, GPT‑4.1 mini, Haiku 4.5 and the lookup fall from 0.73, 0.82 and 0.75 to 0.60, 0.71 and 0.59. In these tests, what the store wrote down limited every reader we gave it to. All stores are synthetic; control, whether a better judgment yields a better answer, is left to a later study.
Authors:Matteo Nerini, Xuekang Liu, Bruno Clerckx
Abstract:
Analog computing with microwave signals can perform linear transformations directly in the analog domain, as the signals propagate through a microwave network. A fundamental question is which transformations can be computed with a given set of microwave components. In our previous work, we answered this question for networks of hybrid couplers and phase shifters by deriving a necessary and sufficient condition on the transformations these networks can compute, and we showed that the discrete Fourier transform (DFT) satisfies it. In this paper, we take a first step toward the formalization of analog computing with microwaves in Lean, a programming language and proof assistant that is increasingly adopted in mathematics. We formalize the considered components, their series and parallel connections, and the class of networks they can implement. Then, we formally prove the necessary and sufficient condition characterizing these networks, as well as the implementability of the DFT of any size power of two. All proofs are checked by the Lean kernel, and the code is openly available at: https://github.com/matteonerini/formalizing‑analog‑computing.
Authors:Zuleika Redondo Garcia, Andreu Matoses Gimenez, Javier Alonso-Mora
Abstract:
Long‑horizon mobile manipulation requires a task plan and the motion that executes it. Generative planners trained on demonstration produce both in one pass, requiring neither a symbolic domain nor search. To date, however, they have relied on thousands of scripted demonstrations of fixed‑base arms and executed open loop. This paper presents a hybrid flow matching planner: a single network generates the symbolic plan with masked discrete flow matching and the motion trajectory with continuous flow matching. Unlike prior generative planners, we aim to learn from a much smaller set of demonstrations and to execute the plan in closed loop. Two properties of the data compensate for the small dataset. A demonstration resumed from any of its intermediate actions is itself a demonstration, which multiplies the training samples and enables replanning after every action. Objects of the same kind are interchangeable, which turns demonstrations of one goal into demonstrations of every permuted goal. Our base implementation produces valid plans on 68% of held‑out scenes; a training and generation scheme for the discrete plan raises this to 76%, and replanning after every action raises the task completion rate from 40% to 53% in a kinematic simulation. The planner matches the task completion rate of motion‑only flow matching policies while additionally providing the symbolic plan, and it outperforms previous hybrid diffusion formulations on both task completion and plan validity. We validate the planner on a real mobile manipulator. Videos and project page: https://andreumatoses.github.io/research/hybrid‑flow‑planning
Authors:Andreu Matoses Gimenez, Andrei-Carlo Papuc, Chris Pek, Javier Alonso-Mora
Abstract:
Latent world models enable robots to plan by predicting the consequences of actions. Planning long tasks with control‑rate actions requires many prediction steps, which enlarges the search space and accumulates error. Skill‑level actions shorten these sequences, but a symbolic skill vocabulary requires domain knowledge and labeled demonstrations. We construct skill‑level actions from the inputs of a flow‑matching policy trained on demonstrations segmented into complete skills. The policy maps a noise seed and an observation, optionally with a code or label, to a complete skill execution, so one execution is one world‑model transition. On this mechanism we propose four action abstractions with increasing task knowledge: a compressed seed, two discrete codes learned from the demonstrations, and a symbolic label. We evaluate them with a common world‑model training procedure and planning framework on simulated block rearrangement tasks that require up to 14 sequential skills. The symbolic label succeeds in over 90% of the tasks that require up to six skills and degrades beyond. Without any label, an object‑centric learned code matches it on single‑skill tasks and retains half to three quarters of its success on tasks of two to five skills. Ablations attribute much of the label's advantage to its planner knowing which actions are applicable, rather than to the label itself. Beyond six skills the search, not the world model, limits success. Project page: https://andreumatoses.github.io/research/flow‑skill‑wm
Authors:Zihao Zhao, Shuhan Zhang, Kai Wang
Abstract:
Modern models increasingly learn through black‑box oracles such as humans, optimization solvers, and external tools that provide feedback without exposing their internal mechanisms. A common remedy is to learn an (action‑)value function as a control variate. In this paper, we first observe that even an exact action‑value function can be arbitrarily far from variance‑optimal. We show that this gap arises because the value function minimizes the noise in each action's own gradient term, while an action can still affect the rest of the gradient estimator through shared parameters. A simple unbiased correction, at no extra oracle cost, can still reduce its variance by an arbitrarily large factor. Motivated by this, we then prove that the residual variance can be decomposed exactly by actions with no cross terms. This decomposition yields a closed‑form variance‑minimizing correction for neural‑network parameters, which can be computed by a simple projection. Empirically, our correction consistently reduces the variance left by the value function and improves learning across all tasks. The source code for all experiments is available at https://github.com/Zihao‑Kevin/black_box_opt.
Authors:David Minkwan Kim, Runfa Blark Li, Beckham Po-Ju Lee, Nikolay Atanasov, Truong Nguyen
Abstract:
In this paper, we develop a method that enables bimanual dexterous hands to manipulate articulated objects with a high success rate without suffering from an embodiment gap. We observe that the correlation between hand motions and object motions is dictated by the object rather than the hands and can be learned from human‑object demonstrations. Based on this observation, we propose PatternDex, a method that learns this correlation and represents it as a token sequence, which we call an interaction pattern. From this pattern, PatternDex estimates the wrist motions and contact points that fit the target robot, and then trains a reinforcement learning policy that exploits these estimates as guidance. Since the guidance fits the target embodiment, the policy explores only the actions that the target robot can execute and thus achieves high success rates. PatternDex also requires only simple fine‑tuning to train a new robot, since it can reuse the learned interaction pattern. We evaluate PatternDex with bimanual dexterous hands on human demonstrations from the ARCTIC dataset. PatternDex achieves, on average, a 92.8% success rate with Allegro hands, while the state‑of‑the‑art baseline achieves 52.2%. Also, it achieves success rates above 70% with three other robot hands after fine‑tuning alone. Furthermore, we verify that the learned policy transfers well to a real‑world task of opening a microwave. Videos and additional results are available at https://patterndex.github.io/PatternDex/
Authors:Tasnim Jahan, Md Easin Arafat, Swakkhar Shatabda
Abstract:
Accurate thyroid ultrasound segmentation is often challenged by low contrast, speckle noise, and unclear boundaries. Although recent methods have improved segmentation accuracy, many rely on resource‑intensive architectures or lack explicit integration of multiscale features with global biomedical visual guidance. In this paper, we introduce ThyCLIPNet, a lightweight semantic‑guided hybrid encoder‑decoder framework that integrates BiomedCLIP‑derived biomedical semantic guidance into a lightweight multi‑scale CNN segmentation pipeline. The encoder integrates MobileNetV2 with efficient channel attention, while atrous spatial pyramid pooling and a custom convolutional block attention module enrich bottleneck features. The decoder combines hierarchical skip connections and lightweight attention refinement with a BiomedCLIP‑guided gated fusion pathway that projects vision‑only global biomedical embeddings into decoder feature space and selectively integrates them through semantic‑local fusion and spatial gating. To the best of our knowledge, ThyCLIPNet is among the first lightweight thyroid ultrasound segmentation frameworks to use BiomedCLIP's vision encoder alone for image‑only global semantic guidance without text prompting. Experiments on TG3K, TN3K, DDTI, and PKTN achieve dice similarity coefficients of 96.22%, 87.58%, 84.73%, and 80.70%; intersection over union scores of 92.72%, 77.91%, 73.51%, and 67.64%; and 95th‑percentile hausdorff distances of 3.75, 16.38, 18.23, and 10.86, respectively. ThyCLIPNet uses 8.55M parameters and 22.99G FLOPs. Overall, the results support integrating global biomedical semantic guidance with lightweight multi‑scale CNN representations for robust and computationally efficient thyroid ultrasound segmentation. Source code: https://github.com/Tasnim‑Jahan/ThyCLIPNet. [Abstract shortened for arXiv. See PDF for full abstract.]
Authors:Ramil Khafizov, Ilya Statsenko, Ruslan Rakhimov, Artem Komarichev, Peter Wonka, Evgeny Burnaev
Abstract:
Sparse‑view novel view synthesis is a central problem in 3D content creation, but diffusion‑based approaches remain limited by iterative denoising, making multi‑view generation expensive at inference time. We introduce NAMVIS, a diffusion‑free framework that reformulates multi‑view image synthesis as geometry‑conditioned next‑scale autoregression. Instead of generating target views through repeated denoising, NAMVIS predicts discrete visual tokens through a small number of coarse‑to‑fine scale steps, while sampling all tokens within each scale and across target views in parallel. To anchor this generation process to explicit camera geometry, we propose Multi‑scale Projective Pose Encoding, which injects source and target camera transformations into both target‑view self‑attention and source‑to‑target cross‑attention at every resolution. NAMVIS further combines global conditioning with dense geometry‑aware cross‑attention, enabling the model to preserve source‑view appearance while maintaining target‑view consistency. Across Objaverse, GSO, and OmniObject3D, NAMVIS outperforms diffusion‑based baselines in PSNR, SSIM, and LPIPS, while running over 3 times faster than the evaluated diffusion baselines under the same evaluation setting. These results suggest that geometry‑conditioned next‑scale autoregression is a promising and efficient alternative to diffusion for sparse‑view multi‑view synthesis. Additional qualitative results, videos, and resources are available at https://corl‑team.github.io/namvis/
Authors:Bangwei Guo, Xujiang Zhao, Shengyu Chen, Yanchi Liu, Wei Cheng, Xi Zhu, Guoning Zhang, Dimitris N. Metaxas, Haifeng Chen
Abstract:
Structural diagrams are widely used to represent complex systems and relational information across scientific, engineering, procedural, and spatial domains. Recent vision‑language models (VLMs) have become increasingly capable of recognizing diagram elements and reasoning about their content, while complete diagram topology extraction remains comparatively underexplored. In this paper, we study diagram‑to‑graph topology extraction: extracting all diagram entities and the complete relations among them. To enable large‑scale supervised training and systematic evaluation of this task, we introduce Knossos, a benchmark of 19,200 diagrams across six diverse domains, with 245,179 nodes and 439,740 edges. Its symbolic generation process provides exact alignment between rendered diagrams and annotations of complete topology, relation types, and connector geometry. To address the modeling challenge of complete topology extraction, we also present Ariadne, a structured framework that decomposes the task into node inventory extraction and source‑conditioned edge prediction. Extensive experiments show that training on Knossos substantially improves complete topology extraction in smaller open‑source VLMs. Ariadne further improves over one‑step extraction under matched supervision, demonstrating the additional benefit of structured decomposition. It achieves the highest average Edge F1 among the evaluated methods on Knossos, while both backbone variants also improve over their unadapted counterparts on the real‑world external benchmark. Code and benchmark are available at https://github.com/bangwayne/knossos_Ariadne_Public.
Authors:Haoxuan Wang, Gengyu Zhang, Ramana Rao Kompella, Gaowen Liu, Yan Yan
Abstract:
Memory is essential for long‑horizon robotic manipulation, where successful actions may depend on past events that are no longer recoverable from the current observation. As episodes grow longer, however, retaining the full history becomes increasingly costly, creating a fundamental scalability challenge for memory‑augmented policies. Existing approaches address this challenge by either storing selected past observations in a bounded memory bank or compressing interaction history into a fixed‑size parametric state through Test‑Time Training (TTT). Yet these formulations do not explicitly distinguish between historical information that can already be recovered from the policy's current context and information that must persist beyond it. We introduce TTT‑RM, which repurposes TTT as Residual Memory, using fast weights not to generically compress history but to complement a bounded memory bank by preserving task‑relevant historical information that cannot be recovered from the policy's current context. Concretely, TTT‑RM learns a history decoder that reconstructs historical representations from the current observation and retrieved memory. The resulting reconstruction residual captures what this context fails to explain and serves as the learning target for TTT. The TTT slow weights are optimized against this residual target so that online fast‑weight updates learn to encode complementary historical information over time. The fast‑weight state is then queried to produce a residual memory representation that conditions action generation. Extensive experiments on memory‑intensive simulation benchmarks and real‑world tasks show that TTT‑RM consistently improves across multiple memory designs, outperforms diverse baselines, and supports sustained execution on a three‑minute, eight‑stage stowing task.
Authors:Baowen Zhang, Wei Fan, Ruman Wang, Hangting Ye
Abstract:
Answering questions over imperfect tables requires handling errors that can affect the answer. We investigate two challenges for large language models (LLMs): whether error discovery depends on where errors appear in a table, and whether providing their locations is sufficient for accurate question answering (QA). Using human‑reviewed instances from RADAR‑T, we conduct controlled studies across three LLMs by varying row order and comparing original, error‑marked, and repaired tables. First, reordering rows changes error discovery even when the table contents and gold answer remain unchanged. During direct inspection, LLMs are more likely to discover all rows containing relevant errors when these rows appear later in the table or are grouped more closely together. Second, providing verified error locations alone is insufficient for accurate QA: with code execution, accuracy on repaired tables exceeds that on error‑marked tables by 39.0‑59.1 percentage points across the three LLMs. As a practical application of these findings, we combine error discovery across shuffled table views with explicit guidance for verifying and handling the reported errors in a simple workflow, Geometry‑Balanced Discovery and Intervention (GBDI). On RADAR‑T, GBDI improves QA accuracy by 3.8‑18.5 percentage points over a code‑agent baseline across five LLMs (paired 95% confidence intervals exclude zero for four), at the cost of additional inference. These results highlight the importance of both reliable error discovery and effective error handling in QA over imperfect tables. Code is available at https://github.com/645‑t/GBDI‑ICLR‑2027.
Authors:Zihao Zhao, Tunyu Zhang, Haizhou Shi, Yusong Zhao, Xinxi Zhang, Hao Wang
Abstract:
Multi‑agent debate can improve reasoning, yet often fails to beat simple majority voting. We argue that successful debate requires two distinct mechanisms: proposal supply must surface a correct answer, and readout must identify that answer when voting misses it. We formalize the first requirement through recoverable headroom, which measures cases where a correct proposal is available but the majority answer is wrong. For the second, we develop Latent Verification Debate (LVD), an accounting model in which candidate proposals receive answer‑specific verification evidence before final generation. Controlled fixed‑proposal interventions estimate this latent effect in equivalent peer‑support units and show that correct evidence changes answer probabilities and generated decisions while proposal supply remains fixed. To improve proposal supply, we construct societies from neural‑thicket agents using labeled and label‑free coverage objectives. Across two backbones and matched‑budget reasoning benchmarks, coverage‑selected societies increase complementary proposal supply and improve aggregate accuracy in repeated stochastic evaluations. Round‑level controls further show that interaction provides gains beyond applying the same finalizer directly to the initial proposals. These results identify proposal coverage and truth‑sensitive evidence use as complementary conditions for debate to outperform voting. Code is available at https://github.com/Wang‑ML‑Lab/when‑debate‑helps.
Authors:Jiuzhou Lei, Chang Liu, Dayou Li, Zhiyuan Zhang, Xiao Liang, Yu She, Zhiwen Fan, Minghui Zheng
Abstract:
Vision‑language‑action (VLA) and world‑action models (WAMs) often degrade under out‑of‑distribution task variations despite retaining partial task capability. To recover such capability, we propose RoboIRS, an inference‑time internal representation steering method that uses successful and failed rollouts to train linear classifiers, select outcome‑relevant intervention locations, and derive task‑specific steering directions without updating policy parameters. On 15 simulation tasks with a frozen π0.5 policy, RoboIRS improves the average success rate from 44.4% to 66.2%, outperforming alternative inference‑time intervention baselines while adding little inference time. We further validate RoboIRS on real‑robot manipulation using the same π0.5 policy and demonstrate its applicability to a world‑action model Cosmos Policy, where the average success rate improves from 35.4% to 55.4%. These results show that directly steering internal robot‑policy representations can improve the performance of robot policies at inference time. Project website is available at https://rollingoat.github.io/roboirs/.
Authors:Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu
Abstract:
Large language models (LLMs) have progressively evolved into the core of autonomous agents. Building on this progress, LLM‑based multi‑agent systems (MAS) coordinate multiple agents into a synergistic team to accomplish complex tasks that exceed the capabilities of individual agents. The effectiveness of such systems depends not only on the agents themselves, but also on how collaboration mechanisms are designed and organized. Note that real‑world collaboration is typically partially observable, where each agent can only access partial information about the environment due to physical or privacy‑related constraints. However, many existing multi‑agent benchmarks assume global observability, and leave limited support for systematically evaluating collaboration mechanisms. To bridge this gap, we introduce MASBench, a multi‑agent collaboration benchmark designed under partially observable constraints. It is organized into three progressive task categories: Reasoning, Scheduling, and Game. Through this structure, we progressively evaluate three representative collaboration mechanisms: Protocol, Memory, and Routing. MASBench further provides deterministic evaluation metrics, including performance score, communication cost, and cost effectiveness, to characterize both collaboration outcomes and communication overhead. Experiments across diverse LLM backbones and mechanism configurations offer empirical guidance for effective MAS design. Code is available at: https://github.com/BUPT‑GAMMA/MASBench
Authors:Yang Dai, Longfei Zhang, Wei Tao, Li Shen, Jincai Huang, Qing Tao
Abstract:
Transferable adversarial attacks are arguably the most practical black‑box threat model. Under the same perturbation budget, stronger transfer attacks attain higher attack success rate (ASR), yet their imperceptibility also tends to degrade. Under such a fixed‑budget protocol, transferability and imperceptibility therefore appear to trade off against each other. We argue that this conflict is an artifact of fixed‑budget evaluation, not an intrinsic trade‑off. When attacks are compared on the ASR‑‑imperceptibility Pareto frontier obtained by sweeping ε, stronger transfer attacks already attain better imperceptibility at matched ASR than weaker ones. To exploit this latent advantage, we introduce the stealthy transfer attack ST, a plug‑in primal‑dual wrapper that adds an L_\infty saturation regularizer to the standard constrained objective and resolves it through a two‑step primal‑dual update: a projected primal step on the perturbation coupled with an L_1‑ball projection on a dual variable that absorbs the regularizer through Fenchel duality, requiring no auxiliary models or handcrafted perceptual priors. Empirically, ST extends the Pareto frontier across different base attacks and additional surrogate architectures. At ε=16/255, average imperceptibility gains over each base attack are 17% on LPIPS and 14% on NIQE while ASR is preserved or improved. At matched high‑ASR levels, the strongest ST variants further Pareto‑dominate dedicated stealth‑oriented transfer attacks, confirming that the latent imperceptibility advantage of strong transfer attacks can be unlocked by a primal‑dual optimization wrapper without sacrificing transferability. Code will be made available at \urlhttps://github.com/AndssY/ST.
Authors:Ron Aluf, Alon Canfi, Eliya Nachmani
Abstract:
Neural audio codecs compress waveforms into compact discrete tokens that underpin speech language models, real‑time communication, and large‑scale audio storage. Almost every dominant design, including residual vector quantization, finite scalar quantization, and single‑codebook variants, follows the VQ‑VAE template by partitioning the encoder latent through a learned codebook or a fixed scalar grid. We ask whether this partition is necessary. We introduce GS‑Codec, a neural speech codec whose bottleneck is a parametric signal decomposition rather than a quantizer. We adapt Gaussian splatting from 3D scene reconstruction to one‑dimensional latents. An inner optimization loop fits each encoder segment as a weighted sum of 1D Gaussian primitives. The decoder then reconstructs the waveform from the rendered sum. To avoid the cost of this iterative inner loop at inference time, we additionally train a lightweight GS Predictor Net that regresses the primitive parameters in a single forward pass. The encoder and decoder are trained end‑to‑end through the inner loop, with no quantizer anywhere in the training pipeline: the bottleneck is the decomposition itself, and scalar quantization is applied only post‑training to the fitted parameters. Rather than relying on discrete codebook stages for bitrate control, our representation exposes a fine‑grained rate‑quality tradeoff: a single trained checkpoint supports post‑training bitrate control by varying the number of primitives and the per‑parameter bit depth, with no retraining required. GS‑Codec matches or exceeds well‑established open‑source codecs such as EnCodec and DAC on speaker similarity (SIM), intelligibility (STOI), and perceptual quality (UTMOS) at comparable bitrates, while achieving comparable semantic performance (WER). Code and audio samples are available at https://ronaluf.github.io/gs‑codec/
Authors:Michael Rathmayr, Ádám Kovács, Gábor Recski
Abstract:
Detecting responses that retrieval‑augmented generation does not ground in its context trades speed against accuracy: surface checks miss paraphrased fabrication, sampling‑based methods cost extra generations. Hidden‑state probes sit between the two, but every existing one reads the generating model's own activations, so a change of generator invalidates the detector and a closed‑weight generator is out of reach. This paper removes that coupling. The Grounding Probe is logistic regression over the mean‑pooled middle‑layer hidden states of an observer language model that reads the context, question, and response in one forward pass and generates nothing, with the recipe it needs: pool over response tokens, read a middle layer, and control capacity, which closes the train‑test AUROC gap from 0.087‑0.202 to 0.009‑0.013. Asking the observer outright, rather than reading its hidden state, costs at least +0.166 AUROC in every one of four models. Fitted on 15,090 annotated responses it reaches 0.879‑0.894 AUROC on RAGTruth test across four observers, and 0.924 AUROC with 0.820 F1@0.5 averaged with a supervised span detector, 0.060 above that detector alone. One probe holds across six generators, and hold‑out controls, including one in which no evaluation prompt appears in training, bound the cost of removing a generator at about 0.02 AUROC. Code, probes, and predictions are released.
Authors:Zhe Tao, Feiran Wang, Gaowen Liu, Ramana Rao Kompella$, Yan Yan
Abstract:
Robots need to anticipate how their actions will change the world, since manipulation success hinges on the resulting contacts and object motions. However, existing Vision‑Language‑Action (VLA) policies that predict future observations from shared features leave the forecast decoupled from the actions the policy will actually execute, and impose no physical constraints on how the scene may evolve. We introduce ForeAct3D, a framework for policy‑grounded future world modeling within VLA policies. Learnable geometric queries decode depth, semantic segmentation, and camera pose from the policy representation into current and future semantic 3D scene states, and the future queries are conditioned on the policy‑generated action chunk to ground the forecast in the planned interaction. A physical‑consistency closure relates the two states through background staticity and instance‑level rigidity, and anchors the wrist‑camera pose to end‑effector kinematics. These objectives shape the shared representation used for action generation during training, and no future prediction is required at inference. Without robot pretraining, ForeAct3D achieves 98.3% average success on LIBERO and an average task length of 3.73 on CALVIN, outperforming its base policy on every suite. Ablations show that semantic 3D supervision, physical consistency, and action conditioning each improve manipulation performance, and that action conditioning substantially improves future object localization. Real‑world experiments on spatial placement, object insertion, and sequential manipulation further raise average success from 6.7% to 37.8% over the base policy. The project page and code are available at https://github.com/anthonytao80‑crypto/ForeAct3D.
Authors:Xiaoming Gong, Ruoyu Wu, Zhenhong Sun, Chunlin Chen, Daoyi Dong, Huadong Mo, Zhi Wang, Hongdong Li
Abstract:
Primitive‑based 3D segmentation offers a compact and explicit alternative to dense surface prediction, naturally supporting structural abstraction and boundary localization. However, geometric decomposition alone does not determine how primitives should be organized into semantic parts: a single part may span multiple primitives, while geometrically similar or touching primitives may belong to different parts. We therefore introduce RePart (Reinforcement Part Reasoning), which formulates primitive‑to‑part organization as a finite‑horizon Markov decision process and learns semantic organization through trajectory‑level reinforcement reasoning. RePart constructs a Composable Primitive Workspace from fine‑grained superquadrics and applies a merge‑and‑stop policy whose decisions are optimized by their downstream effects on the resulting partition rather than local primitive compatibility. The inferred part identities are then mapped back to the original mesh through Boundary‑Aware Surface Labeling, preserving accurate surface boundaries beyond the primitive approximation. On PartNet, RePart achieves the strongest results across all four aggregate partition metrics; on 3DCoMPaT++, it obtains the highest RI and SC without target‑dataset fine‑tuning. These results demonstrate that reinforcement reasoning provides an effective mechanism for organizing geometric primitives into semantic parts while retaining dense segmentation accuracy. Code is available at https://github.com/EngineeringAI‑LAB/RePart.
Authors:Francesco Correnti, Gabriele Magrini, Marco Mistretta, Niccolò Biondi, Pietro Pala, Alessandro Ramalli, Simona Fiori, Andrew D. Bagdanov, Matteo Lenge
Abstract:
Magnetic Resonance Imaging (MRI) is widely regarded as the optimal sensor for fetal brain analysis due to its superior soft‑tissue contrast and anatomical detail. However, its high cost and operational burden make it invasive and difficult to obtain at scale. Ultrasound (US), in contrast, is cheap, safe, and routinely acquired, and as a result it has produced substantially larger datasets and a growing ecosystem of pretrained models. This asymmetry raises a natural question: Can we teach a US‑only model to understand fetal MRI from only a limited set of examples? The standard recipe, training a foundation model on subject‑to‑subject paired MRI‑US scans, is not viable since no such paired fetal dataset is publicly available. In this paper we address this gap with Weakly Aligned Spatiotemporal Pairs (WASP), a framework that formulates cross‑modal correspondence as an entropic Optimal Transport problem driven by clinical metadata, in particular Gestational Age (GA) and diagnostic planes, enabling the fitting of a lightweight alignment module that lifts MRI representations into the US latent space, without fine‑tuning the backbone. Empirically, WASP yields its largest gains when MRI is unseen by the model during pretraining (on USFM, GA estimation error drops from 21.9 to 17.4 days and standard plane classification accuracy climbs from 61.9% to 69.0%), while providing smaller, backbone‑dependent refinements for backbones pretrained on both modalities (e.g., BioMedParse GA estimation error from 6.5 to 6.0 days and SAM‑Med2d plane accuracy from 83.3% to 88.1%). Code is available at https://github.com/miccunifi/WASP.
Authors:Hoigi Seo, Byung Hyun Lee, Minjun Kim, Dohyun Mah, Jongho Lee, Se Young Chun
Abstract:
Multi‑modal large language models (MLLMs) achieve strong modality understanding by pairing a large language model (LLM) with an encoder for a target modality such as vision, video, or audio. However, improving an MLLM's capability for a given modality typically requires additional training on large modality‑specific datasets, incurring substantial data collection and compute costs. Model merging offers an alternative, but it is often infeasible for data‑scarce, large per‑sample size, or domain‑specific modalities (e.g., audio and video), where same‑modality model variants are rarely available. In this work, we characterize an intriguing asymmetric phenomenon: merging a well‑aligned, data‑rich source‑modality MLLM into a data‑scarce target‑modality MLLM substantially improves the target on its own benchmarks. Our theoretical and empirical analyses show that this gain stems from enhanced alignment between modality‑specific and textual tokens, induced by the stronger donor modality. Specifically, we derive a mutual‑information lower bound that is monotonic in alignment‑related quantities and strongly correlated with downstream MLLM performance. Building on this principle, we propose Directional Cross‑modal Alignment Transfer (DCAT), a novel framework that transfers textual alignment from a strong, well‑aligned source (donor) modality to a weak target (recipient) modality, boosting target‑modality performance without further fine‑tuning. We further show that the alignment‑enhancing objective admits a closed‑form weight‑space solution computed from only a small calibration set. DCAT outperforms existing model‑merging methods, offering an efficient path toward cross‑modal alignment transfer. Project page with code is available at \urlhttps://seohoiki3215.github.io/DCAT_project_page
Authors:Tim Aebersold, Soheyl Massoudi, Mark Fuge
Abstract:
Precise constraint satisfaction is a prerequisite to deploying learned models in many areas, motivating methods that repair raw neural predictions with a repair procedure. Current methods unroll multiple repair steps in training and softly penalize constraint violations that remain after the unroll. This is compute‑ and memory‑intensive, lacks robustness when the repair fails to converge, and surrenders most of the constraint satisfaction work to the repair. Our central finding is that, contrary to common practice, a single detached projection step suffices in training. We accomplish this with a Projection‑Adaptive Loss (PAL), which uses the constraint residual after this single step to adaptively weigh constraint penalties on the raw prediction. In experiments, PAL is the only method that retains virtually perfect feasibility on extremely nonlinear constraints, and matches or outperforms current methods on synthetic and engineering benchmarks. Because it only requires a single detached projection step, PAL trains 2.5x faster than the canonical repair‑based method (DC3) on its own ACOPF benchmark. PAL can also be trained when constraints are expensive to evaluate (e.g., via neural surrogates), a setting where current unrolled methods are memory‑intractable.
Authors:Bowen Chai, Tianbao Zhang, Shuyu Wu, Dexin Zuo, Zhaoxin Fan, Danping Zou
Abstract:
Recovering detailed geometry from high‑resolution images is critical for precise perception of the surroundings and objects. However, existing methods which use latent‑space modeling and VAE reconstruction can compromise geometric details. Furthermore, decoding from latent codes introduces substantial inference overhead. To address those issues, we present EagleDepth, an efficient framework for high‑resolution monocular depth estimation that combines the geometric priors of latent diffusion with fine‑grained pixel‑space generation. Our key idea is to retain depth‑aware latent representations as guidance while generating the final depth map directly in pixel space. We train the latent and pixel components sequentially: first, we fine‑tune a pretrained latent diffusion model using paired RGB‑‑depth supervision; then, we adapt a pretrained pixel diffusion decoder, PiD, to predict depth conditioned on the learned features. Training of the pixel component starts at 1024 resolution and continues across multiple resolutions up to 4K. The latent branch processes resized, lower‑resolution RGB images, while the pixel branch generates depth at the target resolution, bypassing the original VAE decoder. This design preserves learned geometric knowledge without requiring the latent backbone to operate at the output resolution. On five commonly used depth estimation datasets and the high‑resolution Synth4K dataset, our framework achieves state‑of‑the‑art depth estimation performance, with faster inference and better preservation of fine structures and object boundaries.
Authors:Amir Sabbaghziarani, Bradley Thomas Baker, Theodore J. LaGrow, Sergey Plis
Abstract:
Many groups now delegate label ontology and metadata harmonization to agentic LLM pipelines. We built one and audited it. Our aggregate scores looked healthy, but the pipeline kept failing in ways they did not show, so we set out to find what they hid. Label agreement asks whether a proposed label matches a reference. It does not ask whether the agent was entitled to propose it, whether the output was complete enough to act on, or whether the label moved when the evidence moved. We measured those three separately on COBRE and FBIRN, two schizophrenia and control neuroimaging cohorts from different consortia, and they come apart, from label agreement and from each other. Showing the agent an upstream proposal barely moves label agreement, 0.857 to 0.870, while agreement on the chosen action doubles, 0.409 to 0.830. Output that parses as JSON still drops a required field on 10% of one model's cases and 33% of the other's. And an agent that replays its first answer scores perfectly on original cases and zero once we change the evidence that decides them. Downstream, a row‑order error that none of these metrics reports erases most of the diagnostic signal. So we measure these properties apart, pair each with a control, and gate commitment on the result, which makes failures visible and easy to route to a person. None of this prevents failure. Our reference labels are rule‑derived, so agreement with them means consistency, not correctness. Code is available at https://github.com/amir‑sbg/Label‑Agreement‑Does‑Not‑Measure‑Authorization.
Authors:Kaipeng Xu, Xianli Yan, Yan Wang, Xiang Liu, Shan Liu
Abstract:
Deep time‑series forecasting models have rapidly diversified, yet adapting them to a specific task still requires extensive expert effort in model selection, mechanism diagnosis, architecture design, implementation, and evaluation. Existing AutoML methods are constrained by predefined search spaces, while general‑purpose LLM research agents lack reliable control over experimental protocols and model promotion. We introduce EvoCast, a fully autonomous research‑agent system for iterative forecasting architecture evolution. EvoCast first establishes and diagnoses a task‑specific baseline through executed mechanism ablations, then generates evidence‑grounded research directions from dataset characteristics, diagnostic results, prior rounds, and failure records. Its central design, cognition‑authority separation, assigns open‑ended hypothesis generation and code implementation to LLM agents, while deterministic program authorities control source‑edit boundaries, canonical evaluation, and promotion decisions. Experimental outcomes are accumulated as evidence to guide subsequent rounds. Results show that EvoCast completes complex architecture modifications with higher implementation success and lower agent‑side token/time cost, and develops task‑specific architectures that outperform selected baselines, strong forecasting models, and agent baselines in three real‑world forecasting cases. The code is available at https://github.com/18e0‑x/EvoCast.
Authors:Hasan Farooq, Murtaza Taj, Mehwish Nasim, Arif Mahmood
Abstract:
Medical Vision‑Language Models (Med‑VLMs) have demonstrated strong capabilities in clinical tasks. However, they often struggle to understand anatomical structures and spatial positioning, which are crucial for medical reasoning. To address this, we propose a localization‑aware enhancement to the Med‑VLM pipeline, introducing improvements at three levels: data,architecture, and alignment. First, we introduce localization lens, a set of expert‑validated representations that provide richer anatomical and positional context. However, as these representations increase input complexity, we integrate pixel shuffle within the model architecture to filter and refine representations, enhancing spatial information processing while preserving anatomical continuity. Lastly, to effectively align the localization lens representations with textual features, we incorporate decoupled contrastive loss (DCL) alongside the standard loss function. This ensures better feature discrimination and robustness, particularly in data limited medical settings. Through extensive evaluations on medical visual question answering (Med‑VQA) datasets, we show that our methodology improves localization‑driven performance across different Med‑VLM architectures. Our analysis of localization‑based questions further reveals that improvements in anatomy and spatial reasoning directly enhance the overall accuracy of Med‑VQA upto 6.2%. The proposed approach is model‑agnostic and can be seamlessly integrated into existing Med‑VLM pipelines. The dataset, code, and trained models will be made publicly available at https://github.com/CVLABLUMS/localizationlens.
Authors:Chenhang Cui, Xu Xie, Linrui Xu, Xiaohao Liu, Xingyu Zhu, Fei Shen, Tat-Seng Chua
Abstract:
As large language models (LLMs) grow in parameter count, model storage and parameter memory traffic have become major bottlenecks to efficient deployment. Codebook‑based weight quantization reduces these costs, but imbalanced nearest‑codeword assignments during fitting can leave some codewords insufficiently updated, limiting effective codebook utilization. To address this limitation, we propose Balanced Assignment Refinement for Quantization (BARQ), which improves quantization quality through balanced fitting of existing codebooks. Specifically, we first compute joint soft assignments between weight blocks and codewords through entropically regularized optimal transport with uniform marginals and curvature‑weighted reconstruction costs, ensuring equal positive fitting mass for every codeword in the exact solution. We then refine the codewords through an assignment‑weighted barycentric update, which we prove minimizes the fitting objective for fixed assignments. For finite Sinkhorn iterations, the implemented update retains this optimality provided all codeword masses exceed the denominator floor. Finally, we discard the soft assignments and use the refined codebook for standard hard nearest‑codeword encoding, with our analysis establishing sufficient conditions for reducing hard‑quantization distortion and evaluation loss. Across multiple LLMs, BARQ achieves lower perplexity and higher mean zero‑shot accuracy than the evaluated baselines at comparable bit budgets. The code is available at https://github.com/chenhangcuisg‑code/BARQ.
Authors:Xiaosu Su, Yun Cao, Yiping Ni, Xiaowei Yi
Abstract:
Partial‑spoof detectors must reject synthetic content while accepting benign edits. We diagnose temporal anchors and boundary‑consistency training using frozen WavLM features, nonlinguistic unit controls, and source‑locked evaluation. Genuine‑genuine and genuine‑fake splices contrast editing false alarms with synthetic‑content misses; they do not isolate a unique causal artifact. With layer‑6 features, phone units have higher PartialSpoof evaluation frame EER than soft frames (11.18% versus 9.59%). In 500 retrospective PS‑eval cases, augmentation reduces genuine‑splice false alarms; synthetic‑core misses increase at 5% PS‑development FPR but not demonstrably at 1%. A 200‑case Llama A follow‑up retains the false‑alarm reduction but does not establish a miss‑rate increase. Constructed‑case ranking can improve while fixed‑threshold misses rise. Consistency gives no uniform gain, and encoder‑layer rankings change across corpora. We report frame and event localization with three seeds, including word units, and treat PS evaluation as retrospective. Audit code, selected training scripts, and result summaries are available at https://github.com/mysxs/partial‑spoof‑diagnostics.
Authors:Leyao Gu, Yingjie Xiong, Zirui Tang, Jiangtao Zhou, Yeye He, Chunwei Liu, Xuanhe Zhou, Fan Wu
Abstract:
Spreadsheet form filling requires agents to consolidate external evidence, ground values to precise cells, and preserve irregular template structure. Errors in early edits can overwrite labels or misalign fields, undermining later decisions. We propose CRAFT, a template‑aware agent framework that connects reflective validation to constrained local repair. Instead of treating reflection as a free‑form request to regenerate the workbook, CRAFT grounds detected errors to spreadsheet regions, restores corrupted template state when necessary, and re‑grounds plausible writable slots before subsequent edits. A Rectangle‑Aware Slot Grounder (RASG) proposes writable cells, while label‑slot hints and protected regions constrain subsequent edits. We introduce FormFillBench, with 327 forms across Instruction‑Only and Multi‑File tracks. Compared with the strongest baselines, CRAFT improves pair accuracy by 8.51 and 23.38 percentage points on these tracks, respectively. Component‑removal experiments support structural adjudication and slot re‑grounding within the pipeline, and the framework retains its relative advantage among the methods evaluated with a second backbone. The code and benchmark FormFillBench are available at https://github.com/Glllllly/CRAFT.
Authors:Yangze Liu, Zhongyi Han
Abstract:
A coding agent is a language model wrapped in a harness: the system prompt, the tool set, and the context management that turn a chat model into something that can work inside a repository. Production harnesses ship releases daily, vendors advertise pass‑rate gains from harness changes, and leaderboards mix harnesses freely. What is rarely measured is how much the harness itself moves the score when the model is held fixed. We run five models through three production harnesses, Claude Code, mini‑SWE‑agent, and OpenCode, on SWE‑bench Verified, and rerun the same configurations to calibrate how much a score moves when nothing changes but the run. On 447 tasks and the two models we ran there, Claude Code and mini‑SWE‑agent, the heaviest and the lightest harness, are equivalent within five points. On a 45‑task hard subset and five models, swapping the harness flips as many tasks as rerunning the same harness, 13% in both cases, and the tasks a harness wins in one run are not the tasks it wins in the next. The one harness effect that clears the noise is a loss, not a gain: OpenCode trails by up to 9 points on the large pool, and on one model half of that gap sits in runs its output cap cut short. What the harness does decide is the bill. With the same model, the same tasks, and one price list, cost per task differs by up to 3x across harnesses. The gap is set at the first call, by the preamble of system prompt and tool schemas each harness sends with every step, and scaled by the number of steps; per‑step growth and per‑call tool output differ far less. The provider's price for cached input scales the bill and does not reorder it. The rerun data also give the resolution a harness comparison needs: at the discordance we observe, 45 tasks catch a 13‑point gap only half the time and no gap with 80% power, and 447 tasks resolve 5 points, still coarser than the gains many harness changes claim.
Authors:Jinzhou Tang, Zijun Zhang, Jing Yang, Yuchen Yan, Kun Zhou, Lingjun Mao, Ruobing Han, Jinglin Cao, Wenpeng Xu, Lukun He, Minghao Fu, Fan Feng, Biwei Huang
Abstract:
Building interactive simulators from real‑world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manual environment construction and calibration. We study whether frontier foundation models and coding agents can automate this process end to end. We formulate \emphautonomous video‑to‑simulation as a software engineering task in which an agent observes an embodied video, constructs the corresponding simulated environment and robot behavior, and iteratively refines the result through execution feedback. To evaluate this capability, we introduce Video2World, a benchmark comprising 222 reconstruction instances derived from 189 robot and human demonstration videos. Video2World measures reconstructed worlds along geometric fidelity, dynamic fidelity, and functional correctness, capturing spatial perception, physical reasoning, and executable interaction. Evaluating 9 frontier coding‑agent systems reveals a sharp improvement in Task success beginning with Claude Opus 5, rising from below 5% to over 15%, while substantial gaps to human‑assisted reconstruction remain. We further find that worlds that look better could work worse: better visual fidelity does not always lead to higher task success. This echoes the broader gap between perceptual realism and factual correctness observed in generative models.
Authors:Yangze Liu, Zhongyi Han
Abstract:
An agent that lacks the information it needs should ask rather than act, and the task definitions of agent leaderboards say so. BFCL multi‑turn builds two of its four categories around a turn on which the model is supposed to ask, and its scorer never looks at that turn: the gold trajectory there is empty, the checker skips it, and the scripted user cannot answer, so asking earns nothing, guessing costs nothing on that turn, and asking twice loses the item. The benchmark also contains the control experiment for that decision. A should‑ask item is a base item with one piece of information removed from one turn, so the same request appears twice at the same turn index, once complete and once not: on the first the model should make the call that changes the world, on the second it should ask. We score one decision per pair, whether the model attempted a world‑changing call on that turn, read off the stored trajectories with no LLM judge; acting always and asking always both score 50. On the 223 pairs that pose this decision, gpt‑5.4 attempts the call on 83.4% of the complete turns and holds back on 78.0% of the incomplete ones, the best decision accuracy of seven models at 80.7%; on the same items the official score ranks it sixth and puts first a model that lands in the middle here. One added line telling gpt‑5.4 not to ask pushes it toward acting on both sides of the pair, so its decision accuracy shows no detectable change, while its official score rises by 13.5 to 23.5 points on the two should‑ask categories and on the base twins; the opposite line, telling gemma‑4‑31B‑it to ask first, improves its decision by 4.5 points and gains no official score. The score moves with the push toward action, not with the decision. We release the pairs, a turn‑level scorer that runs on any BFCL output directory without an API key, and 31 manually verified bad items.
Authors:Saeed Abdul Muizz, Aayat Rafiq, Iqra Altaf Gillani, Janibul Bashir
Abstract:
Machine unlearning seeks to remove the influence of designated training data from a trained model without retraining from scratch. Retrain‑free methods such as Selective Synaptic Dampening (SSD) and its label‑free variant LFSSD avoid full retraining but still require backpropagation and parameter importance computed over the entire dataset. We ask: what happens when a deletion request arrives with only a few images of the class to be forgotten? To answer this question, we introduce UnAct, a gradient‑free class‑unlearning method that needs only forward passes over the forget images. UnAct scores late‑layer units by their responses, attenuates the most responsive connections, and repeats this for up to 20 rounds using no gradients, no labels, and no retained data. On ResNet‑18 trained with CIFAR‑10, CIFAR‑20, and CIFAR‑100, UnAct is competitive with SSD and LFSSD when forgetting entire classes and, unlike them, never collapses the network when forget data is scarce. On ResNet‑18, across all tested sizes, UnAct's retain accuracy stays within 2.5 points of retraining, while SSD and LFSSD, at their full‑class operating points, lose up to 86 points on some classes. With five forget images on CIFAR‑10, UnAct's distance to retraining is 0.21 points, against 67 for LFSSD and 90 for SSD, and re‑selecting SSD's threshold at each size with an oracle does not close the gap. In preliminary transfer to ViT‑B/16, UnAct's distance to retraining is 11.5 against 33.7 for SSD, and a request is 19x faster than SSD when SSD computes its importance at request time. The code is available at https://github.com/abdulmuizz0903/UnAct
Authors:Abdulla Alshehhi, Zongyan Han, Rao Anwer
Abstract:
Agricultural visual models are typically evaluated for either recognition or localization, but reliable diagnosis requires identifying what is present and localizing the evidence. Agricultural visual question answering (VQA) datasets carry rich semantic labels but rarely link them to image regions, and adding such annotations by hand is costly at scale. We introduce AgroGround, a large‑scale dataset for grounded agricultural recognition: identifying plant diseases and other agricultural targets and localizing their image regions. An automated pipeline converts the labels of eight agricultural VQA datasets into annotations for disease lesions and whole objects, producing 794,850 instruction examples. Healthy images provide negative supervision for disease queries, teaching the model to return empty predictions. We fine‑tune a shared vision‑language model on known‑target grounding instructions combined with instructions requiring both recognition and localization. We evaluate predicted identities, regions, joint correctness, and healthy‑image abstention on 1,480 human‑verified images disjoint from all training data. Grounding‑only fine‑tuning reduces recognition accuracy from 51.8% to 29.1%, while adding recognition‑and‑localization instructions raises it to 72.6%. With images and annotations held fixed, combining the two formats raises joint accuracy from 19.2% to 43.3% at comparable grounding. Healthy negatives raise abstention on healthy images to 95.0%, and reinforcement learning improves lesion‑level grounding. The resulting 2B model exceeds its annotation teacher in grounding F1 on our benchmark and on the external PlantSeg test set. AgroGround establishes a benchmark for grounded agricultural recognition, measuring joint correctness of identity and localization along with abstention on healthy images. The code is available at https://github.com/AB‑Abdulla/AgroGround.
Authors:Saurabh Kumar Singh, Yogeshwar Singh Dadwhal, Malhar Vedak
Abstract:
Post‑training quantization to the GGUF format's mixed‑precision K‑quants is commonly how open‑weight language models reach consumer hardware, yet its effect on fine‑grained lexical competence is uncharacterized. We audit 27 quantized artifacts across 13 families and four architecture backbones, 0.35B‑14B parameters, evaluated down their published ladder to Q2_K (about 2.6 bits per weight), on 429 frequency‑validated rare English words under two probes: surface inclusion of a prompt‑supplied word and its one‑sentence definition, scored by a tiered multi‑synonym matcher, its error measured by a blind LLM‑judge census of every definition, with human verification. Three regimes emerge at Q2: total collapse into unusable builds, severe semantic dissociation in sub‑2B models, and mostly robust preservation above about 3B. In every sub‑2B artifact, definitions fall 20‑67% below the artifact's baseline, typically several times the inclusion loss. Two controls separate rarity from task difficulty: within the rare set, loss rises with rarity in six of seven sub‑2B artifacts, and on a 100‑word common‑word set rare words lose more than common words in all eight, significantly in six. Tokenizer vocabulary size does not predict the damage (Spearman rho=0.12); parameter count dominates (rho=0.72), confirmed within five of six same‑tokenizer families. Q4_K_M remains lexically clean at >=1B. The damage is frequency‑graded, provider‑dependent, and not calibrated by WikiText‑2 perplexity: across nine artifact‑matched ladders, near‑identical Q2 penalties (44.7%/47.6%) separate an artifact keeping its definitions (3.6%) from one losing them (43.6%). Aggressively quantized small models can keep generating fluent text while no longer knowing what it means, risking hardware‑constrained deployments in domains where semantics carries consequences. Validation must be per artifact.
Authors:Zhanxun Liu, Yifan Duan, Hengtao Wu, Chen Yang, Qinyuan Cheng, Kun Wang, Xingyu Zeng, Xipeng Qiu, Chaochao Lu, Xie Chen
Abstract:
General turn‑taking behavior in real‑time dialogue systems requires deciding whether to keep listening or start responding while listening, and whether to continue or stop while speaking. Existing turn detectors use heterogeneous, task‑specific label spaces and are often trained on limited annotations or evaluated on isolated utterances, making them difficult to use as a unified causal controller with comprehensive context. We propose XTurnix, a compact text‑based model that formulates turn control as two binary decisions conditioned on the AI's current listening or speaking state and predicts a single control token from the complete dialogue history. XTurnix is pretrained on 5.5 million causal action examples automatically derived from timestamped two‑speaker transcripts, then fine‑tuned on synthetic multi‑turn examples with a flatter distribution across the four state‑action labels. We evaluate XTurnix on four public benchmarks and a balanced self‑curated benchmark. Across the public benchmarks, XTurnix achieves the best results on all SemanticVAD and LiveKit splits, ties the native Smart‑Turn model on Smart‑Turn Bench, and achieves the highest incomplete‑turn accuracy on Easy‑Turn. On the self‑curated benchmark, it reaches 89.06% accuracy, more than 20 percentage points above the strongest third‑party baseline at 68.75%, while maintaining F1 scores between 84.21% and 90.63% across all four categories. These results demonstrate unified listening‑ and speaking‑state turn control in a single compact model. Code is available at https://github.com/xcc‑zach/xturnix, with an interactive demo at https://huggingface.co/spaces/xcczach/xturnix‑demo.
Authors:Nahom Birhan, Mehrdad Rostamzadeh, Sidhant Narula, Mahmoud Nazzal, Mohammad Ghasemigol, Daniel Takabi
Abstract:
Large language models increasingly mediate tool use in Model Context Protocol (MCP) systems, where adversarial influence may enter through user instructions, tool schemas, tool outputs, or protocol messages. Existing benchmarks often evaluate deployed agents, conflating model susceptibility with guardrails, orchestration, and general task capability. We introduce COPEX (COntext Provider EXploitation), a controlled benchmark that isolates the model as an MCP client by fixing the surrounding agent stack and varying only the tool‑selecting model. COPEX covers 25 attack types instantiated as 125 scenarios across four entry surfaces: model/agent, client, server/tool, and transport. Across nine models and 3,375 trials, the mean attack success rate is 64.4%, with surface‑level means ranging from 58.3% to 71.4%. Some client‑ and transport‑level attacks succeed partly outside the model's observation or control, separating system exposure from model susceptibility. Combined input and context scanning reduces mean attack success by 49.6% on an eight‑attack defense subset relative to the undefended setting. The benchmark is available at https://github.com/inspire‑center/copex.
Authors:Xizhe Zhang, Jingfeng Zhang, Zirun Zhou, Hong Jia
Abstract:
Transferring visual representations pretrained on human videos to robot manipulation requires learning reliable correspondences between human and robot demonstrations. However, paired demonstrations can differ in execution rate and in the proportion of non‑key frames that do not directly reflect task progress. Frames at the same relative timestamp may therefore represent different task stages, which can cause correspondence learning to fail. To address these issues, we propose Frame‑Level Temporal Alignment (FLTA), a framework that uses two temporal priors to adapt visual encoders pretrained on human videos for robot manipulation. It learns shared task‑progress representations without frame‑level correspondence annotations, allowing frames at different relative temporal positions to match. A global progress prior combines normalized temporal positions with visual similarity to construct a soft correspondence target. A local temporal order prior penalizes backward transitions while allowing stays and varying forward rates to accommodate execution‑rate differences. With ResNet‑50 and ViT encoders, our method achieves relative improvements of 46.93% and 65.96%, respectively, over the best baselines in average simulation success rates and achieves higher task success rates on real‑world manipulation tasks. These results also suggest that effective human‑robot adaptation depends less on the number of parameters updated than on which parameters are selected. Our project page is available at https://rtx5090ultra.github.io/FLTA‑Project‑Page/.
Authors:Marharyta Shvets
Abstract:
Data quality now matters as much as compute for training language models. Much training data comes from human annotation of text, and interpretive annotation has no ground truth that could settle what is "accurate". Two lines of work respond to this. One combines annotators into a "ground truth" and measures how well they agree with each other; the other treats their disagreement as a signal. Both compare different people at one point in time. We measure something else: how well one reader reproduces their own reading of the same text over time. One expert human reader and three LLM families segmented three Sumerian myths and labelled the causal function of each segment with one of seven states. Across runs months apart, the human cut the text in much the same places but named the segments differently, in every myth. The models show no such consistent pattern: their gap between the two layers is positive in some myths and negative in others, and its size varies. The human's label changes are not random: the runs go through much the same functions but start them one step apart, while model runs start them at the same places. We argue that this pattern is a usable measure of data quality and a contamination check: a "human" annotation whose labels are as stable as its boundaries, and whose functions start in sync, looks like a model's.
Authors:Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini, Shengrui Lyu, Kamal Ndousse
Abstract:
Designing effective reward functions in RL typically requires substantial expertise and trial and error. Recent work automates this process with LLM‑based systems that generate and iteratively improve reward code using policy feedback. However, these methods are often hard to compare because they differ in implementation details, feedback assumptions, and evaluation environments. To address this, we introduce a Benchmark for Iterative Reward Design (BIRD) that expresses existing methods in a unified configuration and evaluation space. This lets us compare algorithms directly, ablate individual design choices, and prototype new components under matched feedback conditions and policy‑training budgets. Across MuJoCo, Meta‑World, Assistax, and HumanoidBench, we identify a small set of simple design choices that consistently improve performance. Combining these choices yields significantly better performance than the evaluated methods from prior work. Our results highlight the strength of simple baselines and motivate further study of when additional algorithmic complexity improves iterative reward design. Code is available at https://github.com/safety‑research/bird.
Authors:Yuhang Wang, Kai Luo, Yuanfan Zheng, Kailun Yang
Abstract:
Comprehensive 3D scene understanding for autonomous driving requires modeling geometry, semantics, and motion. However, camera‑based occupancy and scene flow prediction are sensitive to unreliable spatial and temporal aggregation, caused by semantically incompatible image features, misaligned historical observations, and incomplete voxel structures. To address this issue, we propose SelectOccFlow, a selective spatiotemporal aggregation framework that progressively refines contextual evidence across image, temporal, and voxel domains. To obtain semantically compatible image evidence, we design Semantic‑Guided Sampling (SGS) to regulate feature sampling with semantic priors. Since reliable image evidence alone cannot resolve temporal inconsistency, we then present State‑Conditioned Temporal Aggregation (SCTA) to selectively retrieve historical evidence according to voxel states. To further enhance the structural completeness of voxel representations, we introduce Extent‑Aware Spatial Aggregation (ESA), which exploits directional structural support to refine foreground geometry. Experiments on OpenOcc demonstrate that SelectOccFlow achieves a state‑of‑the‑art OccScore of 44.9, improving the previous best by +4.2%. It also maintains competitive occupancy performance on Occ3D‑nus and improves the mean OccScore under nuScenes‑C corruptions by +11.1%, demonstrating improved robustness to visual corruptions. The source code will be made publicly available at https://github.com/muchen1021/SelectOccFlow.
Authors:Haopeng Deng, Shuo He, Dayuan Wang
Abstract:
Calibrating microscopic traffic models for digital twins is an expensive black‑box optimization problem: tuning car‑following and lane‑changing parameters requires a full simulation run, affording only a tight budget per recalibration window. Matching raw trajectories yields a rugged objective that sparse surrogates cannot learn, reducing sequential acquisition to near‑random probing. We present FLAT, which couples what to optimize with where to sample next. An eight‑dimensional behavioral fingerprint smooths the parameter‑error landscape, making the objective learnable from a few dozen samples; annealed lower‑confidence‑bound (LCB) acquisition then spends each remaining run where it most reduces error. The surrogate, interchangeable among a Gaussian process (GP), random forest (RF), or multi‑layer‑perceptron (MLP) ensemble, plugs into the same LCB loop. Across six heterogeneous real‑world scenes, FLAT‑GP achieves the lowest scene‑averaged behavioral error, winning 6/6 scenes against SPSA, GA, and CMA‑ES and 5/6 against TPE under the matched budget. Some baselines need up to 4.4 times more simulations to match. Ablations show objective choice shifts final behavioral error by 81% on average, removing sequential LCB raises the six‑scene mean by 20%, and surrogate choice shifts it by at most 4.2%, confirming gains trace to objective geometry and sequential allocation rather than surrogate capacity.
Authors:Yinghao He, Mengyu Xu, Haixiang Sun, Donghan Li, Yibo Wang, Lixu Wang, Kezhen Chen, Chi Li, Chunwei Liu, Bharat Bhargava, Chongyang Gao
Abstract:
Reliable language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade‑off, we introduce CoPE‑Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training‑free framework that uses the model's own judgments to identify relevant tokens, suppressing user pressure and amplifying contextual information through token‑level attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases joint‑condition updating by 5.5 percentage points relative to the strongest baseline in the main comparison. In two‑turn dialogue, it improves joint‑condition updating by an average of 13.2 percentage points over the strongest prompting baseline. The source data and codes can be found at https://github.com/03Grant/sycophancy‑and‑stubbornness.
Authors:Xinyi Liu, Xuan He, Danny H. K. Tsang, Yize Chen
Abstract:
Machine Learning (ML) methods promise a fast solution process for Optimal Power Flow (OPF). While inconsistent test cases, implementations, and evaluation metrics across existing studies make it challenging to determine which algorithmic advances are most critical for real‑world deployment. To this end, we propose ML‑OPF‑Bench, a unified benchmark for AC‑ and DC‑OPF that evaluates representative ML algorithms under a consistent pipeline, stress‑tests them across system sizes, distribution shifts, and resource budgets, and ranks them with a multi‑objective framework. We find that prediction accuracy alone is not a reliable indicator of operational feasibility. Under heavily loaded, congested conditions, even the strongest in‑distribution performers lose their advantage, while feasibility is maintained largely by post‑processing that enforces the target constraints rather than by the underlying pure ML predictor. Data scaling shows that prediction accuracy and constraint violations follow different trajectories, whereas compute scaling shows that returns diminish and that larger models do not consistently perform better. These results expose critical trade‑offs among ML methods' speed, accuracy, and feasibility, and offer practical guidance for future ML‑OPF design. We open‑source the benchmark as an extensible Python package for integrating new learning‑based OPF algorithms and evaluating them under the same standard as the existing baselines.
Authors:Shiyu Feng, Xihua Sheng, Lingyu Zhu, Chunyang Fu, Shiqi Wang
Abstract:
Lossless triangle mesh compression must preserve both vertex positions and connectivity. Octrees support learned point cloud geometry coding and progressive refinement, but extending them to meshes requires a compatible connectivity representation. Unlike the eight occupancy decisions of a voxel, a parent edge can develop into varied child connections, making edge refinement difficult to model with a compact prediction prior. We propose OctMesh, a learned framework that codes geometry and connectivity on a shared octree hierarchy. Its key observation is that octree pooling produces parents with either one child or two to eight children. Child edges are then grouped by their endpoint parents' types and whether the endpoints share a parent. Each candidate group contains children from just one parent or two connected parents. The resulting four categories define small, fixed‑shape prediction tasks: connections uniquely determined by the parent graph are inherited without bits, while three neural predictors estimate probabilities for the remaining candidates. These probabilities guide arithmetic coding of the actual edge symbols. Binarized predictions of within‑parent connections provide context for predicting connections between different parents. A graph‑aware parent feature extractor combines local geometry, parent connectivity and global shape. The connectivity models use dedicated weights at coarse levels and share weights at fine levels. Residual edges and a finest‑level face‑selection payload complete the reconstruction. On 256 frames from eight MPEG V‑DMC test sequences, OctMesh losslessly recovers the finest‑level vertex‑coordinate, edge and unoriented face sets at an average of 7.033 bits per face, 12.8% below V‑Mesh. The same hierarchical representation supports nine levels of progressive vertex‑and‑edge refinement.
Authors:Zixuan Wang, Hao Li, Fengyu Gao, G. Edward Suh, Yi Zeng, Yevgeniy Vorobeychik, Ning Zhang, Chaowei Xiao
Abstract:
Large language model (LLM) agents are increasingly deployed in tool‑augmented environments, but their reliance on external inputs makes them highly vulnerable to prompt injection attacks that can hijack task objectives. Existing safety alignment methods rely on static expert trajectories or preference optimization, limiting their ability to generalize to adaptive attack patterns. In this work, we propose Self‑Reflection Fine‑Tuning (SRFT), a training framework that enables agents to improve robustness by learning from their own failure experiences under adversarial conditions. Instead of passively imitating expert behaviors, SRFT exposes the agent to compromised trajectories constructed via injected attacks, and leverages an expert model to generate structured self‑reflection reasoning that contrasts unsafe and optimal actions. This reflective supervision teaches the agent to identify malicious instructions, reason about their consequences, and maintain alignment with the original user intent. We instantiate this framework in SR‑Agent, built on Llama‑3.1‑8B‑Instruct and Qwen3‑8B, and evaluate it on both static and adaptive prompt injection benchmarks. Experimental results show that SRFT substantially reduces attack success rates while preserving task performance, and demonstrates strong generalization under adaptive attacks. These findings suggest that learning from failure via self‑reflection is a promising direction for building robust and secure LLM agents. Our code is released at https://github.com/Eden‑Wang1710/srft‑repo.
Authors:Xingyu Zhao, Yiqiao Zhong
Abstract:
Subliminal learning is a phenomenon where a student language model acquires a teacher model's behavioral traits by training on semantically unrelated outputs. It is a subtle statistical phenomenon as trait transmission relies on weak statistical patterns in the generated data. To understand trait transmission between teacher‑student pairs, we study cross‑trait transfer: how data generated under one teacher trait changes the student's preferences of other traits. To this end, we introduce a directed trait‑transfer matrix that quantifies these effects using log‑probability gains for student answers. We find that the trait‑transfer matrix reveals clusters of related traits, with students sometimes developing preferences for traits similar, but not identical, to the teacher's trait. Such cross‑trait structure can be partially captured by output distribution metrics and representation‑based metrics. Further, we analyze trait development and interaction: learning dynamics shows a progression from broad shared shifts toward more trait‑specific transfer, and multi‑trait experiments suggest that opposed traits can enhance such differentiation. Together, our findings reveal salient statistical structures over trait transfer and competition, thus providing a broader view of how hidden preferences are transmitted in subliminal learning.
Authors:Jiayi Xin, Evan Qiang, Zihan Zhu, Xiang Li, Weijie J. Su, Qi Long
Abstract:
Uncertainty quantification (UQ) for large language models (LLMs) aims to provide reliable measures of predictive confidence, yet current methods are often unstable under meaning‑preserving perturbations. Semantically equivalent paraphrases can induce substantial variability in predictive confidence, even for methods with formal guarantees, such as conformal prediction. To address this issue, we propose a paraphrase‑aware UQ framework robust to semantic rewordings. Our approach trains a lightweight proxy model on LLM hidden states and aggregates its predictions across paraphrases to construct label‑wise nonconformity scores. Under score exchangeability, conformal calibration retains marginal coverage. This guarantee can also hold under test‑only rewording, provided that the paraphrase pipeline satisfies an additional distributional alignment condition. We evaluate three settings (normal, fully reworded, and semi‑reworded) which apply rewording to neither dataset, both calibration and test datasets, or only the test dataset, respectively. Across seven multiple‑choice QA benchmarks and multiple model families, our method produces compact prediction sets with empirical coverage generally near the nominal target, even in the semi‑reworded setting. Ablation studies show that the learned proxy accounts for most of the reduction in set size, while paraphrase‑augmented training and inference‑time aggregation improve stability under rewording. Code is available at https://github.com/Raina‑Xin/PA_Score.
Authors:Yuxuan Li, Fanhu Zeng, Hao Tang
Abstract:
LoRA‑based continual learning methods mitigate catastrophic forgetting through various mechanisms, yet nearly all complement these with small learning rates as a heuristic to restrict gradient scaling magnitude. Such fixed heuristics lack theoretical guidance on how the strength of this restriction should evolve as tasks accumulate. We reveal that even under directional constraints such as nullspace projection, finite‑precision updates inevitably leak into the subspace of accumulated prior knowledge along multiple directions. While small learning rates attenuate such leakage, they cannot prevent the accumulated forgetting from intensifying as the effective rank of historical knowledge grows. We show that the optimal magnitude restriction should adaptively increase with this effective rank to balance stability and plasticity, i.e., preservation of previous knowledge and acquisition of new task information. Under an anisotropic leakage model, we derive a pacing law s^=\sqrtR/c that characterizes the optimal scaling of gradient steps, i.e., the magnitude restriction itself, where R is the effective rank of past updates. Based on this insight, we propose PaLoRA, which compresses historical knowledge via adaptive SVD truncation, projects gradients onto the nullspace of prior tasks, and applies rank‑aware adaptive pacing. Experiments demonstrate consistent improvements over prior methods, with particularly strong performance in long‑horizon settings, achieving substantial gains of 4% accuracy on challenging 50‑task ImageNet‑A and ImageNet‑R benchmarks.
Authors:Younghyun Noh, Minje Kim, Tae-Kyun Kim
Abstract:
Surface reconstruction under sparse‑view settings remains challenging due to limited geometric cues. Volume rendering methods based on signed distance functions often produce over‑smoothed surfaces, while 3D Gaussian Splatting (3DGS), though time‑efficient, suffers from incomplete geometry due to the lack of reliable depth supervision and the limitation of being optimized only from given input views. In this paper, we present Sparse‑GS2Mesh, a stereo‑aware framework for surface reconstruction from sparse views. While 3DGS and stereo matching have been leveraged for surface reconstruction under dense view settings, we extend them to operate effectively under sparse view conditions by first initializing 3DGS using epipolar depth priors to mitigate the 3DGS overfitting problem, followed by our three key components: (I) adaptive baseline selection, (II) fine‑tuning with a stereo matching network, and (III) 2D/3D co‑regularized fine‑tuning. Given a warmed‑up 3DGS initialized with epipolar depth, the adaptive baseline selection automatically determines a baseline to synthesize for each sparse view. We then fine‑tune 3DGS by backpropagating depth‑refining gradients from the stereo matching network, effectively specializing the 3DGS for stereo matching. The 2D/3D co‑regularization further helps obtain stable reconstruction, addressing weak geometric cues in close stereo views. Sparse‑GS2Mesh achieves a 15% improvement over state‑of‑the‑art methods in little‑overlap settings and comparable results in large‑overlap settings. Codes will be publicly available.
Authors:Feiran Wang, Bin Duan, Junyi Wu, Gaowen Liu, Yan Yan
Abstract:
Video world models aim to preserve scene structure and predict how dynamic objects evolve beyond visual observations. We present Kepler4D, a framework for future video generation through explicit 4D scene state evolution. Given a monocular video, Kepler4D constructs a shared 3D representation of background geometry, object motion histories, coarse spatial supports, and semantic context. Chain‑of‑Motion summarizes observed motion and uses a vision‑language model to select structured speed and heading decisions and decide whether to bound object‑center height from below. A deterministic rollout converts these decisions into future object trajectories for inspection and editing before synthesis. We render the evolving proxies into geometric controls for a pretrained video generator, separating coarse object motion from the synthesis of appearance and articulation. Experiments on real‑world videos demonstrate that Kepler4D enables controllable object motion and plausible future rollout while preserving scene consistency.
Authors:Seyedmahdi Kazempourradi, Ramtin Mojtahedi, Behrang Mohseni
Abstract:
Content‑provenance standards such as C2PA let a platform recover a stripped manifest through a soft binding: an invisible watermark read from the content, or a fingerprint looked up in a registry. We benchmarked both families under one protocol, restricted to openly available models whose licences we audited, on public media, with false‑match rates calibrated on held‑out negatives and source‑level bootstrap intervals for performance estimates. For watermarking we evaluated 25 image, 7 audio and 7 video configurations from 12 methods on perceptual quality, robustness, false positives and cost; for fingerprinting, 35 methods on registries of up to 98,985 images, partial edits and adversarial attacks. PixelSeal gave the best balance for image and video watermarks and AudioSeal for audio, but the error‑correcting detector of every TrustMark variant fired on 5.9 to 15.3% of unmarked images, so a verifier should test the expected payload. Among fingerprints, copy detectors trained on non‑commercial data detected up to 75.5% of transformed images at a pair‑level false‑match rate of 10^‑7 and DINOv2, the best permissively licensed method, 65.0%; near‑copies in a product catalogue dominated the false matches, and no detection improvement from geometric verification was observed under a matched calibration false‑binding constraint in the evaluated image pipelines. On the same attacked copies the two families failed differently: the union of watermark and fingerprint successes covered 69% of image copies; fingerprints covered more audio queries, whereas expected‑key watermark verification covered more video queries. Embedding a watermark moved the ISCC code of 62.0% of images past its match threshold, and platform‑dependent colour conversion changed watermark bits between x86 and ARM hosts.
Authors:Feiran Wang, Xiaoqi Wang, Ziwei Li, Wenbin He, Yan Yan, Liu Ren
Abstract:
Predicting future spatial states supports collision avoidance and timely decision‑making in dynamic environments. However, existing vision‑language models (VLMs) and benchmarks for spatial reasoning primarily focus on observed scenes, leaving predictive spatial reasoning beyond the observed interval underexplored. To this end, we introduce SpatialMind, a metric‑scale VLM for spatial reasoning and future prediction. Its metric depth adapter anchors spatial reasoning to real‑world scale, while its progressive state chain establishes current spatial states and observed dynamics as the foundation for future prediction. Given a video prefix, SpatialMind predicts distances, motion directions, and spatial relations in both observed and unseen future frames. For training and evaluation, we build a scalable data engine that grounds entity descriptions in metric geometry to generate question‑answer pairs and state supervision. Using this engine, we construct the SpatialMind‑30K dataset and the SpatialMind‑2K benchmark, both covering driving and everyday egocentric scenes. The benchmark spans eight tasks across three levels: current‑state understanding, observed‑dynamics understanding, and future prediction. Experiments show that SpatialMind substantially outperforms both general and spatially specialized models on our benchmark while achieving competitive zero‑shot performance on VSI‑Bench, OSI‑Bench, and VLM4D.
Authors:Sushrut Ghimire
Abstract:
Semantic Knowledge‑driven Contrastive Learning (SKCL) uses a language model to decide which classes are related, and pulls each image towards the prototypes of its semantic neighbours. On CIFAR‑100‑LT (beta = 100) it reports 54.02% top‑1 accuracy, 2.01 points above Balanced Contrastive Learning (BCL), the method it builds on. The code and the class descriptions are not public. I reimplement SKCL, BCL and ConCutMix in one framework, check it against the public baseline code, and run every configuration with three seeds. The two baselines reproduce within 1.5 points, but SKCL built on BCL, as the paper describes it, ends up 0.56 points below BCL. To find out why, I add SKCL to the authors' own ConCutMix code. Trained for the paper's 300 epochs, it reaches 53.69, only 0.33 below the published number. At the same budget, however, the semantic graph adds just 0.23 points over ConCutMix, while training ConCutMix for 100 more epochs adds 1.04. Together with ConCutMix's published lead over BCL (1.15), this explains the claimed gain. A BCL model that never sees the graph already shares 41.2% of the graph's top‑2 neighbours with its own most‑confused classes (2.0% by chance), which shows why the graph adds so little on these benchmarks. I also test several changes to SKCL. Combining it with the CutMix branch improves it by 1.06 points, and an adaptive version of the graph improves it slightly (+0.34 and +0.28 in two codebases), although these gains are within seed noise.
Authors:Jafar Badour, Maurice van Keulen, Elena Mocanu
Abstract:
Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Training (DST) promise to reduce these costs, but most implementations rely on binary masks over dense tensors and recover little of the theoretical compute, memory, or energy savings. We propose SNACK, a truly sparse GPU layer that stores and computes only non‑zero connections. SNACK exposes a simple PyTorch API for restructuring connections and backpropagating gradients entirely in the sparse paradigm, and ships SNACK‑COO, a custom COO‑format SpMM CUDA kernel with a batch‑to‑Streaming‑Multiprocessor mapping tuned for the small‑batch, high‑sparsity regime typical of large‑model training and single‑stream inference. At the kernel level, SNACK is up to 7x faster than the masked dense baseline (Dense+Mask) and competitive with cuSPARSE, Sputnik, and FlashSparse at 95% sparsity. At 90% sparsity, a single SNACK layer accelerates training by 8x and 3.7x, and inference by 4x and 2x, over Dense+Mask and fully dense layers, respectively, while using 72% less memory than dense and substantially less energy. End‑to‑end, SNACK reduces GPT‑2 peak training memory by up to 40% and graph‑style inference latency by 4.8x over Dense+Mask at 99% sparsity.
Authors:Abhinav Rajput, Acey Vogelstein
Abstract:
Coding agents operating under ambiguous instructions or user prompts must decide whether to ask clarifying questions or attempt a solution directly. While clarification from the user may improve the correctness of the agent's solution, each back‑and‑forth interaction incurs user and system costs, forming an explicit accuracy vs. efficiency tradeoff. Existing works study clarification behavior but do not train policies under enforceable clarification budgets; penalty‑based approaches typically require separate coefficient tuning swept across all clarification budget levels. We formulate clarification as a Constrained Markov Decision Process (CMDP) and post‑train Qwen2.5‑Coder‑7B‑Instruct with PPO‑Lagrangian to optimize coding accuracy, subject to an expected question‑budget constraint. Evaluated on HumanEvalComm with a GPT‑4o‑mini oracle simulator, the resulting policies reveal that untuned clarification behavior is Pareto‑inefficient: budget‑constrained policies can simultaneously achieve higher accuracy and lower clarification rates than the baseline model. Across budget levels, we observe a log‑shaped Pareto frontier with diminishing returns to additional clarification. Gains arise not from simply asking more questions overall, but from improved question targeting and better code generation under ambiguity. Without explicit supervision, trained policies learn to allocate clarification budget non‑uniformly, asking more frequently on tougher (multi‑degradation) tasks. These results suggest that unconstrained interactive LLM systems may systematically use clarification inefficiently.
Authors:Dmitry Kamenetsky
Abstract:
Balanced discrete optimal transport between n sources and n targets of unit mass is exactly the minimum‑cost assignment problem‑a bipartite perfect matching‑and is therefore solvable exactly by industrial matching engines in milliseconds to seconds. We ask when the exact approach beats the standard approximate alternatives, entropic Sinkhorn and its accelerated variant Greenkhorn, and make the sparse‑exact side certified by a textbook LP dual‑feasibility clip. Three contributions. (i) Measurement: on dense 2‑D instances, exact matching (Jonker‑Volgenant) is faster and strictly more accurate than either approximate method throughout the moderate‑n regime (0.01 s at n=500 to 11.5 s at n=8000); reaching a 1% quality target on the same hardware requires roughly 10‑80 min for Greenkhorn (factors 4e2‑6e4 over exact; plain Sinkhorn is 20‑650x slower still), a rough power‑law projection beyond the measured range. Greenkhorn's measured speedup over plain Sinkhorn is only 1.0‑1.5x on most converged cells. (ii) A simple kNN‑pool gap certificate: given a pool matching and its Blossom dual, a one‑pass O(n^2) clip produces a dense‑feasible lower bound; combined with the Sinkhorn dual potential (valid at every iterate, not just at convergence), the bound is valid on all 45 measured configurations and tightens monotonically with k. (iii) A multi‑robot task‑allocation sanity check where the discrete plan is the deliverable: per‑round exact assignment costs 0.1‑68 ms, while a Sinkhorn‑plus‑hardening pipeline costs 0.12‑15.9 s and accumulates 6‑27% extra travel over 15 rounds. The Sinkhorn family's large‑n dense regime is acknowledged and left untouched. Code, data, and results under MIT: https://github.com/dimkadimon/OT‑Blossom.
Authors:Yilin Zhuang, Noah Zambrano, Karthik Duraisamy
Abstract:
The quadratic attention cost of Vision Transformers (ViTs) forces a trade‑off between spatial resolution and rollout horizon, particularly for fine‑scale PDEs where shocks, reaction fronts, and material interfaces occupy small, evolving regions of the domain. Conventional neural surrogates also lack mechanisms to adapt resolution dynamically. We propose WAMRViT, a ViT that tokenizes inputs as balanced quadtrees using a wavelet‑inspired refinement criterion, jointly encodes position and refinement level with 3D rotary positional embeddings, and regrids in cell space during inference for stable long‑horizon rollouts. A multi‑scale variant retains each leaf at its native source resolution and lets the model learn across resolution levels. Unlike fixed‑budget adaptive‑tokenization methods, WAMRViT imposes no predetermined token count and supports fully adaptive topology throughout autoregressive rollout. To our knowledge, it is the first machine‑learning surrogate to natively tokenize multi‑level Adaptive Mesh Refinement (AMR) data. On uniform‑grid benchmarks, uniform‑patch WAMRViT improves finest‑level region‑of‑interest VRMSE over a finest‑patch uniform ViT while using substantially fewer tokens. The multi‑scale variant achieves the lowest first‑step full‑field RMSE and VRMSE on both benchmarks and improves rollout‑averaged full‑field and refined‑region accuracy at long horizons. With parallelized regridding, its end‑to‑end rollout cost lies between finest‑patch and approximately token‑matched coarser‑patch ViTs. On a complex AMR combustion problem whose finest features cannot be represented natively by the evaluated uniform‑grid baselines, WAMRViT operates directly on adaptive cells and substantially reduces finest‑level error at matched transformer capacity. Code: https://github.com/tonyzyl/wamrvit
Authors:Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek
Abstract:
Sparse Autoencoders (SAEs) began as an unsupervised tool for decomposing neural representations into sparse, interpretable features, and are increasingly used not only for passive analysis but also for active interventions such as unlearning, bias mitigation, and concept editing. A central challenge for these editing and steering methods is reliably matching features to target concepts; most current approaches address this by computing post‑hoc scores over an already‑trained, frozen dictionary. We instead introduce ClasSAE, a novel method that both automatically assigns classes to features and guides the encoder toward class‑separable representations during training. Specifically, we apply a differentiable top‑k operator to a trainable feature‑‑class affinity matrix with per‑feature budgets, coupling the features selected for each sample to the classes they are trained to represent. Because gradients flow through the selection of active features rather than only through their magnitudes, the encoder and the affinity matrix co‑adapt rather than being fit in separate stages. The result is a dictionary that is both class‑separable and class‑annotated, with no need for post‑hoc probing. We propose three variants for enforcing sparsity within this framework, which achieve comparable overall performance with slightly different trade‑offs. Using CLIP ViT‑L/14 embeddings on ImageNet, we show that the learned affinity matrix agrees closely with an independently estimated post‑hoc feature‑class matrix computed on held‑out data. The model also supports direct class prediction from the encoder and affinity matrix alone, without a separately fitted classifier, and its more class‑aligned encoder yields improved separation in Targeted Probe Perturbation evaluations. https://github.com/St0pien/ClasSAE.
Authors:Hangzhan jin, Mohammad Hamdaqa, Doina Precup
Abstract:
Reinforcement learning with verifiable rewards improves reasoning, while the allocation of learning signal shapes which solutions remain accessible under repeated sampling. Group‑relative objectives assign equal advantages to equally rewarded responses, making aggregate credit proportional to sampled mode frequency. We introduce Exploration‑Preserving Policy Optimization (ExPPO), a lightweight advantage‑shaping rule that redistributes credit using prompt‑relative, length‑normalized response surprisal and prompt pass rate. ExPPO combines bounded shaping with shared normalization to preserve verifier polarity and approximately maintain each prompt group's total absolute sequence‑advantage mass. Our analysis characterizes response‑level credit allocation alongside sampled mode updates, deriving local conditions for gains in entropy and correct‑mode discovery. Experiments show improved in‑domain and out‑of‑domain reasoning coverage, higher aggregate response accuracy, and strong coverage at large sampling budgets. A controlled multi‑answer evaluation further demonstrates increased correct‑mode yield and gains in diversity among verified‑correct responses. Code is available at https://github.com/jinhangzhan/ExPPO
Authors:Kushagra Gupta, Jingqi Li, Cade Armstrong, Lasse Peters, Ross E. Allen, Ufuk Topcu, David Fridovich-Keil
Abstract:
Zero‑sum Markov games arise in a wide variety of sequential decision‑making problems such as adversarial learning and planning against modeled uncertainties. However, prior work on finite‑sample guarantees on the learned state‑action value function (Q‑function) for zero‑sum Markov games is largely restricted to finite‑action settings, or to continuous‑action games in which agents have linear dynamics and quadratic rewards (i.e. linear‑quadratic, or LQ, games). A central challenge in extending such guarantees to more general continuous‑action zero‑sum Markov games is that the associated Bellman operator involves a minimax problem that need not admit a tractable saddle‑point solution. To this end, we first introduce a class of neural network function approximators for the Q‑function that is convex‑concave in the players' actions, which guarantees that this minimax problem admits a pure‑strategy saddle point. We then study an online variant of fitted Q‑iteration employing this function class and establish, to the best of our knowledge, the first finite‑sample guarantees for non‑LQ zero‑sum Markov games with continuous states and actions. Our code can be found at https://github.com/CLeARoboticsLab/QCanPlayThatGame.
Authors:Eeham Khan, Abdulmoumen Al-Atrash, Ali Ayub
Abstract:
Few‑Shot Class‑Incremental Learning (FSCIL) requires models to continuously learn new classes from limited samples while retaining prior knowledge, under strict constraints on compute and memory. Existing approaches lie along a difficult trade‑off: simple fine‑tuning is computationally efficient but suffers from catastrophic forgetting, replay‑based methods mitigate forgetting at the cost of substantial compute and memory, and exemplar‑free methods often reduce forgetting by freezing most of the backbone, improving efficiency at the expense of adaptability. We propose Selective Backpropagation (SBP), a deterministic parameter budgeting framework that bridges this gap. SBP restricts gradient updates to a pre‑allocated subset of network parameters, freezing past knowledge and preserving unbiased capacity for future learning, enabling rapid adaptation without costly mask optimization. We show that SBP achieves strong performance across standard FSCIL benchmarks while requiring training time close to that of naive fine‑tuning and substantially lower training time than prior SOTA methods. Crucially, our experiments expose a limitation of standard FSCIL evaluation: performance on short, distribution‑consistent benchmarks does not necessarily predict behavior under distribution shift or over substantially longer learning horizons. We therefore evaluate FSCIL methods in cross‑domain settings and over an 80‑session ImageNet‑1K stream. SBP remains strong across these regimes while maintaining low training cost, providing a favorable stability‑plasticity‑efficiency trade‑off. Our code is available at https://github.com/PaInt‑Lab/sbp‑main‑public.
Authors:Yuchen Li, Kaiyuan Deng, Chaoran Feng, Zhenyu Tang, Li Yuan
Abstract:
Text‑to‑video (T2V) diffusion models can reproduce copyrighted, violent, or explicit content, which motivates concept erasure: removing designated concepts from a pretrained model while preserving its behavior on everything else. Existing T2V erasure methods leave two problems open. Their frame‑agnostic suppression can leave isolated frames in which an erased concept resurfaces, a frame‑reactivation gap that clip‑level averages obscure; and they are usually evaluated with one target concept or category at a time. We propose Frame‑Aware Diffusion Erasure (FADE), a multi‑concept video unlearning framework. FADE first applies a joint closed‑form key/value edit that suppresses all target concepts, then trains per‑concept frame‑aware low‑rank adapters whose strength is gated by the frame index and the denoising timestep to remove residual per‑frame leakage. Each adapter is trained with the other targets' prompts as hard negatives, which keeps the concept‑specific components of different adapters well separated, and a similarity‑based soft router combines the adapters according to the prompt. With 16 concepts (objects, artistic styles, and nudity) erased from a single Wan2.1‑T2V‑1.3B backbone, FADE reduces the residual accuracy on the object benchmark to 4.9%, against 15.5% for the strongest of eight baselines, while keeping the VBench average within 0.9% of the unedited model. The ranking is unchanged under a VLM judge and a blinded human study, and the advantage over the strongest baseline carries over to prompts that combine several erased concepts, to 30 simultaneously erased celebrity identities, and to Wan2.1‑T2V‑14B, CogVideoX‑2B, and HunyuanVideo‑1.5.
Authors:Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi, Mohammadreza Pourmirzaei, Dong Xu
Abstract:
Autoregressive transformers remain comparatively weak for protein sequence and structure generation. We study the role of target representation: amino acid tokens encode residue identities without explicit contextual semantics, while backbone coordinates require a discrete representation in our framework. We introduce two learned latent protein languages. Protein Latent Language (PLL) maps sequences to a 4,096‑state contextual alphabet built on a frozen ESM‑2 encoder, with one token per residue. Structure Latent Language (SLL) adapts GCP‑VQVAE Lite with auxiliary sequence and confidence supervision while retaining decoding to backbone coordinates. We separately pretrain autoregressive transformer models on PLL and SLL tokens using next‑token prediction, yielding PLLM and SLLM. Under matched downstream sequence training, PLLM has a fitted compute‑scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model. In unconditional sequence generation, PLLM reduces the fraction of samples below a heuristic 1.5‑bit residue‑composition entropy threshold by 54% relative to the amino acid model across sampling temperatures. For sequence‑to‑structure prediction, replacing the original GCP‑VQVAE Lite tokenizer with SLL reduces best validation perplexity by 34% under matched training. For long proteins, latent‑token sampling is approximately 1,000 times faster than MSA‑based AlphaFold2 in our measurements. In backbone generation, SLLM compares favorably with other generative models on diversity and novelty. We also observe early signs that using SLLM's internal token confidence for inference‑time sampling can improve sequence‑to‑structure prediction quality beyond a single decoded sample. These results position learned latent protein languages as a promising substrate for autoregressive transformer scaling and inference‑time sampling in protein generation.
Authors:Huiyun Peng, Ricardo Calvo, Kelechi G. Kalu, James C. Davis
Abstract:
Software performance optimization requires identifying bottlenecks and improvement opportunities, and empirically evaluating the effects and trade‑offs across performance objectives. Autonomous AI coding agents submit performance‑oriented pull requests (PRs) to open‑source projects, yet it remains unclear whether these changes reflect established optimization practices and adequately substantiate their claimed value. Extending our MSR 2026 Mining Challenge pilot study of 407 performance PRs, we analyze a sample of 1,130 agentic and 1,130 human‑authored performance PRs collected through June 2026. We compare adoption outcomes and patch characteristics between the two groups, and examine their optimization practices and validation behaviors. The results show that agentic PRs are merged less often than human‑authored PRs (54.4% vs. 73.3%), but the two groups use similar optimization strategies and report validation at comparable rates. Agentic PRs rely more on static reasoning and less on benchmarks (46.1% vs. 57.2%), although benchmark use converges by the end of the study period. Agentic PRs also report validation for code‑smell refactorings as often as for resource‑targeting changes, whereas human‑authored PRs do so less often. Across both groups, about half of validated PRs report no quantitative performance metric, and relevant trade‑offs are rarely quantified. Agentic optimization practice has become increasingly similar to human practice, but the supporting evidence remains incomplete. Establishing whether agent‑proposed optimizations are supported by rigorous, quantitative, and multidimensional evidence remains an important challenge. These findings motivate evaluation infrastructure and review criteria that measure intended effects and relevant costs rather than treating the presence of validation alone as sufficient.
Authors:Durmus Karatay, Richard Newman
Abstract:
Many inference workloads evaluate a trained tree ensemble on row groups that share feature values: discrete‑time survival models expand each patient into G time steps, click‑through‑rate models score every item in a search session, and scenario analyses vary a few inputs while holding the rest fixed. Standard inference treats each row independently and repeats the shared work G times. We present TreeWalker, which applies partial evaluation to grouped inference: constant features are static, varying features dynamic. It walks each tree once per group, partitions a row bitmask at varying splits, and skips empty subtrees. Training is unchanged: TreeWalker reads standard LightGBM and XGBoost models. We prove a structural work decomposition: per‑tree work splits into the constant‑projected subtree size |T_c|, G leaf writes, and a predicate‑mask provisioning cost Q. For the trace evaluator, per‑row work approaches a (d_v+1)/(d+1) fraction of a row‑independent walk as G \to \infty. On Intel, TreeWalker is 2.5‑3.2× faster than a row‑independent traversal at the reference configuration (T=500, L=8) and 6.8‑7.8× faster at G=128 on the survival datasets, with larger gains on Arm. On a scenario‑analysis benchmark it is faster in all 16 configurations on both architectures. For f64 models, outputs match treelite's GTIL up to summation order; for f32 models, f64 accumulation is closer to a Kahan‑compensated reference than native f32 on 99.98% of rows and never farther.
Authors:Óscar Gómez-Cárdenes, José Gil Marichal-Hernández, Juan Manuel Martín-Doñas
Abstract:
Screen localization and pointing estimation are key to low‑cost interactive devices. Yet developing and evaluating these algorithms requires realistic data: synthetic captures cannot fully reproduce the optical distortion, rolling shutter, motion blur, and display processing of a physical acquisition, and most existing datasets provide static images rather than the video needed to assess continuous pointing. In this paper, we introduce the DABACO Dataset. Developed within the DABACO (Dispositivo Apuntador de BAjo COste, or Low‑Cost Pointing Device) project, this dataset supports the development and evaluation of screen detection and camera‑based pointing algorithms for embedded systems. It comprises video sequences captured with multiple low‑cost camera sensors, including monochrome global‑shutter and color rolling‑shutter modules, on embedded platforms such as the Raspberry Pi 4B and ESP32‑S3. We present a systematic annotation pipeline combining temporary visual watermarking, optical‑flow tracking, manual verification, and marker removal. Both the original marked captures and the marker‑free images, together with explicit corner annotations and modification masks, are released to support auditing and the study of potential reconstruction bias. In addition, we release an open‑source evaluation toolkit with two reference baselines: a classical screen‑detection pipeline based on edge and contour geometry, and a closed‑vocabulary, off‑the‑shelf YOLOv8 segmentation model evaluated without dataset‑specific training. Both are evaluated on a general‑purpose computer using Intersection over Union, corner error, and pointing error, establishing initial reference results for future embedded implementations. Overall, DABACO addresses a gap in existing resources and is intended to support the development and evaluation of new low‑cost screen‑localization and pointing systems.
Authors:Yukai Song, Yangfan Deng, Jijun Yin, Zhi-Hong Mao, Jingtong Hu
Abstract:
Traditional sleep‑staging methods apply the same model to every EEG epoch. Such uniform deployment expends computation on epochs that a smaller model could handle reliably, motivating cascades in which a primary classifier accepts its reliable predictions and defers the remainder to a more capable model. In this paper, we study the first stage of such a cascade: maximizing the coverage of fixed primary predictions subject to a prescribed accepted‑risk target. We propose COVER (COVerage‑oriented Error Ranking), which integrates two key innovations: (i) auxiliary‑informed primary‑error learning, which replaces maximum softmax probability (MSP) with a learned error score while preserving the primary labels, and (ii) fixed‑scale scorer refinement, which builds on this score to directly maximize coverage under an empirical accepted‑risk constraint rather than error‑prediction accuracy over all epochs. We evaluate COVER on Sleep‑EDF‑20 at a 5% accepted‑risk target, with subjects held out from all fitting and selection. Auxiliary‑informed error learning raises mean subject coverage from 31.5% for MSP to 48.8% at similar subject‑equal risk. At equal acceptance volume, with MSP accepting the same number of epochs as the learned scorer in each subject (20,639 in total), errors fall from 1,467 to 867. Fixed‑scale refinement then adds 1.7 percentage points of coverage over its initialization in nested development, and COVER attains the highest mean coverage among eight evaluated scorers, 50.4% at 4.5% subject‑equal risk, above the selective‑ranking baseline SELE (49.3%) and the probability‑fusion comparator DuoF (43.6%). To the best of our knowledge, this is the first work to combine auxiliary‑informed primary‑error learning with fixed‑scale coverage refinement for selective sleep staging, offering a basis for reliability‑aware allocation of computation in cascaded sleep staging.
Authors:Pawan Prakash, Philipp Höllmer, Addis Fuhr, Peter Hirschfeld, P. Ganesh, Stefano Martiniani, Richard Hennig
Abstract:
Inverse materials design is a long‑standing goal of computational materials discovery. Generative models for crystalline materials are typically trained to match the distribution of a structure database, while nothing in their training objective points them at specific design goals such as targeted properties. We use group‑relative policy optimization (GRPO) to align a generative model based on stochastic interpolants and discrete flow matching with general black‑box reward functions through reinforcement learning. Atom types are generated by a discrete flow and the policy gradient of our generalization of GRPO directly acts on the likelihoods of the atom‑type transitions, which differentiates our work from previous reinforcement‑learning approaches for diffusion and flow‑based generative models of crystalline materials. We introduce a reward function that raises the yield of metastable, unique and novel structures (mSUN) from 13.4% for the pretrained model to 45.5% for the reinforced model, as evaluated by a community benchmark. Our reward also improves the performance of a reinforcement learning framework for crystalline materials based on latent denoising diffusion models. At the same time, we find that directly reinforcing atom‑type transition likelihoods enables reward exploitation that has to be prevented with explicit guards. The same analysis also exposes a gap in the community metric. Single‑element structures in distinct packings are counted as metastable, unique and novel materials and inflate mSUN without yielding any new compounds. A stability claim is only as good as its reference hull. We report every result split by the number of reference phases behind it and argue that benchmarks should do the same.
Authors:Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov, Gül Varol, Fabio Pizzati, Ivan Laptev
Abstract:
Text‑driven human motion generation has advanced substantially, yet most methods assume instructions are available before synthesis. Interactive applications require responding to new instructions while continuing ongoing actions, such as answering a phone while walking. Existing approaches address streaming generation or simultaneous composition without explicitly combining streaming instruction arrival with independently timed, overlapping actions. We introduce streaming multi‑track timeline control and propose TimelineControl to incorporate new instructions alongside ongoing actions. Interval‑aware conditioning preserves instruction timing, while causal part‑structured representations and part‑aware denoising coordinate concurrent actions across body regions. We also construct TimelineMotion, a dataset with overlapping instruction intervals and body‑part annotations. Experiments on TimelineMotion and MTT demonstrate improved semantic alignment and temporal adherence over evaluated streaming baselines, including models retrained on the same data. Ablations and human evaluations validate our design, complemented by spatial conditioning and humanoid execution demonstrations. Our code, data and models will become publicly available.
Authors:Jianbin Zhang, Xin Sun, Shanwen Wang, Wei Ye, Susanto Rahardja
Abstract:
Knowledge distillation is crucial for deploying vision‑language models on resource‑constrained devices. However, existing methods typically impose uniform supervision across visual tokens or rely on static token selection, which confuses task‑relevant cues with background noise and degrades cross‑modal reasoning. To address this limitation, we propose Key Visual Evidence‑guided Knowledge Distillation (KVE‑KD), a framework that dynamically focuses feature distillation on task‑relevant visual tokens identified by the teacher model. Specifically, KVE‑KD appoints the final pre‑generation textual token as a unified semantic anchor and identifies the target cross‑modal fusion layer by analyzing changes in the anchor representation through iterative visual‑token contribution removal. Within this layer, KVE‑KD ranks visual tokens via the anchor‑conditioned attention distribution and selects the most informative visual tokens as key visual evidence with normalized entropy. The key visual evidence subsequently guides focused visual feature distillation, making the student align closely with the teacher's task‑relevant visual representations while suppressing irrelevant background information. Extensive experiments on six benchmarks demonstrate that KVE‑KD outperforms state‑of‑the‑art cross‑modal distillation methods, with particularly pronounced gains on tasks requiring complex reasoning and fine‑grained visual understanding. Importantly, these improvements are achieved without introducing any inference‑time overhead. The source code is available at https://github.com/zhangjianbin07/KVE‑KD.
Authors:Ioannis Tsiokos
Abstract:
Capable systems often show jagged competence: low average error alongside failures on particular inputs. We ask where it comes from in a task built from two known layers. A lower layer A computes five per‑slot sums from records; an upper layer B uses the slot‑1 sum and a mode carried over from earlier boards to predict the next board's category, so B cannot be computed from the current A alone: B is a strict extension of A. We train small recurrent networks on B and measure, against exact ground truth, what they acquire of A. The central finding is that learning B gives the network a jagged version of A, measured through probe readability and supervised outputs. It is readable where B needs it (slot 1, 97‑98% by a linear probe) and becomes less readable where B does not (the other slots, 5‑10%); category misreads concentrate near the slot‑1 cutoffs; and when A is trained explicitly it comes out only approximately right. The networks show jagged competence in B: natural KL below 3×10^‑4 bits coexists with a maximum law TV of about 0.25 on constructed histories. Matched experiments show that even a good A is not enough: connecting a learned A to B cuts misreads 1.3 to 17‑fold, the same exact A gives fewer misreads as one‑hot inputs (0‑2) than as numerical inputs (10‑317), one B update makes an exact A inexact unless the B gradient is blocked, and exact A still leaves some B failures. In this task, uneven acquisition, access and preservation of the lower theory explain part of the jagged competence of a network that learns the theory built on it; some failures remain unexplained. Whether the same holds in larger systems is a test to run.
Authors:Saman Rahbar
Abstract:
Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = ‑0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance‑graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.
Authors:Tilen Marc
Abstract:
A reduced word of the longest permutation w_0\in S_m, taken up to commutations, encodes a wiring diagram of m pseudolines, and a braid move σ_iσ_i+1σ_i\leftrightarrowσ_i+1σ_iσ_i+1 flips one of its triangles. We prove that deciding whether two wiring diagrams are within a given braid‑move distance is NP‑complete. Equivalently, deciding flip distance is NP‑complete in the higher Bruhat order B(m,2), for triangle flips of marked arrangements of pseudolines, and for rhombic tilings of a 2m‑gon; this answers an open question on flip graphs of pseudoline arrangements (SODA 2024). The reduction is from Vertex Cover: for every graph G we construct two wiring diagrams at braid‑move distance |D|+2\,\mathrmVC(G), where \mathrmVC(G) is the vertex cover number of G and the Hamming distance |D| is the number of triples of wires whose triangles are oriented differently. The construction glues copies of a seven‑wire gadget whose braid‑move distance exceeds its Hamming distance by 2; flipping either of two disjoint triangles in both diagrams removes the excess. Adding a line at infinity transfers the construction to rank‑3 uniform oriented matroids, where two oriented matroids are adjacent in the mutation graph if their chirotopes differ in one basis; so deciding mutation distance is NP‑complete in rank 3. An iterate of Rambau's expansion of signotopes amplifies the excess and gives NP‑hardness in every fixed rank r\ge 4. By duality we obtain the corresponding results for every fixed corank at least 3. The same instances show that flip distance in the higher Bruhat orders B(n,k) is NP‑complete for every fixed k\ge 2.
Authors:Le Chen, Lixin Liu, Jan Schneider, Zeju Qiu, Simon Guist, Bernhard Schölkopf, Dieter Büchler
Abstract:
World Action Models (WAMs) leverage pretrained video foundation models for robot manipulation, but their large backbones and video‑action co‑prediction are expensive. Although existing acceleration techniques offer many ways to reduce this cost, selecting and composing them requires substantial engineering for each model and hardware platform. To tackle this bottleneck, we present WAMJET, an agentic harness that accelerates WAM inference by equipping coding agents with reusable optimization guidance and measurement and validation tools. WAMJET follows a bottleneck‑driven workflow where the agent profiles inference, modifies targeted code, validates effects, and iteratively refines the acceleration stack as bottlenecks shift, while preserving action quality. Experiments span six WAMs, three coding agents, and two GPU architectures. WAMJET achieves up to 9.95x lossless speedup over upstream implementations. Approximation and hardware‑aware optimization yield additional latency reductions, with comparable success rates. The results show that WAMJET can produce effective acceleration stacks for WAM deployment.
Authors:Maqun Zhang, Feng Gao, Wankun Chen, Hui Yu, Yanhai Gan, Junyu Dong
Abstract:
Accurate modeling of the daily evolution of sea ice concentration (SIC) is central to improving the credibility and operational forecasting capability of deep learning‑based sea ice prediction. However, existing deep learning methods often couple the underlying sea ice evolution relationships and data errors within high‑dimensional nonlinear mappings, making it difficult to construct a stable and verifiable evolution core and apply it reliably to practical forecasting. To address this issue, this study proposes a reanalysis‑forecast dual‑domain decoupled framework for learning sea ice evolution operators. The framework builds upon a lightweight physical baseline to generate daily evolution trajectories, employs a temporally constrained joint multi‑lead compensation network to com?pensate for unresolved processes, and introduces an ice‑mass?aware transport mechanism to suppress numerical dissipation. In the forecasting stage, the parameters of the base evolution core are fixed, while a lightweight variable‑semantic adaptation mechanism calibrates inter‑domain distributions and evolution responses, thereby separating forecast‑domain errors from base evolution errors. Experiments show that the constructed base evolution core can accurately and stably simulate daily sea ice evolution at both short‑term and annual scales under reanal?ysis forcing, and can be effectively transferred to the forecast domain through lightweight adaptation, achieving stable prac?tical forecasting capability while preserving the base evolution structure. The source code will be made publicly available at https://github.com/zhangmaqun65535/SNM upon acceptance of this manuscript.
Authors:Carl Vincent Kho
Abstract:
Objective. We compared how strongly speech‑muscle sources couple to electrodes on the jaw and around the ear, and tested sensitivity to source orientation, electrode count, and anatomical detail. Approach. Using reciprocity in MIDA (116 labelled compartments, 500 micrometres), we evaluated 22 electrode positions against ten segmented articulators. Muscle was modelled as both source and conducting tissue. Comparisons used equal electrode counts and a hemisphere orientation sweep, with anatomy‑derived fibre models for temporalis and lateral pterygoid. Results. Orbicularis oris, buccinator, mentalis, depressor anguli oris, and platysma favoured the jaw at every sampled orientation and tested electrode subset. Temporalis showed a 2.57 dB ear advantage under the uniform sweep, reduced to 1.15 dB with the derived fibre field. Its jaw‑minus‑ear interval [‑1.45, +5.46] dB crossed zero; 50.5% of four‑site ear subsets favoured the ear. The derived lateral‑pterygoid interval [‑5.33, ‑1.12] dB favoured the ear at the fixed jaw basis. Both derived constructions have unresolved anatomical limitations. Simplifying non‑muscle soft‑tissue conductivities preserved montage assignments while changing gaps by up to 3.27 dB. Significance. In this head model, jaw placement better captures lip and chin activity. Ear‑side preferences depend on source assumptions and electrode selection. Targeted four‑site placement improved lateral‑pterygoid coupling by 1.07 dB over the median random ear montage. These coupling predictions require experimental testing.
Authors:Adithya Bhaskar, Jeffrey Cheng, Danqi Chen
Abstract:
Modern chess engines are silent experts: they play at a superhuman level, but do not offer explanations for their play. On the other hand, language models (LMs) can generate plausible‑sounding explanations, but their weak playing strength limits the utility of their explanations. We introduce Queen, a 4B‑parameter chess‑language model that can explain its moves and plans while playing at the level of a typical Grandmaster. Our novel framework enables domain‑specific reasoning through complementary components: an encoder‑decoder architecture and an iterative distillation algorithm. This architecture integrates a silent expert chess encoder with an instruction‑tuned LM through cross‑attention, which we train via a question‑answering curriculum to extract chess concepts from the encoder's representations. Building on this domain‑adapted model, we iteratively improve its explanations with a natural‑language analog of the Bellman update: the model analyzes the positions after its top candidate moves and consolidates them into an explanation of the current position, which is then distilled back into the model. Over seven iterations, our model gains over 900 Elo points (1782 to 2697), substantially surpassing all frontier models on both playing strength and puzzle accuracy, despite containing three orders of magnitude fewer parameters. Furthermore, LM‑based evaluations show that our explanations are fluent and approach GPT‑5.6‑Sol (high) in coherence. The generality of our architecture and training procedure suggests a recipe for applying language models to domains where silent expert encoders are available, like games, robotics, and computer use.
Authors:Zhanke Wang, Chengfeng Zhao, Qing Shuai, Jingzhong Lin, Heng Li, Zeyu Ling, Yuxin Wen, Jing Li, Di Kang, Chunchao Guo, Linchao Bao
Abstract:
We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning‑based methods typically regress human motion directly from video and train the network with geometric supervision. However, recovering human motion from monocular video is inherently ambiguous in depth, and direct regression tends to collapse toward an averaged solution. Moreover, the recovered motions are not guaranteed to be physically plausible, so physics‑based tracking of them often fails. To address these challenges, we formulate video motion capture as a video‑conditioned motion generation problem and first pretrain a flow matching model for this task. Given an input video, the pretrained model generates diverse motion candidates, but not all of them are faithful to the video or physically trackable. We therefore post‑train the model using Group Relative Policy Optimization (GRPO) with two rewards. A fidelity reward encourages consistency with the input video. A tracking reward favors motions that a physics‑based controller can track successfully. Together, these rewards shift the model's output preference, so the post‑trained model stays faithful to the input video while producing more physically plausible motion. We further introduce Wild‑4K, a large and diverse dataset of about 4K internet videos, for evaluating human motion recovery in the wild. Qualitative and quantitative experiments on Wild‑4K show that our method outperforms state‑of‑the‑art methods in overall motion fidelity and achieves a physical tracking success rate of 82.47%, compared with 62.82% for the strongest baseline, GVHMR.
Authors:Yunus Serhat Bıçakçı
Abstract:
We examine the transfer of a pretrained SigLIP2 image encoder to seven class fire risk classification from aerial imagery. We introduce a reproducible partition of the public FireRisk training mirror and an implementation that records data provenance, preprocessing and model selection. Two initial runs compare a frozen encoder probe with full model adaptation. On the validation partition, full adaptation reaches 63.05% accuracy and 58.94% macro F1, compared with 55.95% and 50.19% for the probe. Both runs use one training seed and select their checkpoint on the same validation partition. These development results support further evaluation of SigLIP2 but do not establish performance on an independent test set or unseen regions. The accompanying code provides a common framework for repeated experiments and comparisons with additional visual encoders.
Authors:Fedor Sergeev, Markus Heinonen, Daniel Waxman, Tim Cooijmans, Ricardo Baptista, Dmitry Batenkov, Eli Bingham
Abstract:
The dynamics of cells, organisms, and fluids are often modeled as probability distributions evolving over time. Reconstructing and extrapolating this evolution from unpaired snapshots requires assumptions about the underlying process. Wasserstein gradient flows are a common choice, but they cannot describe conservative or periodic dynamics. Lagrangian mechanics in Wasserstein space covers both, but existing methods for learning it are simulation‑based: they run a numerical solver at every training step, which makes training expensive. We propose Double‑Stitch, a simulation‑free method that learns these mechanics by penalizing the residual of the equation of motion along a learned population path. We derive this equation from a Clebsch variational principle that does not require gradient velocities, and show that the residual vanishes exactly when the equation holds. We test Double‑Stitch on synthetic, single‑cell and ocean vortex datasets and find that it matches or outperforms gradient‑flow methods and simulation‑based WLM on most tasks, while training 4‑14 times faster than WLM. We provide a JAX implementation of Double‑Stitch at https://github.com/BasisResearch/stitching.
Authors:Linghui Shen, Tinghui Zhu, Sheng Zhang, Muhao Chen
Abstract:
Autoregressive (AR) video models excel at causal generation, but their reliance on next‑chunk prediction confines them to a short‑sighted, reactive paradigm. This limitation is particularly consequential for reasoning‑oriented generation, where achieving a target outcome through valid intermediate states matters more than local visual plausibility. To address this challenge, we propose Learning Prospective Reasoning with Autoregressive Video Models (ProAR), a novel framework that transforms autoregressive video generation into a goal‑oriented reasoning process. ProAR introduces two key components: (1) To anchor generation to the long‑range outcome, we integrate goal‑frame prediction into the autoregressive loop via an asymmetric attention mask, enabling the predicted goal frame to guide the generation of intermediate states without being disrupted by them. (2) To guide short‑range transitions, we introduce future representation self‑alignment to encourage current hidden states to anticipate upcoming temporal dynamics. By leveraging teacher‑forcing in AR training, we extract clean future representations in a single forward pass and align current representations with them using a lightweight, training‑only predictor. Together, these two mechanisms seamlessly combine explicit, sparse target supervision with implicit, dense step‑wise guidance, promoting coherent, goal‑directed reasoning progress with modest computational cost. Experiments show that ProAR's complementary components consistently improve performance across diverse visual reasoning benchmarks. The framework proves highly training‑efficient, surpassing fully trained standard AR baselines using only 25% of the training steps. This paradigm also demonstrates promising applicability to embodied reasoning tasks.
Authors:Ziqi Ma, Shreya Sharma, Mohamed El Banani, Katja Schwarz, Chongjie Ye, Chao-Yuan Wu, Li Fei-Fei, Ben Mildenhall, Georgia Gkioxari, Justin Johnson, Gowthami Somepalli
Abstract:
Camera‑controlled video models are rapidly advancing toward long generation horizons and complex camera control. A key failure mode is 3D inconsistency: as the camera moves, objects lose permanence and scene structures shift. Existing post‑training techniques, which assign a single scalar reward to the entire generation, are poorly suited to correcting these inconsistencies over long horizons. We introduce LoGo, which blends global and spatially localized rewards for camera‑controlled video models. The local reward provides fine‑grained credit assignment, which substantially improves 3D consistency, while the global reward preserves camera following and video quality. Across three base models, LoGo shows a clear advantage on DL3DV and TrajectoryBench, a new benchmark for long‑horizon, complex‑camera‑control generation that current evaluations lack. LoGo effectively reduces local object shifts, artifacts, and global scene changes, illustrating the importance of credit assignment in post‑training video models. Project website: https://ziqi‑ma.github.io/logo‑website/
Authors:Yudong Lin, Haoyuan Deng, Zhuoxuan Yuan, Zaijia Yang, Yuanjiang Xue, Ziwei Wang
Abstract:
Online reinforcement learning (RL) enables robot policies to improve through physical interaction, but the assistance they require changes as their competence evolves. Existing intervention strategies based on offline estimates or fixed decision rules can therefore become mismatched to the current policy. To address this, we propose UniIntervene++, an adaptive intervention agent that learns to allocate control between autonomous execution and heterogeneous assisted behaviors during online RL. Specifically, UniIntervene++ first formulates the evolving RL policy, trajectory correction, and a task‑structured CodePolicy as Options in a unified semi‑Markov decision process and learns their relative values online. Building on this, competence‑adaptive intervention periodically probes the RL policy through unassisted execution, keeping control allocation responsive to its evolving capability. Finally, coupled experience learning allows assisted behaviors to improve the RL policy, whose evolving outcomes in turn reshape future intervention decisions. In this way, UniIntervene++ jointly determines when to intervene, how to intervene, and when to return control as the RL policy improves. Across five real‑world manipulation tasks, UniIntervene++ achieves an average success rate of 89.67%, outperforming all baselines by at least 6 percentage points, while reducing human intervention to 0.77%, a relative reduction of at least 94.6% from the best baseline. Code is available in our \hrefhttps://github.com/dannyyudong/An‑Adaptive‑Intervention‑Agent‑for‑Efficient‑Real‑World‑Reinforcement‑LearningGitHub repository.
Authors:Zhiming Liu, Yikun Miao, Ying Chen, Hongrui Yin, Fangqi Zhu, Xiaoyi Pang, Quanxin Shou, Zhengyang Yan, Haodong Wang, Song Guo
Abstract:
Learning general‑purpose robot policies requires large‑scale real‑world interaction data, yet collecting robot demonstrations remains expensive and difficult to scale. Egocentric videos offer abundant human interaction experience with task‑relevant semantics for robotic manipulation, but direct transfer is challenging for two reasons: latent actions inferred from frame reconstruction can be dominated by nuisance variation such as ego‑camera motion, and human and robot behaviors often exhibit different temporal dynamics. We propose WING (World Action Learning via INteraction‑Centric Spectral Latent Guidance), a framework for transferring interaction knowledge from egocentric videos to robot policies. WING first separates observer‑induced motion from hand‑object interaction and distills the interaction‑centric component into latent actions. It then exploits the observation that cross‑embodiment task semantics are concentrated in slowly varying temporal structures, identifying shared low‑frequency components between egocentric latent actions and robot behaviors in the spectral domain and using them to guide action generation. WING achieves average success rates of 99.20% on LIBERO, 93.80% on RoboTwin 2.0, and 57.7% on RoboCasa‑GR1, and also performs strongly across four real‑world manipulation tasks under diverse generalization settings. These results show that interaction‑centric spectral guidance provides an effective and scalable way to transfer physical interaction knowledge from human egocentric video to robot control. Project page: https://mikuz12.github.io/wing/
Authors:Antonio Canela, Jordi Sànchez-Riera
Abstract:
High‑fidelity 3D head avatars have reached near‑photorealistic quality. While recent methods enable text‑driven manipulation, they struggle to provide fine‑grained localized control, often entangling features or lacking geometric consistency. Modifying geometry through natural language currently requires slow per‑prompt optimization or compromises identity and rigging. We present ManifoldSplat, the first end‑toend framework for language‑guided semantic shape editing of animatable 3D Gaussian Splatting avatars reconstructed from monocular videos. By performing edits within the structured FLAME manifold rather than directly optimizing an unstructured Gaussian cloud, we strictly preserve identity and animation. We introduce DeltaRegion, a per‑region disentangled Conditional Variational Autoencoder (CVAE) delivering feedforward shape deltas, alongside a refining stage to recover view‑consistent details. ManifoldSplat reconstructs and edits an avatar in ~90 seconds on a consumer GPU, rendering at ~800 FPS. Extensive evaluations demonstrate our approach sets a new state‑of‑the‑art in localized prompt alignment, geometric coherence, and identity preservation. Project page and code: https://a‑canela.github.io/manifoldsplat/
Authors:Wangshu Zhu, Xueqi Cheng, Liang Wu, Yushun Dong
Abstract:
Understanding and assessing natural hazards is essential for disaster preparedness and risk reduction. Recent advances in large language models have spurred growing interest in AI agents for hazard analysis, particularly their ability to integrate scientific data, models, and tools into automated workflows. However, effective automation requires agents to determine which scientific methods are appropriate for a given event and executable with the available data and tools. As new evidence and execution results become available, these conditions can change, requiring agents to reconsider their choices. We formulate this problem as state‑dependent scientific route selection and introduce HazardWeaver. Specifically, HazardWeaver first leverages the Hazard Knowledge Compiler to extract evidence‑linked conditions governing scientific applicability, then its Hazard Capability Graph represents executable scientific capabilities and checks compatibility between their inputs and outputs. Using these complementary representations, the Hazard Weaver Agent component selects applicable and executable routes, carries out their workflows, and revises its decisions as the analysis state changes. To evaluate both the scientific outputs and the decisions that produce them, we introduce the Hazard Weaver Benchmark, comprising 141 instances across seven single‑hazard domains and four multi‑hazard interaction classes. The benchmark accommodates multiple valid scientific routes and evaluates output correctness, route validity, and justified abstention. Extensive experiments on this benchmark show that HazardWeaver outperforms existing agent systems, with the largest gains on tasks with multiple eligible scientific routes. Our code is publicly available at https://github.com/LabRAI/HazardWeaver.
Authors:Shuo Yang, Lihao Fang, Yi Zhang, Haixiang Wang, Xincheng Ye, Shufan Chen, Jipeng Guo, Youqing Wang
Abstract:
Diffusion Transformers (DiTs) can generate high‑quality images and videos, but generating each sample requires multiple costly DiT forward passes. Two common ways to accelerate DiT sampling are step distillation, which reduces the number of sampling steps, and caching, which skips some DiT evaluations by reusing a tensor computed at an earlier step. Most caching methods decide in advance which tensor to reuse. After distillation, adjacent sampling steps are farther apart. Reusing a tensor across this larger gap introduces more error, so choosing what to cache becomes especially important. We therefore introduce AutoTarget, a method that chooses the cached tensor for a given model, solver, and reuse schedule. AutoTarget uses a small set of runs without cache reuse to measure the error caused by reusing each candidate tensor, then selects the candidate with the lowest error. We also analyze how an error at one reuse step affects the final sample. For Euler sampling, we identify cache targets that produce the same trajectory and show why a stored solver update may not. Experiments on distilled image and video DiTs show that the best cache target changes with the model, image resolution, and solver. AutoTarget reduces DiT evaluations and retained cache storage. Generation quality remains close to the corresponding uncached run. On the tested PixArt‑LCM and FLUX.1‑schnell settings, its calibration ranking matches the ranking from held‑out cached runs. To help others reproduce the method, we provide its core implementation on GitHub at https://github.com/wali1024‑offical/AutoTarget.
Authors:Yuxuan Hu, Shilin Shan, Jianfei Yang, Feng Xu
Abstract:
Contactless heart‑rate sensing with millimeter‑wave (mmWave) radar requires assessing whether individual measurements support reliable estimation. We study learning to assess heartbeat observability, defined as the readability of the heartbeat component in an acquired phase spectrum, for selective heart‑rate estimation. Coherent superposition of scatterer returns can suppress this component even under similar macroscopic observation geometry, motivating assessment directly from acquired measurements. To obtain training supervision across different observability conditions, we develop a controllable multi‑scatterer frequency‑modulated continuous‑wave (FMCW) simulator. Agreement between the dominant heartbeat‑band peak and the known heart rate provides an automatic observability label for each simulated measurement. We propose HEAR (Heartbeat Estimation with Assessed Reliability), a compact dual‑task Transformer that jointly predicts an observability score and heart rate. Its input combines spectral magnitudes with frequencies relative to the respiration fundamental, providing context for respiratory harmonics. Trained solely on simulated observations, HEAR transfers zero‑shot to two public real‑world datasets collected at 60 and 120 GHz from 134 subjects. The same learned score supports selective prediction with both HEAR's own heart‑rate head and multiple existing estimators. On the 120 GHz dataset, score‑based selection reduces the HR head's mean absolute error from 17.9 BPM at full coverage to 1.6 BPM at 50% coverage. The complete pipeline achieves an end‑to‑end processing latency of 50.8 ms on an edge device. Project page: https://yuxuanhu9.github.io/HEAR/.
Authors:David L. Condrey
Abstract:
We describe the Writerslogic team's participation in the CLEF 2026 SimpleText shared task, addressing Task 1 (text simplification) and Task 2 (complexity spotting). For Task 1, we develop a multi‑candidate generation pipeline using GPT‑4o‑mini that produces five simplification candidates per sentence at varying temperatures, then selects the best candidate using a reference‑free scoring heuristic that rewards compression, source word retention, Cochrane Plain Language Summary vocabulary usage, and lexical simplicity. On Task 1.1 (sentence‑level simplification), our Claude Sonnet 4 submission achieves SARI 47.43 and BLEU 14.21, the top‑ranked sentence‑level system (3rd on the combined Task 1 leaderboard, behind two document‑level submissions). For Task 2, we fine‑tune a DeBERTa‑v3‑large NLI model on 350K labeled (source, sentence) pairs, framing hallucination detection as natural language inference. The model reads the most relevant source sentence as premise and the candidate as hypothesis, directly learning to distinguish grounded from hallucinated content. On Task 2.1 (binary overgeneration identification), our fine‑tuned DeBERTa system achieves 0.8081 document‑level macro F1 (0.8085 in our best ensemble), the top‑ranked entry within the identification track and 2nd among teams overall, behind AIIR Lab (0.8197). On Task 2.2 (multi‑class error classification), our best submission reaches 0.804 multiclass accuracy, ranking 2nd among unique teams behind AIIR Lab (0.827). We evaluate both tasks on English and multilingual biomedical text from Cochrane systematic reviews.
Authors:David L. Condrey
Abstract:
We describe the Writerslogic systems for three PAN at CLEF 2026 shared tasks (Reasoning Trajectory Detection, Voight‑Kampff Generative AI Detection, and Multi‑Author Writing Style Analysis), unified by a shared analytical framework: feature robustness under distribution shift is governed by support overlap between training and test distributions, not by training‑set effect size. This yields a taxonomy (domain‑anchored, domain‑portable, domain‑invariant) that explains why generator‑specific features die under domain shift while vocabulary fingerprints (hapax ratio, Yule's K, Heaps' exponent), compression measures, and character n‑grams survive. On Reasoning Trajectory Detection, where training was entirely mathematics and 84 percent of test was unseen domains, the framework guided system design to 1st place in source detection (0.85 macro F1 via Opus‑Sonnet agreement) and 3rd place in safety classification (0.66 macro F1 via query‑refusal decomposition). For Voight‑Kampff, we built a calibrated ensemble of DeBERTa‑v2 (ONNX), multi‑seed LightGBM with 44 domain‑portable stylometric features, and SVM on n‑gram TF‑IDF, combined via learned stacking with isotonic calibration; the best configuration achieved 0.891 on the PAN 2026 test set with balanced sub‑metrics (0.853 to 0.902 across all evaluation dimensions). For Multi‑Author Writing Style Analysis, we describe a system fusing spectral clustering over character n‑gram similarity graphs, normalized compression distance for local boundary detection, and SmolLM‑135M perplexity for neural change‑point detection; a platform mix‑up meant our run never reached the official evaluation, so we report the design and its a priori predictions. Across all three tasks, features measuring generation process properties are designed to outperform features measuring generated content properties under domain shift.
Authors:Ferran Hernandez Caralt, Simon Heilig, Adrián Bazaga, Asja Fischer, Moshe Eliasof, Pietro Liò
Abstract:
Empirical claims about the connection between over‑squashing and long‑range interactions in GNNs, can only be trusted if the benchmarks used to validate them genuinely require long‑range interactions. The de‑facto standard, the Long Range Graph Benchmark, has been repeatedly shown to be saturated by tuned short‑range models, with existing synthetic alternatives being tied to specific topologies. As such, there is a lack of principled certificate of long‑rangedness on arbitrary graphs. This state reflects the absence of a precise characterization of long‑ranged benchmarks. We address this fundamental gap by introducing four verifiable axioms: Predictability, Tightness, Strictly k‑Range, and Topology‑Invariance, that any task claiming to test k‑hop interactions must satisfy. We formally prove that violating any one of them admits failure modes that undermine conclusions drawn from the task. Based on these axioms, we introduce TRIP (Truly Ranged Interactions Problem) and its generalisation GRIP (Generally Ranged Interactions Problem), constructive procedures that turn any graph into a provably long‑ranged task by drawing features from stable distributions. Moreover, by construction, GRIP admits a closed‑form, per‑range Maximum‑Likelihood oracle that yields the first a priori per‑range lower bound on test error available on any benchmark. Using our framework, we: (i) audit 4 common long‑range benchmarks and identify their failures modes with respect to our axioms; (ii) on TRIP‑instantiated topologies, we find a popular notion of curvature is uncorrelated with GNN performance, supporting topological‑vs‑computational bottleneck distinction; and (iii) we show that a novel benchmark's over‑squashing measures factors beyond pure long‑rangedness. Code to use the framework and reproduce experiments is released https://github.com/ferranhernandezc/graph‑grip.
Authors:Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing, Ruchang Yao, Runtao Liu, Shijie Zhao, Tianfan Xue
Abstract:
Streaming video generation has benefited from distribution matching distillation (DMD), which matches the joint distribution of video frames to a video teacher's approximation of the real video distribution. Although this joint matching mitigates drift during autoregressive rollouts, limitations remain in visual quality and semantic alignment. To address these limitations, we propose DuoMatching, a distribution matching framework that approximates the real video distribution through a unified joint‑marginal formulation. On top of existing joint matching formulations, the additional marginal matching objective provides dedicated frame‑level supervision from an image generator, transferring complementary visual and semantic priors from it. To apply this frame‑level supervision in video generation, we introduce LatentBridge to resolve the latent representation mismatch between the video student and the image teacher. Latent Variation Sampling further distributes such frame‑level supervision across distinct temporal segments, reducing redundancy. Experiments demonstrate that DuoMatching improves visual quality, composition, and semantic alignment while largely preserving motion dynamics. Human evaluations show overall preference rates above 80% against all evaluated baselines. The project page is available at https://johnzhan2023.github.io/DuoMatching/.
Authors:Peng Liu, Jingyan Wang, Qipeng Ye, Wen Li, Jinya Su, Zuo Wang, Shihua Li, Yunda Yan
Abstract:
LiDAR‑based quadrotor navigation in cluttered environments remains challenging under external disturbances, particularly when obstacle‑aware motion generation and disturbance‑rejection control are handled in separate layers. This article presents disturbance‑resilient integrated planning and control (DR‑IPC), which combines lightweight path guidance with nonlinear model predictive control (NMPC) to directly generate angular velocity and thrust. An interconnected extended Kalman filter and nonlinear disturbance observer jointly provide filtered state estimates and reconstructed disturbances for NMPC prediction. The resulting formulation unifies nonlinear quadrotor dynamics, actuator constraints, local motion generation, and penalised safe‑flight‑corridor residuals without requiring a separate trajectory‑optimization stage. Gazebo and MARSIM simulations, together with indoor and outdoor experiments, validate DR‑IPC under wind, suspended payloads, narrow passages, ball impacts and reactive avoidance of a dynamic obstacle. In multi‑goal navigation with disturbances, DR‑IPC increases the number of completed missions from 1/10 to 9/10 in Gazebo and reduces the altitude RMSE from 0.34 to 0.01 m in experiments. The complete system operates onboard at 100 Hz. Supplementary videos are available on the project page https://drpp316.github.io/DR‑IPC‑Page/, and the source code will be released.
Authors:Shivam Shrivastava
Abstract:
Deep tabular generative models are benchmarked on datasets with tens of thousands of rows; clinical datasets have hundreds. We preregistered and ran a size‑ladder benchmark to find where the two regimes diverge: 8 public datasets subsampled from 200 to 20,000 training rows, seven generators (independent marginals, Gaussian copula, SMOTE, unconditional SMOTE, CTGAN, TVAE, TabDDPM) with a fixed 20‑trial tuning budget and 5 evaluation seeds, plus 4 natively small clinical datasets at true size, for 2,220 committed runs in total. The primary metric is the AUROC of fixed classifiers trained on synthetic and tested on real data. In 23 of 24 (dataset, deep model) pairs no deep model ever beats the best trivial baseline by more than seed noise, at any training size we measured. The best baseline wins 40 of 49 (dataset, size) cells. Our preregistered prediction that the deep models' ranking would be unstable at small sizes is falsified: mean Kendall tau between adjacent rungs below 5,000 rows is 0.806, above our 0.8 threshold, and stability is highest at the smallest sizes rather than lowest. One caveat bounds all of this: in 81% of cells the gap between the top two methods is smaller than the variation between seeds. Finally, method rankings on natively small clinical datasets agree only moderately with rankings on subsampled large ones (mean tau 0.57 to 0.64), which questions whether a subsampled large dataset can stand in for a small one. All 2,220 result files, the preregistration and its hash, and the code that regenerates every figure and number from those files are public.
Authors:Chenzhi Liu, Yue Zhang, Jiehong Lin, Jianan Wang, Bo Wang, Zhongrui Wang, Xiaojuan Qi
Abstract:
Long‑horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and arm control. While recent Vision‑Language‑Action models excel at short‑horizon tasks, they lack the hierarchical reasoning required for multi‑stage objectives. Furthermore, existing hierarchical agents suffer from rigid sub‑task mapping, inflexible replanning, and a lack of continuous learning. To address these limitations, we introduce MobiAgent, a dual‑loop agentic framework that bridges robust deployment execution and recursive policy self‑improvement. During deployment, the Inner Loop decouples high‑level reasoning from low‑level control through highly composable atomic skills. It employs Vision‑Language models for receding‑horizon planning and visual reflection, dynamically composing skills to ensure robust error recovery. These skills are executed by specialized flow‑matching experts that share a unified VLM backbone, maximizing reusability while mitigating capacity interference. Concurrently, the Outer Loop drives automated lifelong learning by autonomously segmenting and verifying deployment rollouts, clustering them to discover atomic skills, and continuously fine‑tuning the skill library without human annotations. Evaluations on RoboCasa, BEHAVIOR‑1K, and real‑world tasks demonstrate the effectiveness of MobiAgent. It outperforms π_0.5‑TA by 22.5 percentage points on BEHAVIOR‑1K and enables robust recovery from execution failures. Through autonomous data recycling, success improves from 7.50% to 27.50% on RoboCasa and from 32.5% to 57.5% on Astribot S1.
Authors:Daikun Liu, Xin Zhan, Teng Wang, Xiaoping Wang, Changyin Sun
Abstract:
We propose UniDynamics, a diffusion‑based framework for future 4D dynamic scenes (RGB, depth, and optical flow) generation from a single event‑RGB pair, without requiring long histories or control priors as in existing methods, while explicitly modeling future motion fields. The core idea is to leverage event streams to offer an alternative motion prior for single‑RGB extrapolation, and to enforce geometric and motion constraints throughout generation via multimodal modeling. Specifically, we design an Event Latent Enhancement (ELE) module to align and enhance event latents into diffusion‑injectable conditioning features, providing robust initial motion priors and reliable texture/structure cues. We further introduce a Perceptual Dynamics Space (PDS) embedded in the multi‑scale U‑Net, which decouples and adaptively interacts depth and flow while continuously feeding back constraints to appearance features, improving geometric‑motion consistency for physically plausible and spatiotemporally coherent prediction. Experiments on VKitti2 and DSEC demonstrate state‑of‑the‑art performance, producing high‑quality, temporally coherent, and 4D‑consistent future predictions, especially under challenging high‑speed motion blur.
Authors:Zhe Zhou, Tianhua Tao
Abstract:
Post‑training with verifiable rewards can induce reward hacking, motivating the use of monitors within the training objective rather than solely for offline auditing. We show that a low monitor readout does not identify whether such an intervention controls behavior. In a code‑generation environment whose dominant exploit is available at the start of the reasoning trace, we train policies against three monitors that pass the same offline gate: an in‑domain activation probe and two penalties conditioned on how early the policy commits to its own final answer. The probe score is at its numerical floor from the first recorded training step, and the trained‑score median is zero for every prefix‑trained run at the endpoint. These readouts estimate different quantities, and we do not compare their scales; within each monitor family, however, low values do not establish behavioral control. Within one fixed configuration, prefix‑trained runs with the same zero‑median trained score range, by seed alone, from a mixed regime with a low hacking share to near‑pure reward hacking. All probe runs reach the hacking regime, but their floor‑level readout reflects a mismatch between the position where the probe was validated and the position where it was read during training, not a second instance of this ambiguity. Text‑level analysis identifies a prefix failure mode: generic planning and filler shells postpone the exploit past the cut without eliminating it from the final output. Low measured commitment therefore does not distinguish a low hacking share from delayed commitment to the exploit. Offline discrimination and low monitor‑aligned readouts are insufficient evidence of behavioral control; an out‑of‑band behavioral check is required. We characterize the endpoint readout, not its evolution. Code is available at https://github.com/zhezhou1106/spoof‑cost.
Authors:Zhuowen Liu
Abstract:
LLM agents increasingly screen tool outputs with small prompt‑injection detectors, and teams choose among detectors by their scores on public benchmarks. We ask whether those scores predict how a detector behaves inside an agent. We replay the ground‑truth tool calls of two agent benchmarks, AgentDojo and tau‑bench, without an LLM to obtain tool outputs that are benign by construction, label injected outputs by differential replay, and evaluate fifteen detectors, including Meta's Prompt Guard 2, and two task‑aware LLM judges on these outputs and on the BIPIA benchmark. Detection rankings transfer poorly between benchmarks: the best detector on BIPIA catches 2% of AgentDojo injections at a 1% false‑positive rate, and a detector that catches 72% of AgentDojo injections catches 15% on tau‑bench. False‑positive rates on tool outputs, which range from none to over 90%, do transfer between the two agent benchmarks. Where training data is public, the form of the training inputs explains the results. The BIPIA leader was trained on full BIPIA inputs, but having seen InjecAgent's attack strings as short prompts does not help it find them inside tool outputs; the best detector on both agent benchmarks shares no data with any benchmark and was trained on agent‑style inputs. Evaluations meant to inform deployment should use the agent's own tool outputs, report detection at a low false‑positive rate, and audit what the detector was trained on.
Authors:Antonio Canela, Jordi Sànchez-Riera
Abstract:
We present ChromaGS, a method for real‑time, language‑guided color editing of animatable 3D Gaussian head avatars. Given a trained animatable avatar, users can instantly modify the color of semantic regions through natural language, with edits applied at render time and no retraining required. Our key insight is to augment each Gaussian primitive with learned soft assignments to semantic regions and decompose colors into region‑level base colors and Gaussian‑level residuals. This decomposition enables coherent color transfer: modifying a region's base color propagates naturally through all associated Gaussians while preserving fine appearance details encoded in residuals. A two‑stage language pipeline translates text instructions into target colors, supporting both absolute specifications and relative adjustments. Unlike generative editing methods that may introduce unintended modifications, our approach provides deterministic, precisely localized semantic control. Experiments demonstrate faithful appearance preservation and intuitive interaction across diverse subjects. Project page and code are available at: https://a‑canela.github.io/chromags/
Authors:Chutian Wang, Wenhao He, Jingmin Zhu, Qingyu Yin, Heng Xu, Xiuyu Li
Abstract:
Irregular All‑to‑All communication is a major bottleneck in expert‑parallel Mixture‑of‑Experts (MoE) models. Even with fixed expert routing and placement, uneven utilization of parallel network Rails and incast can limit communication performance. We present RailWave, a phase‑adaptive communication layer built on DeepEP that addresses these bottlenecks below the routing layer through spatial and temporal traffic shaping. RailBalance redistributes source traffic across eligible Rails using source‑local information, while a reusable, topology‑derived permutation schedule limits concurrent senders per receiver without rebuilding demand‑dependent schedules for each communication phase. A lightweight calibrated selector chooses an execution path according to each phase's traffic characteristics and offline profiling results. On training‑derived communication workloads from the 106B GLM‑4.5‑Air model, RailWave delivers up to 5.84x speedup on H800 and 4.36x on H20 over Native. Code is available at https://github.com/CyberSecurityErial/RailWave‑EP.
Authors:Zhihao Zhan, Le Tao, Yifei Tian, Xin Liu, Jie Yuan
Abstract:
Forest point cloud segmentation is fundamental for fine‑grained 3D forest scene understanding, yet remains challenging due to irregular tree structures, severe occlusions, density variations, and ambiguous instance boundaries. Recent query‑based forest segmentation methods have shown promise for unified semantic and instance prediction, but they still insufficiently exploit forest‑specific spatial structure and account for boundary uncertainty. In this paper, we propose ForestQuery, a boundary‑aware and spatially anchored query learning framework for unified forest point cloud segmentation. ForestQuery enhances instance and semantic query learning through two complementary designs. Specifically, boundary uncertainty is explicitly modeled to guide reliable instance query construction and modulate query optimization through adaptive loss reweighting. Meanwhile, spatially anchored semantic query enhancement (SA‑SQE) introduces learnable 3D anchors encoding forest vertical stratification priors to enrich semantic queries with explicit spatial references. We evaluate ForestQuery on multiple public forest point cloud benchmarks and a self‑collected annotated real‑world dataset. Extensive experiments demonstrate consistent improvements in both individual‑tree segmentation and semantic segmentation across diverse forest scenes. Code and data are publicly available at https://zhan994.github.io/ForestQuery
Authors:Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
Abstract:
World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action‑annotated robot trajectories. Observation‑only videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to robot commands. We present NAVA‑WAM, which introduces native action‑prior learning by directly pretraining the action policy from observation‑only videos, avoiding indirect representation‑to‑control transfer or a separate latent‑action model. Our training consists of two stages. First, we pretrain on observation‑only videos, where future‑video flow‑matching supervision over visual transitions is propagated through transition‑structured joint attention to optimize the Action‑DiT and learn action‑relevant priors. Second, we use action‑labeled demonstrations to post‑train the Action‑DiT for robot control through joint video‑‑action flow matching, while asymmetric attention decouples the visual branch from iterative action denoising and enables efficient action‑only inference. Extensive experiments show that NAVA‑WAM consistently outperforms prior approaches under both in‑distribution and out‑of‑distribution settings, while demonstrating strong action‑label efficiency and effective real‑robot generalization. These results establish native action‑prior learning as an effective approach to directly pretrain action policies from observation‑only videos, providing a scalable path beyond action‑labeled robot data.
Authors:Jiawen Lu, Tongtong Wu
Abstract:
Rejection policies must remain useful as candidate sets and tasks change. We compare Laya, Jev and Qwen2.5‑7B‑Instruct using public reference labels, testing Laya/Jev policy transfer at equal calibration budgets and all three models on artificial omission, natural retrieval misses and public out‑of‑scope queries. Source calibration often fails to preserve the target operating point. A Jev policy calibrated on DBpedia rejects 69.3% of covered Emotion test inputs, while an Emotion policy loses detection entirely. Retrieval exposes a different tradeoff: with ten intent candidates, Laya detects 99.0% of out‑of‑scope queries but rejects 48.8% of covered queries. Separating missing‑answer sources reveals these costs alongside retrieval coverage. The benchmark provides shared inputs, explicit decision and failure categories, and reproducible scoring to assess rejection policies under the conditions in which they are reused. Code and benchmark artifacts are available at https://github.com/luckykevvv/Decision_Model_Benchmark.
Authors:Tian Liang, Zishan Shao, Yiran Chen
Abstract:
Diffusion language model (dLLM) compression faces a known challenge because calibration is typically performed on clean, fully visible activations, whereas inference traverses partially masked intermediate states. For low‑rank compression, this raises two questions. First, can low‑rank optimality still be characterized when approximation quality is measured over trajectory‑distributed states, and second, does the choice of calibration states affect mathematical reasoning preservation under compression? We address these questions by formulating a trajectory‑aware low‑rank objective over corruption levels and masking realizations. To estimate this objective efficiently, we propose Traj‑MC, which estimates the trajectory second moment through Monte Carlo sampling and yields exact sampled‑state optimality and population consistency. Under matched compression budgets, trajectory‑aware calibration improves reconstruction over the generation trajectory and preserves substantially more mathematical reasoning than clean calibration on mathematical reasoning benchmarks. Our results connect trajectory‑aware low‑rank optimality to the reasoning capability retained after dLLM compression. Our code is available at: https://github.com/Zishan‑Shao/traj‑mc.git.
Authors:Erik Wikingsson, Martin Andrae, Tomas Landelius, Fredrik Lindsten
Abstract:
Flow‑ and diffusion‑based generative models have recently emerged as flexible and highly efficient forecasting models for dynamical systems. When combined with inference‑time guidance, they offer a promising route to high‑dimensional non‑Gaussian data assimilation (DA), the problem of combining forecasts with observations to estimate latent system states. Existing filters, however, condition on a fixed history and assimilate only the most recent observation, leaving them unable to revise past states when new observations arrive. Estimates then stay tethered to a history that later observations may contradict, and errors accumulate over the assimilation run. To this end, we introduce DAWIS, a unified DA method covering filtering, fixed‑lag smoothing, and block smoothing within a single framework. DAWIS replaces the single flow time of a state‑level prior with a multitask stochastic interpolant over a window of consecutive states, assigning a separate flow time to each. An assimilation cycle inverts the window to a vector of per‑state turning points and regenerates it under observation guidance, with the turning points controlling how strongly each state is held fixed, revised, or generated from scratch. The same construction can also absorb the forecast into the assimilation cycle, removing the need for a separate forecasting model. Experiments on challenging nonlinear systems show that DAWIS improves on both filtering and smoothing baselines under sparse, noisy, and nonlinear observations. The code for DAWIS is available at https://github.com/Erik‑Wikingsson/DAWIS
Authors:Atsuhiro Noguchi, Tianhan Xu, Yiming Liang, Yuta Kikuchi, Masahiro Ishiyama, Shintaro Takagi, Hitoshi Murai, Eiichi Matsumoto
Abstract:
We reconstruct high‑fidelity 3D scene meshes from posed multi‑view images without per‑scene optimization, across scales ranging from single objects to large outdoor scenes. Per‑scene optimization methods lack the learned 3D prior needed when observations are sparse or surfaces are glossy or transparent. Existing generative methods leverage such priors to complete geometry in sparsely observed regions, but typically operate at a fixed resolution over a limited spatial extent, trading spatial coverage against detail. Reconstructing a large scene therefore often requires partitioning it into independently processed overlapping local regions, making it difficult to maintain global geometric consistency. To address these issues, we propose T3lescope, which applies a single fixed‑resolution generator across scene scales in an inference‑time coarse‑to‑fine cascade. A coarse level establishes the scene layout, and finer levels perturb and denoise geometry inherited from the coarser level within progressively finer spatial cells to recover surface detail. The model is trained on individual cells at multiple scales and shares its weights across all levels, so no hierarchy is fixed during training, and the number of levels, cell scales, and cell locations are determined at inference time. On indoor, outdoor, and city‑scale scenes, T3lescope outperforms feed‑forward and generative baselines, matches or surpasses per‑scene optimization, and recovers fine structures as well as glossy and transparent surfaces. These results show that our method generalizes across diverse scenes, view counts, and image resolutions. Project page: https://pfnet‑research.github.io/t3lescope/
Authors:Zhendong Li, Akwum Onwunta
Abstract:
Uncertainty quantification (UQ) for random partial differential equations (PDEs) is ubiquitous in computational science and engineering. However, classical spectral solvers for this class of problems face the curse of dimensionality, and existing neural solvers often ignore the stochastic structure that makes moments and calibration tractable. We introduce a stochastic separable physics‑informed neural network, dubbed S^2‑PINN, that represents the solution u(t,\mathbfx,\mathbfZ) of a random PDE with a learnable Gaussian spatial dictionary, Fourier temporal features, and a generalized polynomial chaos (gPC) stochastic basis, coupled by a low‑rank Canonical Polyadic (CP) tensor decomposition core. The method is trained with a hybrid strong‑form and gPC‑projected residual loss. Our theoretical analysis establishes that the separable class is dense in L^2 under mild conditions, and the projected residual corresponds exactly to a stochastic Galerkin constraint. Furthermore, we show that mini‑batch projection coefficients are logarithmically dependent on the number of gPC modes, and that the orthogonality penalty controls the conditioning of the learned spatial dictionary. Using four manufactured random PDE benchmarks, we show that S^2‑PINN outperforms nine baselines in terms of mean and variance accuracy, as well as calibration, while using significantly fewer parameters. Further evaluations on non‑manufactured Poisson and Darcy problems, a stochastic Navier‑‑Stokes problem, a diffusion scaling study of higher random dimensions, and two stochastic inverse problems reveal the generalization capabilities of the proposed structure. Together, these results support stochastic separability as an effective design principle for physics‑informed neural UQ. The code for the experiments can be found in https://github.com/DMax1314/s2pinn
Authors:Xiangwei Jiang, Yao Mu, Lixin Duan, Wen Li
Abstract:
As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical. Scalable cross‑embodiment learning therefore requires a unified representation that captures shared manipulation structure while preserving morphology‑specific control. Differences in hands, tasks, datasets, and control interfaces prevent existing studies from isolating the effects of representation, pretraining, and architecture. We introduce DexJoCo‑X, a benchmark and toolkit for controlled comparison across seven representative dexterous hands, six single‑arm and bimanual tasks, and 2,100 balanced demonstrations. DexJoCo‑X provides a matched multi‑hand, multi‑task protocol with common scenes, success criteria, and execution interfaces, redesigned glove‑to‑hand mappings, and an automated pipeline that expands reviewed demonstrations across randomized scenes. Using π_0.5, Ego‑Pi, and Being‑H0.5, we examine whether a shared action interface is sufficient for multi‑hand learning. Expanding π_0.5 to an 80‑dimensional bimanual output yields near‑zero success. Ego‑Pi preserves the pretrained action head through interleaved prediction and supports per‑hand multi‑task learning, but remains ineffective for seven‑hand joint training. By contrast, Being‑H0.5 combines cross‑embodiment pretraining, a unified action space, and embodiment‑aware experts, enabling one policy to control all seven hands. Within this architecture, function‑aligned action slots achieve 47.7% mean success, compared with 47.0% for native coordinates and 33.1% for DexLatent. These results show that cross‑embodiment representation depends on the entire learning system: action coordinates, pretraining, and architecture must jointly separate shared manipulation structure from embodiment‑specific control.
Authors:Daniel Eskandar, Ilya A. Petrov, Gerard Pons-Moll
Abstract:
Generative models of human‑object interaction are bounded by the data that exists: everyday activities involve several objects, but most captured datasets record one at a time, as multi‑object capture is combinatorially expensive. Our observation is that interactions are local, so single‑object captures already contain the parts of multi‑object activities. We compose them: contact‑consistent clips of single interactions, mirrored to balance the hands, transfer between bodies, and a language model and geometric checks admit only the pairings that are plausible, semantically and physically. Therefore, the dataset grows combinatorially with the clips rather than recording time. The COSMI dataset holds 222k sequences and 275 hours with up to five objects, nearly thirty times the largest multi‑object capture, and can be extended by adding datasets or even hand‑object recordings. On this data we train the COSMI method, a text‑to‑interaction diffusion transformer that follows how the data is built: weight‑shared object slots generate a variable number of objects, predicted relative to the body parts that move them. On a benchmark with an unseen object and unseen interaction combinations, models trained on the dataset generalize to the unseen combinations. COSMI outperforms baselines in text alignment and contact accuracy, where its margin is largest on the unseen object. Code, models, and the dataset pipeline will be released on the project page: https://ptrvilya.github.io/cosmi.
Authors:Pujun Guo, Yuanfan Zheng, Fei Teng, Mengfei Duan, Guoqiang Zhao, Yuheng Zhang, Kai Luo, Kailun Yang
Abstract:
Panoramic images provide a complete 360‑degree field of view, enabling comprehensive scene understanding for embodied perception. However, heterogeneous embodied platforms exhibit substantial differences in observation viewpoints and spatial layouts, giving rise to cross‑embodiment observation shifts that pose additional challenges to consistent and reliable panoramic perception, while systematic studies of this problem remain limited. To bridge this gap, we introduce a new task, termed Cross‑Embodiment Open Panoramic Segmentation. Meanwhile, we establish EmbPASS, a multi‑platform panoramic semantic segmentation benchmark spanning Vehicle, Drone, Wearable, and Quadruped platforms under a unified semantic taxonomy, providing a testbed for systematically studying cross‑embodiment panoramic perception. We further propose EPONet, an open‑vocabulary panoramic semantic segmentation network that integrates Relation‑Aware Metric Adapter (RAMA) and Content‑Adaptive Semantic Transfer (CAST) to enhance spatial modeling and semantic transfer under heterogeneous embodied observations. Extensive experiments show that EPONet achieves the best platform‑balanced performance on EmbPASS with 35.82% mIoU, outperforming the strongest baseline by 1.10%, while remaining competitive on existing panoramic segmentation benchmarks. The source code and EmbPASS benchmark will be made publicly available at https://github.com/guopj1/EmbPASS.
Authors:Dominik Magiera, Lukas Röhrig, Frank Jäkel
Abstract:
We present WAMpy, a Python framework optimized for synthesizing Prolog programs. Unlike general‑purpose Prolog systems, WAMpy targets workloads that repeatedly generate and evaluate small candidate programs. WAMpy compiles Prolog clauses into NumPy array‑based WAM instructions and supports partial recompilation of hypotheses against fixed background knowledge. Performance‑critical routines are accelerated using Numba just‑in‑time (JIT) compilation. In a benchmark of repeated compilation‑and‑evaluation workloads, WAMpy improves end‑to‑end performance compared with SWI‑Prolog accessed from Python using Janus.
Authors:Jungseob Lee, Seungyoon Lee, Sugyeong Eo, Hyeonseok Moon, Jaehyung Seo, Heuiseok Lim
Abstract:
Large language models fine‑tuned from a shared base can be merged by averaging their task vectors, but some merges collapse far below the base model, and common merge operators give no warning before evaluation. We show that one statistic of the specialists' task vectors both predicts this collapse and calibrates its repair. The power that averaging removes equals the variance of the task vectors across specialists, our measure of interference. Under a working noise model, the disturbance that a merge injects grows with the merge coefficient and with interference, yielding a pre‑merge score. In our experiments on twenty‑two merge configurations from four model families, only destructive merges exceed a threshold on this score. We find that statistics of sign conflict between specialists, a common target of existing merge operators, are anti‑predictive. We then predicted the outcomes of fourteen merges before evaluating them, and twelve predictions were correct, including the destructive outcome of a specialist pair pushed past the threshold by continued pretraining. To address this collapse, we introduce PRISM, an operator that averages the task vectors first and then soft‑thresholds each layer at a level set by the layer's interference. Without data or tuning, PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model, where plain averaging falls at least 14.4 points below it or collapses entirely. We apply PRISM only above the threshold and keep the plain average for merges below it, which include all fifteen harmless ones. Code is available at https://github.com/js‑lee‑AI/PRISM.
Authors:Tingzhu Bi, Ping Wang, Meng Ma
Abstract:
Accident, defect and outage investigations end with a decision that ordinary question answering never faces: whether the evidence gathered so far is enough to close the case. We study this decision for LLM investigators, which request evidence from a case file, revise their hypotheses, and either close the case with a conclusion grounded in what they read or leave it open and name what is missing. This judgment does not come with capability: an untrained 9B model overstates its evidence in 97% of its answers, and a frontier model that identifies the right cause in 84% of cases still overstates in 91% and closes 17 of the 41 cases whose official finding is "cause undetermined". Measuring it is also non‑trivial: the source of a case largely predicts its label, and a rule that reads only the source reaches 83.0 balanced accuracy on our test cases. We therefore evaluate closure with three tests: closure accuracy, reported against this rule and within each source; evidence dependence, which removes the grounds of a conclusion and checks whether the model stops closing; and conclusion and gap quality, a judged checklist of what the model asserts and what it says is missing. We build Nautil, 731 audited cases from aviation, rail, maritime, chemical‑safety and vehicle‑defect reports and production server incidents, with teacher trajectories, an out‑of‑distribution test set and counterfactual evidence versions. Fine‑tuning a 9B model on these trajectories makes its closures follow the evidence: removing the grounds lowers its closure rate by 26 points relative to a matched control, overstatement falls from 97% to 35%, and correct, non‑overstated conclusions rise from 3% to 43%. Reinforcement learning that rewards only the closure decision then raises balanced accuracy from 69.2 to 83.3, on par with the teacher, and within‑source accuracy from 60.4 to 74.1, at some cost in evidence dependence.
Authors:Han Cui, Jianhao Yan, Yun Luo, Hongbo Zhang, Zhizhang Fu, Yue Zhang
Abstract:
On‑policy distillation (OPD) has become an important approach to language model post‑training. However, despite its performance gains, OPD can also collapse into excessively long and repetitive generation, and the mechanism underlying these divergent outcomes remains poorly understood. We explain these outcomes through a reinforcement learning perspective: the teacher implicitly rewards student behaviors, even those it rarely exhibits itself. From this perspective, our experiments show that OPD improves performance without expanding the student's capabilities. When the implicit reward model is reliable, OPD makes correct responses easier to sample. In contrast, when the preference misaligns with quality, reward hacking happens: the implicit reward model amplifies overlong, repetitive student rollouts, even though it rarely generates such text itself. Guided by this diagnosis, we find that masking unhealthy responses during training and using SFT initialization can each effectively mitigate the collapse. Together, these findings show that OPD amplifies student behaviors favored by the teacher's implicit feedback, shifting the focus from how well the teacher generates to how reliably it evaluates student rollouts. Our code is available at https://github.com/HancCui/opd_hacking.
Authors:Aarav Singh, Animesh Pathak, Navyansh Singh
Abstract:
We study hindsight‑guided distillation for rare disease diagnosis on ZebraMap: a 1.5B student is fine‑tuned on chain‑of‑thought traces from a 8B teacher that observes the ground‑truth diagnosis during generation. Absolute accuracy remains low for all models ‑ the task is hard at this scale ‑ but within this ceiling a filtered variant (StudentF) achieves a small, statistically significant accuracy advantage over the teacher (p < 0.001), concentrated in better‑represented diseases. The unfiltered student does not significantly outperform the teacher (p = 0.129), establishing that contamination filtering ‑ not hindsight distillation alone ‑ drives the gain. The gap traces to an artifact we term GT hallucination. Label‑visible generation causes the teacher to embed "ground truth is X" phrases in its reasoning chain; SFT copies the pattern. At inference, the unfiltered student reproduces the phrase in 33.9% of cases, with severe accuracy degradation when the hallucinated label is wrong. A regex filter removing these slots reduces contamination to near‑zero, producing the observed gain ‑ though the effect remains small. We precisely quantify this gain‑cost tradeoff, document frequency‑dependent knowledge transfer absent from the RL‑trained teacher, and characterize a calibration gap that SFT does not close ‑ identifying both as directions for future work.
Authors:Zhiwei Wang, Defeng He, Yuxing Li, Meilu Zhu, Edmund Y. Lam
Abstract:
Cross‑modal image matching establishes stable and accurate geometric correspondences across modalities for planar registration. Existing semantic representations provide cross‑modal consistency, but semantic similarity does not necessarily imply geometric correspondence. Meanwhile, fine‑grained CNN features provide accurate local details but lack global cross‑modal semantic guidance for stable refinement. To address these issues, we propose CDPM, which first establishes geometrically consistent semantic representations and then preserves their dominant role in correspondence estimation during fine‑grained localization. Specifically, we progressively adapt DINOv3 using geometrically consistent cross‑modal patch pairs, enabling feature similarity to better reflect true cross‑modal spatial correspondences. We then construct a DINO‑Centric Feature Pyramid, where multi‑scale DINO representations maintain stable cross‑modal correspondences, while a lightweight CNN branch provides auxiliary structural details for precise local refinement. Extensive experiments on three cross‑modal datasets demonstrate the superior performance of CDPM. On VIS‑IR, compared with the dense matcher RoMa, CDPM improves AUC@3/5/10/20 by 7.36, 13.40, 13.75, and 10.42 percentage points, respectively, and reduces mACE from 5.83 to 2.78 pixels. It also outperforms RoMa v2 across all metrics while requiring 45.6% fewer FLOPs. The online demo and dataset are available, and the code will be released on our project page at https://warren‑wzw.github.io/CDPM/.
Authors:Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis, Maria Vakalopoulou, Jose Dolz
Abstract:
Deep neural networks remain vulnerable to adversarial perturbations, which can distort not only predictions but also confidence scores, undermining uncertainty calibration. While existing certification methods focus on preserving the predicted category, providing guarantees on how calibration behaves under adversarial attacks remains overlooked. In this work, we introduce CalCErt, a simple and efficient post‑hoc strategy that certifies bin‑wise confidence calibration for any pretrained differentiable classifier. Our approach combines empirical calibration estimates, statistical concentration bounds, and local Lipschitz estimates of the confidence function to derive data‑dependent upper bounds on worst‑case miscalibration within an ell_2‑ball of radius R. We evaluate CalCErt across 11 medical image classification tasks and multiple adversarial perturbations, demonstrating substantially higher certified coverage than baseline strategies while maintaining competitive tightness. Our code is available at https://github.com/leofillioux/calcert.
Authors:Inbasekaran S
Abstract:
Serving long‑context autoregressive language models is constrained by the key‑value (KV) cache. Most dynamic eviction methods score token importance per query head and choose tokens independently. This fits poorly with grouped‑query attention (GQA), where several query heads share one physical KV buffer: divergent per‑head selections force the serving engine to retain the union of their choices ‑ inflating the cache by up to the group ratio r ‑ while arithmetic mean pooling dilutes the specialized retrieval heads that carry factual recall. We introduce Page‑EntroKV, a formal framework for KV‑cache eviction operating at the granularity GQA serving actually allocates. Heads within each physical group are pooled by weights derived from sink‑isolated collision (Renyi‑2) entropy ‑ one inner product per head, computed once at prefill with no calibration ‑ so sink heads cannot masquerade as retrieval heads. Pooled scores are projected onto PagedAttention page frames, and eviction executes at the hardware tuple (layer, group, page). We formalize the union overhead ratio (UOR) and intra‑group disagreement, prove an exact identity linking them for two‑head groups alongside two‑sided bounds at every group ratio, prove strict budget preservation and a finite‑context needle‑retention bound that arithmetic mean pooling provably violates, and give exact per‑layer page accounting. On a pilot architecture (Qwen2.5‑1.5B‑Instruct, r=6), head‑independent replay over 2,240 group measurements yields union overhead up to 4.75x at a 2% budget, while Page‑EntroKV holds UOR exactly 1.000; sink isolation removes a 13x sink masquerade; needle recall is 100% versus 0% for mean pooling at a 20% budget; retained cardinality is exact for every page size; and QA and code tasks remain solvable at 20% retention.
Authors:Seunghwan Jang, Jeongyong Yang, Siddharth Ancha, SooJean Han
Abstract:
Generative planners based on diffusion/flow matching can learn to synthesize long‑horizon trajectories from demonstrations. However, real‑world deployment requires (i) enforcing safety constraints during execution and (ii) tight online replanning at fast execution rates. Prior safe diffusion/flow planners generate the agent's full trajectory at once, while repeatedly perturbing intermediate states to satisfy safety constraints. This approach is not only computationally intensive, but also introduces distribution shift since the learned sampling dynamics is distinct from the system's execution dynamics. We propose SafeStreamingFlow, a goal‑conditioned planner that aligns flow sampling dynamics with execution dynamics by sequentially integrating a learned state vector field with hierarchical state prediction. Importantly, we need to enforce safety constraints only for the executed step via high order control barrier functions. Across navigation, racing, and locomotion benchmarks, SafeStreamingFlow reduces planning latency and improves safety compared to existing methods, while maintaining competitive goal‑reaching success.
Authors:Yun Wang, Gad Shaulsky, Tomaž Curk, Blaž Zupan
Abstract:
Biological literature retrieval systems are often developed and evaluated using broad biomedical corpora and general‑purpose search tasks. However, many curated knowledge bases operate in narrower model‑organism domains, where the literature is sparse and terminology is organism‑specific. We introduce a retrieval benchmark from dictyBase for Dictyostelium, a model organism in cell and developmental biology. The benchmark consists of curator‑generated biological queries linked to PubMed‑indexed articles, together with structured gene annotations. Using this benchmark, we study three factors in niche biological retrieval: cross‑encoder reranking, gene‑aware query expansion, and abstract‑only versus full‑text retrieval. We report that reranking and gene‑aware query expansion improve retrieval selectively: reranking is most useful when the model is well suited to biological evidence matching, whereas curated annotations help clarify compact biological queries by reducing vocabulary mismatch. Full‑text chunks substantially improve retrieval when abstracts omit supporting evidence, increasing both candidate recall and top‑rank performance, although these cases are harder than queries supported by abstracts. Data and code are publicly available at https://github.com/fulaibaowang/dictycite, and the benchmark dataset is additionally archived on Zenodo.
Authors:Chaoqun Yang, Qian Wang, Fengbin Zhu, Xinyu Lin, Bingsheng He, Roger Zimmermann, Tat-Seng Chua
Abstract:
Quantitative trading strategy design aims to discover trading programs from historical data that remain effective in future markets, which can be viewed as a black‑box program optimization problem. LLM‑based textual gradients offer a promising approach by providing explicit optimization directions for iterative strategy refinement. However, directly applying textual gradients faces two challenges: (1) optimization is myopic, underutilizing experience from previous evaluations; and (2) aggregate backtest feedback overlooks temporal robustness, potentially favoring strategies that perform well only in specific market periods. To address these challenges, we propose TradeGrad, an experience‑guided textual‑gradient framework for robust trading strategy optimization. TradeGrad leverages accumulated optimization experience to estimate textual gradients and employs multi‑scale revisions for both strategy exploration and refinement. It further introduces the Cross‑Period Robust Objective (CPRO), which emphasizes performance in unfavorable historical periods to promote temporal robustness. Experiments on cross‑sectional and time‑series strategy design in Chinese A‑share and U.S. equity markets show that TradeGrad achieves the best in‑sample and out‑of‑sample performance across all four settings. Notably, its Chinese cross‑sectional strategy achieves 27.99% annualized return, 12.19% maximum drawdown, and a Sharpe ratio of 1.63, approximately 68% higher than the CSI 300 benchmark. Further analyses validate the proposed components and show consistent improvements in both in‑sample and out‑of‑sample performance throughout optimization. The code is available at https://github.com/transcend‑0/TradeGrad.
Authors:Pedro Brandimarte, Nerea Aranjuelo, Marcos Nieto, Oihana Otaegui
Abstract:
Hardware‑aware neural architecture search (NAS) is dominated by evaluation cost: every architecture must be trained before its reward is known. Conformal‑prediction filters cut this cost by pruning candidates whose predicted‑reward upper bound misses a threshold, with a distribution‑free guarantee that at most a fraction δ are wrongly discarded. That guarantee assumes exchangeability between calibration and test candidates, which the surrounding reinforcement‑learning (RL) loop violates: the policy's proposals improve as search proceeds and, in layer‑by‑layer construction, shift within every episode. We replace one‑shot quantile estimation with online feedback control (Adaptive Conformal Inference, with tuning‑free, locally‑adaptive, and group‑conditional variants), restoring steerable coverage: dialing the target delivers it, monotonically and reproducibly, for arbitrary sequences. Across three neural‑network architecture families and both single‑step and sequential search (three seeds), it tracks every requested level to within ~10^‑3 while pruning 25‑50% of evaluations at no measured accuracy cost, whereas static calibration loses control of its coverage and a Gaussian‑process baseline stays conservative regardless of the request. Finally, used as an acquisition function on one constrained testbed, the same optimistic bound beats random search, a gain that fixed optimism already carries and online calibration sharpens. The source code is available at https://github.com/Vicomtech/rl‑hw‑nas.
Authors:Yuhan Liu, Yuxuan Ou, Ruoxi Su, Mohamed Ahmed Zaki, Yunbo Long
Abstract:
Speech delivery varies with both the requested paralinguistic attribute and the linguistic content. We introduce ParaGeo, a matched‑content decomposition of paralinguistic variation in a frozen speech language model. Synthesized audio tokens are replayed with a fixed listening prompt; pooled key/value (K/V) representations are centered and projected into a shared low‑dimensional space. Our GLM‑4‑Voice probe spans 80 requested controls from 12 benchmark families across eight sentences. With a globally fitted calibration basis, content‑held‑out centroid accuracy using this basis is 9.49% versus a 1.25% permutation baseline; same‑label cross‑content cosine similarity is 0.285 versus 0.017, and both conditional permutation tests yield p = 0.001. A separate ten‑scenario, six‑style probe reveals reproducible contrast directions across scenarios. Static, additive, and temporal interventions produce attribute‑, layer‑, and schedule‑dependent response profiles. These results provide a shared coordinate representation for measuring paralinguistic structure and an empirical starting point for latent speech control. Code is available at https://github.com/yuhanlydia/ParaGeo.
Authors:Saptarshi Nath, Inish M. D'Souza, Antonio Carta, Soheil Kolouri, Andrea Soltoggio
Abstract:
In lifelong reinforcement learning, retaining previously learned policies is not sufficient for effective transfer to a new task. Useful knowledge may be distributed across several prior policies, and its relevance may change as the learner acquires experience. One hypothesis is that task similarity can be effectively used in a continual learning setting to find and combine previously learned policies. To test it, Adaptive Mask Selection and Composition (AMSC) is designed to estimate similarity from online experience via non‑parametric Wasserstein task embeddings from state‑action‑reward samples. The z‑score‑normalized sparsemax of the similarity scores are used to derive a variable‑size support to periodically choose and weight policies to form a prior when learning a new task. On CT‑graph and MiniGrid, AMSC achieves higher mean performance and forward transfer than the evaluated modular composition baselines while exhibiting no forgetting. Results on Continual World suggest that identifying relevant prior knowledge and determining its layer‑specific composition may require additional layer‑specific tuning. Ablations show that selecting relevant sources and determining how strongly to reuse them are central to these gains. Independently measured pairwise transfer is also positively associated with task‑embedding similarity. These results indicate that task similarity can be an effective criterion to select and weight specific knowledge for reuse in lifelong reinforcement learning.
Authors:Valentino Maiorca, Walter Nelson, Francesco Locatello
Abstract:
While representation similarity across independently trained language models is well‑documented, how internal mechanics such as attention behave across models remains far less characterized. Inspired by this gap, we examine the structure of post‑softmax attention weights by marginalizing over query positions, mapping them into a joint token‑head "marginal attention space". Evaluating across 60+ diverse LLMs, we find that different properties emerge when reducing this space along its token and head axes. When reduced token‑wise, marginal attention yields a text‑intrinsic signal robustly conserved across models. To explain this property, we empirically connect marginal attention to the input‑output Jacobian of the network, and prove theoretically that under a smoothness assumption, models with similar next‑token distributions are guaranteed to have similar input‑output Jacobian statistics. When reduced head‑wise, it forms a model‑private signature conserved across documents. Practically, this provides a natural way to estimate a per‑head budget for key‑value (KV) cache eviction, effectively decoupling model‑specific budget allocation from text‑intrinsic token scoring. On standard eviction benchmarks, a per‑head budget precomputed offline on pretraining text, combined with a training‑free token score, shows competitive performance with methods that recompute the budget on every document or train it per target. Code available at https://github.com/Flegyas/marginal‑attention
Authors:Siyu Wang, Yifan Wang, Yuecheng He
Abstract:
Large language models are moving from producing trading signals to writing the code that executes them. The failure mode of the second role is silent: generated code runs, a backtest plots, yet the risk logic that the trader described is not the logic being executed. Existing code benchmarks test functional correctness on unit tests and finance benchmarks test forecasting; neither measures whether an implementation behaves like the strategy that was asked for. We introduce MintEval, a benchmark in which reference strategies are generated programmatically from a library of composable building blocks, back‑translated into colloquial trader instructions, and re‑implemented by the model under test. Generated and reference programs are executed bar by bar on identical market data and frictions, and compared on their actions rather than on code similarity or profit: alpha is differenced away. MintEval v0 contains 800 tasks on BTCUSDT 15‑minute data, stratified by an execution‑measured state‑span complexity tau that is decoupled from description length. Low‑cost models reach a mean ActionMatch of at most 0.544 and reproduce at most 0.087 of tasks exactly; on a stratified subset of 200 tasks a frontier model (Claude Opus 5.5) reaches 0.889 and reproduces 0.575 exactly, yet still fails silently on 0.275 of tasks. Given a menu of building blocks, models identify the strategy almost perfectly, yet 79.2% of the implementations whose specification was read correctly diverge on more than 10% of active bars. The LLM judge of a recent strategy‑generation benchmark, applied verbatim, accepts every one of these silent failures.
Authors:Donghyun Lee, Taehoon Lee, Geonhee Ahn, Jieun Kim, Jihyun Park, Suyeon Cho, Yoona Kim, Chaerim Shin, Hoi Ri Moon, Jonggeol Na, Sukho Hong, Jihwan Oh, Soo Kyung Kim
Abstract:
Large language models are increasingly used as reasoning components in AI‑driven materials Co‑Scientists, yet the reliability of the resulting verification pipeline remains unclear. Metal‑organic frameworks (MOFs) provide a particularly challenging setting because structures may appear under different identifiers, synthesis outcomes depend strongly on experimental conditions, evidence is distributed across heterogeneous sources, and some hypotheses require computation rather than literature alone. We introduce a diagnostic benchmark with four task families covering structural grounding, synthesis‑condition verification, evidence‑sufficiency verification, and MLIP‑based computational verification. T‑MOF‑1‑3 are evaluated under closed‑book, retrieval‑enabled, and oracle‑evidence settings to localize failures in knowledge access, evidence acquisition, and reasoning, while T‑MOF‑4 separately evaluates computational verification. Guided by these diagnosed failure modes, we develop MOF‑Verify, a failure‑aware agentic harness that targets structural, literature, evidence‑sufficiency, and computational bottlenecks before producing a final verdict. Across multiple backbone LLMs, MOF‑Verify substantially improves hypothesis‑verification performance over direct inference and retrieval‑based baselines. Benchmark datasets are released at https://github.com/IMMS‑Ewha/MOF‑Verify‑Benchmark.
Authors:Jiangang Han
Abstract:
We present WebFovea, a vision‑based web agent that placed 2nd in the WebRetriever Challenge 2026 with a final score of 57.0 out of 100. The challenge evaluates agents end to end on Protocol III of the WebRetriever benchmark (arXiv:2607.06118): starting from an entry URL on a live website, the agent must operate the site's own interface and return a verifiable answer. A capable multimodal large language model (LLM) is necessary for this, but not sufficient. The model's decisions reach the browser through the harness, the code between the model and the page. At every step, four things must go right: the model's reply must be parsed into the intended action, the action must take effect on the page, the result must be reported back accurately, and the model must be shown the information it needs. On real websites, many of the failures we observed occurred at one of these four stages rather than in the model's reasoning. A coordinate‑space mismatch placed every click at 3/4 of its intended coordinates; actions on native dropdowns, inside iframes, and in text boxes failed silently; and self‑generated chat‑template tokens contaminated 4.9% of task episodes. WebFovea hardens each stage and surrounds the loop with guardrails that keep the agent within the rules and its budget. The four‑stage view does not depend on the model, although some individual fixes do. Because we used the same model in all four submissions, the rise of our official hidden‑set score from 31.0 to 57.0 reflects changes to the harness, up to run‑to‑run variance on live sites. We describe the design, the evidence for each component (including negative results), a failure analysis, the limitations, and a roadmap that includes routing different steps to different models. Code is available at https://github.com/jianganghan/WebFovea.
Authors:Ella Kemperman, Luca Ambrogioni
Abstract:
Diffusion models rely on numerical solvers requiring time‑discretization, which has a large influence on the tradeoff between sampling cost and quality. However, the computational difficulty of the reverse process varies along the sampling trajectory and across data distributions, making the choice of discretization important. We adapt proportional‑integral (PI) step‑size control to diffusion, using our diffusion noise‑normalised error estimator. Unlike existing adaptive methods in diffusion that respond only to the current error, the PI solver also incorporates the previous error, yielding smoother step adaptation. We further show that these per‑sample trajectories exhibit shared structure and can be aggregated into a fixed schedule that retains much of the benefit of adaptive sampling. We evaluate both approaches on natural‑image and language datasets, in terms of quality, measured by FID at a matched number of neural network evaluations (NFE), comparing them with widely used stochastic solvers and schedules. For images, our fixed discretization outperforms the commonly used EDM schedule in terms of sample quality when used with the stochastic Heun sampler, and with the EDM‑churn sampler at low NFE. Additionally, our PI adaptive solver obtains better FID than most stochastic and adaptive baselines, although it does not beat the EDM‑churn sampler at low NFE. Moreover, we find our solver outperforms both the EDM and the entropy schedule on language diffusion at low‑to‑medium NFE in terms of perplexity, with the drawback of lower token entropy. Lastly, we find that the benefit of per‑sample adaptivity is problem‑dependent. It is highly beneficial in 1D toy examples, while only marginal for image and language data, where the average schedule sometimes even outperforms the PI‑adaptive solver. Code is available at https://github.com/ellakemperman/adaptive‑second‑order‑diffusion‑solvers
Authors:Runtong Wu, Fei Teng, Di Wen, Guoqiang Zhao, Kunyu Peng, Kailun Yang
Abstract:
Accurate 3D detection is essential for mobile embodied agents, while Vision Foundation Models (VFMs) offer transferable visual and geometric priors. Yet existing VFM‑based 3D detectors rely on narrow‑view monocular images or discrete perspective views, limiting coherent surround perception; equirectangular projection (ERP) instead encodes a continuous 360 scene in a single image. Direct transfer remains difficult because ERP organizes geometry and visual information differently, making object‑relevant cues hard to model, localize, and preserve. We propose OmniAct3D, a framework that adapts perspective‑trained VFM detectors to ERP while preserving transferable VFM priors. To resolve geometric mismatch, the ERP‑Ray Geometry Adapter (ERGA‑Ray) models spherical viewing rays and periodic spatial structure. To localize evidence in scene‑wide context, the Visual‑Action Reasoning Chain (VARC) grounds each hypothesis in relevant panoramic evidence and converts it into a structured geometric action. To recover local cues lost under fixed token budgets, the Appearance‑Guided Heading Expert (AGHE) re‑encodes object regions at higher resolution for heading estimation. Experiments show that OmniAct3D improves over the previous best 3D detector by 2.96 NDS points on Spheriverse and over the unadapted VFM baseline by 24.87 mAP points on PanoMMOcc. With target‑specific geometry adaptation, VARC retains 95‑‑98% of the same‑configuration mAP, indicating reusable object‑level 3D reasoning across sensing configurations. The source code will be made publicly available at https://github.com/FeiT‑FeiTeng/OmniAct3D.
Authors:Hakjin Lee, Junghoon Seo, Jaehoon Sim
Abstract:
Category‑level object pose estimation predicts the rotation, translation, and metric size of unseen instances within known categories. Many accurate RGB‑D methods rely on external instance segmentation and crop‑based pose estimation, introducing separate stages and object‑dependent processing costs that hinder real‑time inference. To bring accurate pose estimation into real time, we present \ours, an end‑to‑end trainable query‑based RGB‑D set predictor. It jointly detects and segments objects and estimates their \mbox9‑DoF poses without explicit CAD‑derived shape priors or a separately trained instance segmentor. Shared image and scene encoding avoids repeated per‑object crop encoding. A query‑conditioned geometry pathway associates observed 3D points and RGB features with object queries and incorporates shared scene context. Object‑centric refinement uses the resulting point descriptors to update an explicit pose state through pose‑conditioned cross‑attention and recurrent residual corrections. On NOCS, \ours substantially improves on published RGB‑D joint detection and pose estimation results. It achieves competitive performance compared with two‑stage methods under all‑object evaluation on REAL275 and HouseCat6D, while enabling real‑time full‑frame pose estimation at 31.8 FPS on an RTX~A6000. Project page: https://yopo‑series.github.io/RYOPO‑project‑page/.
Authors:Yunji Kim, Yunseok Lee, Hyunwoo Seo, Jaerim Choi, Woojin Lee
Abstract:
Programmable logic controllers (PLCs) coordinate industrial equipment by reading sensor inputs and issuing control commands. Evaluating whether large language model (LLM)‑generated PLC programs satisfy task requirements and safety constraints requires observing how their commands affect device and workpiece states. We introduce PLCWorld, a common closed‑loop execution environment and benchmark that couples Structured Text (ST) execution with simulated plant responses and sensor feedback. Grounded in control relations identified in industrial PLC programs and engineering documentation, PLCWorld contains 100 synthetic tasks and 473 registered task‑condition pairs across Motion Control and Material Handling, with difficulty defined by control‑dependency scope. A common protocol reports Task Success and Safety Violation separately. Validation combines practitioner review, reference and alternative programs, targeted counterexamples, specification‑evaluator alignment checks, and comparisons with independent ST runtimes. Reference and alternative programs satisfy their applicable cases, while all 542 targeted counterexamples activate their designated evaluator rules under at least one registered condition. Execution Gap relates submission‑profile acceptance to subsequent task failure or observed Safety Violation. Across the constructed task groups, direct GPT‑5.5 achieves 82.70% Task Success on Easy cases but 25.10% on Hard cases. Evaluations of six LLMs and four adapted generation‑and‑verification workflows further expose differences between completion, safety, and generation cost. Our code, simulation environment, benchmark tasks, and baseline implementations are publicly available at https://yunji0516.github.io/PLCWorld/.
Authors:Kangjun Noh, Soyu Kim, Kyungwoo Song
Abstract:
Large language model (LLM)‑based self‑evolving search is a promising approach to scientific discovery. However, high‑fidelity evaluation of every candidate is prohibitively expensive in some domains. Self‑evolving systems in such settings therefore rely on low‑cost but imperfect proxy rewards, which may assign high scores to infeasible candidates. These false positives may contaminate both the final output and the feedback used to guide subsequent generations. This motivates statistically calibrated reward intervals for more reliable self‑evolving search. We propose Conformal Interval‑Driven Self‑Evolution (CISE), which constructs candidate‑specific reward intervals using conditional conformal inference and iteration‑wise online density‑ratio estimation. CISE uses conservative interval‑based rewards for evolutionary feedback and returns candidates only when all required property intervals lie entirely within their respective feasible regions. We derive fixed‑iteration coverage results under explicit assumptions of independence and covariate shift. We evaluate CISE on three self‑evolving search tasks in materials science. In our experiments, all candidates returned by CISE are true positives under high‑fidelity evaluation, whereas the baselines return more candidates but include false positives. These results highlight the value of a smaller, more precise shortlist when downstream validation budgets are limited. Our repository is available at https://github.com/MLAI‑Yonsei/CISE.git.
Authors:Rupsa Roy
Abstract:
Large language models (LLMs) can support rubric‑based screening of CVs, but applying a high‑capability model to every candidate and criterion is costly. We present CreateScore, a domain‑theory‑informed Bayesian network for criterion‑level LLM routing. A hand‑specified directed acyclic graph with Dirichlet‑multinomial conditional probability tables converts CV evidence into posterior uncertainty; low‑uncertainty decisions are resolved by a local 8B model and uncertain ones are escalated to a 120B reference model. The graph is causally motivated, but the system performs standard Bayesian conditioning, not causal inference. The escalation threshold is calibrated on a training fold (target: 70% resolved locally) and then frozen. On 200 synthetic Data Science CVs (139 training and 61 test candidates, five criteria), 77.7% of criterion decisions were resolved locally (237 of 305). Relative to a reference condition in which the 120B model adjudicated every criterion, routed escalation reduced token use by 65.2% and raised exact score agreement from 32.8% (8B alone) to 42.6% (95% CI 31.0‑55.1%); at n = 61 the gain was not statistically distinguishable. The uncertainty signal did not, however, identify the decisions on which the 8B model erred: disagreement with the reference was 16.2% among escalated and 19.4% among locally resolved decisions (AUROC 0.47, 95% CI 0.39‑0.56), no better than random selection. We also document how an earlier evaluation was invalidated when truncated reasoning‑model outputs were silently replaced by local labels, and we recommend safeguards for cascade evaluation. CreateScore is supported as an auditable cost‑reduction mechanism, not yet as a targeted error detector, and is not an autonomous hiring system.
Authors:Shuai Fu, Jing Gu, Jian Zhou, Zicheng Duan, Gengze Zhou, Qi Wu
Abstract:
Recent text‑to‑image models have made substantial progress in photorealism, aesthetics, and text‑image alignment. Yet visually appealing images can still violate real‑world plausibility, exhibiting malformed object structures, impossible anatomy, physically implausible interactions, or inconsistent spatial relationships. Such failures are not well captured by existing fidelity, aesthetics, preference, or alignment metrics. To address this gap, we introduce TerraVis, a framework for evaluating world‑grounded visual consistency in generated images. TerraVis defines a structured taxonomy of world‑consistency violations spanning object‑, interaction‑, and scene‑level failures, and employs a multi‑stage evaluation framework to identify and quantify them. Given an image, TerraVis first uses an MLLM to assess its eligibility for evaluation, then detects violations across 18 taxonomy‑defined types and classifies them as minor or major to derive an overall world‑consistency score. Across diverse open‑source and proprietary text‑to‑image models on two widely used benchmarks, TerraVis achieves the strongest correlation with human judgments of world consistency among existing metrics. Our benchmark results further show that models that achieve strong performance on conventional metrics can still exhibit substantial world‑consistency failures. These findings highlight world consistency as a complementary evaluation dimension and demonstrate that TerraVis enables systematic quantification, diagnosis, and comparison of such failures. Our code is publicly available at https://github.com/ShyFoo/TerraVis.
Authors:Jihwan Hong, Woohyeon Park, Jaeik Kim, Jaeyoung Do
Abstract:
Video Object Segmentation (VOS) in complex and long videos is increasingly important for real‑world applications, where target objects often appear only intermittently within long temporal horizons. However, existing benchmarks largely focus on temporally salient objects that remain visible for most of the video. To address this gap, we introduce FaVOS (A Benchmark for Video Object Segmentation with Fractional Temporal Visibility), a benchmark designed to evaluate VOS methods under low temporal visibility. We show that, in this regime, the standard J&F metric can collapse VOS evaluation into absence classification, because empty predictions receive high rewards on target‑absent frames. Consequently, even a trivial empty‑mask predictor can outperform strong models such as SAM 3, revealing a fundamental mismatch between current metrics and practical VOS performance. To mitigate this issue, we propose Volumetric J&F, which evaluates mask sequences as spatio‑temporal volumes and reduces the dominance of target‑absence rewards while preserving sensitivity to segmentation quality and temporal structure. Project page: https://aidaslab.github.io/FaVOS.
Authors:Xiqiao Xiong, Moxin Li, Zhixin Ma, Ouxiang Li, Wenjie Wang, Fuli Feng, Xiangnan He
Abstract:
Agent harnesses play a critical role in defenses by enforcing safety constraints to prevent unsafe actions. However, rapidly emerging attacks outpace manual harness adaptation, motivating automated harness evolution. Yet the signals available for harness evolution are often sparse, such as brief descriptions or a few attack examples in threat reports and preprints. To address this limitation, we introduce HASTE, a multi‑agent framework that evolves agent harnesses from sparse threat evidence through an adversarial interplay between safety‑specification generation and attack‑case generation. Safety specifications guide harness updates toward addressing identified safety vulnerabilities, while attack cases probe for remaining safety vulnerabilities after each update. By feeding evaluation outcomes back into both processes, HASTE enables harness evolution against emerging attacks beyond the initially observed evidence. Experimental results across multiple backbone models, attack types, and evidence forms show that HASTE consistently reduces attack success rates while preserving benign‑task utility. The code is available at https://github.com/xxiqiao/HASTE.
Authors:Drew Edwards, Akira Maezawa, Simon Dixon
Abstract:
Jazz pianists develop distinctive traits that experienced listeners can often identify within seconds, yet the features underlying this recognition resist formal description. We study jazz pianist style through the lens of a pretrained symbolic music transformer, showing that its learned representations already encode pianist identity well enough for highly accurate classification across two benchmarks. We then augment the transformer with cross‑attention over learned pianist identity embeddings, enabling it to generate music conditioned on a specific artist's style. Two evaluation protocols confirm that the generator captures meaningful stylistic structure: a sliding‑window classifier consistently attributes conditioned continuations to the correct artist, far above unconditioned baselines; and a classifier trained entirely on synthetic generations identifies real pianists across 12 classes with 87% chunk‑level and 95% song‑level accuracy. Finally, we repurpose the classifier to locate the most characteristic moments within a performance, surfacing the specific musical gestures that distinguish each pianist's voice.
Authors:Yuting Yan, Shihao Xu, Junhao Yu, Mingcong Zuo, Lu Chen, Nan Xiang, Haiyang Geng, Dongjie Tao, Minghao Wang
Abstract:
Mental health disorders affect a substantial proportion of the global population, yet a persistent shortage of trained practitioners leaves the majority without adequate care. Large language model (LLM)‑based counselors present a promising direction for delivering scalable conversational psychological support. Offline model training alone leaves limited room to adapt to individual clients or to learn from ongoing therapeutic interaction at test time. We introduce PsyEvo, an LLM‑based counseling framework that enables both client‑specific personalization and response‑policy improvement at test time through three components: Hierarchical Bayesian Skill Policy (HBSP) personalizes what intervention to apply by maintaining a per‑client skill posterior updated from session feedback; Inter‑session Listwise Preference Optimization (LiPO) improves how the selected skill is expressed by updating a shared response adapter from cross‑client preference evidence; and State‑conditioned Ordinal Credit Assignment (SOCA) supplies candidate preferences and trajectory credit to the two components through consistency‑checked comparisons and ordinal projection. In simulated‑client evaluation with shared online cohort adaptation, PsyEvo obtains 7.684 Overall on PsychEval and exceeds every component variant in each of three matched runs. Removing individual components lowers mean overall score by 0.138‑‑0.171 under the shared configuration, supporting conditional contributions within the complete scaffold. Our code is available at https://github.com/Lingxi‑mental‑health/PsyEvo
Authors:Daewon Chae, Hyunwon Chung, Changwoo Lee, Hun-Seok Kim
Abstract:
Large‑scale foundation models achieve strong performance across diverse tasks, but their size makes inference costly, largely due to dense matrix multiplications. Prior work reduces this cost by replacing dense weight matrices with efficient structured forms such as low‑rank factorizations. However, these methods approximate weights rather than the output activations that determine inference accuracy. Consequently, small weight‑space errors can be amplified by input activations, producing large output errors. In this work, we propose DyRA, an input‑adaptive method that improves structured matrix multiplication approximation by correcting residual output errors during inference. We show that matrix multiplication can be approximated more effectively by directly optimizing low‑rank factors of the output. DyRA builds on this insight by dynamically approximating and correcting the output error introduced by structured weight approximations. This combines efficient structured computation with input‑dependent correction, yielding a more faithful approximation of full matrix multiplication under the same computational budget. Across vision, speech, and language models, DyRA consistently improves the accuracy‑efficiency trade‑off over structured weight approximations alone. Notably, DyRA achieves a 1.5× end‑to‑end GPU speedup for DINOv3 while reducing accuracy degradation by more than 3× relative to weight‑only baselines.
Authors:Qingtao Xia, Siyao Cheng, Jiahua Bao, Jiaxing Du, Jie Liu
Abstract:
Retrieval‑augmented document question answering assumes that once the right page is found, a vision‑language model (VLM) can read it. We show that this assumption often fails, leaving a retrieval‑reading gap: evidence found but not used. A paired protocol isolates this gap by comparing answers from the retrieved page images alone with answers from the same images plus their extracted text. On FoveDoc‑Bench, our benchmark with traceable evidence, retrieval finds nearly every evidence page, yet adding CPU‑OCR text raises strict accuracy by 13 to 16 points. An exact text layer roughly doubles the gain, which appears across six VLMs from three families and, within one family, narrows with scale without closing. The reader can read this evidence but cannot find it: crops of it recover most of the text gain, boxes around it on the page none. The same protocol identifies two boundaries. Extracted text helps on textual evidence but is neutral or harmful on charts and figures. Its advantage shrinks as retrieval degrades, and unrelated text of the same form adds nothing detectable. Extracted text is an amplifier of retrieval that works, not a substitute for retrieval that does not. Our code is available at https://github.com/atoz03/fovedoc‑sup.
Authors:Yicheng Zhu, Linfeng Tian, Tianmu Zhao, Yang Chen, Fan Zuo, Tao Li, Zilin Bian
Abstract:
Scalable driving simulators typically execute vehicle commands using prescribed behavioral or kinematic rules, overlooking the physics of tire‑road interfaces, thereby limiting their ability to capture how adverse road and environmental conditions alter vehicle execution and propagate through traffic. To address this limitation, we present SceneFactory‑3D, a GPU‑batched, physics‑grounded multi‑agent driving simulator. Vehicles execute acceleration and steering commands via suspension‑ and friction‑limited forces evaluated at each wheel‑contact point. Spatially varying friction, per‑world 3D heightfields, gravity, and rigid contact consistently govern wheel motion and chassis collisions. Per‑world terrain isolation and GPU batching enable SceneFactory‑3D to run matched physical counterfactuals in parallel: traffic scenario setup and vehicle controllers remain fixed while only the road condition changes, enabling the resulting closed‑loop effects to be evaluated across parallel worlds. To demonstrate the advantage of the SceneFactory‑3D‑enabled counterfactual evaluation, we conduct an empirical study on vehicle controllers' sensitivity to road conditions. We study three learned‑policy families on 1,024 matched 12‑vehicle worlds per condition, and two classical planners on a shared 32‑world subset, across 21 friction and grade conditions. When friction drops from 1.0 to 0.18, the share of vehicles that clear the work zone safely falls by 6 to 90 percentage points across learned policies (18‑19 for classical planners), and near‑collision situations become more frequent for every learned policy. Code: https://github.com/SmallWorldLab/SceneFactory_3D
Authors:Wei-Jin Huang, Yuan-Ming Li, Kun-Yu Lin, Wang Luo, Yinlin Zhu, Yue Yu, Shenghao Ye, Junbin Yuan, Fa-Ting Hong, Qing Zhang, Wei-Shi Zheng
Abstract:
Recent text‑to‑motion models have improved motion quality and instruction following, yet many‑step denoising and large model components make deployment slow and memory‑intensive. We present Terminal‑Amplification‑Controlled Distillation (TACD), an on‑policy approach for training efficient motion generators from text prompts and pretrained teachers, without real‑motion training data. Building on segmented on‑policy flow distillation, we supervise clean‑motion predictions along student‑generated trajectories. We identify a failure mode in which velocity matching on a fixed supervision grid repeatedly overweights errors near the denoising endpoint, degrading few‑step generation. TACD ties the latest teacher query to the student's step size, bounding the effective loss weights in clean‑motion space without changing inference. Experiments on HumanML3D and KIT‑ML demonstrate improved few‑step generation, including a 58% reduction in eight‑step HY‑Motion student FID relative to distillation without this bound. For diffusion teachers, the endpoint‑matching form of TACD yields four‑step students with lower FID and matched or improved text‑motion retrieval relative to their 50‑step teachers on HumanML3D. On HY‑Motion and Kimodo, eight‑step students with compact components achieve 7.7‑11.9x end‑to‑end speedups and reduce peak GPU memory by 3.8‑6.7x relative to their teachers. Project page: https://vkgo.github.io/TACD/
Authors:Chunghyun Park, Beomjun Kim, Seungcheol Park, Heeseung Kwon, Yashu Shukla, Seunghoon Sim, Jinwoo Shin, Minsu Cho
Abstract:
World action models jointly learn to forecast world dynamics and predict robot actions, such that the learned internal world dynamics guide accurate actions. Existing approaches typically represent the world as RGB frames or latent counterparts while predicting actions as end‑effector poses or joint angles, but they often struggle to capture the 3D spatial structure and contact geometry central to dexterous manipulation. We introduce Point World Action Model (PointWAM), a 3D world action model that decomposes the world into a scene (i.e., environment) and hands (i.e., actor), and jointly forecasts both as 3D point trajectories within a shared space‑time coordinate frame. This explicit, disentangled representation enables effective pre‑training on large‑scale human demonstration videos without requiring any task‑specific object or keypoint selection. Given a colored point cloud and a language instruction, PointWAM predicts how the scene and hands co‑evolve in 3D space over time, then retargets the forecast hand motion to robot actions. Pre‑training on human videos improves average DexJoCo success by 56.9 percentage points, and scene‑trajectory supervision adds 10.9 points over forecasting the hands alone. With both, PointWAM surpasses the prior state of the art on ten DexJoCo tasks by 11.7 points and outperforms strong VLAs on a real robot.
Authors:Qiyuan Huang, Tianshi Xu, Meng Li
Abstract:
During post‑training of large language models (LLMs) with Reinforcement Learning with Verifiable Rewards (RLVR), GRPO‑style algorithms can exhibit severe late‑stage collapse. Prompt‑based probing reveals that this is not benign strategic pruning, but a harmful contraction of effective strategy capacity that makes distinct reasoning strategies increasingly inaccessible. To characterize this phenomenon, we define strategies through trajectory‑level policy‑update interactions and develop a unified theoretical framework combining optimization dynamics and information theory. We prove that major RLVR objectives progressively concentrate probability mass onto a single strategy, while sustaining nontrivial task accuracy requires a minimum strategy capacity. The conflict between these two results provides a mechanistic explanation for catastrophic collapse. We further derive the Mirrored Entanglement Index (MEI) as a lightweight online warning signal. To prevent collapse, we propose Mesh Learning, which exposes multiple reasoning strategies and prevents any single strategy from dominating optimization. Across AIME26, AIME25, MATH‑500, GPQA, and LiveCodeBench, Mesh Learning consistently outperforms strong baselines across Qwen and Phi model families, with gains of up to 13.4 pp and 11.5 pp, respectively. These results establish strategy preservation as a key principle for stable RLVR. Code is available at https://github.com/Ayanami‑0123/Open‑Mesh‑Learning.
Authors:Wenteng Chen, Jiachen Zhu, Rong Shan, Tianyi Xu, Yuxiang Chen, Congmin Zheng, Teng Wang, Junjie Wu, Weiwen Liu, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
Abstract:
Vision‑language models (VLMs) are powerful listwise rerankers for multimodal retrieval, but high inference costs restrict them to evaluating small local candidate views. Existing multi‑call strategies rely on fixed schedules, wasting expensive VLM calls on uninformative candidate pairs and easy queries. To address this, we propose Adaptive Multi‑view Budgeted Elo Reranking (AMBER), an online, budgeted multi‑view reranking framework that dynamically optimizes global resource allocation. AMBER treats fragmented listwise VLM outputs as local tournaments, using continuous Elo updates to maintain a lightweight global ranking state. Building on this, it allocates computation at two levels: dynamically constructing candidate views with high score ambiguity, and scheduling queries to maximize expected information gain. We show that each Elo update corresponds to a stochastic gradient ascent step on the Bradley‑Terry log‑likelihood, and provide a submodular information‑theoretic motivation for the query‑level allocation strategy. Experiments on CIRR, CIRCO, and PhotoBench demonstrate that AMBER achieves the strongest overall performance among the compared multi‑call VLM reranking methods under comparable VLM‑call budgets, while remaining effective in lower‑budget settings. Our code is publicly available at https://github.com/wnlfc/AMBER.
Authors:Yi Wang, Baicheng Chen, Yu Wang, Jian Zhao, Yilei Chen, Tianxing He
Abstract:
Large Language Model (LLM) watermarking provides a lightweight mechanism for identifying text generated by a specific model, but its robustness remains fragile under post‑processing attacks. Deletion attacks are particularly challenging because they shift token positions and break the alignment between observed tokens and their original watermark positions. We propose Reed‑‑Muller Code Watermarking (RMCW), an LLM watermarking method based on Reed‑‑Muller codes. In contrast to global codeword recovery, RMCW searches for surviving local algebraic structure, leveraging the Reed‑‑Solomon consistency induced by affine‑line restrictions of Reed‑‑Muller codewords. During generation, RMCW injects a Reed‑‑Muller structure into the sequence via a secret‑keyed vocabulary partition. During detection, it maps the given text to keyed vocabulary bins and tests local subsequences for low‑degree Reed‑‑Solomon consistency using Berlekamp‑‑Welch tests. Experiments on C4 and ELI5 datasets with OPT‑1.3B and Llama‑3.1‑8B‑Instruct show that RMCW preserves strong clean‑text detectability and outperforms or matches the baseline methods under several deletion and rewriting attacks. Our code is available at https://github.com/BaichengDanny/RMCW.
Authors:Naing Oo Lwin
Abstract:
Keeping a Python program and its formal guarantees aligned is a continuing maintenance problem. Specifications must describe the implementation that actually runs, and updates must preserve the behavior on which existing callers depend. VeriPy brings these obligations into a common workflow for annotated Python components. Developers and agents express contracts, invariants, and proof hooks as Python comments, then refine checked auxiliary lemmas using source‑located diagnostics. Direct encoders produce Dafny or Lean artifacts while retaining admitted executable bodies and explicit models of their dependencies. A relational product checks whether an updated component still admits old inputs and preserves returned values and modeled exceptions. The resulting workflow connects source‑level proof development with functional verification and backward compatibility, recording the assumptions under which each guarantee applies. The code is available at https://github.com/astrio‑labs/veripy.
Authors:Wenya Su, Kai Luo, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Kunyu Peng, Kailun Yang
Abstract:
Fisheye cameras give mobile robots a single‑sensor, low‑cost view of their surroundings, yet the COCO‑pretrained detectors that practitioners routinely reuse fail on them: strong radial distortion warps local image structure, while boundary compression shrinks objects to near‑invisible sizes. Full fine‑tuning closes much of the gap but requires abundant fisheye labels and compute. We present FUSEye, a training‑light framework that turns a frozen‑backbone COCO‑pretrained extra‑large YOLO26 detector (YOLO26‑x) into a fisheye detector. FUSEye adds roughly 227k new parameters while updating the inserted modules and the pretrained detection head. It addresses the transfer gap at three causally linked levels. At the input level, overlapping grid view generation and box remapping (GridViews) enlarge compressed boundary regions. At the feature level, zero‑initialized residual adapters (Z‑Adapters) correct distortion‑induced feature misalignment. At the decision level, learned cross‑projection agreement fusion (AgreeFusion) promotes low‑confidence detections only when they are supported by consistent evidence across multiple views. On the WoodScape surround‑view fisheye benchmark, FUSEye raises YOLO26‑x from 0.148 to 0.266 mAP50 and retains 84.3% fully fine‑tuned accuracy. Moreover, randomly using only 25% of the labeled training images, FUSEye achieves 0.2597 mAP50, retaining 97.6% of its full‑label performance. FUSEye also consistently improves YOLOv8‑11 detectors, showing that the recipe is architecture‑agnostic. Source code will be available at https://github.com/Su‑wenya/FUSEye.
Authors:Naomi Baes, Jemima Kang, Nick Haslam, Chris Groot, Alsa Wu, Luc Raszewski, Yulia Otmakhova
Abstract:
Mental health stigma has profoundly harmful impacts but its complexity makes it difficult to evaluate. Stigma may involve explicit derogation, but also subtler forms of blame, fear, paternalistic pity, social distancing, structural exclusion, and discrimination. We introduce a theory‑grounded benchmark for automatic evaluation of mental health stigma in online communication, consisting of naturally occurring online news and social media text annotated with a fine‑grained taxonomy of stigma across multiple mental health conditions. Our annotation framework comprises a binary stigma‑detection task and a multi‑level taxonomy covering (i) stigma mode, (ii) domain, and (iii) specific components of certain forms of stigma. We apply this framework to texts mentioning six mental health conditions and evaluate large language models alongside stigma‑related classifiers for detecting sentiment, toxicity, and hate speech. Results show that mental health stigma is not well captured by models trained to detect these neighboring constructs, and that LLMs often overpredict stigma unless given explicit operational rules ‑ mirroring the importance of decision rules in human annotation. We release the publicly available part of benchmark, annotations, prototypical exemplars of stigma and code at: https://github.com/jemimakang/mh_stigma.
Authors:Shivam Gupta
Abstract:
Retaining language‑model prefix states trades recomputation against storage time. Optimizing each cached block independently can overcount savings: a resident block is usable only when the required preceding prefix is also available. We introduce Prefix‑Certificate Retention (PCR), an exact finite‑trace formulation for static, grouped, reset‑on‑access timeouts. Usable‑prefix rewards become nodes whose prerequisites are timeout thresholds and preceding hit certificates. The resulting maximum‑weight closure reduces to one minimum cut, with graph size linear in the number of block lookups and timeout choices. A breakpoint theorem extends the construction to all nonnegative timeouts without discretization error. We also derive a linear‑time‑in‑grid‑size dynamic program for ordered timeouts and bounds that certify the cost of this restriction. Exhaustive small‑instance checks and chronological replay of 39,632 public Mooncake requests validate the formulation. On the fixed grid, ordered timeouts attain the unrestricted training optimum in 118 of 120 trace‑grouping‑price cases. Heterogeneous retention improves several held‑out memory‑time tradeoffs, but finer training optimization does not uniformly improve transfer. The contribution is a tractable optimization model and an auditable benchmark for retention policies; the experiments measure usable prefix blocks and storage time, not GPU latency.
Authors:Shivam Gupta
Abstract:
In structured rating prediction, does a personal AI use historical item‑rating associations, or mainly the user's rating tendencies? We audit this distinction by permuting historical ratings within each user while preserving the exact rating distribution, item support, and metadata. We combine this control with full history, native memory extraction, and matched numerical readers in a publicly frozen evaluation of 400 held‑out user profiles and 6,160 target ratings across Coat and MovieLens. On Coat, the tested Qwen‑written Mem0 pipeline increases user‑macro mean absolute error relative to full history by 0.084 for Qwen and 0.149 for Phi; both family‑adjusted bootstrap intervals exclude zero. Correct historical assignments help both readers on Coat, but the corresponding MovieLens effects are smaller and inconclusive after adjustment. A history‑only ridge reader outperforms Qwen in both domains and Phi on MovieLens, while the Coat Phi comparison is unresolved. All 2,800 reader calls, including 150 invalid outputs, are retained under a fixed fallback rule. A separate implementation verifies inputs, metrics, and all ten primary contrasts. The contribution is a reproducible diagnostic study showing why extraction, association use, output reliability, and reader choice require separate evaluation.
Authors:Fangyuan Wang, Songhao Huang, Haoxiang Sun, Shipeng Lyu, Chengyang He, Anqing Duan, Peng Zhou, David Navarro-Alarcon
Abstract:
Generative robot policies predict short action chunks but lack explicit long‑horizon intent. Recent methods expose longer‑horizon structure through language plans, subgoal images, or video forecasts, which are costly to generate and still need to be translated into robot motion. Predicting future robot motions avoids this translation, but a dense, time‑indexed trajectory requires numerous parameters to cover the full remaining task, and over a short horizon it largely repeats the action chunk and adds little guidance for action generation. We propose Proprioceptive Action Models (PAM), which jointly generate a compact, timing‑free sketch of the robot's remaining joint‑space path and a dense executable action chunk within a single transformer denoiser. The sketch parameterizes the path by arc length rather than time, capturing geometric intent invariant to execution timing. Block‑causal attention and a staggered denoising schedule maintain directed sketch‑to‑action dependence, ensuring the action tokens condition on a progressively cleaner sketch throughout sampling. In simulation, PAM improves over its action‑only counterparts on Push‑T and LIBERO‑Long; on four real‑world bimanual tasks, it raises success from 47.5% to 75.0%. Project page: https://nicehiro.github.io/pam_dp/
Authors:Zhiyuan Zhang, Jingyuan Xu, Yiming Tang, Liu Liu, Dan Guo
Abstract:
Multi‑scale design is crucial for efficient audio‑visual speech separation, yet effectively modeling multi‑scale information for audio‑visual feature fusion remains challenging. We argue that the limited capacity of existing approaches primarily arises from: 1) treating features from different modalities in the same manner, and 2) overlooking the role of global features. To address these issues, we propose a Global‑guided Asymmetric Attention Network (GAANet). Our model introduces two core innovations: first, an asymmetric multi‑scale fusion framework that allows audio and visual streams to extract and interact with features at their respective optimal temporal resolutions, removing the need for symmetric temporal downsampling; second, a global‑guided attention mechanism that compresses each modality into a compact global token with a temporal dimension of one, which then provides high‑level semantic cues to guide both intra‑ and inter‑modal fusion across scales. Experiments on LRS2 and VoxCeleb2 demonstrate that GAANet achieves state‑of‑the‑art performance, reaching 16.5 dB SI‑SNRi on LRS2 and 14.0 dB on VoxCeleb2, while maintaining a lightweight computational profile with only 3.3M parameters and 19.8G MACs. These results highlight the strong potential of asymmetric temporal modeling and global guidance for efficient and robust multimodal fusion. The source code is publicly accessible at https://github.com/redizzy/GAANet
Authors:Minji Park, Seunghyun Yoon, Hyuk Lim
Abstract:
A compressor can keep the facts of a dialogue and still drop the turn that changed them. A user corrects a price, reverses a choice, or adds a constraint. We call this failure turning‑point eviction. One overall retention score hides it, because that score mixes what the user first wanted with what the user wants now. We introduce TPBench, which evaluates three complementary information targets at shared nominal retention budgets. P1 asks for the user's initial goal. P2 asks for the current value of a slot the user revised. P3 asks for both, in dialogues with a late annotated slot update. The current‑value answers come from the human dialogue‑state annotations of MultiWOZ and SGD. The initial‑goal answer is the first sentence of the first user turn. Neither requires new crowdsourcing. The probe‑specific evaluations rank compression methods differently. On the joint probe at a retained fraction of 0.30, every tested compressed method remains below full context with the main Llama reader. Deleting the turn that carries the update sharply lowers current‑value accuracy, while deleting one matched irrelevant turn leaves it unchanged. A Mistral reader repeats the P2/P3 rankings and the joint‑probe gap. Current‑value recovery is tested on an additional corpus, LongMemEval‑KU, and on Chinese RiSAWOZ: full context has the highest accuracy, and recency has the highest compressed‑method mean in both evaluations.
Authors:Arturo Flores Alvarez, Nathan Lintu, Dennis Hong
Abstract:
Closed‑loop learned locomotion is established on commercial quadrupeds but remains uncommon at the sub‑kilogram scale. Continuous‑rotation legs give MiNI‑Q, a 270 g quadruped, access to supporting configurations on either side of the body. We exploit this range with a single posture‑conditioned reinforcement‑learning policy that runs entirely onboard. A continuous joint‑space reference on the torus T^8 and its gravity‑conditioned transformation connect upright walking, inverted walking, and landing recovery without state machines or phase switching. Coordinated posture and release curricula train this behavior family; identified actuation, cross‑engine validation, and embedded execution support hardware transfer. The same sub‑100k‑parameter network tracks forward velocity with RMSE of 0.037 m/s upright and 0.050 m/s inverted, resumes walking in 24 of 30 release trials, and operates with four interchangeable foot geometries across four indoor surfaces. Hardware experiments and simulation ablations connect these capabilities to the representation, conditioning, and training choices that exploit the platform's motion range. Demonstration videos and supplementary material are available on the project website: https://submissionreview.github.io/around‑the‑world/.
Authors:Kaushik Pendiyala, Haris Zia, Trevin Lee, Timothy Legge, Alejandro J. De Leon, Zihan Zhao, Aaron Wang, Abhijith Gandrakota, Jennifer Ngadiuba, Richard Cavanaugh, Javier Duarte
Abstract:
Mixture‑of‑Experts (MoE) models can increase parameter capacity without proportionally increasing active computation, but it is unclear how this trade‑off behaves in particle‑physics transformers. We study dense and MoE Particle Transformers on 188‑class JetClass‑II, varying expert count, routing capacity, top‑K, and auxiliary loss. We find that, when token dropping is avoided, top‑1 MoE models improve over the dense baseline at nearly unchanged nominal forward compute, while further increasing the number of stored experts produces little additional accuracy gain. Activating multiple experts per token yields additional predictive improvements at higher computational cost. Routing analyses show that expert assignments become more strongly associated with particle identity and kinematics in some configurations, but this structure does not increase monotonically with classification performance. These results highlight the need to distinguish stored parameter capacity, active computation, routing capacity, and routing organization when evaluating sparse expert models for jet classification. Code and experiment configurations are available at https://github.com/kpendiyala/MPT.
Authors:Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Shixiong Kai, Mingxuan Yuan, Bo Han
Abstract:
Reliable large language models (LLMs) must not only produce accurate answers but also express confidence that faithfully reflects their probability of being correct. Such calibration is essential for identifying uncertain predictions and supporting reliable decision‑making in real‑world deployment. Recent studies incorporate calibration learning into reinforcement learning (RL), jointly optimizing answer correctness and verbalized confidence using ground‑truth correctness supervision. However, their reliance on labeled data limits their applicability in practical test‑time settings, where ground‑truth labels are unavailable and calibration may need to adapt to newly encountered target tasks. To address this challenge, we propose Test‑Time Calibration Learning (TTCL), a label‑free framework that jointly adapts reasoning accuracy and verbalized confidence directly on unlabeled target‑task data. Specifically, TTCL derives self‑supervision signals for both correctness and calibration from multiple model‑generated responses, enabling calibration learning at test time without ground‑truth labels. Theoretical analysis further establishes TTCL as a bounded surrogate for the ideal calibration objective. Extensive experiments on mathematical reasoning and factual question answering demonstrate that TTCL consistently improves both accuracy and calibration across diverse models and tasks. On base models, TTCL achieves an average relative accuracy improvement of +40.13% and an ECE reduction of +70.80% across eight benchmarks. Moreover, TTCL can further improve both accuracy and calibration for already calibrated models under domain shift, particularly when source‑domain calibration transfers poorly to target tasks. In the math‑to‑factQA setting, TTCL achieves an average relative accuracy gain of +20.35% and reduces ECE by +53.83%. The source code is released at https://github.com/tmlr‑group/TTCL.
Authors:Jie Jin, Ziyin Ma, Min Yin, Jinyu Chen, Haigang Song, Zhikun Pang, Xiaowen Zhang
Abstract:
Full‑duplex voice agents make many small, closed decisions, which current systems answer by slow autoregressive decoding. We propose DuplexJev, which feeds ASR‑encoder hidden states through a small connector into a frozen LLM and reads each question as a single‑token distribution over its options. Nothing is decoded, and an 8‑GPU node answers 80 decisions about eight utterances in about 0.1 s. With a last‑layer connector, spoken QA stays close to reading the transcript (90% vs. 91%). DuplexJev also hears the speaker: gender and emotion accuracy both reach 90% (from 55% and 28%) with a cross‑attention connector, whose spoken QA drops by only 1 point (83% to 82%). We train decisions with cross‑entropy on the read‑out answer token, instead of the usual transcript distillation, whose teacher never hears the voice, and keep distillation for content. Encoders and LLMs are interchangeable; we release weights, training recipe, a batched‑inference pipeline for full‑duplex serving and a bilingual spoken‑QA set.
Authors:Zekai Wang, Yingqiang Ge, Zekun Wang, Hai Wang, Yuhui Xu, Joshua Frandsen, Shancong Fu, Ashia C. Wilson, Chandan K. Reddy
Abstract:
Harness evolution improves an LLM agent's prompts, tools, and workflow, while the optimizer's own tools and procedures often remain fixed. We study whether an optimizer can improve another agent more effectively by also improving how it diagnoses failures, develops edits, and tests their effects. Two observations guide our design. In a controlled study, optimizer self‑evolution fails to improve performance without execution‑based verification, but achieves the best result of that study when verification is available. Across five executors, self‑evolving optimizers build their own tools for failure analysis, verification, training audits, and workflow control. Motivated by these findings, we introduce VERSE, a Verified Self‑Evolving optimizer for agent harnesses. VERSE lets the optimizer test draft edits, replay failures, and perturb suspected steps before submission, while tracking fixes and regressions across rounds. Using this feedback, the optimizer revises both the executor harness and its own prompts, skills, tools, hooks, and notes, while the weights of the optimizer and executor models stay fixed. Under a shared protocol with disjoint training, validation, and test tasks, VERSE improves all four evaluated harness optimizers on held‑out SWE‑rebench tasks and newer out‑of‑distribution tasks in five languages. Its best validation‑selected harness reaches 42.3% and 37.7% accuracy, respectively, against 39.2% and 29.3% for the strongest baselines. Code is available at https://github.com/wzekai/VERSE.
Authors:Anna T. Thomas, Sohum Patnaik, Caroline Cotto, Benjamin Sanchez-Lengeling
Abstract:
Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug and materials discovery. Evaluating whether a novel food tastes like its animal‑based target requires expensive human sensory panels, bottlenecking the design‑build‑test loop. We introduce TasteBench, a multimodal benchmark and privacy‑preserving competition for sensory prediction, spanning two tasks: a food‑level ranking task built on 21K+ human evaluations across 215 plant‑based foods in 24 product categories, yielding 935 within‑category ranking pairs, and a supporting molecular‑level taste classification task over 15K flavor molecules. To enable rigorous interpretation of model performance, we characterize the ground truth: inter‑rater agreement among panelists is low (Krippendorff's α= .077), and the split‑half reliability ceiling of panel‑aggregated rankings is .825, establishing the range within which ML systems on this benchmark should be assessed. We evaluate baselines across four input modalities; on the same pairs panelists rated, the best model achieves .661 pairwise accuracy, competitive with the median individual panelist (.650), and .683 across all within‑category pairs. TasteBench provides the evaluation infrastructure and baselines for measuring progress on computational screening for sustainable protein discovery.
Authors:Chengyang Shi, Xianglin Ji, Jintao Huang, Jicheng Wang, Yifeng He, Jiachen Liu
Abstract:
Agents are increasingly given open‑ended research tasks: discovering an empirical law from self‑designed experiments, improving a heuristic whose optimum nobody knows, or beating a standing record. Their execution logs record every step of this research, yet the runs are still judged by their outcome score. That score alone does not establish whether an agent's claims follow from executed experiments, and a reference answer may be unavailable. We evaluate the agent's epistemic process: how it forms hypotheses, tests them, and revises them in response to evidence. We introduce OEB (Open‑Endedness Bench), a benchmark‑agnostic methodology that reads only the agent's execution record and never a reference answer or an outcome score. OEB compiles the record into a unified epistemic event graph whose edges connect the propositions the agent states to the executed actions that test them; each node carries an exact excerpt that code verifies against the record. One principle governs scoring: prose can state a proposition, but only evidence returned by an executed action can support or refute it, so OEB checks what the agent writes against what it actually ran. From the graph, OEB scores four competence axes (evidence, experiment, revision, and no reward hacking), mostly as the share of opportunities for sound research that the agent took, and profiles six subjective persona traits that describe the agent's research habits. We score 119 existing runs over 12 tasks from three benchmarks: LLM post‑training, chip design, and a training‑speed record. Against logged results, only 16‑29% of the improvements agents claim are real. On 9 of 10 tasks, the best run tries more new ideas in its second half than the worst run. The persona readings follow the model: for every trait, the model that ran explains more of its variance across runs than the task (a median of 43% against 7%).
Authors:Eray Turkel, Mengsha Sun, Kartik Ayyar, Sean Dunigan, Jack Lu, Vlad Shcherban, Hsiang-Shun Shih, Xin Wang, Tiantian Zhang
Abstract:
We present OpenGameEval, a benchmark and evaluation framework for agentic game development inside Roblox Studio. It runs language models as agents in reproducible, stateful game‑engine sessions and scores each run with executable checks, both on the edited scene and in a simulated play session. Most agentic coding benchmarks require exploration but score only final task success. OpenGameEval separates observation tools from editing tools in its eight‑tool action space, so exploration can be measured directly. We measure the pass rates and exploration behavior of 13 frontier models on 84 human‑curated core tasks, with 16 attempts per task. The tasks are hard for current models. The best model solves 51.7% of tasks on a single attempt and 39.4% five times out of five, and no tested model solves six of the tasks. Models at the frontier reach similar pass rates by solving different tasks: splitting tasks by the kind of work they require spreads the top five by 5.0pp on script‑authoring tasks and 12.5pp on scene‑change tasks. Exploration behavior predicts whether a run succeeds. Holding task and model fixed, a run that inspects every object a reference solution touches before acting on it passes 13.4pp more often than a run that inspects none of them on scene‑only tasks, and 9.8pp more often on script‑only tasks. We release the task suite, its place files, the per‑task annotations, a plugin that runs the tasks inside Roblox Studio, and an updated leaderboard under the MIT license at https://github.com/Roblox/open‑game‑eval.
Authors:Fei Zhang, Zhaochong An, Duncan Frost, Yikai Wang, Pengfei Liu, Ya Zhang, Michal Drozdzal, Amir Bar
Abstract:
World action models (WAMs) have emerged as a promising paradigm for robotic control by jointly predicting future visual dynamics and actions from an initial observation and instruction. However, existing WAMs struggle with long‑horizon prediction, as generating dense video rollouts is highly inefficient. Some recent WAMs address this by predicting a single future frame without generating the full video, but this approach neglects how to progress toward the goal. We present ProWAM, a progressive world action model that jointly predicts actions and an ordered sequence of sparse visual sub‑goals, providing explicit visual guidance to anchor action generation throughout task execution. This design scales naturally, as sub‑goal prediction can be learned from large‑scale action‑free videos, allowing the video backbone to offload complex visual planning from the action policy. For efficient action generation, ProWAM executes a single video‑backbone forward pass to cache sparse sub‑goal features, eliminating iterative full‑video generation and requiring only lightweight action denoising during replanning. Across extensive evaluations, ProWAM achieves superior out‑of‑distribution robustness. On simulation benchmarks, it sets new state‑of‑the‑art results on LIBERO‑Plus (85.8%) and randomized RoboTwin (75.7%), outperforming the strongest baseline with relative gains of up to +35.9%. On RoboCasa365, ProWAM achieves a 48.1% success rate and 18.2% on the challenging Composite‑Unseen split, ranking 4th overall. Crucially, in zero‑shot real‑world experiments, ProWAM achieves 70.0% success, outperforming the strongest baseline by +15.0 (from 55.0% to 70.0%, a +27.3% relative gain) in novel scenes. These results demonstrate the value of progress‑indexed visual foresight for closed‑loop control. Our program is in https://sii‑ferenas.github.io/ProWAM‑page.
Authors:Xueqi Cheng, Liang Wu, Kelly Wan, Liangjie Hong, Yushun Dong
Abstract:
LLM compression reduces inference costs and memory requirements, but selecting a method and configuration remains largely empirical because comparable resource reductions can produce different capability losses. We systematically investigate capability scaling‑down laws for LLM compression across pruning, quantization, and distillation. Our framework measures capability loss in mathematics, code generation, and question answering, and relates these measurements to model size, training stage, compression settings, data availability, and training exposure. We develop simple predictive relations and evaluate their accuracy, measurement efficiency, and generalization to unseen configurations and model states. Sharing the density response across pruning levels halves the configuration measurements needed to fit a pruning predictor: on new Pythia states, on pre‑registered OLMo‑2 test states and under Wanda pruning, the compact relation matches a regression fitted with all measurements on math and code to within 0.020 nats per token, with coefficients refitted for each setting. Controlled distillation experiments show that the cost of heavy data reuse recurs across question‑answering distributions, while the net benefit depends on the evaluation distribution. We further evaluate the decision value of these predictions by comparing numerical selection with configuration medians and fixed method priorities. Independent evaluations across two model families show that selection captures most of the available cross‑method benefit for question answering within the tested candidate sets, where a fixed method priority attains the same regret, with smaller opportunities for mathematics and code. These results clarify the predictive scope of capability scaling‑down laws and their use in compression method selection. Our code is publicly available at: https://github.com/LabRAI/scaling_down_law.
Authors:Zhaoyang Chu, Earl T. Barr, Claire Le Goues, Peter O'Hearn, Mark Harman, Federica Sarro, He Ye
Abstract:
Coding agents are extending their reach into the physical world by writing and executing robot control programs. One might expect the agents to use the existing mature software stack that engineers have developed over decades to access sensors and control motion. Yet prior work primarily engineers complex custom harnesses to orchestrate agents for robot use, particularly by prescribing specialized workflows and providing bespoke interfaces. This raises the question: "Is such additional harness engineering necessary?" We introduce OpenRUA, a zero‑abstraction harness that bypasses bespoke abstraction layers by providing off‑the‑shelf coding agents with only terminal access to the robot's native software interface ROS 2. OpenRUA employs a minimalist workspace‑as‑harness design, only offering ROS 2 documentation and basic tools while leaving the coding agent to organize its own work without orchestrating any agentic workflow. Within this workspace, OpenRUA recasts perception as file I/O and manipulation as coding. With Claude Code powered by Claude Opus 5, OpenRUA achieves success rates of 99.0% on CaP‑Bench and 87.0% on LIBERO‑PRO, demonstrating that an off‑the‑shelf coding agent can serve as a zero‑shot visuomotor policy through the robot's native interface, without bespoke primitives or task‑specific training. Under this minimalist design, further analysis reveals striking emergent behaviors of coding agents: (1) For perception, the agent spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes. (2) For manipulation, the agent spontaneously builds motion‑control clients (e.g., gripper control) in 95.87% of episodes and closed‑loop control programs (e.g., adjusting motion based on sensor feedback) in 50.13% of episodes. Our code is available at https://github.com/terminalworld/OpenRUA.
Authors:Omar Farouk Zouak, Houssam Eddine Boukhalfa, Soumaya Lakehal, Shiv Katiyar, Samia Nefti-Meziani
Abstract:
Large language models often solve a theorem forward yet fail to disprove a closely related false one: a falsification gap that supervised fine‑tuning does not close and can actively worsen. We frame counterexample generation as constrained witness emission against a deterministic per‑theorem Python verifier, and release SymCE, a corpus of 4,707 false undergraduate‑algebra and real‑analysis conjectures, each paired with executable verifiers. The verifier also serves as the reward function, making SymCE a training environment. Training Qwen3‑4B with SFT followed by GRPO under this oracle reveals an imitation trap: counterexample‑only SFT collapses true‑theorem recognition from 0.27 to 0.00, while RLVR with a sparse outcome‑only reward repairs this and exceeds the base, to 0.66. The collapse replicates across four seeds and on Gemma‑3‑4B. Sparse and dense rewards yield statistically indistinguishable in‑domain success yet diverge by 33 points on a held‑out calibration probe, a dissociation we trace to the partial‑credit term. Our 4B model outperforms every evaluated 7B open‑weights math specialist, remains competitive with six frontier commercial APIs, and transfers under unchanged prompting to GSM8K, MATH‑500 and MMLU‑college‑math. A human audit of 177 verifier decisions finds 97.7% accuracy. Code, data, verifier modules and annotations: https://github.com/ce‑rlvr/SymCE.
Authors:Nickil Maveli, Antonio Vergari, Shay B. Cohen
Abstract:
Large language models (LLMs) have demonstrated strong performance in code generation, where success depends on both recalling relevant algorithmic knowledge and reasoning about how to apply it. However, existing LLM pipelines are opaque, with no explicit separation between these two components. We argue that for well‑known algorithms whose canonical implementations are widely accessible in pretraining corpora, code generation is better measured as parametric code retrieval: reproducing a named algorithm from internalised knowledge rather than synthesizing a novel one. We introduce AlgoREval, a benchmark of 599 problems spanning classical 77 algorithms across 14 domains, 7 programming languages, and 4 graph‑input representations to evaluate this capability in isolation, and assess 15 models (7B‑‑34B parameters) in a zero‑shot setting. We find substantial variation in retrieval accuracy across languages and input representations, even for widely documented algorithms and show that prompt augmentation with retrieved code snippets or structured algorithmic hints improve accuracy on complex algorithms, while SFT achieves broader language gains and GRPO achieves larger per‑language gains on specific languages. Together, our results establish parametric code retrieval as a distinct, measurable capability and caution against deploying AI‑generated algorithmic code without systematic validation.\footnoteCode and dataset are available at https://github.com/Nickil21/AlgoREval
Authors:Bryce Sandlund
Abstract:
We study long context language models. Instead of training long context natively, or designing a long context harness, we train a model over the simplest possible harness: a tool to call itself with any specified prompt and a tool to read tokens in a range from the input context. We finetune Qwen3.6‑35B‑A3B on a diverse synthetic dataset using this harness. With only 8,000 tokens of context, our small model is as strong as GPT‑5.4 with 1M tokens of context on the OOLONG‑synth benchmark when document length exceeds 40K tokens.
Authors:Yixuan Liang
Abstract:
Electric utilities and grid operators increasingly rely on machine‑learning models to forecast next‑day demand, and those models learn from meter data that is routinely defective: readings go missing, sensors freeze, buildings read zero for hours, and units change by a factor of 100. This report presents a data‑onboarding pipeline that detects and repairs such defects before a model is trained, using only information available at forecast time, and a controlled experiment that measures whether the pipeline protects a 24‑hour‑ahead forecast. On hourly electricity data for twelve U.S. buildings from the public Building Data Genome 2 dataset (210,528 rows, 2016‑2017), seeded, hash‑logged defects touching 0.10% of the training period raised the error of a gradient‑boosting forecaster by 86%; after detection and past‑only repair the error returned to the clean‑data level (mean absolute scaled error 0.760 clean, 1.415 corrupted, 0.729 repaired) while 93% of training targets were retained. At a defect prevalence calibrated to published field studies (1.6% of training rows) the unprotected forecaster's error reached 4.4 times that of a seasonal‑naive rule, and the repaired forecaster again matched the clean baseline. The same pattern held for ridge regression and a random forest and across horizons of 1 to 24 hours. A first version of the pipeline over‑cleaned natural data and made forecasts 25% worse; that result is retained, its cause is traced in the published artifacts, and the per‑building calibration that corrects it is documented as a dated amendment. Every number is reproducible from pinned public inputs with SHA‑256 verification, 84 automated tests and continuous integration.
Authors:Felix X. -F. Ye, Yu Chin Fabian Lim, Naigang Wang, Davis Wertheimer
Abstract:
Streaming GPU solvers for entropic optimal transport (EOT), such as FlashSinkhorn, avoid storing the dense kernel but still evaluate all n× m point pairs in every Sinkhorn iteration. We present FlashSinkhorn~2 (FS2), a solver for squared‑Euclidean cost on low‑dimensional point clouds that solves large discrete EOT problems to a prescribed marginal residual on a single GPU by coupling two stages. A coarse stage solves on cell centroids, lifts the potentials to every point and, when a sampled marginal check rejects the lift, continues on the centroids, replacing most point‑level updates. A block‑sparse fine stage then removes the centroid error that coarse updates cannot. Its Morton‑ordered blocks support screening and fused tensor‑core execution, and a threshold set by the block masses bounds each omitted tile's contribution to every row and column. On synthetic benchmarks, FS2 reaches the target residual on all 32 problems and GeomLoss multiscale on 10. On one A100, FS2 solves discrete EOT between two 1.34×10^8‑particle measures from a cosmological N‑body simulation, at an entropic blur equal to the mean interparticle distance, to an all‑particle marginal residual below 0.01 in under 2.5 hours. To our knowledge, it is the largest discrete EOT problem solved to this accuracy within hours. For reproducibility, we release an open‑source implementation at https://github.com/ot‑triton‑lab/flash‑sinkhorn
Authors:Ran Dan, Si-Tong Wei, Pengfei Xiong, Wei Zhang, Yadong Mu, Peng-Shuai Wang
Abstract:
Existing 3D large language models (LLMs) compromise on two fronts: they compress shapes into latent codebook indices or coordinate text, which removes spatial structure from what the model observes, and they acquire the 3D modality by fine‑tuning the backbone, which overwrites its general language ability. We present OctLLM, which addresses both limitations. Geometry enters as an explicit 3D sequence of octree occupancy tokens. However, full octree sequences grow rapidly with depth; OctLLM therefore randomly empties penultimate‑level nodes and omits descendants while preserving shape, yielding a shorter coordinate‑ and depth‑anchored Sparse Octree (S‑Octree) for position‑aware mask‑modeling generation and 3D understanding. On the other front, existing methods introduce a new modality with full fine‑tuning or LoRA, but full fine‑tuning is costly, LoRA limits 3D capacity, and both modify the language pathway. OctLLM instead adds 3D capacity in parameters separate from the pretrained ones: mesh tokens are routed through independent trainable branches in a subset of blocks while text and image tokens retain the frozen vision‑language pathway, and the two streams interact through shared self‑attention. It trains far fewer parameters than full fine‑tuning, yet sets a new state of the art among unified multimodal LLMs, lowering image‑to‑3D FID by 17.4% and raising render‑grounded captioning by 28.7 points over ShapeLLM‑Omni, while matching the backbone on general language benchmarks.
Authors:Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu
Abstract:
Transfer functions (TFs) control color and visibility in medical volume rendering, but image‑trained Gaussian proxies typically bake one transfer function into their appearance. We present FactorSplat, a per‑scene N‑dimensional Gaussian splatting (N‑DGS) proxy that accepts region‑specific intensity‑to‑RGBA curves at inference. A local lookup applies the authored color and opacity change, while a shared functional encoder and low‑rank per‑Gaussian factors learn the residual appearance response. Geometry and directional appearance remain shared across presets, with visibility control and TF‑aware pruning preserving the ability to hide and reveal structures. On seven CT and MR scans, FactorSplat improves mean PSNR and changed‑region error over region‑aware VEG across validation, interpolation, unseen composition, and out‑of‑distribution (OOD) edits. Across these four splits, seven‑scan mean PSNR gains over VEG range from 1.10 to 1.52 dB. One checkpoint per scan supports unseen edits without retraining. At 1600^2, the cached fast renderer averages 524 FPS with 1.17 ms TF switches. Project page: https://gaozhongpai.github.io/FactorSplat/.
Authors:Jisung Park, John Le, Heath Cooper
Abstract:
GraphRAG pipelines construct auxiliary structures during offline indexing‑‑semantic summaries, hierarchical edges, and pre‑computed scores‑‑that determine how retrieval is prioritised at query time. Prior attacks target only instance‑level components (nodes, edges, triples), overlooking these schema‑level structures. We formalise Auxiliary Schema‑Level Entity as a novel attack surface and propose the 3S Framework (Semantics, Structure, Scoring) for its systematic exploitation. Our Hop‑Decayed Influence (HDI) attack identifies high‑impact targets through query‑aware influence propagation and corrupts their auxiliary structures post‑indexing. Across two benchmarks (HotpotQA, 2WikiMultiHopQA) and two architectures (Microsoft GraphRAG, HippoRAG2), HDI achieves 88‑94% attack success rate while modifying as few as 0.016% of auxiliary structures. Each modification affects up to 6.00 queries (Schema Leverage Ratio), demonstrating 1:N amplification unavailable to instance‑level attacks. Manipulated structures evade perplexity and paraphrase defenses with over 99% evasion rate, as they remain linguistically coherent system‑generated artifacts. These results reveal that auxiliary schema‑level entities receive implicit trust without runtime validation, constituting a structural blind spot in current GraphRAG defenses. https://github.com/Jisung‑Pacific/HDI‑GraphRAG‑Attack.
Authors:Kevin Zhai, Siva Rajesh Kasa, Soumya Roy, Sumit Negi, Mubarak Shah
Abstract:
Text‑to‑image generation enables users to explore several images generated from the same prompt. For these generated images to be useful, each one must reflect the user's preferences, measured by a learned reward, and differ visually from the others to maintain diversity. Existing methods are limited: they either address reward and diversity separately or combine them in one aggregate score, enabling high diversity to offset low rewards. In this paper, we address these limitations by formulating generation as satisficing: every image (candidate) must satisfy a reward floor and the batch of images must satisfy a diversity cutoff. The reward floor controls the balance between worst‑candidate reward and batch diversity; we show that varying this floor defines a Pareto frontier. To traverse this frontier, we introduce SatisDive, a training‑free inference‑time method. SatisDive uses a batch‑relative reward cutoff to distinguish lower‑ from higher‑reward candidates, emphasizing reward improvement for candidates below the cutoff and diversity among candidates above it. On Pick‑a‑Pic, at matched DreamSim, SatisDive improves worst‑candidate reward over FK steering by up to 0.43 with FLUX.1‑dev as the base model and HPSv3 as the reward, and by up to 0.70 with SANA‑1.6B as the base model and ImageReward as the reward. More broadly, across their overlapping DreamSim ranges, SatisDive's satisfaction‑diversity curve Pareto‑dominates FK steering's curve in each setting.
Authors:Zifan Zhang, Mingzhe Han, Kannan Athreya, Yuchen Liu
Abstract:
Massively parallel GPU simulators train multi‑robot policies in thousands of environments, and many fleets use private Fifth‑Generation (5G) networks, where each robot's delay depends on its teammates' traffic. Network‑in‑the‑loop training places a simulated 5G network inside this loop. However, GPU robot simulators reduce the network to an independent delay per message, while packet‑level simulators run one scenario per CPU process and cannot keep pace with thousands of parallel environments. To bridge this gap, we present Isaac‑Net, a GPU‑batched 5G New Radio (NR) module that advances the uplink of thousands of environments in lockstep with Isaac Lab physics. Isaac‑Net simulates every slot, the 0.5~ms interval in which the base station decides which robots transmit, for all environments at once. Extensive experiments confirm that its NR engine reproduces the median delay of ns‑3 5G‑LENA across loads, with a median delay 5‑‑10% low on an unseen carrier and 9% high at 32 robots per environment in closed loop. The engine also reproduces the Age of Information (AoI), the age of each robot's newest delivered report, while an independent delay per message leaves the AoI tail about three times too light. In a configuration validated against 5G‑LENA, Isaac‑Net keeps the network in the loop for about one million robots on one GPU at 83% of the Isaac Lab rate without the network, measured under a random policy. Isaac‑Net is open source at https://github.com/ZzZTripleZzZ/isaac‑net
Authors:José Lucas De Melo Costa, Seong Woo Ahn, Fabrice Popineau, Arpad Rimmel, Bich-Liên Doan
Abstract:
Joint‑Embedding Predictive Architectures (JEPAs) are prone to representation collapse, typically mitigated through empirical heuristics. We develop an early‑training stability theory that unifies these heuristics. Linearising the coupled JEPA gradient flow around the trivial fixed point reveals two competing effects: a driving force (γ) and a decay effect (σ). Under approximate spectral decoupling, a per‑mode stability ratio μ_i = γ_i / σ_i factorises into independent data‑side and predictor‑side terms and the count of unstable modes tracks the rank of representations that can emerge. The framework predicts a phase boundary, which we confirm empirically across more than 800 Tabular‑JEPA configurations. It also unifies predictor scaling, masking ratio, and EMA as distinct mechanisms for shifting μ. Guided by this analysis, we introduce ResidualPred, a transformer predictor whose attention is biased toward the identity at initialisation; it improves both effective rank and downstream accuracy on tabular benchmarks and in I‑JEPA pretraining on CIFAR‑10, CIFAR‑100, STL‑10, and ImageNet. Our framework connects empirical collapse‑avoidance heuristics to an explicit dynamical picture, yielding theory‑driven stabilizers. Code is available at https://github.com/jose‑melo/drive‑vs‑decay.
Authors:Chaoyi Zhou, Zhongpai Gao, Anwesa Choudhuri, Meng Zheng, Benjamin Planche, Run Wang, Terrence Chen, Siyu Huang, Ziyan Wu
Abstract:
Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between camera motion and tissue deformation. We present SCOPE‑4D, an endoscopic 4D geometry foundation model that jointly predicts camera parameters, dense geometry, and 3D tissue trajectories from monocular RGB video in a single forward pass. Our curation and annotation pipeline constructs SCOPE‑5K, a collection of approximately 5,000 clips spanning real and synthetic gastrointestinal endoscopy and laparoscopy. The collection provides rich geometric supervision and includes newly collected phantom and real‑colonoscopy evaluation sets. Geometric supervised fine‑tuning on SCOPE‑5K learns endoscopic priors that improve camera and depth estimation. Common‑‑Residual Motion (CRM) further constrains local deformation relative to common tissue movement. Together with geometric supervision, CRM and trajectory supervision further improve camera and depth estimation over geometric fine‑tuning alone while enabling dense 3D tissue tracking. Evaluations on public and newly collected benchmarks demonstrate strong in‑domain and out‑of‑domain geometry, superior 3D tracking, and more stable long‑sequence colon reconstruction. A blinded user study further supports the perceived reconstruction quality on real clinical video. Together, these results demonstrate the value of large‑scale endoscopic supervision and motion constraints for joint geometry estimation and tissue tracking.
Authors:Juntao Ren, Yifan Hou, Shuran Song
Abstract:
Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful. Trajectory stitching offers a way to compose these behaviors into improved training data, but identifying useful connections and verifying their feasibility is difficult in high‑dimensional robot data, where many prior methods rely on low‑dimensional state representations. We introduce NEEDLE, an offline dataset‑augmentation algorithm that addresses these challenges by adding short, verified action bridges between recorded observations in high‑dimensional robot demonstrations. First, NEEDLE identifies and creates connections that bypass suboptimal detours, broaden action coverage, and augment the original dataset with failed trajectories, using only RGB images, proprioception, and episode‑level outcomes, without new environment interaction or privileged object state. Next, we present a sampling technique that incorporates accepted bridges into policy training without synthesizing intermediate images or discarding the original demonstrations, allowing policies to learn alternative actions while retaining the original dataset's coverage. On real‑robot tasks, NEEDLE improves success rate over the strongest baseline on each task by an average of 21 percentage points. Videos and supplementary materials are on https://needle‑work.github.io/.
Authors:Max Ku, Nok-Kan Law, Yu-Chien Tang, Shih-Ying Yeh, Ping Nie, Andy Zheng, Tat Hei Lai, Fei-Yueh Chen, Nikko Yu, Wei-Chieh Sun, Suzy Huang, Chiao-Wei Hsu, Chih-Chuan Huang, Chak-Wing Mak, Ho Yin Sam Ng, Edisy Kin Wai Chan, Min-Hung Chen, Ho Kei Cheng
Abstract:
Interactive world models are increasingly capable of generating environments and acting within them, yet deliberately editing an existing executable world remains underexplored. We formulate world editing as intervening on an existing world while preserving properties that should remain unchanged, and introduce intervention depth as an axis describing how strongly an edit couples world entities, dynamics, and systems. We instantiate this capability through industry‑grade game modding and introduce IGMWorld, together with IGMBench, a benchmark of 110 tasks and over 1.1K executable state and behavioral criteria across Minecraft and Terraria. The tasks span property, entity, dynamics, and system interventions and are evaluated through deterministic executability, behavioral, preservation, and visual checks. Frontier coding agents already exhibit substantial world‑editing capability: the strongest configuration solves 78.2% of tasks under a strict task‑level criterion, while criterion‑level performance reaches 94.8%. Reliability generally decreases with intervention depth, and this pattern persists even among tasks with similar numbers of evaluation criteria. Most failed edits still build and load successfully, suggesting that the main difficulty is making the edited world behave as requested. Visual consistency remains a separate weakness, with all evaluated configurations below 50% joint visual pass rate. These results show that world editing is a distinct capability from world generation and interaction, and that executable games provide a practical testbed for studying it.
Authors:Jie He, Wei Li, Junwen Tong, Rui Shao, Wei-Shi Zheng, Liqiang Nie
Abstract:
Flow‑based Vision‑Language‑Action (VLA) policies generate action chunks by transporting samples from a task‑agnostic isotropic Gaussian source. As this source is conditioned on neither recent execution nor predicted future evolution, (i) it discards the local continuity established by recently executed motion. (ii) Even when predictive world representations are introduced, they often only condition the transport dynamics rather than determine where generation starts, how far it may deviate, or along which action directions it may expand. Building on this observation, we introduce ProAct, a world‑calibrated proposal‑to‑action framework that makes the generative source itself predictable. (i) To preserve motion continuity, a lightweight Proposal Expert converts recent actions into a scene‑aware hypothesis via one motion‑anchored endpoint flow‑matching step, initializing generation near the demonstrated action manifold. (ii) To jointly capture intended scene evolution and proposal‑future compatibility, a prospective World Expert treats the hypothesis as a soft motion prior while predicting the task‑consistent latent future. (iii) From this compatibility, the model calibrates a proposal‑centered anisotropic source, where a bounded per‑step extent controls the allowed deviation and a trace‑normalized low‑rank geometry under a condition‑number budget allocates refinement over coupled translation, rotation, and gripper directions. Compared with π_0.5, ProAct improves performance across simulation and real‑world tasks while reducing denoising steps by 50%, inference latency by up to 25.8%, and increasing throughput by up to 34.8%.
Authors:A. Said Gurbuz, Ahmed Nassar, Sunghwan Hong, Marc Pollefeys, Peter W. J. Staar
Abstract:
Computer‑use agents need to reliably ground action targets in complex desktop scenes, where multiple applications, overlapping windows, and visually similar controls compete for attention. Existing training data rarely pair such scenes with dense annotations or vary them in a controlled way. We introduce DeskForge, a controllable desktop environment that composes and explores real applications to generate large‑scale supervision for computer‑use agents. It varies application states, content, window layout, appearance, and resolution, and fuses screenshots, accessibility trees, and window geometry into dense element annotations while recording the outcome of each executed action. Using this environment, we construct DeskForge‑1M, a corpus of 1.2M annotated desktop observations containing 159.7M element instances. We fine‑tune four vision‑language models on 200K grounding examples drawn from DeskForge‑1M. All four improve across held‑out desktop conditions and on all five external GUI grounding benchmarks; for Qwen3.5‑4B, accuracy increases by 11.51 percentage points on ScreenSpot‑Pro and 10.11 points on OSWorld‑G. The gains also translate to long‑horizon task completion: under a fixed planner, the fine‑tuned action models solve more WebArena‑Infinity and OpenApps tasks, with Qwen3.5‑4B increasing from 31 to 50 of 119 tasks and from 3 to 15 of 100 tasks, respectively. These results show that controllable composition of real desktop environments provides a scalable source of supervision for improving both GUI grounding and long‑horizon computer use. The framework code, the dataset, and the fine‑tuned model are available from the project page: https://saidgurbuz.github.io/deskforge/
Authors:Ruiqi Zhang, Jiahao Wang, Mingxuan Li, Haichen Luo, Chaoting Wang, Guoyu Mou, Keyu Lai, Hanchao Lv, Jiaxu Wang, Yibo Zheng, Aijun Yang, Xiaohua Wang
Abstract:
Existing Simulink benchmarks mainly evaluate whether generated models compile, execute, or resemble a reference model. These criteria do not establish whether a model satisfies its engineering requirements. We introduce SimuVerity, a benchmark of 101 text‑to‑executable Simulink model‑generation tasks across ten engineering domains. For each task, executable‑system profiles ground the engineering specification and four families of native simulation scenarios. A hierarchical evaluator first checks artifact delivery, native executability, and engineering qualification, then scores qualified models across six dimensions covering accuracy, output quality, mechanistic fidelity, control and causal integrity, operating‑domain robustness, and dynamic response. We evaluate six agent systems with SimuVerity. The best system achieves an overall score of only 42.86. The results show that structural similarity is a poor proxy for engineering performance: capability bottlenecks arise both in producing qualified implementations and in satisfying multidimensional requirements after qualification. Meanwhile, some high‑scoring models still exhibit severe visual‑layout disorder. SimuVerity provides a systematic basis for assessing agents' engineering capabilities and diagnosing failures in executable Simulink model generation.
Authors:Ruihan Yu, Yu-Ju Tsai, Muyao Niu, Runyi Li, Lian Fu, Hanqing Liu, Zheng-Hui Huang, Yonghao Yu, Sho Kuno, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
Abstract:
3D editing methods are usually tested on a single edit, yet an asset is built through a long sequence of revisions, each of which must implement the requested change while leaving everything else unchanged. We introduce EditHero, to our knowledge the first benchmark for long‑horizon, part‑level 3D editing, with natural‑language instructions and target images for both geometry and texture. A deterministic assembly engine produces the exact target after every edit, and every sequence is reviewed by hand. We use EditHero to compare 2 opposite approaches to 3D editing. Non‑agentic methods operate top down, regenerating the object from a learned 3D representation and inferring what to keep. In contrast, LLM/VLM agents operate bottom up, editing through code that inspects the mesh and rewrites only the parts required by instructions. The non‑agentic methods often miss the requested change and disturb regions that should stay fixed. Most LLMs follow instructions more closely, and all of them preserve the unedited parts better, but each of their edits takes minutes. We will release the engine and the edit sequences to support research on reliable iterative 3D editing.
Authors:Sait Furkan Teke
Abstract:
HakemBench is a Turkish benchmark of typed decisions, in which the model under test reads a text, a question and a fixed set of options and returns a probability for every option. Version 1.0 is released fully open under CC BY 4.0, with 2,346 items and 4,275 choice, yes/no and score questions in seven tracks (fact‑check triage, education, guardrails, legal routing, moderation, spam and phishing, and customer support). One harness scores decision quality (macro F1), calibration (from the normalised Brier score) and selective automation (from the normalised area under the generalised risk‑coverage curve), combines them by a geometric mean and reports intervals from 2,000 bootstrap draws; probes for option order, paraphrase, English translation and substituted names are reported alongside. Most gold labels come from blind passes of one AI model family compared with the votes of a panel of large language models from other model families; they are not human‑verified. On a board of 16 rows the leader scores a composite of 0.888 and the lab's own model is 7th at 0.660. Its numbers are not blind. Earlier runs' test results shaped its training data, so its guardrail, moderation and customer support numbers are flagged; with every model scored on the other four tracks only, its composite is 0.678, 6th of 16.
Authors:Bart Jaworski
Abstract:
Societal resilience research relies on access to useful and actionable data, which motivates our main research question: Can annual reports, processed at scale with LLMs, provide a useful signal about how companies disclose their response to AI? We test this by applying a reproducible two‑stage classification pipeline to 9,821 annual reports from 1,362 UK listed companies (2020‑2025, with partial 2026 data). We first validate the method against 474 human‑annotated passages, finding high recall and moderate label‑level agreement. We then report three empirical patterns: (i) between 2020 and 2025, the share of reports mentioning AI risk rose from 2.8% to 41.2%, while AI adoption disclosure also rose, from 13.8% to 45.2%, and named vendor mentions cluster around a small set of major providers led by Microsoft; (ii) disclosure varies substantially by Critical National Infrastructure sector and market segment: AIM reports disclose AI risk at far lower rates than Main Market reports, and sectors such as Energy and Data Infrastructure lag behind the rest in AI risk disclosure; and (iii) harm disclosures are near‑absent (seven reports across the entire corpus). We develop a substantiveness classification to assess the quality of the disclosure and find that most AI risk disclosure is not substantive: in 2025, 41.2% of all reports mention AI as a risk, but only 4.3% contain AI risk disclosure we classify as substantive.
Authors:Xinye Yang, Zhusi Zhong, Scott Collins, Grayson Baird, Xuyu Wang, Zhicheng Jiao
Abstract:
Medical vision‑language model (VLM) evaluation is sensitive to workflow design, prompting strategy, and benchmark construction, yet most studies treat these factors in isolation. We introduce a reliability stress test for chest X‑ray interpretation built on two balanced datasets (a private report‑backed set and a curated MIMIC subset). Three medical VLMs (CheXagent, MedGemma‑4B, and MedGemma‑27B) are evaluated across three prompt styles and two workflows (single‑VLM and multi‑agent), producing 36 configurations. We show that exact‑match accuracy alone can overstate the effectiveness of conservative models that default to "Normal" predictions. Diagnostic reliability also depends heavily on model family and scale: multi‑agent reasoning helps some configurations but hurts others. Building on these observations, we propose a decision‑time routing framework that selectively escalates to multi‑agent inference only when beneficial, improving the cost‑quality trade‑off over fixed workflows. Our results highlight the need for evaluation protocols that jointly consider prompt sensitivity, failure‑mode diversity, and workflow choice before clinical deployment.
Authors:Xinye Yang, Zhusi Zhong, Scott Collins, Michael Bernstein, Grayson Baird, Terrence Healey, Michael Atalay, Mahesh Jayaraman, Xuyu Wang, Zhicheng Jiao
Abstract:
Emergency chest X‑ray (CXR) triage has a structural modality gap: reports arrive after triage decisions, yet multimodal foundation models require image‑text inputs. We present Variational Risk Minimization (VRM), a distillation framework that treats LVLM‑generated report variants as Monte Carlo samples of latent clinical interpretations. Rather than distilling from a single teacher target, VRM learns from a variationally marginalized teacher distribution, enabling uncertainty‑aware supervision under missing‑modality constraints. Under matched encoder families, VRM outperforms direct fine‑tuning baselines and improves calibration with strong recovery from hallucinated supervision. Marginalized supervision reduces report‑selection instability. In our compact edge‑student instantiation, a confidence‑gated cascade reaches AUC 0.941 at 103ms average latency with 20.3% cloud escalation, yielding an explicit reliability‑latency operating point for cloud‑edge clinical workflows.
Authors:Jiawei Li
Abstract:
Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input carries an injection. System‑1 decision models answer such questions in a single forward pass with class probabilities, promising large cost and latency savings over LLM calls. We present a paired evaluation of an open‑weight (Laya) and a hosted (Jev) System‑1 model on 11 agent decision points built from 18 public sources: 7,283 base cases plus 6,640 robustness variants, with byte‑identical inputs, paired tests, and cross‑hardware and cross‑day reproducibility checks. Jev is significantly more accurate on 9 of 11 decision points (+10.8 to +46.0 pp). Neither model beats chance on zero‑shot model routing, and they tie on RAG relevance gating. Laya changes 30% of its answers when the option order is reversed and degrades sharply with many or similar candidates (31% at 50 nearest‑neighbour tools, vs. 98% for Jev on items with a unique correct tool). We also audit our own pipeline. Three analysis errors and one design confound distorted headline deployment claims: an omitted pre‑screen cost (reported 23.9% saving, actual 4.3%), gate accuracy reported as end‑to‑end quality (58% vs. 98%), in‑sample thresholds (5% target, up to 17% held‑out misses), and a "channel effect" on injection false positives that vanishes with channel‑native content. Two other suspected confounds did not change the conclusions. All cases, raw outputs and analysis code are available at https://github.com/David‑DL‑Space/sys1‑eval.
Authors:Jiyun Kong, Jungwoo Kim, Enes Eray Demirtas, Touradj Ebrahimi, Jong-Seok Lee
Abstract:
Neural video codecs derive motion and temporal contexts mainly from RGB frames, leaving room for cross‑modal guidance from complementary temporal observations. Event streams can provide such observations by recording brightness changes between frames. In this work, we propose an Event‑guided Neural Video Codec (ENVC) that uses events shared by the encoder and decoder to improve RGB compression efficiency. For motion coding, ENVC forms an event‑guided motion prior and codes the remaining motion residual. For frame coding, an event‑conditioned predictor supplies multi‑scale features for gated temporal context refinement. To support training and evaluation on standard video datasets, we synthesize paired RGB‑event data and assess its predictive utility through comparisons with real events. Across six benchmarks, ENVC achieves average BD‑rate savings of 39.13% using PSNR‑RGB and 67.63% using LPIPS relative to DCMVC. Further analyses show that our gains persist on large‑motion sequences and that ENVC effectively learns to integrate event information. These results demonstrate the potential of events as a complementary modality for reducing the RGB coding rate. Our model and code are available at https://github.com/kjungwoo03/ENVC.
Authors:Anidipta Pal
Abstract:
Operational land surface forecasting systems built on Mamba‑family Structured State Space Models absorb non‑stationary confounding events (unrecorded irrigation booms, dam‑operation shifts, sensor recalibrations) into their state‑transition matrices, silently biasing NDVI, LST, and crop phenology predictions long after the physical cause ends. This paper introduces SSU‑LSF (State‑Space Unlearning for Land Surface Forecasting), the first machine‑unlearning framework purpose‑built for geoscientific Mamba‑based SSMs. We develop EKFac influence functions specialized to the Mamba state matrices via a closed‑form matrix‑exponential gradient, use spectral‑radius‑weighted elbow thresholding to localize a temporal confounding footprint Φ, and apply Hessian‑free projected gradient ascent within a KL‑divergence trust region augmented by spatial total‑variation (TV) regularization. Proposition 1 establishes that residual confounding is bounded by \mathcalO\big((1‑ρ(\barA)^T_c)/((1‑ρ(\barA))μ)\big), which grows with the window length T_c. Across three heterogeneous benchmarks and eleven baselines, SSU‑LSF achieves confounding reduction rates of 0.773 (CropHarvest), 0.821 (NDVI‑LST), and 0.859 (ERA5), with worst‑case clean‑domain RMSE degradation of 4.2% on ERA5, converging in 3‑‑5 epochs at 8.4× lower GPU‑cost per unlearning request than full retraining. Code: https://github.com/Anidipta/SSU‑LSF
Authors:Gautam Khosla
Abstract:
Algorithm‑based fault tolerance detects corrupted matrix multiplications by comparing a checksum of the product against a checksum recomputed from the factors, flagging a fault when the residual exceeds a threshold. The classical threshold is tau = Ku, derived from a dot‑product rounding bound that requires Ku <= 0.01. At bfloat16 with contraction length K = 4096, Ku is approximately 32, so the bound does not hold and the threshold is not defined. Prior work therefore evaluated checksum ABFT only at float32. We supply the missing calibration. This paper is a characterization of a detector under an output‑level, post‑GEMM perturbation model; it is not a characterization of faults in silicon, and no rate reported here estimates fault incidence in deployed hardware. We fit a Generalized Pareto distribution to 7.99e7 clean per‑row residuals and extrapolate a threshold at a per‑GEMM false‑positive rate of 1e‑6, obtaining 3.1445e‑6, which sits between the two clean sample maxima we observed. The independence assumption behind the per‑GEMM conversion is checked empirically rather than asserted, at a variance ratio of 0.985 against a binomial reference. Against this threshold the detector catches 304 of 400 single exponent‑bit flips (76%), failing a bar of 90% that was registered before any data was collected, and catches 0 of 600 low‑mantissa flips. Detectability is invariant in K across a 16x range even though the clean noise floor falls as K^‑0.4999: signal and floor scale together, so bit position governs detectability and contraction length does not. This falsified the mechanism the study was designed around. We report four registered predictions that failed, including that one, and four retractions of our own earlier findings, two of which were caught by pre‑publication audit rather than by prediction. Code, data, and timestamped pre‑registration documents are released.
Authors:Kaiyu Yue, Sean McLeish, Ruchit Rawal, Brian Bartoldson, Menglin Jia, Tom Goldstein
Abstract:
Sphere Encoder is an autoencoder that generates images by decoding random points from a high‑dimensional latent sphere. We identify two limitations of the original formulation that reduce its generation quality. First, random points concentrate near the equator relative to the pole on an encoded latent, but the training rotation never reaches this region, leaving a gap that limits one‑step generation. Second, training for generation with pixel‑wise reconstruction loss encourages the decoder to average over plausible images, producing blurry images that lack high‑frequency details. We present Sphere Encoder 2 to address both limitations, substantially improving image generation quality while maintaining the speed and simplicity of a autoencoder. Models are released at https://github.com/kaiyuyue/sphere2.
Authors:Jichao Jiang, Cristian McGee, El Houcine Bergou, HanQin Cai, Aritra Dutta
Abstract:
Full‑parameter fine‑tuning of large language models (LLMs) incurs substantial optimizer state memory overhead, limiting the model sizes that fit on modern GPUs. Existing approaches either compress optimizer state, abandon first‑order gradients, or change the update geometry while retaining dense state. The recently introduced Muon optimizer reduces optimizer memory through matrix‑valued updates. Still, its geometry differs from AdamW and can lead to performance degradation when fine‑tuning AdamW‑pretrained models. To reduce optimizer memory without sacrificing accuracy or computational efficiency in LLM fine‑tuning, we propose Ternary Absolute‑max Column‑wise One‑sparse optimizer, or TACO, which follows Muon's operator‑norm steepest‑descent view but takes the geometric route further. TACO computes the exact steepest‑descent direction under a dimension‑normalized 1\to1 operator norm by selecting the sign of the largest magnitude entry in each column of two‑dimensional weight matrices. This retains first‑order gradients while making optimizer state memory nearly negligible. Our practical TACO optimizer maintains only a small set of low precision gradient components per column, reducing persistent optimizer state by 174× relative to AdamW8bit (from 27.7 GB to 0.16 GB) and peak training memory by 2.9× (from 80.6 GB to 27.5 GB) on OPT‑13B, while achieving comparable accuracy and runtime. TACO further enables full‑parameter fine‑tuning of 30‑32B‑parameter models on a single 80 GB H100 GPU across multiple model families and tasks.
Authors:Sebastian Sanokowski, Alireza Sarmadi, Majid Khadiv
Abstract:
Several state‑of‑the‑art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic. However, critics are typically trained to predict returns, and accurate value predictions do not necessarily yield accurate action derivatives, potentially leading to unreliable policy updates. We propose Forward Entropy‑Regularized Policy Optimization (FERPO), an on‑policy maximum entropy reinforcement learning algorithm that performs policy improvement using critic values without differentiating the critic with respect to actions. FERPO derives an optimal target action distribution from a policy‑improvement objective regularized by entropy and Kullback‑Leibler (KL) divergence. We then fit the actor to this target by minimizing a forward‑KL objective, estimated using self‑normalized importance sampling (SNIS) with actions drawn from the rollout policy. By limiting the target distribution's deviation from the rollout policy, the KL regularization helps keep these importance weights well behaved. In contrast to reverse‑KL objectives, which can favor a subset of the target distribution's modes, the forward‑KL objective encourages coverage of multiple high‑value modes and thereby promotes exploration. Experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample‑efficiency gains. Computational benchmarks also demonstrate faster actor updates than Relative Entropy Pathwise Policy Optimization (REPPO).
Authors:Zhuo Lin, Sirui Xu, Liuyu Bian, Yu-Xiong Wang, Liang-Yan Gui
Abstract:
We study test‑time evolution for humanoid loco‑manipulation: solving tasks that a controller was never trained for by repurposing its existing skills, improving from its own attempts, and retaining what it learns, without retraining. Our key insight is that a broad controller already holds much of the competence a new task needs, and that this competence becomes accessible through an interface between planning and control that is expressive enough to specify contact‑rich, multi‑stage interactions, yet executable and measurable enough that execution feedback can guide planning from experience. InterEvolve realizes this interface with two components. First, we develop an object‑aware forward‑backward (FB) behavioral foundation model, whose object residuals on a frozen body prior turn a new reward about the body or objects into loco‑manipulation behavior at test time. Second, we specify tasks as reward programs: staged rewards with completion conditions and tunable constants. A large language model (LLM) agent revises the program structure in context, drawing on execution feedback and a skill library of verified programs, while a numerical optimizer tunes its constants. With every candidate verified across parallel simulation scenarios, the program explores new ways to induce, repurpose, and compose the controller's existing motor competence for the task at hand, and thus improves over iterations. Experiments show that human‑designed rewards leave much of the FB model's loco‑manipulation competence untapped, whereas the programs InterEvolve evolves release it, sometimes through novel strategies. It further produces behaviors for diverse tasks, complex scenes, and long‑horizon compositions in simulation, and evolved skills run autonomously on a physical Unitree G1 from egocentric onboard perception.
Authors:Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma
Abstract:
Distribution Matching Distillation (DMD) trains a few‑step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log‑density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution‑matching gradient underlying DMD, through the classical identity linking discriminator logits to log‑density ratios. We further introduce gap‑based reweighting, which adapts teacher supervision across noise levels from the real‑data head's empirical logit gap between real and teacher samples. DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one‑step generation on ImageNet‑64x64, 14.47 with four‑step SDXL on COCO‑10K, and a VBench total score of 85.15 with four‑step Wan2.1‑T2V‑14B, the best values among the compared few‑step methods and the multi‑step teachers. On MiniMax‑H3‑33B, our four‑step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio‑video generation, excluding ties. Our code, models and demos are available at https://yzmblog.github.io/projects/DMAD.
Authors:Mahindra Rautela, Alexander Scheinker, Ayan Biswas, Diane Oyen, Nathan DeBardeleben, Earl Lawrence
Abstract:
Simulation and experimental measurements provide complementary data for learning spatiotemporal physical systems, but standard simulation‑to‑experiment fine‑tuning optimizes only the experimental objective after transfer and can degrade simulation performance. We formulate simulation‑‑experiment prediction as a multi‑objective learning problem with domain‑specific simulation and experimental risks. On four fluid systems from RealPDEBench and two model capacities, we compare Simulation only, Experiment only, Sim\rightarrowExp, and Joint training, evaluating every final model on both held‑out domains. Sim\rightarrowExp tends to specialize more strongly to experimental data at the cost of simulation‑domain forgetting. Joint training consistently achieves the best balanced performance over a broad range of simulation‑‑experiment evaluation weightings, while substantially improving simulation retention over Sim\rightarrowExp. Joint also better preserves simulation‑only fields absent from experimental measurements. Project page: https://mahindrautela.github.io/morph.
Authors:Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang, Rui Yang, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh, Reid Simmons, Joerg Liebelt, Xiang Kong, Zhongyu Jiang
Abstract:
Multimodal agents commonly generate free‑form reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection‑based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open‑ended generation. SSR represents recurring high‑level reasoning as pre‑specified, reusable natural‑language candidates. At each turn, the model selects from these reasoning candidates based on their likelihoods given the current context, without requiring an auxiliary task head. Using pre‑specified reasoning traces enables parallel scoring, where teacher‑forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluate SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine‑tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per‑turn reasoning latency by over 90% and total per‑question model inference latency by 28‑54%. Project page: https://zfy0314.github.io/ssr‑webpage/.
Authors:Liuzhenghao Lv, Yuyang Liu, Yuyang Gao, Li Yuan, Yonghong Tian
Abstract:
Protein mutation effect generation asks a model to describe the functional consequence of a point mutation in natural language. Existing protein‑to‑text systems typically encode mutation information into undifferentiated representations, overlooking the organization of mutation‑induced evidence across structural and biochemical factors. We propose RipplePLM, a mutation‑aware generation framework centered on Direct‑Distal Cross‑Attention (DDCA). By constructing a residue‑level Mutation Perturbation Field from pre‑trained protein language models, DDCA leverages predicted contact maps to organize mutation representations into two pathways: the mutation site's immediate contact neighborhood and its multi‑hop distal context. To complement this structural decomposition, we further introduce the Property Latent Chain (PLChain), which injects expert‑guided supervision of biochemical property changes (e.g., thermostability and optimal pH) into the LLM hidden‑state pathway through latent property tokens. On MutaDescribe, RipplePLM improves over mutation‑specific baselines on temporal and structural splits; under a matched‑backbone comparison, average structural‑split ROUGE‑L increases from 22.23 to 35.65. Expert evaluation further shows a higher proportion of biologically accurate or relevant descriptions than the mutation‑specific baseline. Additional ablations, representation diagnostics, and low‑N fitness regression experiments further support the effectiveness of the learned mutation‑aware representations. Code: https://github.com/Lyu6PosHao/RipplePLM.
Authors:Yohan Chatelain, Pablo de Oliveira Castro
Abstract:
Should low‑precision transformer inference use stochastic rounding (SR) or round‑to‑nearest (RN)? The answer depends on where in the network you look. We isolate this effect by holding the numerical format fixed and varying only the rounding rule at individual operation sites. To enable experiments at freely chosen precisions, we extend the PRISM vectorized rounding library to arbitrary virtual precision via a variable‑precision stochastic rounding (VPSR) algorithm, proving that the rounding decision is evaluated exactly in hardware floating point. We develop two analyses providing complementary insight into this site‑level trade‑off. First, a probabilistic forward‑error bound for linear projections shows that SR's error envelope grows as O(\sqrtn u) in reduction length n, versus O(n u) for RN, a gap that widens rapidly at low precision and is most pronounced in the long multilayer perceptron (MLP) down‑projection. Second, a second‑order decomposition of expected cross‑entropy loss change at the output softmax into signed drift, drift curvature, and a Fisher‑weighted variance penalty reveals why the two sites behave oppositely: MLP noise is predominantly a uniform logit shift to which softmax is invariant, so SR's variance is largely discounted; head noise is non‑uniform across the vocabulary and is not. On DistilGPT‑2 at t=6 significand bits, observations match theory: SR in the MLP raises perplexity to 1.15x the full‑precision reference, versus 2.21x for RN. At the language‑model head, the ordering reverses because SR introduces non‑uniform variance, whereas deterministic RN carries none. In a mixed‑precision configuration (MLP output at t=6), assigning SR to the MLP and RN to the head brings perplexity within 1.10x of the full‑precision reference, a 28% reduction over matched‑bit RN.
Authors:Zheng Fang, Yongmin Li, Yichang Zhang, Dongming Jin, Haoyu Wang, Shuai Wang, Zhi Jin, Ge Li
Abstract:
Coding agents can generate code that appears correct but implements behavior the user never intended. This mismatch can arise when an agent silently resolves underspecified requirements through its own assumptions. As subsequent development builds on these assumptions, correcting the resulting behavior can become increasingly costly. Early clarification can help prevent such mismatches, but unnecessary questions can interrupt developers and slow down development. Existing methods struggle to identify key clarification questions while avoiding unnecessary ones. Therefore, we propose CONTRA, a training‑free method that combines broad question discovery with semantic and execution‑based question qualification. CONTRA first generates candidate questions and filters out those unrelated to required behavior or already resolved by the requirement. For each remaining question, it generates programs conditioned on two plausible answers and checks for stable behavioral differences on shared inputs. It then uses the interaction history to select among qualified questions or stop asking. Experiments on ClarifyCodeBench show that CONTRA achieves the highest F1 with all four coding agents, exceeding the best baseline macro‑average F1 by 13.88 percentage points. With the same LLM and evaluation protocol, CONTRA also achieves higher clarification recall and F1 than the coding harnesses Claude Code and OpenHands. To support practical use, we also implement CONTRA as a Claude Code plugin that integrates selective clarification into everyday development.
Authors:Jakob Paul Zimmermann, Andrei Balakin, Moritz Grillo, Georg Loho
Abstract:
Understanding loss landscapes is central to explaining neural‑network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias‑free ReLU networks in the teacher‑‑student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non‑negative student output weights once the student network is at least as wide as the teacher network. In contrast, without the skip, we construct a fixed teacher network with positive output weights and only three hidden neurons in input dimension two whose spurious local minima persist at every student width at least three. Thus, a learned linear skip can remove spurious minima that persist under arbitrary overparameterization. Furthermore, we show that a positive output weight student network always learns the subspace spanned by the teacher features: student features at local minima with non‑negative student output weights lie in the span of the teacher features. For ReLU networks in two dimensions, even heavily overparameterized student networks have effective width controlled by the teacher width: every critical point with positive student output weights has at most twice as many distinct student feature directions as teacher neurons. Finally, we transfer the benignity result to empirical minima over parameter balls of any prescribed radius, with the required sampling accuracy depending on that radius.
Authors:Abdelrhman Werby, Fabio Scaparro, Kai O. Arras
Abstract:
Enabling robots to estimate the kinematic parameters of articulated objects unlocks a wide range of capabilities for interaction and manipulation. The estimation has to happen from the information the robot currently observes, often just a single RGB image of an object it has never seen before. Current single‑image approaches couple articulation part segmentation with articulation estimation, making their predictions vulnerable to missed detections and incorrect part associations, and they regress metric 3D geometry that a single view fixes only up to scale. We present QueryArt, a model that estimates articulation parameters from a single RGB image, a 2D query point, and camera intrinsics. QueryArt is trained to estimate the 3D articulation geometry relative to the queried point and in units of its depth, which keeps its target identifiable from the image alone. A single depth measurement at the query point then supplies the scale and recovers the metric parameters. We train QueryArt on a curated mixture of synthetic and real‑world articulation datasets. We evaluate QueryArt on several benchmarks and compare it against recent baselines. QueryArt outperforms recent baselines on most articulation metrics, including on out‑of‑distribution data. To demonstrate the model's capabilities in real‑world settings, we evaluate QueryArt on a mobile manipulator across 57 manipulation trials spanning 16 object parts and five viewpoint classes, achieving a 70.2% success rate. We provide code and videos at: https://abwerby.github.io/queryart/
Authors:Haochen Chai, Xinbi Luo, Zining Liu, Fangfang Jiang
Abstract:
Wrist electrodermal activity (EDA) differs in amplitude from one person to the next, so affect‑recognition models normalize their input before classification. Studies that test such models on held‑out subjects seldom report where the normalization statistics come from, yet statistics computed from the held‑out subject's own recording give the model information that a device does not have when it is first worn. We asked how this choice alters the measured benefit of pretraining. A compact convolutional network, SAFE‑EDA, was pretrained on expert artifact annotations from 43 subjects and compared with the same network trained from scratch on the Wearable Stress and Affect Detection (WESAD) dataset (15 subjects, leave‑one‑subject‑out), with two normalization sources crossed with four window hops. When the statistics came only from training subjects, pretraining raised macro‑F1 by 0.078 to 0.227; when they came from the held‑out user's full recording, the gain fell to between 0.020 and 0.050 and was no longer significant. Artifact supervision was far more useful than self‑supervised pretraining on the same recordings (0.078 versus 0.008). Across 13 configurations in two datasets, the pretrained network was better in 12, but on the second dataset (26 subjects) per‑user normalization increased the gain instead of reducing it, so the interaction depends on the data. Only five of 50 published WESAD studies state which data were used for normalization. Reporting this choice is necessary to separate first‑use performance from performance after calibration.
Authors:Dominik Wojcikiewicz, Diego Paez-Granados
Abstract:
Mobile robots operating among pedestrians need trajectories that become available quickly, remain spatially credible through missed observations, preserve identity, and fit within an embedded computing budget. Aggregate tracking scores provide limited insight into when and how trajectories fail, while varying detector inputs can confound tracker and detector quality. We present a deployment‑focused, tracker‑only evaluation protocol that uses shared detections to isolate tracker behavior and directly evaluates initialization, detector‑gap continuation, identity recovery, close‑neighbor association, and load‑dependent tracker‑step runtime, while Higher Order Tracking Accuracy (HOTA) is retained as a complementary aggregate measure. We apply the protocol to the JackRabbot Dataset and Benchmark (JRDB) using six open‑source trackers and our lightweight Pedestrian Reference Tracker (PedRefTrack), together with a GT‑assisted variant that estimates the remaining tracker‑side gap under idealized association and motion. Under fixed detections, the non‑GT trackers span only 24.26%‑29.67% HOTA yet exhibit markedly different capability profiles. After 1.0 s without detector support, no tracker without GT assistance maintains spatially correct, same‑identity output in more than half of eligible cases, making missing‑observation continuation the dominant limitation among the tested properties. Close‑neighbor failures are smaller and increase mainly at the shortest separations. Tracker‑step runtime on an NVIDIA Jetson Orin is heavy‑tailed and load‑sensitive, causing several trackers to fall below the 10 Hz real‑time target in crowded frames. The protocol provides a reproducible way to characterize tracker behavior and deployment suitability in pedestrian‑centric environments. Code and evaluation scripts are released at https://github.com/SCAI‑Lab/tracker_eval.
Authors:Marek Gałązka, Hanna Wdowicka
Abstract:
An assignment of tasks to servers can resist every small improvement and still make tasks wait longer than necessary. We determine exactly how inefficient such an assignment can be when each task requires one unit of service and the eligibility constraints permit all tasks to use distinct servers. For every move size r and maximum current server load K, we give a closed formula for the worst ratio between locally optimal and globally optimal total completion time. Local optimality here allows every feasible reassignment changing at most r tasks. Every finite‑cap bound is attained on a tree where each task has at most two eligible servers. Thus the worst behavior already occurs under simple eligibility constraints. At load cap two, the exact ratio is 1+1/(r+2), attained on a path with r+2 tasks. Without a load cap, the worst‑case supremum is 3/2 for single‑task moves and approximately 1.294503159 for two‑task moves; its excess above one is 1/(r+2)+O(2^‑r/r) as r grows. The proof uses an explicit rational potential on a comparison graph and matching extremal constructions. These results give sharp guarantees for bounded‑size local search on matchable semi‑matchings, including exact guarantees under degree bounds.
Authors:Ahmad Samuel Gali, Shamsuddeen Hassan Muhammad
Abstract:
Yorùbá is a widely spoken tonal language that depends on diacritics to avoid lexical ambiguity. However, it is often written without these diacritics, thereby hindering downstream Natural Language Processing (NLP) tasks. In this paper, we introduce Yo‑ByT5, a byte‑level Automatic Diacritic Restoration (ADR) model fine‑tuned from ByT5‑small. We evaluate Yo‑ByT5 alongside five publicly released Yorùbá ADR models and one open‑weight large language model (LLM) on the YAD benchmark under a consistent protocol. Our results demonstrate that Yo‑ByT5 matches the performance of the strongest existing model, mT5‑base, with a DER of 10.14% and a CER of 3.48%. Furthermore, it exhibits superior text fidelity despite using approximately half the parameter count of mT5‑base. We also release our training code and model outputs, as well as call for the development of a larger, purpose‑built benchmark for Yorùbá diacritic restoration.
Authors:Xindi Yang, Baolu Li, Liam Lee, Zhenfei Yin, Songxin Zhang, Zhuoyang Song, Xu Jia, Jianfei Cai, Tien-Tsin Wong, Bingyi Jing, Mengyue Yang
Abstract:
Generative video world models can now synthesize open‑ended environments that agents can navigate and interact with in simple ways. Yet open‑ended generation does not imply full interaction: as a generated world expands, newly created content through navigation should expand what the agent can act upon, and as the agent changes the world, those changes should become persistent parts of the environment rather than transient visual effects. We characterize these two requirements as Open‑World Interactivity, where newly generated or encountered entities are incorporated into the actionable world, and Persistent State, where interaction outcomes are committed to the world state and continue to influence subsequent observations and interactions. We present Oneira, an interactive video world model that closes the loop between generation and interaction through an explicit, extensible world state managed by a coding agent. Given the current observation and an action or high‑level goal, the agent reads the world state, grounds the relevant entities, plans the interaction, and writes its outcome back into a world state table. When exploration reveals new objects, the agent incorporates them from generated observations, allowing the interaction space to expand with the generated world. Meanwhile, previously induced state changes are carried across video segments, making the consequences of interaction persistent parts of subsequent world evolution. The updated world state is rendered along the camera action trajectory into a coarse conditioning video, from which a video generator fills in the appearance, motion, and interaction details not represented in the state. Experiments show that Oneira enables direct and consistent interaction with newly generated objects, while preserving the effects of prior interactions over long horizons. Project page: https://madaoer.github.io/projects/oneira
Authors:Guy Amit
Abstract:
Decision models often score a variable‑sized set of candidate actions encoded in a single sequence. This setting is increasingly relevant for System 1 components inside generative systems, where candidates may be proposed or ordered differently across runs. Standard causal cross‑encoding is expressive, but it can make a candidate's score depend on serialization order rather than on the underlying decision problem. We introduce candidate‑independent block‑causal attention, which preserves causal computation within the shared context and each candidate while blocking cross‑candidate information flow and resetting candidate positions. We compare this architecture with standard causal attention and complementary invariant baselines across Gemma 3 1B, Qwen3 1.7B, and Qwen3 4B backbones. Candidate‑independent attention consistently reduces permutation sensitivity while retaining competitive decision quality; ablations indicate that candidate isolation is the primary source of the effect, with position resetting completing the intended symmetry. A larger Qwen3‑4B study further examines the behavior of the proposed architecture with substantially more training data. Code is available at the \hrefhttps://github.com/guyAmit/ci‑decision‑models\textcolorblueproject repository, and the \hrefhttps://huggingface.co/Guy‑Amit/qwen3‑4b‑ci‑decision‑4096‑poc\textcolorblueQwen3‑4B model artifact is available on Hugging Face.
Authors:Youngwoo Shin, Yusung Ro, Minseo Kim, Junmo Kim
Abstract:
Video Large Language Models (VideoLLMs) receive frames in sequential order and interpret how visual content evolves along the temporal axis, yet temporal reasoning remains a persistent weakness across architectures. Reversing the frame order of a video, a transformation that should invert temporal answers, often leaves the final prediction unchanged. We investigate where this failure originates by defining the temporal divergence vector τ_l, the layer‑wise representational difference induced by reversing temporal order. Tracking its magnitude across layers reveals a consistent temporal divergence profile where the divergence peaks at intermediate layers and progressively diminishes toward the output. We confirm this peak is specific to temporal reasoning and functionally critical for predictions, establishing that VideoLLMs acquire temporal information at intermediate layers but fail to maintain it to the output. This progressive fading motivates our method, Temporal Activation Injection (TAI), which extracts τ_l at the peak of the profile for each input and reinjects it into subsequent layers following the measured decay. TAI requires no training and consistently improves temporal reasoning across three VideoLLMs and four benchmarks with negligible impact on non‑temporal tasks. Code is available at https://github.com/Youngwoo‑git/Before‑It‑Fades.
Authors:Kabir Murjani, Nisarg Patel
Abstract:
Non‑Markovian environments are often modeled as Regular Decision Processes (RDPs), where dynamics depend on the interaction history through a finite automaton. Existing offline guarantees for RDPs rely on a distinguishability assumption on the behaviour policy but provide no means of verifying it. When the assumption is violated, distinct models may explain the data equally well. We study when data collected under a fixed behaviour policy can distinguish two candidate RDPs. We prove that the posterior odds between observationally equivalent candidates remain equal to the prior odds at every sample size, even when the policy visits every automaton state, and verify both results formally in Lean 4. We then characterize this equivalence exactly and derive PEC, an algorithm that decides it in time linear in the size of the product automaton. The distinguishability assumption of prior work fails on three of our four test environments, and the experiment identified by PEC restores it in each case.
Authors:Samuele Bortolotti, Weixin Chen, Han Zhao, Andrea Passerini, Stefano Teso, Antonio Vergari
Abstract:
Neuro‑Symbolic (NeSy) predictors incorporate prior knowledge into the prediction process of neural networks, ensuring that outputs satisfy specified constraints, making them particularly suitable for high‑stakes applications where compliance with domain knowledge is essential. A key bottleneck in this paradigm is the acquisition of symbolic constraints: encoding domain knowledge into logical formulas remains a manual and expert‑intensive process. In this work, we investigate the extent to which auto‑formalization via LLMs can systematically translate textual knowledge into symbolic knowledge that can be plugged into NeSy predictors. To this end, we introduce auto‑nesy‑bench, a new benchmark for evaluating constraint formalization and its impact on downstream accuracy of NeSy predictors. Through an extensive evaluation across several domains, we find that LLMs can formalize constraints to a meaningful extent, generating formulas that are often similar to those provided by human experts. Moreover, when the generated formulas are syntactically valid, they can lead to high‑quality downstream predictions. The code and benchmark are available at https://unitn‑sml.github.io/auto‑nesy‑bench/.
Authors:Jolle Verhoog, Ali Burak Ünal, Holger Caesar
Abstract:
Robust perception in intelligent vehicles demands 3D object detectors that remain dependable under domain shifts, such as changes in time of day, location, or weather. However, due to costly annotation and rare shifts, some environments lack sufficient data to train a standalone detector. Federated learning offers a privacy‑preserving framework for collaborative model training, enabling clients to benefit from shared learning across diverse environments. Yet, this framework traditionally relies on a single global consensus model, which struggles to perform across heterogeneous local data distributions. Local conditions are better captured by adapting a subset of the model, but many personalization approaches rely on predefined layer partitions or fixed personalization ratios, thereby limiting adaptation to client‑specific divergence. To reduce this rigidity, we propose FedCKA, a Centered Kernel Alignment (CKA)‑based strategy that dynamically handles the personalization‑globalization trade‑off. Specifically, FedCKA computes layer‑wise feature similarities between local client models and the global consensus model during training. By converting layer‑wise similarity scores into client‑specific aggregation masks, FedCKA selectively shares representation‑consistent layers. Evaluation on a unified multi‑domain benchmark based on nuScenes shows that FedCKA outperforms established federated baselines, including FedBN, FedRep, and FedSelect, improving average NDS by 7 percentage points over the strongest baseline. The findings offer both a comparative benchmark and a promising direction for robust federated 3D perception across shifts in location, weather, and illumination. Code is available at https://github.com/j‑verhoog/FedCKA.
Authors:Yuxuan Zhang, Dong Li, Zezhou Sun, Yuxuan Xu, Siyu Teng, Yuchen Li, Jianjian Yang, Long Chen
Abstract:
This report presents the OpenSpace Lab's solution to the Competition on Intelligent Information Gathering for Single and Multi‑Robot Systems Workshops, organized as part of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026. Our team reached 1st place in the Single‑Robot Public Track and 3rd place in both the Single‑ and Multi‑Robot Private Tracks. The single‑robot framework utilizes pre‑trained map completion predictions for global planning to prioritize unexplored areas. To reconcile map coverage with limited operation time, we introduce a remaining‑time‑based exploration strategy that integrates homing constraints into the decision‑making process. For multi‑robot exploration, we utilize a utility‑driven target selection strategy that balances observation gains, movement costs, and budget constraints, leveraging shared map and intent data to eliminate redundant search and maximize coordination efficiency. Our solution reached a 61.04% coverage rate in the Single‑Robot Public Track, while reaching 39.53% and 39.91% coverage in the Single‑ and Multi‑Robot Private Tracks, respectively. An extended full‑length paper based on this report is currently being prepared for submission, and the source code will be released upon acceptance of the full manuscript at https://github.com/OpenSpace‑Lab/Indoor‑Exploration‑IROS2026.
Authors:Yu Huang, Jungang Li, Zhiyuan Wang, Yonghua Hei, Song Dai, Jiayu Yang, Deyuan Liu, Xiang Zheng, Xiaoshuang Shi, Hao Cheng, Kaidi Xu
Abstract:
Recent video generation models can produce highly realistic videos from natural language instructions, with visual quality approaching cinematic standards. Existing evaluation benchmarks, however, predominantly assess visual quality, aesthetic appeal and physical plausibility, while paying limited attention to text, an essential medium for conveying information in everyday scenes. A generated video may appear visually compelling and feature lifelike subjects, yet still render the text within the scene incorrectly. To address this overlooked dimension, we introduce VTR‑Bench, a systematic benchmark for evaluating the Visual Text Rendering capabilities of video generation models. VTR‑Bench situates text within concrete application scenarios, such as advertisements and scientific videos, with 300 carefully constructed prompts spanning five scenario categories. We develop an automated evaluation pipeline with human alignments that separately assesses text fidelity through carrier‑specific transcription and scene and motion requirements through a prompt‑specific chain of query. Beyond evaluation, we introduce a Keyframe‑Guided Agentic Framework in which a Director agent coordinates image and video generation with visual evaluation, guiding iterative refinement and candidate selection through visual feedback. Experiments on 11 state‑of‑the‑art models reveal widespread difficulties in accurately rendering scene text, with the best‑performing model recording an overall word error rate (WER) of 0.250. We further analyze text rendering failures to characterize the challenges faced by current video generation models. These findings highlight visual text rendering as a key challenge for video generation and demonstrate a practical path toward improvement. Code is available at https://github.com/hardenyu21/VTR‑Bench.
Authors:Mohammed Hafsati, Ahmed Loughzali
Abstract:
Backchannel prediction has been studied almost entirely in dyadic conversation. We introduce a multi‑party benchmark based on the AMI corpus, comprising 682 masked‑listener views from 171 meetings, 190 speakers, and 18,697 backchannel events, with a person‑disjoint held‑out split. A state‑of‑the‑art dyadic model applied zero‑shot to meeting audio performs at chance (AUROC 0.499); nevertheless, its frozen acoustic features remain informative: a linear probe reaches 0.704, and retraining the predictor raises performance to 0.751. Retraining reveals a second limitation. Listener conditioning improves prediction for listeners seen during training but not for unseen listeners, and the gap remains under capacity reduction, listener‑adversarial training, per‑listener adaptation, and oracle lexical conditioning. Adversarial training removes only part of the speaker‑identity information, while stronger removal hurts prediction, suggesting that identity is entangled with cues that are useful for backchanneling. A within‑model control helps explain this pattern: with the same features and data splits, turn‑onset prediction transfers to unseen listeners, while backchannel prediction does not. Backchannel rates also vary about twice as much across individuals as turn‑onset rates. Since backchannels occupy only about 1% of frames, frame‑level F1 is strongly affected by the base rate. We therefore report AUROC alongside event‑F1 on listener‑active regions. We release the benchmark and evaluation tools at https://github.com/HafsatiMohammed/bc_multiparty_release.
Authors:Aviral Gandhi, Jinglue Xu, Jialong Li, Hitoshi Iba
Abstract:
Low‑cost NAS must both explore high‑performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training‑free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data‑driven method that combines a brief fair hard‑path warm‑up with discrete search under a single CMA‑ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate‑conditioned supernet updates. On NAS‑Bench‑201, LESS achieves \(93.189\pm0.467%\) CIFAR‑10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately \(1/24\) of its source‑reported search time. Matched controls show that calibration improves selected validation accuracy by \(0.577\) percentage points while changing best‑visited accuracy by only \(0.054\) points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR‑100 and ImageNet16‑120 with \(69.615\pm1.139%\) and \(43.720\pm1.697%\) accuracy. Applied without tuning to the larger DARTS space, LESS achieves \(96.95\pm0.14%\) on CIFAR‑10 and \(82.43\pm0.80%\) on CIFAR‑100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data‑dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.
Authors:Zhaoxu Meng, Yiming Sun, Mingyuan Gao, Jiachang Zhang, Zhuhan Dai, Yipeng Du, Zheng Lian, Jian-Qiao Zhu
Abstract:
We often plan ambitiously yet act habitually and wonder, in retrospect, whether we would have planned differently had we known what we would actually do. Hindsight offers a valuable perspective on past decisions, although we often wish we could have simulated hindsight at the moment of choosing. If a system could generate plausible trajectories from one's personal history, such previews might help people formulate more realistic plans and make better informed decisions. We introduce NextMe‑800, an approximately 800‑hour first‑person dataset from one volunteer over 126 days with 1 Hz images, gaze, and audio, captioned at five hierarchical abstraction levels from atomic actions to major activities. We formulate personalized action anticipation as open‑vocabulary K‑step sequence prediction and construct NextAct, a 1,500‑point benchmark combining NextMe‑800 with the multi‑person EgoLife dataset. Using an embedding‑based soft edit distance as the metric, we evaluate how well different models can anticipate personal behavior across abstraction levels and prediction horizons. NextMe‑800 and NextAct provide a months‑long resource and evaluation framework for studying how far ahead personal behavior can be anticipated from egocentric observation.
Authors:Bin Chen, Yumeng Xue, Patrick Paetzold, Yunhai Wang, Oliver Deussen
Abstract:
UMAP achieves scalable layout optimization through stochastic negative sampling. However, this stochasticity can lead to unstable embeddings across reruns and downstream reuse, as the estimated repulsive forces depend on the ordering of sampling events. We present ibUMAP, a coherent field‑based alternative that evaluates attraction and repulsion from a shared embedding snapshot and applies them synchronously. Its degree‑weighted repulsive field is motivated by the conditional expectation of negative sampling for a fixed embedding and represented by three scalar moments, which are evaluated efficiently on CPUs and GPUs using an interpolation‑based FFT scheme. This formulation avoids explicit all‑pairs computations while inducing optimization dynamics that differ from those of standard online UMAP. Controlled experiments show that synchrony and kernel capping alter the local‑global fidelity trade‑off, whereas FFT evaluation produces small average changes in final quality. End‑to‑end benchmarks show median speedups of 3.29x unseeded and 5.79x seeded over umap‑learn on CPU, and 1.44x over cuML on million‑scale datasets under unseeded GPU execution. These gains accompany greater run‑to‑run stability and measurable fidelity trade‑offs.
Authors:Minho Jeong, Dooho Lee, Jinmo Lee, Jaemin Yoo
Abstract:
Tabular foundation models (TFMs) achieve strong predictive performance through in‑context learning, yet repeatedly conditioning on labeled data makes inference expensive. Knowledge distillation can reduce this cost by transferring their predictive ability to lightweight, dataset‑specific students. However, the dependence of TFM predictions on both a labeled context and a query introduces two design questions: how to construct teacher supervision and whether expanding query coverage improves distillation. We examine these questions across two TFMs and both neural and tree‑based students, and derive an effective distillation recipe. The recipe uses the full labeled training set as teacher context and trains students solely on teacher predictions for observed and synthetic queries. On TabArena, the resulting students outperform their supervised trained tuned‑and‑ensembled counterparts by 57‑98 Elo points. Applied unchanged to TALENT, the same recipe improves matched default students on 236‑258 of 300 datasets and reduces median primary error by 4.0‑6.4%. The distilled students also achieve median inference speedups of 3.0‑21.6 times over their teachers, offering a practical trade‑off between predictive performance and repeated inference cost. Code is available at https://github.com/nums‑ai/TFM_Distillation .
Authors:Xudong Wang, Hao Wu, Haozhe Hu, Peiran Yin, Xinghao Chen, Yunpu Ma, Wei Zhang, Xiaoyu Shen
Abstract:
Multimodal large language models (MLLMs) incur substantial inference costs when processing long visual‑textual sequences. While existing operation compression methods exploit modality‑level redundancy, they largely treat computation within attention heads and shared feed‑forward network (FFN) channels as unified units, leaving finer‑grained redundancy underexplored. We find that redundancy varies both across modality‑interaction paths within the same attention head and across visual and textual executions of the same FFN channel. Based on these findings, we propose Modality‑aware Width‑wise Operation Pruning (MWOP), which independently prunes visual‑to‑visual (V2V), text‑to‑visual (T2V), and text‑to‑text (T2T) attention paths within each layer, and separately selects FFN channels for visual and textual inputs. A first‑order Taylor criterion guides the pruning process, with FFN importance re‑evaluated after attention pruning and LoRA‑based recovery training. To translate the resulting fine‑grained sparsity into practical acceleration, we further develop path‑sparse Triton attention kernels and compact visual‑side FFN execution. MWOP preserves the token sequence while reducing attention and FFN computation, making it complementary to token compression and enabling simultaneous reduction of sequence length and per‑token computation. On LLaVA‑OneVision‑7B, MWOP alone achieves a 1.6× prefill speedup with 99.7% average performance retention across 12 benchmarks. Combined with two representative token compression methods, it further increases their prefill speedups from 2.0× and 1.9× to 2.9× and 2.7×, respectively. Results on Qwen2.5‑VL‑7B further demonstrate its applicability across architectures. The code is available at https://github.com/EIT‑NLP/MWOP.
Authors:Ziqi Jiang, Zhenqi He, Long Chen
Abstract:
Trajectory crossing remains a critical bottleneck in Flow Matching (FM), and previous works typically view these crossings from a theoretical optimization perspective causing velocity averaging. They attempt to address it indirectly by post‑hoc distillation or endpoint coupling, without explicitly regulating the intermediate trajectories. In this paper, we introduce a new network learning perspective: crossing points inherently induce large local Lipschitz constants in the target velocity field, leading to two drawbacks. First, high Lipschitz constants correspond to high‑frequency signals in the velocity field that neural networks struggle to fit due to spectral bias. Second, they also imply drastic velocity variations, leading to severe numerical integration errors in few‑step inference. To alleviate this, we propose CoFlow, a framework that introduces the contrastive learning paradigm into FM to explicitly repel trajectories during training, thereby lowering the local Lipschitz constants of the velocity field. Specifically, we formulate CoFlow from a Stochastic Differential Equation (SDE) perspective by injecting a repulsive drift term. This drift actively guides the forward process of positive samples away from negative trajectories, effectively reducing the local Lipschitz constant. Furthermore, we derive an equivalent stochastic interpolant formulation from this SDE, providing a simple and tractable design space to control the influence of negative samples. Extensive experiments on ImageNet 256x256 demonstrate that CoFlow significantly reduces FID compared to standard FM in few‑step inference (e.g., 20 steps), with no added training overhead. The code can be accessed at: https://github.com/HKUST‑LongGroup/CoFlow
Authors:Javid Baydamirli, Ismayil Ismayilov, Kaan Oktay, Didem Unat
Abstract:
GPU‑initiated communication lets GPU threads post RDMA operations directly to the NIC. It underpins NVSHMEM, NCCL GIN, and DeepEP, which serve the fine‑grained, latency‑critical communication of Mixture‑of‑Experts (MoE) models, yet its performance characteristics and optimizations remain scarcely documented beyond source code, and library comparisons fail to separate the costs of the hardware mechanism from those of the library around it. This paper dissects GPU‑initiated communication at the GPU‑NIC boundary. We first detail the GPU‑side network path: queue placement, work‑request construction, doorbell ordering, and completion semantics. We then introduce mini‑gda and mini‑proxy, minimal transports for the GPU and CPU‑proxy submission paths, and measure them alongside NVSHMEM IBGDA, NCCL GIN, DeepEP, UCCL‑EP, MSCCL++, and fabric‑lib on NVIDIA H100, H200, B200, and GB200 platforms. A minimal GPU path issues an operation in 0.7 μs and completes in 4.0 μs; libraries add up to 4.6 μs of issue time through queue management, memory ordering, and completion scope, and issue time scales with the SM clock. A tuned CPU proxy matches or beats the GPU path at idle, at the cost of a dedicated core whose operating state sets its latency and throughput. On either path, sharing a queue with bulk traffic raises latency by one to three orders of magnitude. Reaching the 260 M msg/s ceiling of our InfiniBand platform requires doorbell batching and queue parallelism, and both have resource costs: communication code can reduce GPU block residency even when unused, and all‑to‑all traffic loses 59% of its NIC message rate at about 3,000 active connections. The submission path alone therefore does not predict communication performance. Our experiment code and results are available at https://github.com/ParCoreLab/Dissecting‑GPU‑Communication‑Experiments.
Authors:Wojciech Gromski, Patryk Krukowski, Jan Miksa, Maciej Zieba, Przemysław Spurek
Abstract:
Continual personalization of text‑to‑image diffusion models requires sequentially acquiring new concepts while retaining previously learned ones. However, existing methods either suffer from catastrophic forgetting or rely on storing additional concept‑specific parameters and spatial components, causing their parameter footprint to grow with the concept stream. This limits their ability to scale to long sequences of personalization tasks. We propose a rehearsal‑free approach that uses a single fixed‑size hypernetwork to continually personalize a frozen diffusion model. Instead of expanding the model as new concepts are acquired, the hypernetwork dynamically produces the concept‑specific adaptations required for personalization while preserving previously learned concepts. Our framework further integrates spatial control into the personalization process, allowing users to specify where a personalized concept should appear without introducing additional per‑concept components. This formulation enables continual personalization with a parameter footprint that remains independent of the number of learned concepts, aside from compact concept representations. Experiments demonstrate strong retention of previously learned concepts and reliable spatial grounding, matching or improving upon existing methods while scaling effectively to long streams of personalization tasks.
Authors:Duong Hien Chi Kien, Thanh Trung Huynh
Abstract:
Reinforcement learning for trading often struggles to balance upside participation with drawdown control. Profit‑only policies can collapse toward passive long exposure on upward‑drifting assets, while aggressively risk‑penalized rewards can become too defensive during volatile periods. This paper proposes PPO‑HRAP, a hybrid regime‑aware policy that combines Proximal Policy Optimization with an interpretable regime prior. The agent observes both market features and portfolio‑state variables, receives a reward combining portfolio log return, VIX‑conditioned drawdown‑increase penalty, target‑exposure deviation, and turnover cost, and executes a blended action between the PPO actor output and a regime‑derived target exposure. On the held‑out 2020‑2022 SPY test window, PPO‑HRAP achieves 27.62% total return, 8.48% annualized return, 0.6447 Sharpe ratio, 0.8588 Sortino ratio, and 0.4592 Calmar ratio, while reducing maximum drawdown from 34.10% for Buy and Hold to 18.47%. Across five SPY seeds, PPO‑HRAP remains stable with mean total return 0.2725 \pm 0.0109 and mean Sharpe ratio 0.6219 \pm 0.0565. Single‑run cross‑asset tests on QQQ and DIA further show that the proposed method ranks first on total return and Sharpe ratio for all three reported assets. These results suggest that blending learned actions with a volatility‑aware regime prior is a practical way to improve risk‑adjusted trading behavior, although the current policy still incurs high turnover and cross‑asset robustness beyond SPY remains limited to single‑run evidence.
Authors:Stephanie Finley, Liudas Panavas, Thomas Mikkelson, Cam Hinton, Stacey Ganss, Bradley Monton, Emily Kendall, Michelle Spradlin, Lydia Bye, Michael O'Brien, Lauren Ylvisaker, Derek Ray, Suhaas Garre, Sushant Mehta, Edwin Chen
Abstract:
Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds. We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80). The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance. Each task is a containerized Harbor environment with an expert rubric of binary criteria (median 47.5 and 57.5 per task) that an agentic judge applies to the delivered files, and an attempt passes only if it meets every criterion. Across 30 model configurations from 13 developers, the strongest, Claude Opus 5.5, passes 24.7% of healthcare and 23.9% of finance attempts, and the median configuration passes 0.6% and 2.5%. In case studies, agents accept premises that the record contradicts and carry wrong inputs through otherwise consistent analyses. We release all healthcare tasks, 50 of the 80 finance tasks, the evaluation harness, and the leaderboard.
Authors:Ruofan Yan, Na Lu, Shu Peng, Wenlong You, Zhige Chen, Yuxuan Yan, Yan Liu, Kay Chen Tan, Jibin Wu
Abstract:
Stimulus‑driven auditory attention determines which sound gains priority when multiple sources compete without an explicit listening goal, yet most computational studies focus either on acoustic salience or on decoding predefined attended targets. This study investigates instruction‑free auditory competition using the Stimulus‑driven Auditory Attention (SAAD) paradigm and develops a neurophysiologically informed framework that integrates stimulus‑derived sound priority with trial‑specific EEG evidence. Behavioral analysis using a Bradley‑‑Terry model showed that sound priority estimated from previous competitions generalized to unseen sound pairings, improving held‑out prediction from an AUC of 0.577 to 0.718. EEG analysis further revealed mid‑to‑late centro‑temporal lateralization associated with the reported selection side, with neural information remaining predictive beyond acoustic asymmetry. Guided by these findings, the proposed model first estimates a latent priority for each competing sound and forms relative stimulus evidence from their difference. A multi‑scale EEG pathway with complementary signed and power‑based readouts then extracts trial‑specific neural evidence, which is incorporated through gated decision‑level integration. The framework is evaluated using mirror‑constrained and pairing‑held‑out protocols, together with representative acoustic, EEG, multimodal baselines, and systematic ablations. The results support a computational account in which spontaneous auditory selection reflects the interaction between generalizable stimulus priority and trial‑specific neural variability.
Authors:Karol Dziekan, Przemysław Spurek, Dawid Malarz
Abstract:
Concept erasure suppresses a target concept while preserving behavior on unrelated inputs. Existing closed‑form methods were designed for 2D image diffusion and assume a single generative pathway, so one edit must cover geometry and texture at once. Native 3D generators, which synthesize structured 3D representations directly rather than by lifting 2D samples, violate this assumption. We show that shape and object concepts must be erased in the structural stage of the pipeline and material concepts in the appearance stage. We therefore formulate erasure in native text‑to‑3D as a stage‑aware editing problem and introduce STAGE, a training‑free, closed‑form framework. STAGE confines each edit to the low‑dimensional subspace spanned by the differences between erase and anchor embeddings, and relaxes the norm‑preserving (orthogonal) constraint of prior editors into a least‑squares affine correction that maps target activations onto safe anchors subject to a penalty on the displacement of retained prompts. The correction applies to the structural stage, the appearance stage, or both. We find that the stage an edit must reach is determined by concept type. On TRELLIS, the standard open native 3D generator, across 15 shape, material, and object concepts, STAGE reaches 66.7 on a composite score that balances forgetting the target concept against preserving everything else, aggregating CLIP‑based semantic and physical metrics, versus 53.2 for the strongest adapted baseline. Code: https://github.com/gmum/STAGE/ Project Page https://gmum.github.io/STAGE/
Authors:Giulio Schiavi, Andrei Cramariuc, Michael Pantic, Roland Siegwart
Abstract:
Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual‑learning framework for single‑view 6‑DoF grasp synthesis for a parallel‑jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real‑world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6‑DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long‑horizon continual learning with limited forgetting. In real‑world experiments, our method reaches over 90% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.
Authors:Lianjun Liu, Shipeng Li, You Huang, Weiqi Yan, Mingte Qiu, Huazhong Liu, Xiaofeng Zhu, Yunshan Zhong
Abstract:
Mixture‑of‑Experts (MoE) Diffusion Language Models (DLMs) offer flexible parallel decoding and increased model capacity, but their large number of expert parameters incurs substantial computation and storage costs. Existing low‑rank MoE compression methods largely rely on static factorization and fixed rank allocation, which overlook the distinctive properties of MoE DLMs. Specifically, we identify two properties: cross‑mode non‑uniform redundancy, where parameter redundancy and sensitivity to rank truncation vary across the input, output, and expert modes, and token‑wise utilization variation, where hot and cold tokens exhibit distinct spectral characteristics and expert activation patterns. To address these challenges, we propose ITC‑MoE, an Importance‑guided Token‑aware Compression framework for MoE DLMs. ITC‑MoE consists of two complementary components. First, Importance‑guided Adaptive Tucker Compression (IATC) incorporates activation and gradient importance into expert weight transformation, jointly factorizes expert weights across multiple modes, and adaptively allocates ranks under a fixed parameter budget. Second, Token‑aware Compensation and Routing (TCR) applies lightweight low‑rank compensation to compression‑sensitive hot tokens and restricts the candidate expert set for cold tokens with concentrated routing patterns. By jointly adapting compression capacity and inference execution to both parameter redundancy and token‑wise variation, ITC‑MoE substantially reduces the computation and storage costs of MoE DLMs while preserving their generation quality. For example, on SDAR‑30B‑A3B‑Chat‑b32, ITC‑MoE maintains an accuracy of 96.33% on MultiArith under a 30% compression budget, while achieving up to a 7.22x end‑to‑end speedup. The code is publicly available at https://github.com/lianjunl13‑sudo/ITC‑MoE.
Authors:Sihan Lv, Zhen Li, Zhiqi Cao, Jinshan Zhang, Ying Li, Meng Xi, Jianwei Yin
Abstract:
Audio decisions often depend on evidence that a transcript does not preserve, while their probability estimates can depend on how answer options are ordered. AudioJev maps a waveform, question and supplied alternatives directly to a candidate distribution through one shared full‑parameter model. We define order calibration as preserving an answer's probability under meaning‑preserving option permutations. Random‑derangement SKL training pairs each question with a reordered view in which every alternative changes position, supervises both answers, and aligns the two distributions before applying a symmetric KL penalty. Inference retains a single candidate‑scoring forward, with no calibration head or order ensemble. Across three training seeds, AudioJev reaches 68.88%/55.33% mean accuracy on complete MMAU/MMAR and reduces random‑order SKL by 43.8%/60.5% relative to the single‑view removal ablation. The same model handles intent, environmental sound, note properties, speech activity and conversational transitions. Paired removal ablations and multi‑order evaluation measure predictive accuracy and probability stability together, establishing a direct audio interface whose calibration objective acts on candidate meaning rather than presentation position. Inference code and model weights are available at https://github.com/SihanLv/AudioJev‑Inference and https://huggingface.co/shlv/AudioJev.
Authors:Lingyi Zhou, Yunke Wang, Mengyu Zheng, Wenbo Wang, Zijian Wang, Chang Xu
Abstract:
Reliable shelf monitoring is an important capability for retail automation, yet existing out‑of‑stock detection methods mainly operate in image space and lack metric 3D localization for downstream robotic systems. We formulate shelf monitoring as object‑level 3D change detection: given two RGB‑D observations captured at different times, the goal is to identify changed products and localize each change with a 3D bounding box. To support this task, we introduce ShelfChange3D, comprising 145K synthetic and 5K real‑world paired RGB‑D observations with object‑level 3D change annotations. We further propose ChangeBox, an end‑to‑end framework that jointly reasons over paired observations and predicts object‑level 3D change boxes. To improve localization accuracy, we introduce a geometry‑based refinement stage that exploits depth and gravity prior to estimate relative pose and refine predicted boxes. Experiments show that ChangeBox outperforms existing change detection baselines, with further gains from refinement and effective transfer from synthetic to real‑world observations.
Authors:Jian Hu, Shujing He, Leixin Chang, Zongze Li, Ding Huang, Chaoyang Shi, Chengzhi Hu
Abstract:
Autonomous colonoscopic navigation can reduce operator burden and the risk of loop formation or tissue trauma, but remains challenging due to deformable anatomy, weak‑texture and specular endoscopic visuals, and contact‑rich viscoelastic interactions. Existing methods either rely on geometry‑driven pipelines, which are efficient and interpretable yet brittle due to manually engineered features and switching logic, or adopt learning‑based policies, whose inferred depth/geometry can become temporally inconsistent or overly smooth under weak texture and specular highlights while simulation‑trained variants (e.g., deep reinforcement learning) may further suffer from a sim‑to‑real gap. We propose ColoACT, an autonomous navigation system that integrates an RGB‑D‑E based Action Chunking Transformer policy (ColoACT policy) for a compact self‑propelled Bevel‑Gear‑Based Endoscopic Robot (BGER). The ColoACT policy augments RGB with estimated relative depth and a gradient‑based pseudo‑elevation map to enhance fold‑ridge saliency and other high‑frequency geometric cues, and enables smooth continuous control of the BGER by predicting overlapping action chunks and fusing them via temporal ensembling. In different ex‑vivo porcine colons (approximately 60 cm), our system achieves success rates of 85.4% and 72.5% in straight and curved segments, respectively, and achieves 70% success in 90‑degree turns and 60% in double‑bend sequences, with feasibility further demonstrated in challenging triple‑bend segments. The project page is available at: https://Adamhu1.github.io/ColoACT/.
Authors:Yiqiao Jin, Yiyang Wang, Lucheng Fu, Bing He, Siheng Xiong, Yijia Xiao, B. Aditya Prakash, Josiah Hester, Srijan Kumar, James Evans, Jindong Wang
Abstract:
Scientific progress emerges from a longitudinal ecosystem in which researchers, institutions, funding agencies, collaboration networks, and the scientific literature co‑evolve. As AI becomes increasingly involved throughout the scientific research cycle, understanding these interconnected and evolving processes becomes increasingly important. We introduce SciUtopia, a persistent, closed‑loop LLM‑agent simulation framework for studying academic research ecosystems. SciUtopia models interconnected scientific processes such as research‑direction choice, collaboration, submission, peer review, resubmission, citation, funding, and researcher attrition, while maintaining evolving states across simulated years. Its configurable institutional mechanisms and information channels provide a controlled testbed for matched counterfactual experiments and targeted interventions. Across 61 simulation worlds, SciUtopia simulates over 40,000 researchers from 8,000 institutions, producing around 400,000 publication decisions and 1.2 million LLM‑generated peer reviews. Using these longitudinal simulations, we find that rejection‑driven resubmission substantially amplifies reviewer burden beyond population growth alone, cautious exploration balances citation impact with career success and long‑term topic diversity, and resource inequality can emerge even without detectable cumulative advantage from narrowly winning early funding. Code is available at https://github.com/Ahren09/ScienceUtopia.
Authors:Huichan Seo
Abstract:
Vision‑language models (VLMs) increasingly act as judges that pick the best of several generated images, so their choices decide what users see. Such judges are usually validated by score agreement with human ratings, not by the images they return. We audit VLM judges as decision‑makers: on 300 culturally situated prompts, we compare the returned image with human ratings the judge never sees and with random choice from the same candidates, and repeat every decision with the candidates reordered. A 4B‑parameter judge barely beats random and falls short of a CLIP similarity baseline. It picks the first image shown in 49% of calls (chance: 28%), and reordering changes its choice on 60% of prompts. For this judge, agreement across orders is informative: decisions that survive reordering are much better than random, whereas agreement with a weaker second judge keeps the wrong ones. An 8B judge shows almost no position bias and outperforms CLIP, yet for it the same filter mostly discards good decisions. Agreement helps only when it targets the judge's failure mode, so filters must be re‑audited whenever the judge changes. The 4B judge's slight rise in stereotype ratings is no longer detectable after aggregating across orders or with the larger judge.
Authors:Tarm Kalavantavanich, Teerawut Ponarchar, Pattaramanee Arsomngern, Jenta Wonglertsakul, Watcharakorn Chuthong, Chiraphat Boonnag, Knot Pipatsrisawat, Titipat Achakulvisut
Abstract:
Automatic SOAP note generation can ease the documentation burden on physicians, but existing reasoning methods often omit clinically important information and generate unsupported content. Progress in Thai is further hindered by the lack of publicly available datasets. We propose ASCRIBE, a physician‑inspired reasoning framework that ascribes a clinical‑significance level to each extracted atomic fact in the conversation before summarization, making a general‑purpose LLM a more reliable scribe. We also release ThaiClinicBench, the first de‑identified Thai clinical summarization benchmark of real encounters, together with a synthetic training corpus derived from real clinical notes. As a prompt, ASCRIBE outperforms chain‑of‑thought prompting on GPT‑5.4 and Gemini 3.1 Pro across the physician‑aligned LLM‑judge metrics and improves on standard prompting by up to 10.3 points on the completeness LLM‑judge metric. As a GRPO reward, it enables a Gemma‑4‑E4B model trained solely on synthetic data to match Gemini 3.1 Pro in factual precision and surpass it in completeness. Code and data can be found at https://github.com/loolootech/ascribe.
Authors:Lianjun Liu, Tiantian Zheng, You Huang, Weiqi Yan, Mingte Qiu, Huazhong Liu, Xiaofeng Zhu, Yunshan Zhong
Abstract:
Efficient long‑context inference is essential for large language models (LLMs), yet it poses a severe computational bottleneck. Hash‑based retrieval offers an efficient alternative by encoding queries and keys into binary codes and using Hamming distance for key selection. However, this leads to a critical mismatch between Hamming distance and attention relevance. Query‑Key logits depend jointly on directional similarity and feature magnitudes, whereas hash binarization discards magnitude information, causing both false‑positive retrieval of low‑logit keys and false‑negative omission of high‑logit keys. To address these failures, we propose Hierarchical Hash Retrieval (HHR), a coarse‑to‑fine framework that progressively improves retrieval accuracy through Geometry‑Aware Key Routing (GKR) and Learned Hash Projection (LHP). GKR learns a head‑wise orthogonal transformation to redistribute feature magnitudes and derive more discriminative page‑level logit bounds, enabling effective pruning of low‑logit keys while preserving important candidates. LHP then learns a head‑wise projection space that aligns Hamming distance with the true Query‑Key relevance ranking for fine‑grained retrieval. By combining GKR and LHP, HHR suppresses false positives and recovers false negatives, substantially improving the fidelity of hash‑based sparse attention. Extensive experiments across diverse LLMs and benchmarks demonstrate that HHR achieves superior performance over existing methods. For example, on LongBench, HHR improves the average score by 1.10 points and, at a context length of 128K, achieves up to a 3.30x decoding speedup and a 2.83x end‑to‑end speedup for Llama‑3.1‑8B‑Instruct. The code is publicly available at https://github.com/lianjunl13‑sudo/HHR.
Authors:Ziyi Chen, Yan Zhang, Jianhui Wei, Daoan Zhang, Zuozhu Liu
Abstract:
When training large language models with reinforcement learning, terminal rewards provide little guidance about which steps matter. Common methods for assigning step credit overlook that work built on uncorrected mistakes is wasted while independent work remains valid. With only a final success/failure reward, every step in a failed episode has zero total future reward, even when it made progress. We propose Dependency‑Aware Reward Shaping (DARS), which represents task progress as predicates linked by prerequisite relations and assigns step‑level credit over the dependency graph. An annotator marks which predicates each step verifies, invalidates, or repairs. Verified predicates are discounted according to graph distance from the nearest broken prerequisite, while independent predicates are unaffected. Repairs update these weights based on any errors that remain; invalidated predicates need re‑verification to regain credit. A fixed potential converts these annotations into signed per‑step rewards. A common reward and annotation interface allows DARS to integrate with a range of reasoning and agentic training methods, such as GiGPO and ARPO/AEPO, without changing their rollout strategies or optimizers. Across five task families and models from 1.5B to 8B, DARS improves success by up to 10 points over GiGPO trained with the same budget and harness (ALFWorld), raises the WebShop task score and Search‑R1 QA accuracy, complements AEPO's entropy‑based training on AIME24/25 with a Python interpreter, and exceeds OmniOPD in controlled tool‑free reasoning comparisons at 1.7B and 4B. Ablations show that step‑level credit, dependency attenuation, and graph topology each contribute. On ALFWorld, a distilled 8B annotator matches the API annotator, enabling DARS to run efficiently without a frontier judge. Code is available at https://github.com/JianhuiWei7/DARS.
Authors:Yihao Ding, Daniel Yitian Su, Yiran Zhang, Christopher M. Gonzalez, Wei Liu
Abstract:
Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation. Mineral‑exploration drillholes provide a natural but largely unexplored setting for this paradigm: as drilling proceeds, lithology is revealed sequentially from shallow to deep, making prediction of deeper strata inherently autoregressive. Existing drillhole modelling, however, is dominated by spatial interpolation and reconstruction, or largely rely on masked modelling, leaving strictly autoregressive prediction largely underexplored. We introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next‑layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum, from local prediction through spatial shift to cross geological province transfer. Benchmarking classical, geostatistical, and neural models reveals a clear \emphtransfer boundary: spatial and geochemical conditioning provides large local gains but deteriorates sharply under stronger shift, whereas lithology‑sequence autoregressive models transfer more robustly. Guided by this finding, we develop a backbone‑agnostic recipe combining large‑scale pretraining on historical drillholes with spatial retrieval of neighbouring lithology. Retrieval is most effective in weathered cover, when local spatial continuity remains informative, whereas pretraining contributes more strongly in bedrock and under broader geological shift. Together, they retain strong local performance while improving generalisation under spatial and cross‑province shift, most markedly on the most distant splits. The benchmark and code are available at https://github.com/yihaoding/drillbench.
Authors:Pramish Paudel, Ajad Chhatkuli, Luc Van Gool, Danda Pani Paudel
Abstract:
Object permanence, keeping track of an object's identity and position while it is occluded, is central to video representations that track, predict and plan. Trackers that achieve it learn from boxes, track identities and visibility labels. On the other hand, self‑supervised object‑centric methods discover objects without labels: through slot attention, it represents a video as slots that bind to objects and follow them across frames. However, these slots are lost under occlusion, making the desired permanence impossible. Reasoning permanence is a hard problem because it requires to detect when an object becomes occluded, re‑identify when object reappears, and keep the object's hidden position continuous, using reapperance as the only learning cue. To address this, we propose iSEE, a novel framework that offers all three aforementioned requirements, without any labels whatsoever. We built iSEE using the following three proposed components: (i) Object evidence modelling: a slot's attention, compared with its own past, reveals when its object is hidden. (ii) Appearance‑position separation: two slot streams let the appearance be held for re‑identification while the position keeps changing. (iii) Permanence from reappearance: a walker follows the hidden object's position, trained only on where the object reappears. On LA‑CATER static, iSEE returns a reappearing object to its own slot after 86% of occlusions, against 32% for SlotContrast, and localises it while hidden within 4.1 mAP of the label‑trained SoTA RAM. The two streams also allow downstream planning, with the position stream as the action of a world model. Project page: https://insait‑institute.github.io/iSEE/
Authors:Faias Satter, Noor Masrur, Sk. Md. Masudul Ahsan
Abstract:
An optical character recognition(OCR) system can scan paper and extract text, making people's jobs easier. While numerous OCR systems are accessible in the software sector, finding a dependable equivalent solution for Bangla is tough. When it comes to handwritten texts, the case is even more rare. The first fundamental step to any OCR is to segment words from text images. If this stage fails, the total OCR's performance will be poor no matter how promising the later stages perform. This research aims to segment words in a handwritten Bangla text image. This research can be implemented on any smartphone‑captured image, irrespective of the color and type of paper and ink. Furthermore, as smartphone‑captured images can create shadow interferences, the custom dataset built for this research is created in such a way that every possible obstacle that can be faced is included. For 7374 words, a total of 7278 bounding boxes are generated, which have recall of 90.60%, precision of 91.80%, and F1‑score of 91.20%. The system can be further improved with nested operations on bounding boxes containing several words or by adjusting the adaptive thresholding and dilation filter sizes to a more precise level.
Authors:Di He, Pengxiang Li, Da Chang, Qingyan Meng, Lu Yin, Shiwei Liu
Abstract:
Looped Transformers introduce recurrent depth as a new scaling axis for LLMs: by repeatedly applying shared Transformer blocks, they increase effective depth without increasing parameter count. However, the benefits of looping remain unclear for large MoE LLMs under FLOPs‑matched comparisons. The main reason is that the gains from additional iterations diminish quickly and can even turn into degradation, so the extra FLOPs spent on looping yield little substantial improvement. Consequently, prior work typically settles on two loops. We identify two main obstacles to scaling looped MoE. First, looping inherits and amplifies the curse of depth: hidden‑state variance grows with each iteration as residual updates accumulate, which destabilizes deep recurrence and causes representations to drift. Second, looped MoE suffers from expert selection collapse: routers repeatedly select the same experts across loops, so extra iterations add computation without adding computational diversity. Guided by this diagnosis, we propose LOOM, built on a single principle: each loop should contribute new computation while keeping the recurrent state stable. LOOM stabilizes recurrence by scaling residual updates to bound variance growth and re‑injecting the input embedding at every loop, and diversifies it through per‑loop routers that engage different experts and a Looping Residual that carries earlier outputs forward. Experiments across 100M‑1.7B models show stable scaling to 9‑12 loops. Under near‑iso‑FLOP, the 700M model performs best at 5 loops, reducing perplexity from 18.36 to 16.54 and improving average zero‑shot accuracy from 38.84% to 39.53% over the non‑looped baseline. Without FLOP matching, the 1.7B model trained on 60B tokens peaks at 9 loops, reducing perplexity from 9.62 to 7.77 and improving average zero‑shot accuracy from 42.4% to 47.7%. Code is available https://github.com/hed‑ucas/LOOM.
Authors:Faias Satter, Sk. Md. Masudul Ahsan
Abstract:
An optical character recognition (OCR) can scan a paper and extract text using technology, making people's jobs easier. While various OCR systems are available in the software industry, finding a reliable equivalent solution for Bangla takes much work. When it comes to handwritten texts, the situation is much more unusual. Recognizing words from word images is the most critical stage in any OCR process. It is the second stage after segmenting words from text pictures. If this stage fails, the overall performance of the OCR will be poor, regardless of how well the other phases perform. This study aims to recognize words using deep learning in a handwritten Bangla word image. Three object detection models, SSD with MobileNetV2, Faster R‑CNN with InceptionResNetV2, and an ensemble model of these two, have been used to train and test handwritten word images. A modified Non‑Maximum Suppression has been introduced to enhance the effectiveness of the models' results. A customized dataset of 9841 handwritten Bangla word images has been compiled, featuring diverse handwriting styles from various individuals. All three models' performances have been checked against the test dataset, and the ensemble model has been the most impressive, with an F1‑score of 92.61%. Also, at the word level, the ensemble model correctly recognizes 96.12% of the words to some extent. The system can be further improved by introducing a post‑processing phase to correct errors generated by the system.
Authors:Yoonkyu Woo, Woojin Lee, Jin-Xia Huang
Abstract:
End‑to‑end research agents can now produce complete scientific papers, yet manuscript claims often diverge from executed experiments. This gap is structural: research state, failure histories, and claim‑evidence alignment are not maintained as persistent, verifiable state across long‑horizon pipelines. We present YouRA (Your Research Agent), an architecture for stateful, evidence‑traceable autonomous research. YouRA preserves research state, execution evidence, and failure history across the research trajectory by integrating three components: a Verification State Architecture (VSA) that tracks hypotheses, gates, and evidence pointers; an Independent Controller that turns state and reflection records into lifecycle, recovery, and debate/review control while separating control from execution; and Stateful Reflection that logs failures as structured lessons and routes recovery through bounded repair, redesign, or reset. On MLR‑Bench's predefined ten‑task end‑to‑end subset, YouRA improves over both MLR‑Agent and AI Scientist V2 on scalar Overall across all three matched backbones. An automated diagnostic using MLR‑Bench's hallucination taxonomy reports intersection/union counts for four fact‑based failure types, and data‑provenance diagnostic shows more real‑data‑based outputs. Ablating each of the four components (the VSA, the Independent Controller, MCP tool access, and reflection‑guided recovery) supports their separable contributions. Removing either core‑state component drops YouRA below the full system. Code: https://github.com/PrayPrey/Your‑Research‑Agent.
Authors:Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang
Abstract:
In‑context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fixed activation vectors extracted from specific layers or positions, but these input‑independent interventions fail on complex tasks where the output depends on fine‑grained interactions with the input. By analyzing the ICL forward pass, we show that each attention head's output is an affine transformation of its context‑masked counterpart, and that the parameters of this transformation are empirically stable across samples for a given task. Building on this, we introduce Task Operator (TO), which replays this transformation as an analytically derived update to the attention output projection. Across lexical, algorithmic, and reasoning tasks, TO achieves the best overall performance among prior methods and substantially narrows the gap between zero‑shot inference and ICL. We further show that the extracted knowledge concentrates in a task‑specific sparse circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables effective many‑shot scaling without expanding the context window. Our code is available at https://github.com/gzxiong/task_operator.
Authors:Xuehang Guo, Haoyu Wang, Haifeng Chen, Yangyi Chen, Zhenhailong Wang, Qingyun Wang
Abstract:
Tackling complex real‑world tasks can exceed the capabilities of a single large language model (LLM), motivating the use of multi‑agent workflows that coordinate specialized agents to work together on these tasks. Recent methods train LLMs to construct better workflows from execution outcomes, but they optimize only the workflow generator, while the other agents that build or execute each workflow remain fixed even though every outcome depends on all of them. However, extending training beyond the generator is challenging: the agents are coupled, and a workflow's outcome is a single sparse score that cannot tell which agent causes a failure. We propose FloWright, which leverages the workflow as a harness to optimize workflows. By introducing a hierarchical, structure‑aware reward paradigm, FloWright enables one role to self‑evolve and two or more roles to co‑evolve, with no additional models, labels, or executions. Considering the limitation that workflows are commonly trained and evaluated on data that a single agent can already handle, we further propose DataWright, an adaptive data hardening approach that converts existing datasets into workflow‑level tasks with increased difficulty. Across document, slide, chart, code, math, and finance tasks, small open models trained with FloWright achieve improved performance by up to +7.41%, with co‑evolving (+5.03%) more roles gaining more than optimizing one of them alone (+2.83%). Our project page: https://xhguo7.github.io/FloWright/.
Authors:Hao Wu, Shengju Qian, Weiyan Wang, Fan Xu, Fan Zhang, Yuanpeng He, Qingsong Wen, Yuxuan Liang
Abstract:
Robotic world models predict action‑conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate‑specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory‑Conditioned Search‑Guided Policy Optimization), which converts video trajectory rewards into group‑relative advantages to optimize the world model. On RT‑1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32‑frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical‑flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander‑wu/FutureWorlds.
Authors:Xuehang Guo, Haoyu Wang, Shengyu Chen, Zach Chen, Wei Cheng, Qingyun Wang, Haifeng Chen
Abstract:
Large language models (LLMs) increasingly construct multi‑agent workflows that decompose a complex task and assign specialist agents from a pool. However, building such a workflow well remains challenging: how finely to divide the task, which agent to trust with each subtask, and when to create a new specialist are all critical decisions a workflow constructor needs to settle up front. Thus, whether each subtask succeeds remains unknown until the workflow runs. Yet, improving a workflow is costly. Locating a fault usually requires a reference answer, a graded outcome, or a trained assessor, and the fix is applied to the whole workflow through re‑execution, re‑search, or retraining. We propose InFlowOp, which prices every decision in one label‑free cost that weighs how well an agent's competence meets what a subtask demands against how much that agent takes to run. Before execution, InFlowOp bidirectionally determines the granularity of task decomposition and agent assignment following from the cost rather than from a fixed template. During execution, InFlowOp corrects a fault with the cheapest move via the same cost that serves the workflow both as it is built and as it runs. Facing the workflow‑level evaluation challenge, we introduce Braid, a benchmark whose tasks require multi‑agent coordination beyond single‑agent capability. Across various domains and backbones, InFlowOp outperforms single agent baselines by up to +11.97%, achieving +9.64% with in‑flow optimization. Our project page: https://xhguo7.github.io/InFlowOp/.
Authors:Zekai Zhang, Yunjie Tian, Yanjin He, Xiaoyan Zhang, Dongdi Zhao, Qing Qu, Di Fu
Abstract:
Diffusion language models (DLMs) offer a promising alternative to autoregressive (AR) language generation. Recent advances in continuous DLMs, which apply latent diffusion to continuous text embeddings, raise a practical question: which embedding makes the best latent space, i.e., the most diffusible? To answer this, we search through different embeddings and find that scaling the embedding model to stronger ones within the same family (T5 to T5Gemma‑1 to T5Gemma‑2) greatly improves generative performance. But the raw T5Gemma‑2 embeddings are still not optimal. They are so discriminative that even the embeddings of plausible alternative words are separated, which makes the generation vulnerable to imperfect sampling. Consequently, continuous diffusion often fails to reach any of them and ends up at an invalid embedding instead. To address this, we distill T5Gemma‑2 into a student encoder that learns the teacher's decoded probabilities as soft labels. Learning from such soft labels makes the student pull the alternative embeddings closer while maintaining the encoding‑decoding mechanism. The distilled embeddings form a more connected and diffusible latent space, improving over the vanilla T5Gemma‑2 embeddings. As a result, our medium‑sized DLM achieves Gen. PPL 17.8 (against real‑text PPL 15.4) at real‑text entropy on OpenWebText, outperforming GPT‑2‑M on Gen. PPL.
Authors:Junyi Wu, Fanqing Kong, Leyang Chen, Shaoqiu Zhang, Yulun Zhang
Abstract:
Feed‑forward 3D vision models such as VGGT have achieved remarkable progress, unifying camera estimation and dense scene reconstruction in a single pass. However, their quadratic global attention makes long image sequences expensive, while existing sparse methods may favor highly attended yet value‑redundant regions. To address these limitations, we introduce VASC, a training‑free sparse attention method combining value‑aware block selection and execution‑aware cross‑layer memory. Our value‑aware block selection integrates pooled query‑‑key relevance with neighboring value contrast, reducing redundancy while preserving query‑relevant and distinctive content. Cross‑layer memory tracks unserved demand across layers and updates this state according to actual execution, enabling previously underserved blocks to compete under a fixed computation budget. Experiments on 7Scenes and NeuralRGB‑D with VGGT and π^3 demonstrate improved pose estimation and reconstruction quality compared with FasterVGGT, together with up to 2.29× faster inference than dense VGGT. Code is available at https://github.com/kosakayamahoo‑design/VASC.
Authors:Gunwoo Lee, Yoori Oh, Yoseob Han
Abstract:
Video‑to‑speech synthesis aims to generate natural‑sounding speech from silent talking‑face videos while ensuring phonetic accuracy. A fundamental challenge in this task is the inherent one‑to‑many mapping problem, where visual dynamics often lack sufficient information to uniquely determine the corresponding utterance. To address this, we propose Watch Your Speech (WYS), a video‑to‑speech synthesis framework that incorporates textual conditioning as an explicit linguistic cue to mitigate visual ambiguity. Our framework features an attention‑based embedding fusion module that synergistically integrates textual context with video sequences, coupled with a conditional flow matching objective for high‑fidelity speech generation. Extensive experiments on the LRS2 and LRS3 datasets demonstrate that WYS achieves superior performance, establishing new state‑of‑the‑art results in audio‑visual synchronization (LSE‑C/D) while maintaining highly competitive textual accuracy (WER). Subjective evaluations further confirm that our model generates speech with near‑human naturalness, validating the effectiveness of textual conditioning in content‑controlled video‑to‑speech synthesis. Project page: https://github.com/gunwoo5034/Watch‑your‑Speech
Authors:Sharath M Shankaranarayana, Davor Runje, Jan Jannink
Abstract:
Decision models return probabilities intended for routing, abstention and automated action. Calibration makes those probabilities useful on average, but does not establish whether low confidence reflects chance or missing knowledge, nor whether confidence falls when a model moves beyond what it knows. We audit this distinction in Jev, a decision model, with over 15 public datasets and 6 generated task families, with paired interventions that vary the information supplied for a fixed item. Jev's confidence is calibrated on familiar closed‑choice tasks but fails as an indicator of missing knowledge: with no answer‑relevant information it assigns up to 0.80 to a salient option, and on news beyond an observed knowledge boundary it exceeds accuracy by 0.21‑‑0.33, a gap that recalibration on earlier months does not close. Targeted yes/no questions give sharper readouts of the case: whether an outcome is settled (AUROC 1.00) and whether the evidence suffices (0.95, against 0.85 for confidence on the same items). Asking whether Jev knows the answer appears to flag fabricated entities and post‑boundary news (0.91), but with realistic names or with dates removed it shows no advantage over answer uncertainty. Black‑box knowledge audits therefore need explicit controls for surface cues. Code: https://github.com/Syntheme/beyond‑answer‑confidence.
Authors:Alan Estrada Flores, Nelson Rosa
Abstract:
In this paper, we explore the impulsive dynamics common to single‑joint, two‑link models of walking and brachiating gaits with respect to slope and switching time. In particular, we investigate how the stability of a gait and bifurcations encountered within a family of gaits change under time‑based and state‑based switching of the impulsive dynamics.
Authors:Jungseob Lee, Sugyeong Eo, Seongtae Hong, Seungyoon Lee, Chanjun Park, Jaehyung Seo, Heuiseok Lim
Abstract:
Knowledge distillation aims to transfer the factual knowledge of large language models to smaller models for efficient deployment. Yet a teacher may recall a relation in one direction while failing to generate the answer in the reverse direction. Distillation from its generated answers can therefore propagate this directional limitation to the student. The same teacher can nevertheless recognize such an answer by scoring the relation in the direction it knows. We introduce directional label distillation, in which frozen teachers score candidate answers in that known direction and the best‑scoring candidate becomes the student's training target. On facts about parents and their children, known‑direction scoring yields more accurate labels than scoring the requested direction, even after tuned corrections for name priors. With prior‑corrected scores, the better direction depends on the facts rather than the template, and reverses on mined facts whose notable entity is the parent rather than the child. With the evaluated children's forward facts withheld, students trained on known‑direction labels improve open‑ended accuracy on their trained queries by 13 to 15 points over students trained on prior‑corrected reverse labels. After generated answers are matched to a fixed name list by lexical similarity, students reproduce nearly all selected labels. Their accuracy largely follows label quality. The label advantage holds on unscreened queries and when candidates are retrieved without inserting correct answers. Our findings show that directional verification mitigates the transfer of errors from teacher‑generated answers to students by providing more accurate training targets. Code is available at https://github.com/js‑lee‑AI/directional‑verification.
Authors:Xianda Du, Max Ku, Weiming Ren, Zhi Rui Tam, Chunlin Ren, Ping Nie, Min-Hung Chen, Wenhu Chen
Abstract:
Existing synthetic image evaluators typically provide only a scalar quality score and do not identify the image regions that support it. We introduce VIEScore2, a unified evaluator for image generation and editing tasks with optional conditioning images. VIEScore2 represents an image as an N x N grid and jointly predicts quality scores and defect locations in a single model pass. Its text‑native grid representation provides a common interface for heterogeneous spatial supervision and enables directly verifiable post‑training objectives. We train on 38K examples spanning score‑only, localization‑only, and joint supervision across generation and editing tasks. Starting from supervised fine‑tuning, we further apply GRPO to improve defect localization using rewards that combine cell‑level Dice overlap, score accuracy, and output‑format validity. A parameter‑free parser converts the structured predictions into readable explanations. On the primary suite, VIEScore2 achieves an overall‑score SRCC of 0.601, compared with 0.491 for Gemini‑3‑Flash, the strongest zero‑shot general‑purpose VLM baseline under matched inputs. For defect localization, VIEScore2 outperforms both general‑purpose VLMs and specialized spatial evaluators on three of six benchmarks in per‑image grid IoU and ranks among the top three on five, including datasets beyond its training sources.
Authors:Chao Li, Shigeng Wang, Anbang Yao
Abstract:
Post‑training quantization (PTQ) methods typically use sequential quantization that partitions a pre‑trained LLM into a series of units (e.g., transformer blocks), with one unit quantized at each stage. State‑of‑the‑art PTQ methods are predominantly learning‑based, optimizing auxiliary quantization parameters (e.g., scaling factors, rotation matrices, clipping thresholds, and adapters) via gradient descent to minimize a reconstruction loss. A common practice is to use mean squared error (MSE) as the reconstruction loss function, yet its induced optimization behavior remains largely unexplored. In this work, we take a holistic view of sequential quantization and systematically investigate how optimization evolves from the first quantization stage to the last, aiming for a deep understanding of optimization in learning‑based PTQ schemes. Through extensive empirical studies spanning representative learning‑based PTQ methods, LLM families, model scales, architectures, quantization settings and various tasks, we consistently uncover Optimization Imbalance: reconstruction loss magnitudes vary dramatically across stages, accompanied by highly uneven gradient magnitudes and parameter updates under MSE. We term the cross‑stage range of loss magnitudes the reconstruction loss scale, and reveal that MSE translates the unexpectedly large reconstruction loss scale into highly uneven gradient magnitudes, which in turn lead to uneven optimization strength across quantization stages. This finding suggests a general principle for improving learning‑based PTQ: optimization strength across stages should be decoupled from the reconstruction loss scale. Theoretically, we show that root mean squared error (RMSE) variants defined at the sample, channel, token, and element levels naturally realize this principle through implicit gradient normalization, outperforming MSE significantly as a drop‑in replacement.
Authors:Shota Kobayashi, Koki Seno, Daichi Yashima, Komei Sugiura
Abstract:
We focus on language‑conditioned flow‑based manipulation, where robot flows (robot velocity fields) serve as embodiment‑agnostic, motion‑centric representations for leveraging data collected from multiple robot platforms. This task is crucial because language‑conditioned manipulation is essential for practical robotic systems, yet scaling robot foundation models remains limited by the labor‑intensive collection of embodiment‑specific data. Existing methods either coarsely approximate robot flows with sparse keypoint displacements, or cannot handle language‑conditioned manipulation. To address this limitation, we propose NarrativeFlow, which models robot flows as continuous velocity fields using a flow‑matching formulation conditioned on language. Accordingly, NarrativeFlow generates robot flows that are physically consistent with real‑world manipulation. To validate NarrativeFlow, we have conducted experiments on standard datasets for language‑conditioned manipulation. The experimental results show that NarrativeFlow outperforms representative baseline methods on standard evaluation metrics. Furthermore, through real‑world experiments, we show that NarrativeFlow achieves higher success rates than baseline methods across multiple manipulation tasks. The project page is available at https://shota0520.github.io/NarrativeFlow‑project‑page/
Authors:Zijian Liu, Yangzhixin Luo, Junyu Lu, Yi Li, Yu Chen, David Xu, William F. Shen, Xinchi Qiu, Xisen Wang
Abstract:
Multi‑agent AI scientists have shown improving performance across a diverse range of tasks. Yet a common approach is design‑time agentic orchestration, which typically relies on fixed workflows. In contrast, human scientists coordinate and adjust their division of labor at runtime. We therefore ask: can AI scientists also coordinate at runtime? To this end, we introduce Runtime Agent Coordination (RAC), which selects agents from existing AI‑scientist hosts during execution, assigns scoped work contracts, and provides artifact‑grounded verification. Verification informs subsequent agents without blocking transitions or discarding artifacts. We conduct a single‑seed exploratory evaluation across Agent Laboratory, EvoScientist, and ARK on ResearchClawBench, preserving host models, tools, and permissions under host‑calibrated budgets. Four cumulative conditions separate native execution, runtime communication, runtime selection, and the combined addition of contracts and verification. Runtime selection yields the highest observed mean score for each host; adding contracts and verification reduces these means, with host‑dependent outcomes relative to native execution. These results motivate runtime coordination while exposing the limits of additional coordination mechanisms under constrained budgets. Code is available at https://github.com/systemind‑team/Runtime‑AI‑Scientist.
Authors:Arman Behnam, Binghui Wang
Abstract:
Causal representation learning aims to discover robust features by exploiting the causal structure underlying data generation. Existing methods require specifying the causal structure a priori, yet different structures demand fundamentally incompatible invariance constraints, and misspecification leads to representations that discard predictive information. We introduce SaCRL, a framework that jointly identifies the causal structure and learns the corresponding invariant representation without prior structural knowledge. Our approach formulates structure selection as a soft optimization over candidate invariances using HSIC‑based violation metrics, with adaptive weights that automatically concentrate on the achievable structure. We provide theoretical guarantees for structure identification, including under random‑feature approximation, invariance satisfaction, and out‑of‑distribution generalization. Empirically, SaCRL recovers the true structure on synthetic and semi‑synthetic Bayesian‑network benchmarks, outperforms fixed‑invariance baselines on Colored MNIST, achieves state‑of‑the‑art accuracy on three DomainBed benchmarks (PACS, VLCS, OfficeHome), and degrades gracefully under structural misspecification and limited environment diversity. Code is available at: https://github.com/ArmanBehnam/sacrl.
Authors:Enxin Song, Yinuo Xu, Shusheng Yang, Wenhao Chai, Jiatao Gu
Abstract:
A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. We introduce the attack pyramid, five levels of shortcut attacks with increasing access to each item, and audit 115 video benchmarks with it. On 35 benchmarks, attackers that never see a frame approach full‑video accuracy. On 51 benchmarks with temporal probes, shuffled frames keep a median 96% of full‑video accuracy. Near‑duplicate questions make up at least half the items in 63 benchmarks. We screen 505,518 question‑answer pairs from 112 of them into an audited pool. Agents turn evaluation requests into specifications, and a deterministic selector with a red‑team gate composes reproducible benchmarks. We release Video‑Index, the 210 hardest verified items under these attacks in each of four capability groups, 840 items from 76 sources. With the same fixed input, Claude Opus 5 outscores every open‑source model by over 37 percentage points, and agent tools add about 20 more, yet all systems leave room to improve efficiency and accuracy. Blog: https://www.enxinsong.com/blog/video‑index/ GitHub: https://github.com/Espere‑1119‑Song/Video‑Index Hugging Face: https://huggingface.co/datasets/Video‑Index/Video‑Index
Authors:Geyi Yang, Zikun Qu, Xiang Li, Zhiyong Wang, Min Zhang, Shipei Zeng, Zhongxiang Dai
Abstract:
The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termination are controlled. Compared with harness optimization for non‑GUI agents, automatically optimizing this harness poses three coupled challenges: reconciling model intent with observed visual effects, diagnosing failures under variable execution outcomes, and identifying recurrent failure patterns across tasks and translating them into reusable runtime changes. We introduce GUI‑HARVEST, an automatic harness optimizer that enables self‑improving GUI agents with frozen backbone models. First, to ground diagnosis in observed action effects, it aligns model outputs and executed actions with before‑and‑after screenshots, tying findings to specific interface transitions. Second, to account for execution variability, it treats repeated runs of the same task as a joint evidence unit, using within‑task comparisons to locate outcome‑relevant behavioral differences. Third, it consolidates verified findings across tasks into recurring failure patterns, maps them to bounded source‑code edits with predictions recorded before evaluation, and checks the predicted behavioral effects alongside task performance through repeated execution. Experiments on OSWorld‑Verified show consistent held‑out gains across six general‑purpose open, GUI‑specialized open, and proprietary backbone models; Qwen3‑VL‑32B‑Instruct gains 12.33 points on the full suite. Frozen‑harness transfer improves GPT‑5 by 13.87 percentage points on WindowsAgentArena at 50 steps without further optimization. With the same backbone and initial harness, GUI‑HARVEST outperforms Self‑Harness and Meta‑Harness, suggesting that GUI‑specific diagnosis and validation help harness improvements generalize to unseen tasks. The code is available at https://github.com/GaryYang12345/GUI‑HARVEST.
Authors:Shawn Bowers, Martin Caminada, Haoyang Liu, Bertram Ludäscher
Abstract:
ABDA‑NL adds a natural‑language interface to ABDA, a system for argument‑based discussion using ASPIC‑ knowledge bases under grounded semantics. Users see which conclusions are accepted, rejected, or undecided, open an interactive rendering of the grounded discussion game to learn why, explore what‑if alternatives by suspending assumptions and rules or changing preferences, ask questions that are answered from a scenario's reference documents, and author new facts, assumptions, and rules in plain English. A large language model provides the bridge between language and formalism: it answers questions from the documents and the current state of the scenario, and it translates plain‑English edits into candidate formal statements. The deterministic ABDA engine remains the sole source of arguments, attacks, and acceptance labels, and every proposal of the model is validated and confirmed by the user before it takes effect.
Authors:Peihao Chen, Qing Wang, Lichun Fan, Yufeng Hao, Zhifeng Kong, Mengyao Zhu, Hengyi Hong, Hang Chen, Hang Su, Yujie Jian, Chao-Han Huck Yang, Shichao Hu, Jun Du, Jian Luan, Ke Li
Abstract:
Embodied assistants in domestic environments must infer what happened, where and when it occurred, and how to respond. To address this, we introduce RMS‑AQA, a spatial audio question answering (SAQA) benchmark for real‑world domestic environments. The benchmark features a two‑stage question‑answering (QA) format to comprehensively assess the ability of audio‑language models (ALMs) to first ground audible sound events and subsequently perform complex spatio‑temporal reasoning based on that grounding. To maximize acoustic realism, our dataset combines authentic real‑world first‑order Ambisonics (FOA) recordings with high‑fidelity synthetic data generated using measured room impulse responses (RIRs). Furthermore, we provide a lightweight spatial plug‑in that injects FOA‑format data into frozen audio‑language backbones. Experimental results reveal that the primary challenges stem from concurrent sources, far distance, and sim‑to‑real domain gap between RIR‑synthesized and authentic recordings.
Authors:Chengkai Xu, Yiming Cui, Jiaqi Liu, Yicheng Guo, Cheng Qin, Geyuan Zhang, Xinwei Dong, Shiyu Fang, Peng Hang, Jian Sun
Abstract:
Autonomous driving is a cornerstone technology for the future of intelligent transportation, where end‑to‑end learning has emerged as a transformative paradigm that directly maps multimodal sensory inputs to driving actions through unified differentiable models. While offering advantages, the effectiveness of end‑to‑end autonomous driving (E2E‑AD) is ultimately determined by the quality of its training ecosystem. This paper provides a comprehensive review of training methods and ecosystem for E2E‑AD. We introduce a Data‑Strategy‑Platform taxonomy that conceptualizes training as an interdependent system. The data layer defines what can be learned, the strategy layer governs how learning aligns with driving objectives, and the platform layer supports scalability and continuous evolution. Within this framework, we survey recent advances across data‑centric pipelines, learning paradigms, and training infrastructures, and analyze their interplay in shaping model performance, robustness, and deployability. Finally, we reflect on current limitations and articulate a forward‑looking vision that emphasizes a shift from data quantity to data value, from isolated optimization to foundation‑driven generalization, and from static training to integrated training‑testing loops, aiming toward robust, scalable, and trustworthy autonomous driving systems. We maintain a continuously updated repository tracking cutting‑edge literature and works at \hrefhttps://github.com/Jiaaqiliu/Awesome‑Training‑Ecosystem‑for‑E2E‑ADOur Project Page.
Authors:Jungmin Lee, Niamat Ullah, Yoseob Han
Abstract:
Gaze estimation under natural head‑eye motion underpins applications from driver monitoring to human‑computer interaction. Single‑frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi‑frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory‑Aware Gaze Estimation), a causal multi‑frame framework built around an explicit first‑order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation‑invariant in gaze space, this token carries subject‑invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross‑attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0^\circ on Gaze360 and performs on par with the strongest baseline on EVE (2.56^\circ vs. 2.58^\circ). Within‑model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze‑history prior. Our code is available at https://github.com/peter8366/EyeTAG.
Authors:Yixuan Li, Yiyun Zhou, Yao Long Teng, Fuchao Yang, Yanchen Deng, Zhiyi Lyu, Xuyu Dong, Feng Chen, Bo An
Abstract:
Choosing an agent system means choosing both a language model and the harness through which it acts. We ask whether a strong model, harness, or pairing stays strong when the setting changes. We evaluate 66 configurations: four configurable harnesses (OpenHands, DeepSeek Harness, PI, and openJiuwen) paired with five models on TUA‑Bench, ALE‑CLI, and Terminal‑Bench 4, plus the native Codex‑GPT and Claude Code‑Claude pairings. Model rankings reverse across harnesses. On Terminal‑Bench 4, Claude leads GPT by 7.94 points in OpenHands but trails it by 30.16 points in PI. For four of the five models, the best harness changes from one benchmark to another, yet some pairings hold: openJiuwen gives Kimi its highest score on all three benchmarks, by 5.61 to 11.11 points. A model's own vendor harness is not reliably its best, and higher cost does not reliably buy a higher score. On Terminal‑Bench 4, GPT scores higher under PI than under DSH at less than a quarter of the cost per task. Matched trajectories suggest why fit varies. Models start almost all repairs themselves, so much depends on whether the harness hands failures back in a form the model can use. GPT does best with PI's lean scaffold, while Kimi, which often issues malformed tool calls, does best in openJiuwen. We argue that the model, the harness, and the task should be evaluated together, and we release the harness adapters, evaluation code, and all 6,204 scored trajectories at https://github.com/liyix/finding‑the‑right‑fit and https://huggingface.co/datasets/yixuanli97/finding‑the‑right‑fit.
Authors:Dmitry Kamenetsky
Abstract:
The discovery of Chair44 (Tsiokos, 2026) settled the three‑dimensional einstein problem with a strongly aperiodic polyhedral monotile in \mathbbR^3. This note extends the underlying mechanism‑‑‑the rep‑2^N chair C_N = [0,2]^N \setminus (1,2]^N with corner/socket markings‑‑‑to \mathbbR^N. Besides expository material (the rep‑2^N dissection and a conditional strong‑aperiodicity theorem under lattice registration and hierarchical enforcement), the note makes a new computational contribution. We introduce a frame‑marking formalism in which the marking of a tile is its full orientation frame and the matching rule is the contact language generated by the substitution itself; this makes the search for matching rules finite in every dimension. We give a finite certificate (coarsening closure, tightness, and a two‑shell enclosure analysis) whose validity implies that every lattice‑registered tiling by the marked tile is uniquely hierarchical, hence strongly aperiodic. For N=3 the certificate passes: it yields explicit facet matching rules on the 24 panels of C_3 (135 admissible facet‑contact triples) and reproduces, from first principles and independently of published constructions, the Chair44 statistics 2388 \to 44 admissible contacts (30 occurring), 33 one‑shell clusters, 15 extendable, each forcing a unique supertile. Among the 2187 homochiral frame assignments of the 3D substitution with a translated central child, the certified one is unique up to conjugation. For N=4 the same pipeline is run on several structured families of frame assignments (canonical, D_4‑, Z_2× Z_2‑ and Z_4‑symmetric, and a lift of the 3D solution); none is coarsening‑closed, and we report the failure data. A self‑similar marking of C_4 thus remains an explicitly finite, open computational problem, which we state precisely. Code: https://github.com/dimkadimon/Monotile‑RN
Authors:Yufa Zhou
Abstract:
Human reasoning depends on how objects are related within propositions. How do relations organize the language representations of contextual contents? We give an LLM a list of facts in its context (e.g., \emphAlice eats an apple. Bob eats a pear.) and measure how its hidden state changes when the question switches from what Alice eats to what Bob eats. Averaged over many lists, this change is a steering vector, which we call the \emphordinal vector. It points to a fact by its \emphorder of mention, the order in which the facts were stated in the context. We find that LLMs represent the fact a question asks about by its order of mention, not by the name the question contains. We state this as the ordinal addressing hypothesis: each order of mention has a \emphfact address in the model's state, shared by all contexts, and a question moves the state to the fact address of the fact it asks about, while the context supplies what that fact says. Across Qwen, Gemma, and Llama, fact addresses are (1) \emphordered by mention: query states are organized by the order of facts, not of names, even when one fact has multiple subjects; (2) \emphsteerable: added to a question about the first fact of a new list, the ordinal vector makes the model answer with the second fact of that list; (3) \emphlow‑rank: they span a low‑rank subspace in which the first‑mentioned fact is the easiest to reach, surprisingly similar to human recall; and (4) \emphemergent: they are shared in late‑middle layers, hold from 1.5B to 32B parameters, and form early in pretraining. Language models reach a stated fact by where it was mentioned, deepening our understanding of LLM reasoning.
Authors:Sungho Park, Wonjoong Kim, Jue Zhang, Wook-Shin Han, Pengfei Gao, Chanyoung Park, Yongqiang Yao, Rao Fu, Elsie Nallipogu, Qingwei Lin, Victor Rühle
Abstract:
Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. ActiveSaddler models the evolving curriculum as a non‑stationary bandit with dynamically instantiated optimization targets. It abstracts recurring failures into reusable failure‑pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co‑evolve with the harness. Experiments on GAIA2 and Terminal‑Bench 2.0 show that ActiveSaddler consistently discovers stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer using a scenario order fixed before optimization, respectively. Ablations further show that these gains depend on dynamically constructing optimization targets, estimating their evolving utility, and balancing continued optimization with new failure discovery. Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization.
Authors:Keisuke Shirai, Tomohiro Motoda, Hanbit Oh, Ryoichi Nakajo, Roman Mykhailyshyn, Ryo Hanai, Shotaro Miwa, Yukiyasu Domae
Abstract:
Autoregressive Vision‑Language‑Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with standard next‑token objectives. FAST has substantially improved this representation by compactly encoding action containing diverse temporal frequencies into relatively few tokens. However, while such compression reduces the number of action tokens required for autoregressive prediction, it does not necessarily improve the efficiency of policy learning from limited demonstrations. In particular, FAST typically assigns a single deterministic tokenization to each quantized action sequence, although multiple token sequences can represent and decode to the same robot motion. We investigate whether exploiting this representational redundancy can improve policy learning. In this paper, we propose TOkenization of Action sequences with STochastic sampling (TOAST), a stochastic action tokenization method that samples alternative tokenizations of the same quantized action sequence during policy training. This diversifies the discrete supervision while preserving the underlying robot action and requires no additional demonstrations. Experiments on LIBERO show that TOAST consistently improves over its deterministic counterpart, with the improvement increasing as training data decreases, achieving a 6.8 point gain in success rate when only 1/16 of training data is available. Across four real‑robot manipulation tasks, TOAST further improves mean success rate by 15.8 points over the deterministic counterpart. These results demonstrate the effectiveness of stochastic action tokenization for autoregressive robot policy learning, particularly when training data are limited.
Authors:Mohamed Mady, Yupei Li, Johannes Reschke, Björn W. Schuller
Abstract:
Robust detection of AI‑generated text under deployment conditions is challenging: distribution shifts across domains and generators, adversarial perturbations of the input surface, and the absence of target‑domain labels for threshold calibration all degrade detectors that perform well in‑domain. We present DeBERTa‑ConPara, a deployment‑oriented detector combining attack‑aware Unicode preprocessing with a contextual transformer encoder trained over HC3 Plus, M4, MAGE and RAID. Our central finding is that preprocessing acts in opposite directions depending on where it is applied: normalising the training corpus deduplicates it, collapsing 35.4% of RAID rows into copies of their clean siblings and deleting the adversarial supervision, whereas normalising at inference is an effective defence. A factorial varying the two placements independently identifies raw training with normalised inference as the best configuration, reaching 99.61% AUROC, 99.01% TPR@5% FPR and 96.57% TPR@1% FPR on the official RAID hidden test, alongside 93.14% average balanced accuracy across HC3 Plus and MAGE under a fixed threshold. The gain is confined to two of twelve attack classes: homoglyph and zero‑width‑space insertion rise from 11.05% and 1.12% to 96.98%. The same signature reproduces in a zero‑shot detector of different architecture, showing the effect belongs to the attacks rather than to our model. We additionally report two negative results: semantic‑invariance augmentation through paraphrasing and supervised contrastive learning (ConPara) does not improve the best configuration, and the handcrafted feature‑fusion branch is inert in distribution and harmful outside it.
Authors:Pengfei Qi, Haoran Lin, Sizhuang Chen, Kai Luo, Sirui Zhang, Xinqi Liu, Fei Cheng, Wenrui Chen, Liming Yin, Kailun Yang
Abstract:
General‑purpose embodied robots should support both navigation toward language‑specified destinations and dynamic person tracking under arbitrary initial target azimuths. However, existing methods typically rely on forward‑facing observations and address these tasks with separate policies, limiting omnidirectional perception and unified closed‑loop control. We present UniTrackPLA, a unified panorama‑language‑action model for instruction‑guided navigation and dynamic person tracking. Its Panoramic‑Aware Encoding (PAE) preserves the temporal and azimuthal structure of perspective views projected from each panorama, enabling perspective‑pretrained visual encoders to process omnidirectional observations. A shared vision‑language backbone grounds instructions in the panoramic context and predicts continuous robot‑centric waypoint chunks for both tasks. World‑Action Consistency (WAC) further predicts action‑conditioned future visual states and verifies waypoint prefixes online, allowing reliable actions to be reused while triggering replanning upon inconsistency. We also introduce OmniTrackNav‑Bench, comprising 5,000 simulated tracking trajectories, 10,000 simulated VLN routes, and 96 verified real‑world routes, providing 919,978 waypoint‑supervision instances. UniTrackPLA improves overall tracking SR from 23.50% to 35.00% and Omni‑VLN SR/SPL from 13.00%/12.77% to 19.75%/19.29%. Incorporating 76 real‑world routes further improves held‑out EP@0.2m from 42.92% to 92.08%. Closed‑loop experiments on a Go2‑W robot demonstrate unified panoramic tracking and navigation across indoor and outdoor environments. The project page is at https://tw5775.github.io/UniTrackPLA.
Authors:Jiawei Fan, Sifeng Wang, Yuqing Hou, Anbang Yao
Abstract:
In this paper, we study how to achieve one‑step action generation in Robotic Foundation Models (RFMs), aiming to overcome the high inference latency of multi‑step flow matching. MeanFlow provides a promising framework for this goal, yet its direct application leads to performance collapse. We discover that this stems from two distinctive dynamics exhibited in the RFM velocity field: (1) the ``local acceleration" exhibits stability early on, but surges sharply towards the end of the denoising process, and (2) the spread of its magnitudes across samples widens as denoising progresses. To address these issues, we introduce Kinematic MeanFlow (K‑MF), a novel one‑step action policy tailored for RFMs. Specifically, grounded in a kinematic identity, K‑MF decouples the time derivative term in the MeanFlow formulation into two sub‑interval terms separated by an intermediate point. This decoupled formulation enables the two terms to capture early‑stage and late‑stage denoising dynamics, respectively, while mitigating the error amplification across the process. As a result, our K‑MF empowers RFMs to achieve one‑step action generation in both training from scratch and fine‑tuning paradigms across diverse tasks, while outperforming multi‑step flow matching in most settings. In terms of inference efficiency, K‑MF reduces action‑head latency of GR00T‑N1.6 by 67.5%~74.4% across L40 and Jetson Orin in eager and compiled modes, yielding end‑to‑end latency reductions of 30.3%~54.9%. Code will be available at https://github.com/IntelChina‑AI/K‑MF.
Authors:Haojian Huang, Pukun Zhao, Zexi Li, Yehang Zhang, Yangkai Wei, Wenqian Li, Han Yang, Kaiwen Zhou, Ying-Cong Chen, Yinchuan Li
Abstract:
A robot generalist must interpret a scene, act on it, and satisfy every condition of a task. How far do frontier vision‑language models (VLMs) meet these demands? We introduce Embodied Agent Arena, evaluating ten VLM agents across Geometry, Spatial Reasoning, Affordance, Task Planning, and Manipulation. Its 1,000 cases combine 32 established sources with GeoProbe, our 168‑case benchmark for camera, motion, depth, and scale estimation. A minimal harness leaves task decisions to the model through source‑native interfaces. Closed‑weight models lead all five domains, whereas open‑weight VLMs show useful spatial reasoning but substantially lower planning and manipulation success. Beneath these averages, similar geometric errors conceal different scale and shape distortions, and valid region estimates need not identify usable action points. During execution, agents can achieve early subgoals yet lose object control, bind an action to the wrong receiving region, or stop before the required physical state holds. Additional computation does not uniformly resolve these failures: code repair can restore execution without completing the task, while self‑review can reverse correct judgments. Failure costs also differ: the strongest planners concentrate failed‑token expenditure in prolonged body‑control tasks, whereas other models fail broadly across procedures. Matched traces show how successful recovery uses feedback to restore a missing prerequisite and advance the remaining task.
Authors:Shuze Liu, Kaixiang Zhao, Runyang Xu, Jingzhi Chen, Nathan Wu, Yu Wang, Yushun Dong
Abstract:
Large language models (LLMs) deployed through text‑only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access assumptions, model configurations, query budgets, and security objectives, limiting comparability across methods. To address this gap, we introduce a unified benchmark covering six extraction attacks, ten defenses, and two adaptive attacks that paraphrase or back‑translate protected responses before surrogate training. The benchmark controls model configurations, query data, budgets, and held‑out evaluation conditions within each comparison while preserving attack‑specific querying and training procedures. We measure surrogate capability, fidelity to the victim, output quality using Rep‑4, and query‑budget sensitivity; defenses use their own security metrics paired with surrogate performance. For the adaptive attacks, we jointly measure provenance‑detector scores and the capability and fidelity of surrogates trained on rewritten responses. The benchmark thus provides a reproducible basis for comparing extraction methods and their interactions with defenses under text‑only access. Code and artifacts are available at https://github.com/sliu11‑byte/MEA‑Bench.
Authors:Sachin Gupta
Abstract:
Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four‑gate checker for declared evidence IDs, entities, numbers, and claim types. It checks a fixed schema; it does not verify every fact in the prose. At r=0 and r=1, we test 900 instances per setting (450 grounded‑ungrounded pairs) with GPT‑4o‑mini, Llama‑3.3‑70B, and Claude Sonnet 4.6. Under this schema‑level contract and before repair, 80.3% of mini claims and 47.9% of Sonnet claims fail. These are verifier rejection rates, not prose‑hallucination rates. One repair pass raises claim survival from 19.7% to 28.0% for mini and from 52.1% to 54.3% for Sonnet. Verified claims per example change by +0.14 for mini, ‑0.71 for Llama, and ‑0.47 for Sonnet, so survival and output volume must be reported together. A second Sonnet pass gives no clear gain. At r=1, Gate 4 covers 97.0%, 98.7%, and 100% of failing claims for mini, Llama, and Sonnet. Small human studies support the rules but show gaps between schema checks and correct prose. A domain‑specific GPT‑4o judge test shows an order effect, so it is only a usefulness check. We release the code and data.
Authors:Jiamu Zhang, Tianze Yang, Yucheng Shi, Evan Chen, Zixiang Nie, Kelly Wan, Liangjie Hong, Ninghao Liu, Liang Wu
Abstract:
A typed decision is a choice among a fixed set of options, returned as a probability rather than as text. Systems that need typed decisions today use models trained for that purpose. This report describes AnyJev, which reads a typed decision from one prefill of a pretrained instruction‑tuned language model. The readout restricts the next‑token distribution at the answer position to the option tokens. It has two defects: the model assigns higher probability to some labels whatever the input, and to some positions in the option list. AnyJev corrects both with no gradient steps and no parameter changes: it divides out a label prior estimated from unlabelled inputs, and it averages log‑probabilities over the K cyclic rotations of the option list. On two 20‑option tasks the rotations lower the order‑flip rate from 0.33 to 0.14 and from 0.33 to 0.18, and raise accuracy on 11 of 11 models on both. Reading every rotation requires K prefills. A stopping rule selected against the full‑rotation decision on unlabelled states cuts that. Selecting the threshold on one unlabelled split and bounding its disagreement on a second, it reads 10.6 rotations of 18 at a verified 0.008 bound on two of four cells; selected and bounded on one split, as our serving run did, it reads 7.3 and serves 2.2 times as many decisions per second on vLLM. The code is open source.
Authors:Samuel Liu, Youngsun Kim, Martin Matak, Gilwoo Lee
Abstract:
Simulation can expand scarce real demonstrations for co‑training, yet how world fidelity and similarity to human behavior affect policy performance remains unclear. We distinguish world grounding, which aligns simulation with the real system, and behavior grounding, which aligns simulated trajectories with human motion. We build a real2sim2real pipeline that varies these axes independently to generate data for co‑training. On a dynamic dexterous pick‑and‑sort task, fully grounded co‑training raises success from 52% to 86%; averaged across configurations, world grounding improves success by 18 percentage points and behavior grounding by 10. Deployed policies behave like a mixture of real‑derived and simulation‑derived policies, imitating real demonstrations in covered states and relying on simulated behavior elsewhere, which we examine through latent‑space analysis. Together, these results suggest complementary roles: world grounding lets policies use simulated experience beyond real‑data coverage, while behavior grounding matters mainly when world grounding is imperfect. Grounded simulation remains beneficial when co‑training foundation models.
Authors:Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, Tianzhi Li, Fei Tao, Pooyan Fazli
Abstract:
Video generation has rapidly progressed from short, low‑quality clips to high‑resolution, long‑duration sequences with complex spatiotemporal dynamics. Despite strong generative priors learned through large‑scale pretraining, pretrained video models often fail to reliably follow human intent, maintain temporal coherence, or satisfy physical and safety constraints. Compared with image and text generation, alignment in video generation presents unique challenges, including error accumulation over time, motion‑appearance coupling, multi‑objective trade‑offs, and limited supervision for temporal properties. These challenges motivate systematic post‑training strategies that adapt pretrained models without retraining them from scratch. In this survey, we present the first comprehensive review of post‑training and alignment in video generation models. We frame post‑training as a unifying framework and distinguish between implicit alignment and explicit alignment based on how alignment signals are enforced. From this perspective, we organize existing approaches into four broad categories: supervised fine‑tuning methods, self‑training and distillation methods, preference‑ and reward‑based methods, and inference‑time methods. This taxonomy provides a coherent view of how alignment signals shape model behavior across both training and deployment. Beyond methodological advances, we review commonly used datasets, benchmarks, and evaluation practices, and discuss open challenges such as scalable reward design, long‑horizon temporal consistency, stability‑expressiveness trade‑offs, and safety‑aware generation. This survey aims to provide a structured conceptual foundation and practical guidance for advancing controllable and reliable video generation models.
Authors:Marcelo Fernandez
Abstract:
A correctly governed LLM agent can reach a state in which neither continuing execution nor automatically halting is admissible: the system has detected a persistent failure of its observability or drift‑detection layer, but cannot itself decide who has the authority to resume, deny, or recalibrate the deployment. We call this an identity‑bound governance event and formalise the mechanism that resolves it. We introduce the Accountability Proof Block (APB): a system‑constructed evidence block, a human‑supplied decision block, and an ed25519 signature binding both to a registered principal. The system cannot forge the signature, and the principal cannot alter the evidence undetected. We prove four theorems: Protocol‑Bounded Governance Completeness, Non‑Repudiability, Impossibility of Anonymous Re‑Authorization, and Finite‑Time APB Construction Termination. The implementation uses RFC 8785 JSON canonicalization and a UUID4‑based replay predicate. Empirically, governance completeness holds over 3,812 halt events with zero unresolved cases; the verifier detects 100% of 1,800 attacks across 9 adversarial vectors; a k‑of‑n multi‑principal variant shows 0 false acceptances in 2,000 single‑key capture attempts. A study of six open LLMs finds the drift threshold T stable within model (sigma/T < 2%) but varying across models, refuting size‑monotonicity: the largest model did not drift. T must therefore be measured per deployment, and the APB is the vehicle by which that threshold yields accountable authority transfer.
Authors:Haoyang Jiang, Yuheng Li, Di Yang, Yanhai Xiong, Haipeng Chen, Yi He
Abstract:
Flow matching (FM) learns generative transport by fitting continuous‑time motion from a simple source distribution to the data distribution. Most existing methods use first‑order bridges: once a source and a target sample are paired, the path is a straight motion with constant velocity. FM with optimal transport (OT) improves the pairing, but the bridge itself remains linear, limiting its ability to model curved motion, acceleration, and changing directions. A natural remedy is to use second‑order phase‑space dynamics; however, learning the bridge requires target‑side terminal‑velocity information that static datasets do not provide. We propose Variational Terminal‑Velocity Flow Matching (VTV‑FM), a second‑order FM framework that derives the missing velocity by minimizing acceleration energy, yielding a closed‑form closure for static data. The same minimum‑acceleration variational construction also defines the OT pairing cost and the acceleration targets used for training. Experiments on low‑dimensional datasets, PDE‑governed physical fields, and CIFAR‑10 show that VTV‑FM improves transport geometry and generation quality over first‑order and high‑order FM baselines.
Authors:Mingzhe Li, Yuefeng Peng, Kejing Xia, Pranav Jeyakumar, Ruolan Leslie Famularo, Shiqing Ma
Abstract:
Image watermarking supports provenance and attribution by embedding verifiable identity information into images. Practical deployments, however, must jointly satisfy requirements for attack resistance, false‑positive rate (FPR), image quality, and latency. Existing watermarking methods are robust to different classes of transformations, so combining complementary methods can provide broader protection than any single watermark. Such composition is challenging, as additional fragments increase distortion and decoding cost and must share the same FPR budget. Therefore, we propose TAILOR, a request‑conditioned watermark composition framework with three stages: (1) offline characterization measures fragment recovery, distortion, and runtime as response curves over embedding strength; (2) joint configuration selection encodes the request as an SMT model over these curves and solves for the lowest‑distortion composition of fragments, strengths, order, and geometric recovery; and (3) live calibration validates the selected configuration on the user's images and refines predictions that fail to transfer. Experimental results across 7,321 distinct requests spanning five scenarios and 20 attack settings show that TAILOR achieves 96.21% scenario‑averaged request satisfaction with a mean PSNR of 41.02 dB, outperforming existing methods in robustness while achieving consistently better image quality. Code is available at [https://github.com/aaFrostnova/Tailor](https://github.com/aaFrostnova/Tailor).
Authors:Peter Nutter, Dani Roytburg, Clément Dumas, Jinghua Ou, Shi Feng
Abstract:
Pre‑training interventions are critical to alignment research, since beliefs formed during pre‑training shape how a model generalizes from later training. One recently popular technique for such interventions is synthetic document fine‑tuning (SDF), which aims to alter what the model believes. Ideally, synthetic documents would be mixed into pre‑ or mid‑training, but every change to a pre‑training corpus must be followed by a full post‑training run before its effect can be measured, making iteration slow and expensive. Common practice instead applies SDF to an already post‑trained model. This is known to leave artifacts and degrade capabilities, and, as we show, it makes the model treat fabricated entities unrelated to the documents as real, a failure we call reality drift. We propose grafting: train the SDF adapter on the pre‑trained checkpoint, then add the learned weight update to the post‑trained model, which approximates the faithful approach while reusing the existing post‑training. We demonstrate this by installing false facts, training misaligned model organisms and applying a constitutional mid‑training intervention, across model families up to 284B parameters. Grafting installs the target belief as strongly as SDF on the post‑trained model while reducing both reality drift and the loss of preference coherence by more than half on average, and it stays closer to a faithful mid‑training run. Because grafting requires no post‑training, the same adapter can be applied to any later checkpoint, enabling researchers to iterate quickly on pre‑training interventions at the cost of a single fine‑tuning run.
Authors:Sanya Verma, Luca Cilio, Velissarios Christodoulou, Tyler Fermelis, Qi Ting Ng, Rachel Chung, Sofia Liang
Abstract:
Simulated evaluation is increasingly used alongside real‑world evaluation of robot policies because it is cheaper and easier to repeat; however, its value depends on how closely its outcomes track the real robot's. We test whether our reconstruction pipeline, combining metrically scaled object geometry, authored physical parameters and scene reconstruction, reduces disagreement between simulated and real robot scores relative to a default open‑source recipe. We constructed two simulated versions of one bimanual robot cell: an authored reconstruction, using object geometry at estimated metric scale, projected textures, authored physics and our own scene splat; and a baseline, referred to as the default reconstruction, using the open‑source recipe of a generative single‑image mesh, engine‑default physics and a Gaussian‑splat scene. Both reconstructions use the same object photographs and scene video. Two policies ran five tasks each, giving ten task‑policy pairs, which we call cells; each cell was run twenty times in each reconstruction with all other settings held fixed. Both reconstructions were scored against the same real trials, graded by a third‑party evaluator. Pearson correlation between the ten simulated and real cell means is r = 0.90 for the authored reconstruction and 0.51 for the default. Mean score error is 6.97 percentage points for the authored reconstruction and 17.54 for the default, a reduction of 10.56 percentage points. These results show that improving the quality of the environment reconstruction through higher visual fidelity, authored physics and metric scale makes the simulation more faithful to the real world and narrows the sim‑to‑real gap. We release the harness, the per‑trial scores, every reported run's configuration, and the assets and scenes of both reconstructions.
Authors:Morgan Byrd, Jacob Blevins, Maks Sorokin, Robert Wright, Sehoon Ha
Abstract:
This paper explores a reward‑based policy to achieve zero‑shot transfer between source and target environments with completely different observation spaces. While humans can demonstrate impressive adaptation capabilities, deep neural network policies often struggle to adapt to a new environment and require a considerable amount of samples for successful transfer. Instead, we propose a novel reward‑based policy only conditioned on rewards and actions, enabling zero‑shot adaptation to new environments with completely different observations. We discuss the challenges and feasibility of a reward‑based policy and then propose a practical algorithm for training. We demonstrate that a reward policy can be trained within three different environments, Pointmass, Cartpole, and 2D Car Racing, and transferred to completely different observations, such as different color palettes or 3D rendering, or Stretch robot navigation in Habitat‑Sim, in a zero‑shot manner. We also demonstrate that a reward‑based policy can further guide the training of an observation‑based policy in the target environment.
Authors:Morgan Byrd, Robert Wright, Sehoon Ha
Abstract:
Controlling an agent with vision requires being able to separate useful information from irrelevant background information. JEPA‑style latent world models seem like a natural approach for this, as they do not perform pixel‑level reconstruction; however, they are still sensitive to these distractor signals and experience latent collapse. In this work, we introduce Controllability Factorized JEPA (CF‑JEPA), a JEPA‑style world model which splits the latent space into controllable and uncontrollable subspaces. This factorization allows us to capture all the distractor information into the uncontrollable region, while we use the control‑relevant latent information for our task. With this, we show comparable performance across 2D and 3D control tasks under nominal conditions and improved performance under distracted conditions, where CF‑JEPA is the only model that does not experience latent collapse. We also validate our model under distracted conditions for a simulated robot task, highlighting the practical application of such a scheme.
Authors:Xisen Jin, Jingheng Li, Zhenglun Chen, Junyi Du, Xiang Ren
Abstract:
As large language model (LLM) agents become widely adopted, they are increasingly deployed for tasks that require persistent monitoring or recurring actions (e.g., market analysis). These agents are expected to operate unattended for days or weeks, act at the right timing, and adapt to the dynamic environment over time. These challenges are not fully captured in the existing long‑horizon agent work, as they often consider a static environment that is not temporally changing. We introduce ReLiveGym, a diagnostic evaluation environment of long‑lived tasks in which agents act sparsely over simulated weeks of chronologically replayed real‑world news, market, and social‑media streams. The tasks span diverse levels of time sensitivity, reasoning intensity, and recurrence. Across eight base language models, we investigate how model choice and harness design affect agent performance on such long‑lived tasks. Our results show that how agents determine when to act arises as an important harness‑design axis for long‑lived tasks; and that the optimal design varies across tasks and sometimes model choices as well. We also evaluate how continuous learning from hindsight feedback affects performance and addresses failure modes observed in these long‑lived tasks. These findings indicate model choice, action timing mechanism, and use of feedback as important considerations in the design of long‑lived agents. Code: https://github.com/SaharaLabsAI/ReLiveGym
Authors:Zhuo Ning, AmirHossein Naghi Razlighi, Sagi Polaczek, Daniel Cohen-Or, Ali Mahdavi-Amiri
Abstract:
Recent joint audio‑video generative models can synthesize realistic videos with synchronized sound, but typically generate audio as a single mixed track. This limits source‑level control and differs from practical audiovisual workflows, where speech, music, sound effects, and ambient sounds are represented as separate editable tracks. We introduce Soundwich, a training‑free framework that transforms a frozen joint audio‑video flow‑matching model into a generator of multiple synchronized, independently editable audio stems coupled to a shared video. Soundwich generates separate audio stems with explicit control over their temporal activity. To keep separately generated sounds coherent, we introduce a shared scene representation that communicates global audiovisual context across stems while preserving their source‑level separation. We further route cross‑modal interactions between each audio stem and its corresponding visual source, improving audiovisual consistency. The resulting stems remain synchronized with the video and can be independently retimed, muted, replaced, or remixed. Experiments and human evaluations show improved temporal control, source separation, and naturalness, while enabling flexible source‑level editing within coherent audiovisual generation. Code is available at https://github.com/CodyNing/Soundwich.
Authors:Elias Rotondo, Lin Duan, Yanming Xiu, Sangjun Eom, Conrad Li, Maria Gorlatova
Abstract:
Advancements in augmented reality (AR) continue to foster innovative solutions, facilitating novel methodologies within educational systems, healthcare delivery, and risk‑mitigation protocols. However, optimizing for end‑user immersion and comfort remains challenging, as AR head‑mounted displays contend with constrained scene geometry, spatial jitter, and temporal instability. User studies are the standard AR evaluation method for visual quality, but their cost, diminishing scalability, and inflexibility pose bottlenecks during iterative application design. To address this problem, we present an automated framework for AR content evaluation and refinement, built on vision‑language models (VLMs), to evaluate and predict the visual fidelity of AR scenes as perceived by users. First, we introduce RateAR, a benchmark of AR images and videos collected across diverse scenes and environmental conditions, with good‑to‑excellent reliability (ICC(2,5) >= .90) across perceptual factors, including object placement, scale, and shadow consistency. Subsequently, we evaluate eleven commercial VLMs on the crafted benchmark. Results support that VLM‑based quality predictions strongly correlate with human subjective judgments, achieving Spearman's rank‑order correlations of up to 0.8695. An ablation study further suggests that, compared to other prompting strategies, our contextual prompting yields better alignment with human ratings while balancing introduced complexity cues. Building on these findings, we construct an automated AR content adjustment system and conduct a 21‑participant user study. More than 90% of participants found that the system improved placement and size coherence of virtual content.
Authors:Bo Yuan, Wenqian Ye, Zelin Zhao, Lama Moukheiber, Henry Kautz, Aidong Zhang, Yongxin Chen
Abstract:
Evolutionary approaches to LLM‑driven discovery often generate new programs from a small set of selected ancestors. This keeps contexts manageable but can omit useful evidence from other experiments, whereas including the full experimental history produces long, redundant contexts. We introduce a simple, single‑agent discovery harness built around LabBook, an agent‑maintained memory that serves two complementary roles: guiding retrieval of relevant evidence from a complete experimental log and informing the generation of new solutions. At each iteration, the same agent combines its memory with retrieved evidence and jointly produces the next program and an updated LabBook. This separates complete history retention from selective context construction, without requiring an explicit population or branching search structure. On 49 Frontier‑CS problems, LabBook improves the observed quality‑cost trade‑off over the evaluated evolutionary baselines with two backbones, while remaining competitive across nine additional mathematical, systems, and heuristic‑design tasks. Code will be released at https://github.com/BoYuanVisionary/LabBook.
Authors:Vu Dinh Xuan, Duc-Hai Nguyen, Minh-Dung Dao, Vu Quynh Giao, Quang Hong Nguyen, Binh-Son Hua, Barry O'Sullivan, David Murphy, Hoang D. Nguyen
Abstract:
Qualitative comparison figures are central evidence in computer vision papers, and vision‑language models (VLMs) are increasingly used to judge them. Yet existing benchmarks score only scalar quality or overall preference, so a judge can be rewarded for picking the preferred image for the wrong visual reason. We introduce VisionQ, the first benchmark built from peer‑reviewed CV comparison figures that grounds every judgment in a named visual criterion: each question states the criterion, and a judge is credited only when it selects the output the authors identify as best on that criterion. We call this task criterion‑conditioned visual discrimination. VisionQ comprises (1) a corpus of 1,409 CVPR and ICCV papers with 1,800+ validated comparison figures and 3,911 hand‑annotated data points linking method crops to author‑stated visual claims; (2) a six‑axis, 51‑leaf taxonomy of the visual criteria behind qualitative judgment; (3) a criterion‑conditioned evaluation protocol that hides method names, captions, and paper identity and reports accuracy per criterion; and (4) VisionQ‑Judge, a DPO‑tuned Gemma‑4‑E4B judge trained on symmetric evidence pairs, which reduces last‑option predictions by 7.0pp and improves accuracy by 2.5pp on a held‑out test set. Evaluating 20 open‑ and closed‑source VLM judges, we find that the strongest reach only 63.1% accuracy (chance 32.2%) and that reliability varies sharply across criteria. Code: https://github.com/ReML‑AI/visionq. Data: https://huggingface.co/datasets/visionq‑anon‑2026/VisionQ‑1k.
Authors:Nikita Vasiliev, Kirill Borodin, Vasilii Kudryavtsev, Maxim Maslov, Grach Mkrtchian
Abstract:
Long‑form text‑to‑speech must retain requested words over extended generations, yet sentence‑level training and evaluation can hide omissions and early stops. We introduce Balalaika‑Longform, an open Russian corpus of 189 hours in continuous units of 30 seconds to 15 minutes. Long units and matched short windows support fine‑tuning comparisons on the same source recordings, and the accompanying evaluation retains every synthesis attempt. We fine‑tune CosyVoice3, Qwen3‑TTS, VoxCPM2 and F5‑TTS on either view and test continuous synthesis on 50 text‑voice pairs with voices unseen in fine‑tuning, at about 75, 300 and 1,200 words. At 1,200 words, CosyVoice3 WER falls from 99.9% after short‑window fine‑tuning to 47.3% with long targets, and to 16.6% with punctuated, format‑matched transcripts; paired intervals favor long targets for Qwen3‑TTS and VoxCPM2, and by a small margin for F5‑TTS, whose absolute WER stays above 90%. The results isolate the effect of training sequence length under a fixed continuous‑generation protocol.
Authors:Oleg Shchendrigin, Egor Cherepanov, Aleksandr I. Panov, Alexey K. Kovalev
Abstract:
In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next decision. Memory architectures and benchmarks for RL mostly test retention, the ability to keep information unchanged until it is needed. We formalize two further requirements. Rewriting sets the decision‑relevant content to a value independent of the old one, and experience fusion transforms the old content by a rule that a later observation specifies. For tasks built from such updates, we count the memory states that a solution needs, and several baselines reach their lowest success rates on compositions that need more states. We introduce ALER (Adaptive Learnable Experience Rewriting), an agent that pairs an LSTM with a slot memory. An independently addressed Gumbel‑Softmax write that concentrates its weight on one slot overwrites that slot, and a learned gate fuses the retrieved content with the recurrent state before the policy and value heads. We also introduce Rune‑Mazes, three environments in which rune observations invert, cancel, reset, or repeat updates of a hidden cue under vector and pixel observations. Against seven baselines, ALER reaches a success rate of at least 0.82 in all sixteen Endless T‑Maze configurations and at least 0.99 on all five Rune T‑Maze compositions, and it has the highest mean success rate on four‑branch Rune Multi‑Corridor with an Invert rune. On pixel‑based Rune MiniGrid Memory, it has a higher mean success rate than PPO‑LSTM in eight of ten configurations. Project page: https://quartz‑admirer.github.io/ALER‑Adaptive‑Learnable‑Experience‑Rewriting/.
Authors:Sahan Paliskara, Nattaput Namchittai, Andrew Lampinen
Abstract:
People are increasingly delegating tasks to AI agents, and those agents are increasingly encountering other people's agents over shared resources such as a codebase, a calendar, or a budget. When each agent acts for a different user with different goals, coordination often fails, and the group ends up worse off than if a single agent had acted for everyone. We study this multi‑user, multi‑agent setting across five frontier models and 77 scenarios in four environments: an API key environment in which agents share a compute budget, a clinic in which they share a calendar, a personal assistant environment in which they share a group order or booking, and a merge queue in which they share a release cutoff. In each scenario, we compare a single agent that serves every user (a coordinator) to a team in which each agent serves one user, with and without a communication channel between the agents. Teams deliver worse group outcomes than the coordinator in every environment: without a channel, they completely collapse in two environments, and even with one, coordination overhead creates substantial gaps. For example, in the personal assistant environment, the coordinator fulfills a targeted user request about twice as often as teams. We identify distinct behaviors associated with this poor group‑level performance, including stalling as teams grow, overriding each other's actions, and fabricating claims. We find effective but environment‑specific mitigations, such as a team lead, explicit procedural instructions, and a platform check that makes an agent read its peers' messages before committing. We will release the API key, clinic, and personal assistant environments as MAMUBench, comprising 74 scenarios for evaluating multi‑user, multi‑agent coordination.
Authors:Ziqing Zhang, Xiao Liu, Kai Liu, Jianze Li, Weihang Zhang, Linghe Kong, Yulun Zhang
Abstract:
Aesthetic image cropping aims to identify the optimal crop of an image in terms of aesthetics and composition. While supervision based on annotated data is fundamental, the field has been hindered by a long‑standing problem: existing datasets suffer from (1) human subjectivity and (2) rigid discreteness confined to fixed sampling grids. These flawed annotations not only limit the accuracy and generalization of trained models but also severely distort fair evaluation. To overcome this, we propose to model human cropping preference as a multi‑peaked, continuous, and sharp field over the crop space. We introduce the Continuous Preference Field (CPF), which recovers a dense preference landscape from discrete annotations through (1) peak clustering, (2) off‑lattice refinement, (3) negative shaping, and (4) field assembly. Based on this, we train CPIC, a VLM‑based cropping model optimized via GRPO with the CPF reward, which overcomes template collapse, achieving state‑of‑the‑art performance and exceptional out‑of‑domain generalization. Finally, to resolve the long‑standing benchmark evaluation crisis, we introduce CPICD, a comprehensive recalibration of existing ground‑truth boxes. By leveraging the CPF to correct grid‑bound artifacts across mainstream benchmarks, CPICD establishes a rigorous and reliable foundation for future cropping research. Extensive experiments and user studies demonstrate the superiority of our CPF, CPIC, and CPICD. Code, model, and data are available at https://github.com/zzqingz/CPIC.
Authors:Zequn Yang, Yu Miao, Haotian Ni, Ziheng Chen, Chengxiang Huang, Dongzhan Zhou, Kai Chen, Qi Zhang, Ji-Rong Wen, Yake Wei, Di Hu
Abstract:
In this paper, we propose Gestalt, a new paradigm of large multimodal model built around multimodal interplay. Despite rapid advances, large multimodal models are reaching a bottleneck: existing approaches focus primarily on accommodating additional modalities while overlooking the distinct characteristics of each modality and the relations among them. Motivated by the multistage property of human multisensory perception, we propose a multimodal interplay pyramid that organizes multimodal modeling as a progression from modality‑specific processing, through cross‑modal alignment, to deeper multimodal integration. Guided by this pyramid, Gestalt adopts a unified discrete diffusion framework and an interplay‑partitioned architecture, with learnable interplay tokens mediating cross‑modal exchange and integration. The pyramid also structures its data organization and training strategy. Strong performance across image generation, multimodal understanding, and text‑only evaluation shows that Gestalt significantly improves cross‑modal integration while preserving modality‑specific information, effectively harnessing the strengths of diffusion‑based multimodal models and offering a promising path toward unified multimodal intelligence. Project page: https://GeWu‑Lab.github.io/Gestalt.
Authors:Tong Zheng, Skylar Zhai, Zhan Cheng, TianMing Sha, Youling Huang, Shuo Zhou, Shaotong Qi, Jingcheng Liang, Xuwei Ding, Pengcheng Xu
Abstract:
Multi‑reward reinforcement learning trains large language models to satisfy multiple behavioral objectives simultaneously. Reward‑wise normalization, as used in GDPO, preserves reward‑specific relative information within rollout groups, but different objectives can still exhibit uneven learning progress. We study this behavior through advantage energy, the sum of a reward's squared advantages over a batch. Under idealized GDPO normalization, we show that this energy is proportional to active‑group density: the fraction of rollout groups in which the reward provides nonzero relative advantages. This reveals a residual batch‑level signal imbalance and provides a basis for calibrating reward contributions. Based on this relation, we propose Density‑Aware Reward Aggregation (DARA). We derive an inverse‑square‑root density correction that gives greater weight to signals from less frequently active rewards. DARA computes its weights from each rollout batch, adapting to changes in reward activity throughout training without modifying the underlying policy optimization objective. Experiments on tool calling and mathematical reasoning show that DARA learns the targeted behaviors faster than GDPO, reaching high format compliance in up to 26% fewer training steps on tool calling and near‑saturated length compliance in up to 65% fewer steps on mathematical reasoning, while remaining competitive in final performance. Our code is available at https://github.com/zhaihaotian/DARA.
Authors:Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu
Abstract:
Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits requires training samples that capture both visits, often spanning minutes. Yet dense attention over the full span incurs quadratic costs, making long‑span supervision expensive. Memorizon breaks this coupling: long spans are needed for supervision, but not for attention, since the two visits can share a forward pass without including every intervening frame. A training sample covers a span of any length but is scored only on its last k chunks. Instead of tokenizing the history before them, each scored chunk retrieves its own top‑K latents by camera co‑visibility, and the union of these requests forms a shared bank. The bank is bounded by kK, so the sequence stays bounded however long the span; at the shortest span the recipe is exactly conventional training. Adding the bank raises the cost of a step once; beyond that, a longer span costs little, and going from 100 to 400 s adds 12% to the step time. Against a sliding‑window baseline, retrieval raises revisit consistency on every split, and a span long enough to reach the first visit of each return adds a further 24% to 30%, at some cost in image quality; beyond that span, more length no longer helps. Filling the bank from another episode lowers revisit correlation by 83%, so the model uses what it retrieves. Project page: https://tingtingliao.github.io/memorizon
Authors:Yerim Oh, Young-Jun Lee, Jaewoo Ahn, Gunhee Kim, Dongyeop Kang
Abstract:
AI‑generated content, often called AI slop, is increasingly common everywhere, particularly in academia. Slop in AI‑generated scientific papers, however, has more complex patterns that cannot be easily detected by existing token‑based AI detectors. Each part of such a paper looks plausible while the scientific reasoning that connects the parts breaks down, which can mislead how readers assess the work. We benchmark these failures as scientific slop through six measures across Structure, Argument, and Artifacts. We construct SciSlopBench with 390 AI‑generated papers, mostly in computer science but spanning the life, social, and natural sciences, each paired with a human‑written paper matched by research problem and contribution type. Our measures identify the AI paper in each pair with 85.9% accuracy, compared with 68.7% for Binoculars. Higher scientific slop accompanies lower ICLR ratings and distinguishes rejected from accepted papers above chance in every year from 2017 to 2025. Reducing these patterns, however, is not as simple as directly optimizing the measures. We therefore propose SciSlopHarness, a harness‑level framework that guides a fixed LLM to revise slop only where the experiment records support the change. While standard revisions leave residual slop and direct slop‑aware prompting triggers reward hacking, SciSlopHarness reduces the remaining AI‑human gap by 63% over the strongest revision baseline without requiring human reference targets. Overall, we demonstrate that AI‑generated scientific papers leave fundamental traces in their global reasoning, and that responsible mitigation demands strict evidentiary grounding rather than mere prose refinement.
Authors:Chengxi Yang, Tej Chajed, Thomas Reps
Abstract:
Modern incremental computation theories like DBSP have enabled efficient incrementalization of general recursive computations. To do so, they require a runtime Fixpoint Detection (FPD) mechanism to detect whether an iterative computation has reached the fixpoint and thus should terminate. However, we show that the commonly suggested "FirstZero" strategy is unsound even in naturally arising cases, and that exact FPD is impossible for arbitrary DBSP circuits with expressive primitive nodes. This issue reveals a fundamental gap between the mathematical specification and implementations of such theories. To fill this gap, using DBSP as a core calculus, we develop a formal theory of convergence detection. Within this theory, we define internal convergence (IntConv) as a declarative criterion corresponding to the internal‑state‑stability strategy used by practical implementations, and prove that IntConv is a sufficient condition for external convergence. We then define the state fixpoint (StFP) predicate and a sound and complete StFP detector. Combining the StFP detector with fixed‑input and zero‑output checks yields a sound and complete IntConv detector. Moreover, for a large class of useful circuits (programs) including Datalog queries, nested while queries, and their incrementally optimized versions, we show that IntConv is not only sound but also complete (meaning any convergence in the theory implies the convergence in our criterion). As a result, our theory provides semantic guarantees for convergence detection on all these circuits. Our results are formally verified in Lean, with the formalization available at https://github.com/Arcadia‑Y/fixing‑the‑fixpoint/.
Authors:Taegeun Yang, Youngju Na, Yoonki Cho, Sung-Eui Yoon
Abstract:
Vision‑language‑action (VLA) models often struggle to generalize to skill combinations absent from their fine‑tuning demonstrations, even when every constituent skill has been demonstrated. We focus on a vision shortcut as one failure mode: during fine‑tuning, visual observations can serve as a proxy for the instruction, so a policy may execute a demonstrated combination associated with similar observations rather than the instructed combination. This motivates training with counterfactual pairs formed by holding a demonstration observation fixed while changing the instruction to specify an undemonstrated combination. These pairs, however, lack corresponding demonstrated action targets. Crucially, the currently required skill has already been demonstrated, but actions from those executions cannot serve as direct targets because the same skill can require different actions across observations. We propose CRAFT, which transfers supervision from demonstrated executions of the required skill to counterfactual pairs using skill representations that can be reused across executions of the same skill. Across three VLA models and two simulation benchmarks, CRAFT improves success on undemonstrated combinations while maintaining high success on demonstrated ones; it also improves compositional generalization on a real robot. Project website: https://taegeunyang.github.io/craft/
Authors:Liang You, Hengyu Shi, Dongwen Ou
Abstract:
Conformal prediction guarantees marginal coverage, but a single calibration threshold can still spread that coverage unevenly, over‑covering easy regions and under‑covering hard ones. SimplexUQ is, to our knowledge, the first benchmark and reproducible protocol for measuring this allocation problem on simplex‑valued predictions; it compares existing conformal wrappers rather than proposing a new one. Its task suite, SimplexTasks‑12, combines six controlled synthetic regimes with six frozen‑predictor real tasks spanning class probabilities, topic mixtures, spectral abundances, cell‑type fractions, age distributions, and emotion mixtures. Each comparison fixes the predictor, score, and response‑free stratification map, varies only the wrapper, and reports marginal coverage, worst‑stratum coverage, max disparity, and within‑task radius and compute. Global calibration can look valid while failing badly: on CIFAR‑10 it attains 0.900 marginal coverage but only 0.542 in the worst entropy stratum, and Mondrian calibration raises that stratum to 0.886 while reducing max disparity from 0.358 to 0.022. No wrapper dominates, however. Under smooth synthetic heterogeneity, several repairs are competitive; fixed‑map analyses show that rankings depend on the evaluation groups and protocol; and in a 12‑task comparison, Mondrian has lower disparity on its single target partition for all 12 tasks, whereas BatchMVP has lower disparity over overlapping groups on five. These are empirical comparisons, not new coverage guarantees. A controlled predictor‑bias sweep shows that removing predictor bias only partly reduces global‑threshold disparity. We release task cards, result provenance, permitted derived arrays, and rebuild instructions, and treat wrapper selection as a diagnostic comparison rather than a universal ranking.
Authors:Gadiel Sznaier Camps, Chengyang He, Guillaume Sartoretti, Eduardo Montijano, Mac Schwager
Abstract:
Autonomously constructing physically realizable 3D structures remains a significant challenge due to combinatorial action spaces, interchangeable components, equifinal assembly sequences, and strict stability requirements during construction. State‑of‑the‑art methods fine‑tune large language models for text‑based generative construction. However, these approaches do not allow for Multimodal (text, image, sketch) conditioning, overlook the practical role of scaffolding for stabilizing intermediate structures, and suffer from slow inference speeds. Therefore, we formulate construction as a probabilistic next‑block generation task with multiple potential assembly actions and multiple potential task conditioning modalities. Concurrently, we explicitly consider the utility of scaffolding by introducing an auxiliary scaffold block token. We present Scaffold Multimodal Monte Carlo (ScaffoldM3C), a multimodal, lightweight, auto‑regressive model for stable block‑based construction, that proposes a set of next‑step candidate blocks. Leveraging these candidates, we utilize Sequential Monte Carlo (SMC) to maintain a population of possible assembly sequences, allowing us to consider multiple, potentially different, assembly directions simultaneously. We train our multimodal architecture by extending the StableText2Brick dataset to contain image conditioning prompts and scaffold‑stabilized build sequences. ScaffoldM3C is 4x smaller than competing baselines, yielding a 5x to 20x speedup during inference, while achieving comparable construction quality to state‑of‑the‑art methods and higher overall stability. We demonstrate the effectiveness of our approach through simulations and real‑world robot assembly demonstrations.
Authors:Rikhat Akizhanov, Yangsong Zhang, Nikolai Kaliazin, Peter Wolf, Yoshihiko Nakamura, Pascal Fua, Fabio Pizzati, Ivan Laptev
Abstract:
Human motion, environmental contacts, and interaction forces are governed by common physical laws, yet existing approaches typically separate visual pose reconstruction from contact and force estimation. This separation limits joint reasoning and can propagate errors between stages. We introduce PACT, an end‑to‑end model that jointly learns to estimate human pose, contacts and contact forces from monocular video. Our approach augments a human reconstruction foundation model with learnable contact‑force tokens and a temporal transformer that integrates visual features with world‑space motion. Joint prediction heads refine human poses and estimate contacts and forces, while physics‑based supervision encourages consistency between the reconstructed motion and interaction forces. To address the scarcity of force annotations, we develop a data annotation pipeline that combines contact labeling with physics‑based motion and force optimization, producing training supervision from synthetic and real‑world videos. We also introduce a real‑world climbing benchmark ForceWall with climbing videos and corresponding ground‑truth contact forces obtained from the force sensors. Experiments demonstrate state‑of‑the‑art contact and force estimation, outperforming staged reconstruction approaches and generalizing to interactions beyond the training distribution. These results support end‑to‑end joint learning as an effective approach to recovering human motion and physical interactions from video.
Authors:Haokai Ma, Chieh Lin, Yupeng Qiu, Ee-Chien Chang
Abstract:
Computer‑use agents increasingly operate as long‑running assistants through persistent workspace memory, which OpenClaw‑style CUAs realize as automatically reloaded files that hold user instructions, system summaries, and external claims at the same privilege level. Here, remembering a claim confers authority over later behavior. This enables a persistent memory attack, in which an attacker who controls only benign‑looking external content induces the CUA to record attacker‑favored claims during a legitimate task, and those claims later govern benign tasks the attacker never touches. The attack chain extends "malicious context ‑> malicious response" into "malicious context ‑> memory injection ‑> malicious execution", making this a cross‑environment threat. Existing defenses studied intervene either before content enters memory or at the action it later induces, not whether stored content may guide action. We propose ZoneClaw, which separates persistence from authority by replacing flat workspace memory with hierarchical trust zones carrying explicit authority levels. External claims persist in a low‑trust zone and acquire action‑guiding authority only by crossing an explicit authority boundary, at which promotion is cross‑checked against zones the attacker cannot directly write. Role‑specific processes of asymmetric privilege enforce this boundary, ensuring that no process both ingests external content and acts outward. Across four attack scenarios, two injection settings, and four backbones, ZoneClaw drives ASR from 372/480 to 6/480 while retaining utility in 458/480 trials, and remains effective against some defense‑aware attackers. Attacker claims still persist in low‑trust memory yet rarely cross the authority boundary, showing that ZoneClaw withholds authority rather than refusing to learn from the environment. Our code is available at: https://github.com/euph00/ZoneClaw‑code.
Authors:Zahra Khodagholi, Niloofar Yousefi
Abstract:
Pairwise guide‑‑transcript scores do not enforce conservation of a finite guide‑loaded RISC pool when they are interpreted independently as occupancies. We formulate a differentiable scalar equilibrium layer: one conservation equation with a unique positive root and exact implicit gradients. It yields a redistribution theorem, a qualified high‑resource limit, an analysis of the retrieval approximation, and a conditional rank‑invariance result: within one construct at one dose, rankings by fractional occupancy cannot distinguish equilibrium from independent scoring. We therefore audit the two experiments that proposition leaves open, dose and cross‑context, on archival off‑target data. Corrected thermodynamic affinities associate weakly with measured repression in the direction a working predictor requires, but a paired permutation test and a construct‑cluster bootstrap do not establish added predictive value from the coupling: what survives their differing permutation‑null baselines is \GapNet, a descriptive \GapNetOverSE of the equilibrium association's cluster standard error. The dose fits are heterogeneous and frequently violate the model‑implied exponent constraint, which is superlinear rather than sublinear, so these data do not identify the competition parameter. A saturable compression of the competitor set holds both accuracy targets on held‑out guide families but is not faster at the size measured. The contribution is a reusable conservation operator and the experimental information needed to test it. The code for this study is available at https://github.com/shadi97kh/One‑Pool‑Many‑Targets.
Authors:Pietro Moriello, Pietro Buzzega, Angelo Porrello, Simone Calderara
Abstract:
We introduce IrekoGPT, a post‑hoc method for converting pretrained LLMs into slimmable models whose width can be adjusted at inference time. Building on SliceGPT, we retain its projection matrices without pruning them, allowing a single model to expose nested subnetworks at different widths. We improve robustness by calibrating each layer across multiple compression ratios, and correct downstream linear layers through gradient‑free ridge regression. Across Llama and Qwen models, preliminary results show improvements over naive PCA‑based slimming, with the largest gains at high compression. Code is available at https://github.com/aimagelab/IrekoGPT
Authors:Maxim Trokunov, Kirill Borodin, Nikita Vasiliev, Grach Mkrtchian
Abstract:
Human mean opinion scores (MOS) are costly to collect, and non‑intrusive MOS predictors degrade sharply outside their training domain. ProxyMOS turns a pool of public MOS predictors into a single stronger model without new human labels. Eight predictors are benchmarked against human ratings; the five most informative enter a subset search under uniform, correlation‑weighted, error‑weighted, MSE‑optimised and adaptive per‑utterance routing; and the best routed four‑model ensemble labels 807k unlabeled utterances that train a wav2vec 2.0 student. On URGENT the student reaches Spearman ρ=0.802 against 0.773 for the best teacher. On mos260, a new Russian TTS benchmark of 4,600 utterances from 38 synthesis conditions, it reaches ρ=0.636 against 0.613 per utterance and 0.95 per condition, matching its own routed ensemble in one forward pass. Adaptive routing is the only rule that does not degrade when weak predictors are added. Model, ONNX exports and mos260 are released. It's about 950 characters; arXiv's limit is 1,920. I kept ρ because arXiv renders it on the abstract page. If you'd rather avoid math, replace ρ=0.802 with rho = 0.802 and do the same for the other ... values.
Authors:Timothée Lesort, Alejandra López de Aberasturi Gómez, Tristan Karch, Tom Veniat, Philippe Modard, Karl Tuyls, Ludovic Denoyer
Abstract:
Evaluating large language models becomes increasingly challenging as their capabilities advance: benchmarks can saturate, public test sets risk contamination, and assessing harder tasks can require expensive grading or execution infrastructure. These challenges are amplified in automatic prompt optimization (APO), where evaluation is repeated throughout the search for better prompts. Studying APO therefore requires a benchmark that is cheap and deterministic to score, hard enough to leave room for improvement, and renewable as models evolve. We introduce a chess benchmark built from 1,118 Lichess puzzles to study APO for frozen LLMs: we optimize their prompts without updating their model weights. Chess combines inexpensive exact‑match scoring, engine‑based evaluation of alternative moves, and a renewable supply of problems with adjustable difficulty. Unlike evaluations that report only success on isolated test items, the benchmark also connects puzzle‑solving gains to short game‑play rollouts within the same domain. We use it to evaluate six APO algorithms on eight target models, measuring not only baseline strength but also how much each model responds to optimization and whether optimized prompts transfer across models and to game play. Chess is thus a well‑suited benchmark for APO: it is (i) challenging, as even the strongest evaluated model, Gemini 3.5 Flash (used as the meta‑model), solves only about 55% of puzzles; (ii) discriminative, revealing gains, unchanged performance, and regressions across methods and models; (iii) renewable, with fresh puzzles to reduce contamination risk and adjustable difficulty to maintain headroom as models improve; and (iv) affordable, as the complete study runs for around \800. We release the puzzles, optimization and evaluation code, and dataset‑renewal scripts (https://github.com/imec‑ailabs/Automatic‑Prompt‑Optimization‑with‑Chess).
Authors:Jiawei Lin, Saibo Geng, Thomas Bourgeat
Abstract:
KV cache pruning reduces long‑context inference memory usage by evicting less important key‑value pairs. KVzip estimates importance through context reconstruction: prompting a model to repeat the context chunk by chunk. This achieves strong compression quality at the cost of additional forward passes. Learned approximations reduce this cost but require model‑specific training. We analyze how KVzip identifies important cached information and show how to approximate its reconstruction scores using information already computed during prefill. These findings motivate EchoPress, a training‑free method that approximates reconstruction attention using queries and keys from standard prefill. For each request, it reconstructs only the first chunk to calibrate importance scores for the remaining context. Experiments on LongBench and RULER with Qwen3‑8B and Llama‑3.1‑8B‑Instruct show that EchoPress matches KVzip in task accuracy across eviction ratios from 50% to 90%, while reducing compression overhead by a factor of 1.7‑19.6 and total prefill time by a factor of up to 2.9. Code is available at https://github.com/ljwljwljwljw/kvpress/tree/echo‑press.
Authors:Felix Oury, Harley Day, Karina Mayoral, Ville-Pekka Seppä, Sejal Saglani, Reiko J. Tanaka
Abstract:
Overnight impedance pneumography (IP) is used to monitor paediatric respiratory health. Its current clinical readout, the Expiratory Variability Index (EVI), compresses each IP recording into a single scalar and achieves an AUC of 0.633 for night‑level wheeze classification. We introduce Wheeze Impedance Pneumography Scalogram Network (WIPSNet), a 3D ResNet operating on stacked continuous wavelet transform scalograms of overnight IP signals. On a 15‑patient cohort (60 nights, 281 hours), WIPSNet achieves an AUC of 0.783 \pm 0.026, outperforming EVI, a state‑space model (Mamba), and two modern sleep‑staging architectures. Performance peaks at a volumetric depth corresponding to 32 minutes of temporal context, suggesting that multi‑scale temporal aggregation is important for modelling nocturnal respiratory dynamics. Overall, these results indicate that structured time‑frequency representations combined with 3D convolutional architectures provide an effective approach for learning from long, irregular physiological time series.
Authors:Luran Wang, Linrui Ma, Hannes Stärk, Regina Barzilay
Abstract:
Inference‑time steering enables pretrained diffusion models to satisfy new constraints without full retraining. However, specificity‑aware generation is difficult: repelling samples from a negative reference distribution can also erode the positive distribution where the two overlap. The key challenge is to suppress negative mass while minimally distorting the positive distribution. We address this problem by formulating specificity‑aware steering as a target‑design problem and deriving a target distribution from an overlap‑based objective. The resulting target keeps the desired reference distribution only in regions where it is sufficiently preferred over the undesired reference distribution, giving a likelihood‑ratio interpretation of specificity. To sample from the corresponding time‑dependent target path, we develop a Sequential Monte Carlo sampler with a variance‑minimized local proposal. We further introduce a practical fixed‑noise optimization procedure with the Jacobian‑‑vector products with the desired and undesired score fields. Experiments on synthetic task, class‑contrastive generation, text‑to‑image tasks and peptide‑MHC (p‑MHC) binder show that the proposed method suppresses undesired regions more effectively, reduces mode shift, and improves sampling stability by decreasing the SMC weight collapse compared with negative‑guidance baselines. Code is available at: https://github.com/WangLuran/Specificity‑Aware‑Diffusion‑Steering
Authors:Shanbin Yu, Shaoyang Guo, Haoran Zhao, Danni Yu, Ziming Liu
Abstract:
This paper studies forking, a generalization failure discovered in NanoGPT autoresearch. Under data replay, models with an over‑encoding n‑gram memory branch show a sharp separation of training and validation loss at epoch boundaries, resembling the shape of forks. We study this phenomenon in a controlled vanilla NanoGPT setting and reproduce it in a DeepSeek‑style model with Engram. Mechanistically, repeated updates sharpen the continuations observed in training while suppressing the probability of unseen continuations, whose loss grows with each pass. The n‑gram module creates weakly interacting context‑specific subspaces, amplifying this effect. Low‑frequency contexts contribute most of the gap, whereas larger training budgets and heavily crowded tables suppress it. We also observe forking in short‑budget, heavily repeated SFT and RL‑like regimes. The contributions of this paper are twofold: (1) Forking reveals yet another curious phenomenon in deep learning, in addition to grokking and double descent. (2) Forking is an unexpected and unpleasant by‑product of tricks proposed by autoresearch agents. While these agents produce an enormous number of results that seem useful, we should always be careful with their results.
Authors:Fabio Mattiussi
Abstract:
The Natural Image Quality Evaluator (NIQE) scores an image by its statistical distance from a model fitted on pristine images, and its distributed model is fitted on photographs. We release the Radiology Image Quality Evaluator (RIQE), a NIQE‑style model fitted on 3,792 full‑dose slices from 158 patients of the public LDCT‑and‑Projection‑data collection, with a declared intensity mapping, a manifest of every slice and a script that reproduces the fit. On 40 held‑out patients, RIQE ranks reduced‑dose reconstructions, simulated by projection‑domain noise insertion, worse than the full‑dose reconstruction of the same slice in 240 of 240 chest and 230 of 240 abdominal pairs, and ranks images with 20% more noise worse than their source in 97.5‑100% of cases. Its preferences among filtered images, however, do not follow lesion signal. With a 4 mm, +10 HU lesion inserted in noisy abdominal slices, RIQE prefers bilateral filtering to the unfiltered image in every image up to a 32 HU residual, at which 29% of the lesion's matched‑filter signal remains and its detectability index falls from 0.51 to 0.33; it never prefers Gaussian smoothing, which at the same 32 HU residual leaves 70% of the signal and a detectability index of 0.48. Fitted on photographs with the parameters published for NIQE, the same code ranks every simulated reduced‑dose abdominal image better than its full‑dose counterpart. RIQE is suited to ranking a degraded image against its source; under the conditions tested it should not be the sole criterion for selecting, comparing or tuning denoisers.
Authors:Rongwen Li, Haixin Xie, Mingyang Wang, Hongwu Liu, Kun Fang, Changjian Chen, Zhuo Tang, Kenli Li
Abstract:
Station‑based weather forecasting supports daily life and economic activity, yet accurate forecasts require modeling complex spatial dependencies among stations. Recent clustering‑based selective modeling offers a promising alternative to dense inter‑station interactions. However, a grouping shared across an observation window may obscure local changes in station relationships, while intra‑cluster interactions alone may miss important global context. The theoretical advantages of selective interactions over dense connectivity also remain insufficiently understood. We therefore propose STCFormer, an adaptive spatio‑temporal Transformer that dynamically groups stations according to their local evolution within each temporal patch. Its Cluster‑Guided Attention Block combines fine‑grained local attention within clusters and global attention over regional state summaries, allowing each station to access information beyond its own cluster. We further show that a derived Lipschitz upper bound for cluster‑conditioned local attention is no larger than its fully connected counterpart, explaining a potential robustness benefit and motivating the design of InfoLoss. Experiments on three real‑world weather datasets spanning eight temperature and wind forecasting tasks show that STCFormer achieves the lowest 24‑hour mean squared error on all eight tasks and ranks first or second in 47 of 48 comparisons across metrics and forecasting horizons. Ablations and case studies further confirm the benefits of locally adaptive grouping and complementary local‑global interactions. Our code can be obtained at https://github.com/hnu‑vis/STCFormer.
Authors:Yunbei Zhang, Saiyue Lyu, Janet Wang, Yingqiang Ge, Jiang Guo, Jihun Hamm, Chandan K Reddy
Abstract:
Multi‑agent systems derive their capabilities from sharing evidence, delegating tasks, and combining information across agents. The same process creates a safety problem: contributions that are admissible in isolation can jointly enable a prohibited use. Blocking every sensitive action avoids disclosure but defeats the purpose of collaboration. We introduce authorization‑paired evaluation, which makes blocking prohibited uses and completing required authorized uses a joint success criterion, and FlowReview, a framework connecting object resolution, permission ranking, and deterministic enforcement. In controlled composition experiments, reviewing combined artifacts reduces the denied‑commit rate from 86.0% to zero with no loss of authorized supply. Our findings show that preserving information and lineage alone does not ensure correct permission attribution. Object identity and permission must remain connected to execution through components whose outputs can be verified. Together, these findings establish a system‑level requirement for multi‑agent safety: govern composed information flows while preserving the authorized capabilities that make collaboration useful.
Authors:Rongwen Li, Xiao Wang, Mingyang Wang, Hongwu Liu, Changjian Chen, Zhuo Tang, Kenli Li
Abstract:
Station weather forecasting is fundamentally shaped by both complex spatial dependencies across stations and strong physical coupling among weather variables. However, existing studies often consider these relationships separately and use different datasets and experimental settings, hindering systematic assessment of their individual and joint contributions. In this paper, we introduce M^2Weather, a benchmark for joint multi‑station and multi‑variable weather forecasting. Through multi‑criteria quality control and station stratification, we collect 2,809 high‑quality stations with 5 physically coupled weather variables across three spatial scales: France, Europe, and Global. This multi‑scale design lets us examine whether conclusions persist from national to global station networks. We also introduce unified training and evaluation protocols to enable fair comparison of different station‑variable modeling paradigms. To further examine the benefits of modeling station‑variable relationships, we design a lightweight, plug‑and‑play adapter. With a trained weather forecasting model, this adapter can introduce missing station or variable relationships without retraining the model. This enables fair and efficient investigation of station‑variable relationships. Systematic evaluation of 16 representative models shows the benefits of jointly modeling station and variable relationships. Completing missing relationships further reduces MSE for all adapted models on all three datasets. Together, these results identify the complementary information across stations and variables as an important resource for improving station weather forecasting. Our code can be obtained at https://github.com/hnu‑vis/M2‑Weather.
Authors:Haoyu Wang, Siyuan Qian, Yanjun Li, Zeyu Zhang, Yandong Guo, Boxin Shi, Hao Tang
Abstract:
Reinforcement learning (RL) for dexterous manipulation must discover finger‑object contacts and then control the object precisely; the action noise that serves the first goal can interfere with the second. In trajectory‑guided settings such as ViViDex, where RL refine hand‑object trajectories from human video, our baseline PPO runs end near their initial action noise after 5M steps, motivating explicit control of exploration scale. DexPolicy makes that scale an explicit function of training steps, annealing from broad to narrow exploration while holding loss, architecture, reward, and optimizer settings fixed. We study three policy‑optimization settings: PPO, critic‑free GRPO continuation, and a flow‑parameterized PPO variant (FPO). Across five YCB objects and three training seeds, mean deterministic Target success rises from 49.4% to 68.1% (FPO), 14.1% to 45.4% (GRPO), and 32.0% to 35.7% (PPO). On a RealMan RM75 arm with an Inspire/RH56 hand, 360 trials over three objects raise mean Target success from 25.0% to 85.0% (FPO), 10.0% to 63.3% (GRPO), and 8.3% to 43.3% (PPO), with one trained model per object‑method condition. PPO component screening favors noise control over the tested optimizer contraction; the selected PPO schedule yields higher mean Target success than linear decay with the same endpoints on three tested objects. Training return, deterministic Target success, and tolerance to execution noise dissociate; schedules should therefore be judged by terminal task success under the intended execution conditions, per task and policy‑optimization setting. Code: https://github.com/AIGeeksGroup/DexPolicy. Website: https://aigeeksgroup.github.io/DexPolicy.
Authors:Bravish Ghosh
Abstract:
AI agents that control wallets read attacker‑reachable content, so they can be steered into proposing harmful transactions. The usual last line of defense is a pre‑signing check: a static allowlist, an LLM reviewer, or a transaction simulation. All three share a gap: the check describes the chain state at check time, but the transaction executes in a later state that an adversary can shape through front‑running, contract upgrades or token‑parameter changes. We call this state drift. We present Proof‑Gated Signing (PGS), which simulates a proposed transaction, extracts its effects, and uses an SMT solver to check a declarative value‑and‑permission policy for every price in an oracle‑uncertainty band. It then compiles on‑chain post‑conditions (wallet balance bounds, payee receipts, allowance caps and ownership) and proves that every execution satisfying them also satisfies the policy. The agent's smart‑contract wallet enforces them atomically, so the guarantee applies to the executed transaction under arbitrary drift. On an open testbed of 260 scenarios (14 attack families including five drift and two adaptive families, and 12 benign families), with harm measured from attacker balances rather than from any policy, PGS prevented 93.6% of the 140 harmful scenarios and passed 97.5% of the benign ones. Simulation‑only checking prevented 57.9% and a static allowlist 71.4%. None of the 50 drift scenarios produced attacker gain under PGS. The only unprevented family, an in‑policy drain, was bounded by the per‑session budget. We also find that giving an LLM reviewer a clean pre‑drift simulation made it more likely to approve a drift attack. Overhead is about 41k gas and 0.1‑0.2 s per check.
Authors:Zhengkai Tu, Mingda Zhang, Zijia Wang, Xiaoying Tang, Jimmy Huang
Abstract:
A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference‑anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real‑world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer‑written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open‑weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open‑weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.
Authors:Bingjun Luo, Jialin Guo, Tony Wang, Siqi Li
Abstract:
As Large Multimodal Models (LMMs) transition toward natively unified architectures, evaluating their safety in synergistic harmful image‑text generation tasks becomes a critical challenge. Unlike unimodal threats, synergistic risks emerge when text and image modalities are coordinated to produce harm that significantly exceeds their individual components. We introduce UnifiedAttack, a novel benchmark designed to evaluate LMM safety in collaborative scenarios by focusing on the harmfulness gain achieved through cross‑modal synergy. The benchmark incorporates samples filtered for their multimodal potential alongside a novel subset of synthesized disinformation queries. To verify identified vulnerabilities, we propose a synergistic hijacking framework featuring In‑Context Reskinning (ICR) and Cognitive Planning Injection (CPI). ICR utilizes few‑shot learning to wrap adversarial intent in benign virtual shells to desensitize safety filters, while CPI hijacks the reasoning path by enforcing a plan‑then‑execute paradigm. By compelling the system to commit to a neutral logical plan, we exploit its internal drive for consistency to induce the synchronized generation of harmful multimodal content. Extensive evaluations on state‑of‑the‑art architectures demonstrate that UnifiedAttack consistently bypasses modern alignment. Our findings reveal that the structural helpfulness and logical coherence of unified models can be systematically weaponized, highlighting the urgent need for logic‑aware defenses in synergistic generation tasks. Code is available at https://github.com/bingjunluo/UnifiedAttack .
Authors:Jungseob Lee, Sugyeong Eo
Abstract:
Speculative decoding accelerates large language model inference by drafting future tokens cheaply and verifying them with the target model in parallel. Block drafters score a whole block of future tokens in one forward pass, yet standard decoding verifies only the top‑scoring chain and discards the other candidates. Because these candidates are already scored, verifying more of them adds target computation but no extra drafting. We introduce CAST (Cost‑Aware Speculative Trees), which packs these candidates into a tree and verifies it in a single target pass, leaving the target model, drafter weights, and decoding rule untouched. To decide how wide the tree should be, CAST adds candidates while the expected gain from the next one outweighs the verification time it adds. The width therefore adapts to each deployment from a latency measurement, without sweeping over widths. We evaluate CAST across five domains on three GPU generations and two model families. At its predicted width, CAST is faster than the standard chain in all eight settings, by up to 43%. We also find that the best width depends strongly on the deployment. Where verification cost jumps at a kernel boundary, a 128‑token tree is only 2% faster than the standard chain, whereas the tree at the predicted width is 20% faster. Furthermore, we prove that CAST leaves the target output distribution unchanged under both greedy and sampled decoding. Code is available at https://github.com/js‑lee‑AI/CAST.
Authors:Jungseob Lee, Dongyub Jude Lee, Sugyeong Eo, Seongtae Hong, Seungyoon Lee, Heuiseok Lim
Abstract:
Fine‑tuning adapts aligned large language models (LLMs) to downstream tasks, but a few dozen harmful examples can remove their refusal of harmful requests. Prior work localizes safety‑related behavior to specific layers, directions, and tokens, suggesting targets for protection. We test whether successful localization and recovery support defenses that survive changes in the attack. Across six checkpoints from four model families, harmful and benign prompts remain linearly separable after attack, and patching full clean hidden states into the compromised model restores refusal at a reproducible transition depth. Building on a prior layer‑freezing defense, we freeze every layer up to this depth and repeat the attack. At a hundred harmful examples, refusal remains near zero on all six checkpoints, with recovery transitions above the frozen boundary. In a second study, removing the update's top two singular directions restores refusal after short attention‑only fine‑tunes on four checkpoints. On Llama‑3.1‑8B, ordinary training changes weaken this repair and an attacker who spreads the update defeats it. A spectral detector calibrated on benign Llama fine‑tunes misses most repair failures on that checkpoint. Localized freezing can nevertheless help preserve refusal when a few harmful examples enter training data unintentionally. These results show that an attacker can bypass a region identified by recovery and defeat a repair that works across multiple checkpoints, motivating five checks for defenses against adaptive fine‑tuning. Code is available at https://github.com/js‑lee‑AI/refusal‑relocates.
Authors:Lingzhao Kong, Yongsheng Zang, Yu Kang, Kailun Yang, Jie Fu, Yukun Zuo, Zhiyong Li
Abstract:
Collaborative perception enables connected agents to share complementary observations for 3D object detection, extending sensing range and mitigating occlusion. Under asynchronous communication, however, cooperative features arrive with temporal delay. Existing prediction‑based methods compensate for these features mainly from the transmitting agent's own history, leaving residual misalignment with the ego agent's current observation; subsequent fusion also often overlooks spatial variations in alignment quality. We propose EgoRefine, an ego‑referenced predictive alignment and reliability‑aware fusion framework for asynchronous collaborative perception. Its Ego‑referenced Predictive Alignment module uses the current ego feature to guide cooperative trajectory‑field prediction and refines the sampling offsets along an ego‑referenced trajectory direction. Its Trajectory‑conditioned Reliability‑aware Fusion module treats the trajectory discrepancy between the ego and cooperative streams and the directional refinement magnitude as alignment cues, using them to condition the relation between aligned features and adaptively reweight the two streams before convolutional fusion. Experiments on V2V4Real and DAIR‑V2X‑Seq show that EgoRefine outperforms TraF‑Align by 1.6 and 2.9 points on average in AP@0.5 and AP@0.7, respectively. The source code will be made publicly available at https://github.com/godk0509/EgoRefine.
Authors:Xiaolong Qian, Zhonghua Yi, Qi Jiang, Kailun Yang, Shuhang Xie, Shaohua Gao, Kaiwei Wang
Abstract:
Simplified optical systems often exhibit residual lens aberrations and Veiling Glare (VG), resulting in spatially varying blur and contrast reduction. Large‑scale Lens Libraries (LensLib) enable reusable aberration correction models by covering diverse Point Spread Functions (PSFs), but their aberration‑only training distribution does not include target‑specific veiling glare. Extending such foundations to compound degradation is challenging because realistic target‑system compound pairs are difficult to obtain. To address this challenge, we propose LensBridge, a two‑stage framework that first establishes a reusable aberration correction foundation and then adapts it to compound optical degradation using only a few unpaired target observations. In Stage I, we build a PSF‑aware one‑step diffusion foundation by constructing discrete degradation priors from LensLib PSFs and learning to retrieve them directly from aberrated images, enabling PSF‑aware correction without requiring explicit PSF at inference. In Stage II, we adapt this foundation to compound degradation through frequency‑domain guidance. At the data level, Frequency‑guided Degradation Completion (FDC) transfers target low‑frequency characteristics to LensLib aberrated images while preserving aberration structures to synthesize compound training pairs; at the model level, Frequency‑guided Pseudo Decomposition (FPD) forms aberration‑ and VG‑dominant pseudo observations to condition separate adaptation branches. Extensive experiments across multiple optical systems demonstrate that LensBridge effectively extends reusable aberration correction foundations to joint aberration correction and veiling glare removal without target‑system paired supervision. All code will be available at https://github.com/XiaolongQian/LensBridge.
Authors:Shivam Gupta
Abstract:
External verification can correct individual outputs while leaving a self‑reinforcing population in the basin of a wrong consensus. We study how the timing and addressing of a fixed verification budget affect recovery in an asynchronous binary register. For a general nonlinear response, we derive the minimum fuel required to cross a basin boundary under a peak verification constraint. For a finite population, an exact birth‑‑death calculation gives the probability of subsequent wrong consensus after a pulse. Our main asymptotic result identifies the critical budget window: a leading term N\log(x_0/b) and a correction of order \sqrt N, with separate variance contributions from repeated verification targets and autonomous amplification after verification stops. The distinction is substantial: with 16 majority‑updated slots and 14 initially wrong, 9 random checks cross the mean‑field budget threshold, whereas 23 are required for 95% eventual recovery in the exact model. A prospectively specified experiment records 13,392 language‑model responses, including calibration and 108 held‑out trajectories. Calibration produces different fitted response regimes, but all four adjusted schedule‑comparison intervals include zero. A distributional audit also finds that modest mean‑prediction error can conceal a large underestimate of terminal consensus occupancy. The results support risk‑calibrated reset scheduling under a specified update contract, while explicitly separating it from distinct‑target checking and unrestricted evidence broadcast.
Authors:Sushovan Majhi, Pramita Bagchi
Abstract:
Memorization audits of generative models read similarity scores against thresholds, with no null distribution, and the conclusions they support can be wrong. By MemBench's rule, the benchmark's mitigations roughly halve Stable Diffusion's memorization; audited with false‑discovery control, two thirds of the certified images are no longer detected under random prompt perturbations, five sixths under attention rescaling, and all of them under embedding optimization. The field's data‑copying test, read against its own null, flags ten of twenty‑four generators that reproduce nothing. We argue that for memorization the null is the hard part, and supply two. For a whole model, training and held‑out images are exchangeable given its samples, and relabelling them is a permutation test, exact for any statistic when the held‑out images are a random split; under it, a nearest‑neighbour preference still fires on seven of those twenty‑four, and a count restricted to the near‑duplicate scale on none (McNemar p=0.016). For single images, the natural nulls fail twice, measurably: ranking an image among random images yields 596 false discoveries among 2,365 controls, and resampling independent generations makes the null three times too narrow. Calibrated against matched controls, the audit certifies 36 of 61 MemBench images at 5% false‑discovery rate, held‑out controls are certified in 0.01% of calibration splits, and on this benchmark two generations per image recover that count. A calibrated maximum, which reads occasional rather than typical copying, certifies 46. As the scale‑restricted statistic we recommend the small‑scale mass of the Intersection Euler Characteristic Profile, which also counts distinct images copied and tests whether two models copy the same ones.
Authors:Shivam Gupta
Abstract:
Model failover restores availability, but changes which financial institutions share decision errors. We formulate outage‑contingent routing through a local market‑impact response matrix and study expected squared price displacement. A symmetric construction shows that a shared backup can leave an order‑one concentration floor as the number of primary endpoints grows, while balanced fallback risk decreases inversely with the surviving endpoint count. For indivisible decisions, we derive the exact second moment of independent randomized routing and an effective‑exposure granularity that determines its gap from fractional allocation. Conditional‑expectation rounding gives a finite‑agent bound without coupled quotas; a separate swap procedure preserves endpoint counts and is assessed against dual lower bounds. Across 60 synthetic portfolio networks and 11,340 scenario evaluations, the latter reduces risk by 6.57% and 10.53% for single and double endpoint removals at the central feedback setting with independent errors. A replay of 1,024 recorded API responses on constructed rebalancing tasks gives a smaller held‑out reduction of 3.30% (paired bootstrap interval 2.07‑‑4.57%). Strongly aligned errors, inferior endpoints, and indivisibility limit diversification. The contribution is an auditable routing stress test and implementation analysis, not an estimate of real‑market crash probabilities.
Authors:Hamza Tariq, Adeel Akhtar
Abstract:
This paper studies decentralized safe path following for multiple quadrotors on intersecting paths, where safety requires both collision avoidance and strict adherence to pre‑assigned routes. The proposed controller reformulates transverse feedback linearization as a constrained quadratic program with four equality constraints: two enforce convergence to and strict adherence to the path, and two prescribe the desired speed and heading. Safety is achieved by relaxing only the along‑path speed constraint, while the path and heading constraints remain hard. Under the stated assumptions, and for admissible initial conditions, the controller guarantees that all agents converge to and thereafter follow their assigned paths, avoid collisions, and avoid attitude singularities. We evaluate the controller in the Drake physics engine on non‑planar intersecting paths and compare it against two nominal‑plus‑safety‑filter cascades. Code and additional results are available at https://gradslab.github.io/safe_multiquad_pf/.
Authors:Esther Xin
Abstract:
Several recent pipelines build RLVR training data by masking a span of real corpus text and asking a language model to invent plausible wrong answers around it. The correct option is therefore genuine human prose; every distractor is synthetic. Correctness and provenance become entangled, and a policy could in principle learn the second instead of the first. We audit that possibility in GooseReason‑0.7M. First we ask whether the asymmetry is visible at all: a classifier reading only five surface statistics (never the meaning) reaches AUROC 0.562 over 315,499 options, barely above chance. The aggregate hides something, though. Code sits at 0.416, below chance, and manual inspection explains why: code distractors turn out to be single‑operator mutations of the gold answer rather than freely written alternatives, so the two classes are nearly identical by construction. Detecting a signal is not the same as showing a model uses it, so we then run an intervention. We build a paraphrase‑matched control corpus, hold training‑set size identical across arms, and train two policies under one fixed budget. The exploitation gap does not favour the unmodified‑data arm: 0.021 against 0.027 for the control. Under our budget, in other words, a detectable artifact went unexploited. We think that dissociation, along with the domain‑specific construction finding, is worth knowing for anyone curating corpora of this kind, and we release the audit as a mostly CPU‑only protocol.
Authors:Jingtai Yang, Yining Wu, Yanjun Li, Zeyu Zhang, Hao Tang
Abstract:
Recent advances in motion generation and whole‑body tracking have enabled humanoid robots to execute increasingly diverse motions, yet the same motion capabilities may be requested repeatedly during continual deployment. Reliable reuse is challenging because intervening motions can change the robot's entry state, making previously successful motions unsafe to replay blindly. Meanwhile, validated capabilities accumulate during deployment, while bounded storage requires deciding which ones are worth retaining. To address these challenges, we present HumanoidTTT, a framework for test‑time capability reuse in continual humanoid control. Specifically, we introduce Selective Full‑Motion Reuse, which authorizes direct reuse of validated complete motions only from certified applicable entry states, allowing accepted reuse to bypass fresh generation. We further introduce Test‑Time Capability Consolidation, which adapts which qualified capabilities persist in a bounded Full‑Motion Store using subsequent deployment reuse as feedback. Experiments demonstrate zero unsafe accepts and a 16.4× end‑to‑end speedup over fresh generation, while online consolidation improves avoided generator calls by 13.2 per 200 requests over its frozen counterpart. Overall, HumanoidTTT enables reliable and efficient reuse of validated motion capabilities while adaptively retaining useful capabilities throughout continual deployment. Code: https://github.com/AIGeeksGroup/HumanoidTTT. Website: https://aigeeksgroup.github.io/HumanoidTTT.
Authors:Alpay Ozkan, Tunc Ozan Aydin, Marc Pollefeys, Jelena Trisovic, Daniel Barath
Abstract:
Voxel‑based volumetric mapping is fundamental to 3D reconstruction, yet fixed‑resolution grids remain inherently inefficient ‑ wasting memory in uniform regions and losing detail in complex ones. Existing adaptive methods, such as MAP‑ADAPT, partially address this by varying resolution based on geometry and user‑defined semantic class lists, but these heuristics require expert tuning, lack generalization to unseen objects, and provide no explicit mechanism to control memory usage. We propose an adaptive framework that refines voxels based on semantic entropy, which captures label uncertainty, together with geometric curvature and texture richness as scene complexity cues, yielding principled resolution allocation without reliance on semantic taxonomies. To make the accuracy‑memory trade‑off explicit and user‑controlled, we further introduce a reinforcement learning agent that learns voxel subdivision policies under a user‑specified target memory budget, replacing hand‑tuned thresholds with a single intuitive control parameter. The resulting multi‑resolution TSDF achieves higher geometric accuracy, better semantic consistency, and improved memory‑accuracy trade‑offs compared to MAP‑ADAPT and fixed‑resolution baselines on both synthetic and real‑world datasets. Our code and models are available at https://github.com/alpayozkan/UnRL.
Authors:Hyojung Han
Abstract:
We study commercial intent inference that runs entirely on the user's device, under three constraints frozen before the work began: the downloaded payload under 3 MiB, Tier‑0 inference under 20 ms at p95, and no raw text, content embedding, or stable identifier leaving the device. Under them we build a retrieval path over a 6,020‑leaf commercial taxonomy: a static embedding table distilled from a Korean sentence transformer, quantized to 4 bits, no inference runtime. Our main result is where that constraint costs accuracy. On real Korean commerce text labelled by others (22,900 AI‑Hub shopping reviews), mid‑category top‑5 on real product names is 75.0% against an 18.4% permutation baseline, but splits on one observable: a query containing some leaf name as a substring scores 83.5%, one containing none 45.2%. A generic 196.6x larger teacher seemed to localize the gap (+20.1 pp without an anchor, +0.1 with). That null was two effects cancelling: the same teacher fine‑tuned on the student's own contrastive pairs reaches 0.8586 and beats the pure‑encoder student by +10.6 pp with an anchor and +20.9 pp without. The cost is not uniform, but it is not free anywhere; where the anchor is absent, task adaptation buys the teacher nothing, so what the constrained encoder lacks there is capacity. The expensive regime is detectable on‑device from the ranker's own score margin: declining the least confident fifth lifts the rest to 0.8296. A second axis we first reported, a manufacturer model code, does not survive source‑category fixed effects (‑4.0 pp, p=0.51); the anchor does (+13.0 pp). Payload is 2,942,652 bytes, all three library links measured. Tier‑0 p95 is 4.431 and 3.670 ms on two iPhones (A14, A16) and 5.080 ms on a budget Android tablet (Snapdragon 695), all slower than three server CPUs on the same code. Taxonomy supervision is mostly synthetic Korean utterances.
Authors:Romain Claret, Michael O'Neill, Paul Cotofrei, Kilian Stoffel
Abstract:
Open‑ended artificial life systems must acquire diverse competencies from a single evolving genotype. Biological brains combine neuromodulation, which reconfigures circuits without changing connections, with diverse neuron types matched to specific computational roles. Can artificial evolution achieve something analogous in indirectly encoded substrates? Using indirectly encoded substrates evolved via CPPNs, we show through more than 10,000 experiments that neuromodulation alone is insufficient: under evolutionary search, monotonic activation functions impose a 75% ceiling on parity tasks that persists regardless of capacity, topology, or population size. This is an evolutionary search barrier, not a representational limit, since Adam gradient descent achieves 100% on the identical architecture. We combine neuromodulation with per‑task activation function selection, matching oscillatory primitives to parity tasks and monotonic to threshold tasks, producing multi‑behavioral evolved substrates. The result: 100% simultaneous 5‑task success across all 30 seeds (median 14 generations). This generalizes across the oscillatory activation class: all four functions reach 100% (30 seeds each). Neither mechanism suffices alone. The barrier extends to higher‑arity and asymmetric tasks, while multi‑layer depth provides an alternative path. For open‑ended evolution, the computational primitive should itself be an evolvable trait. At inference, one evolved genotype expresses many behaviors.
Authors:Jun He, Deying Yu
Abstract:
Persistent AI agents revise beliefs, consolidate memory, and replace execution substrates. Similar successor states can accompany differently authorized transition claims, while legitimate development can change state substantially. We introduce the Cognitive Continuity Test (CCT), a policy‑relative contract for verifying submitted transitions using scoped authority, provenance, deterministic application, semantic predicates, and candidate‑persistence receipts. CCT distinguishes verified admissibility, affirmative violation, and unresolved required evidence. Separation results concern transition claims rather than live runtime identity; soundness is conditional on the specified checker and evaluator assumptions. IdentityLineageBench provides 24 generated transition families. The reference post‑resolution verifier matches all 576 canonical held‑out labels; lexical state similarity and a lineage‑only diagnostic baseline admit 60.0% and 80.0% of invalid fixtures. These comparisons establish synthetic conformance, not superiority to a policy‑aware deployed system. Signed adversarial regressions cover fabricated interaction counts, unsupported belief changes, and mixed missing/contradictory evidence. SIT behavior and actual model migration remain unmeasured. An 18,000‑execution valid‑path study measures a 6.21 ms default median on resident inputs. We specify the additional activation and recovery obligations needed for deployment.
Authors:Luyao Tang, Yingkai Yang, Hanqi Chen, Jiewei Zheng, Chaoqi Chen, Cheng Chen
Abstract:
In real‑world clinical practice, medical images face open‑world shifts: (i) long‑tailed rare diseases, (ii) subtle lesions dominated by normal anatomy, and (iii) hierarchical taxonomies. Yet most open‑world paradigms assume flat, balanced label spaces, leaving these biomedical demands unresolved. We introduce Generalized Biomedicine Discovery (GBD) and a unified benchmark spanning long‑tail, anomaly, and taxonomy‑aware discovery. Our key insight is that dominant known patterns form a visual manifold that masks subtle novelty. Inspired by expert diagnosis, we propose SCAN (Surprise‑evoked Complementary AccommodatioN), which follows a cognition‑inspired perceptual progression: it applies predictive suppression to filter expected norms, triggers surprise‑evoked salience to highlight unexpected deviations, and performs complementary accommodation to integrate these shifts into global representations. Extensive experiments show that SCAN improves novel concept discovery while generally preserving established clinical knowledge, and it plugs into existing architectures to better navigate the known‑unknown trade‑off in medical imaging. Code is available at https://github.com/lytang63/generalized‑biomedicine‑discovery.
Authors:Stefan Abi-Karam, Miaoyan Zhou, Callie Hao
Abstract:
Deep learning and large language models (LLMs) are rapidly gaining adoption in semiconductor design, driving demand for training datasets. Most efforts focus on hardware description languages (HDLs) while designs for high‑level synthesis (HLS), a popular approach to domain‑specific accelerators, remain scarce. HLS dataset efforts emphasize manual curation or design parameterization, seldom addressing high‑quality LLM‑based generation or diversity in code length, hierarchy, design‑space size, latency, resource utilization, and application domain, potentially limiting model generalization. We propose SyntheticHLS, a framework for generating large‑scale, complex, diverse synthetic HLS datasets using LLMs. Its two key ideas are: 1) an iterative feedback‑guided mutation loop that uses paired HLS source code and design‑space specifications to incrementally transform seed designs into more complex, scalable designs; and 2) quantitative metrics of HLS design complexity and design‑space scalability that serve as measurable objectives for LLM‑guided mutation. We systematically cross‑validate an HLS Quality‑of‑Results (QoR) deep learning model trained and tested across common HLS benchmarks, zero‑shot synthetic designs, and iteratively mutated synthetic designs. Synthetic designs transfer well to common benchmark test sets while the reverse does not hold. SyntheticHLS's iteratively mutated designs provide the most generalizable training corpus among the datasets studied. Analysis of the mutation process and dataset shows that metric‑guided trajectories consistently improve targeted complexity and scalability objectives without regressing non‑target metrics. Mutated designs span a substantially broader, more diverse design space than zero‑shot generated designs. Our framework, dataset, and evaluation are open‑source: https://github.com/sharc‑lab/synthetic‑hls.
Authors:Ting Huang, Biao Wu, Ronghao Chen, Zeyu Zhang, Tengfei Cheng, Qizhen Lan, Huacan Wang, Hao Tang
Abstract:
Recent diffusion and autoregressive models have substantially improved text‑to‑video generation, yet producing coherent long‑form story videos with consistent characters and aligned audio remains challenging. Existing methods often suffer from narrative drift, unstable character identity, weak cross‑scene continuity, and audio‑visual mismatch over extended sequences. We propose DramaAgent, a hierarchical, agentic, and model‑agnostic framework for long‑form text‑to‑video‑and‑audio generation. Rather than improving the underlying video backbone itself, DramaAgent introduces an upper‑level control layer that decomposes generation into story planning, persistent character conditioning, scene‑wise synthesis, and reflection‑guided targeted repair. The framework maintains reusable story and character states across scenes, diagnoses failures such as identity drift, missing scene semantics, temporal discontinuity, and cross‑modal mismatch, and repairs problematic clips in a stage‑specific manner. Experiments across multiple video generation backbones show that DramaAgent improves long‑horizon coherence, character consistency, narrative fidelity, and scene‑level audio‑visual consistency over direct generation and strong baselines. These results suggest that hierarchical agentic control is a practical direction for controllable long‑form audiovisual generation. Code: https://github.com/AIGeeksGroup/DramaAgent. Website: https://aigeeksgroup.github.io/DramaAgent.
Authors:Jiahao Chen, Zhou Feng, Oubo Ma, Yichen Yan, Ruixiao Lin, Hangtao Zhang, Linkang Du, Hengyu An, Yong Yang, Jun Liu, Junhao Li, Naen Xu, Chunyi Zhou, Yuan Su, Zehao Jin, Qianli Ma, Leyi Qi, Yiming Wang, Zhe Ma, Yuwen Pu, Mengyao Du, Yuanyi Song, Enhao Huang, Zhihui Fu, Jun Wang, Jinfeng Li, Yuefeng Chen, Hui Xue, Yiming Li, Tianyu Du, Shouling Ji
Abstract:
Large language models (LLMs) exhibit strong general capabilities, yet their parameters typically remain fixed after deployment, limiting learning from new interactions. In open‑ended environments, this motivates self‑evolving agents that continually update reusable state‑including model parameters, memories, tool definitions, skills, and workflows‑from data, feedback, and accumulated experience. This shift changes the safety problem: once experience becomes reusable state, past events become future causes, and information harmless in one context may later influence decisions with greater persistence, authority, or scope. Self‑evolving agent safety therefore asks not only whether a response is aligned or an action authorized, but whether safety properties survive the accumulation, generalization, and cross‑context reuse of locally useful experience. We introduce SAVER, a transition‑centered framework in which Substrate locates reusable influence, Adaptation captures how it changes, Violation identifies compromised safety attributes, Exposure marks where failures become observable, and Response assesses containment, repair, or revocation. Our survey reveals that failures need not originate from harmful information: legitimate state can become unsafe when adaptation expands its persistence, authority, or scope beyond the conditions under which it was valid. Existing work provides comparatively strong evidence for admission, retrieval, activation, exposure, and local containment, but much less for descendant repair and evaluation after adaptation resumes. We therefore argue for longitudinal evaluation that traces unsafe influence to its originating transition, verifies repair across descendants, and tests whether it can re‑emerge under continued evolution.
Authors:Timothy Kassis
Abstract:
Detailed profession‑specific system prompts raise token use and estimated cost per response without a consistent accuracy gain. We evaluate Scientific Agents, an open‑source corpus of 503 profession‑specific AGENTS.md profiles, with Gemini 3.8 Flash via OpenRouter in the Pi agent harness. We compare matched profiles with four controls: a minimal baseline ("You are a helpful assistant"), the profile's opening role sentence, a generic scientific rigor guide, and a profile from an unrelated domain. Across nine text‑based science benchmarks (4,531 sampled questions, 100 matched profiles), 4,488 items completed all five conditions after API‑error retries, scored with automated, rule‑based grading. The average profile‑baseline accuracy difference is ‑0.6 percentage points (95% bootstrap interval [‑1.5, +0.2] across fixed tasks), and no benchmark shows a statistically clear improvement. Matched profiles produced 1.5‑2.3 times as many output tokens and cost 2.2‑4.5 times more per successful call. On 60 tool‑using BioMysteryBench bioinformatics problems (three runs each for baseline and profile), mean solve rates were 46.7% with the profile and 56.7% at baseline, a difference of ‑10.0 percentage points (95% interval [‑16.7, ‑3.3]) driven by more frequent token‑ and time‑limit stops under the profile. Longer prompts had one unexpected operational advantage: on SuperGPQA, frequent provider API drops left the short baseline with a correct first‑pass answer on only 54.0% of items, against 71.6% with the profile. Generic and mismatched prompts were about as reliable, so this gain comes from prompt length or formatting rather than domain expertise. For the tested model and tasks, loading full profession profiles by default does not improve accuracy and costs considerably more; whether selective retrieval of profile sections or open‑ended scientific tasks would change this remains to be tested.
Authors:Aubrey M. Brueckner, Darshil Patel, Yuhuan He, Timothy Kassis
Abstract:
K‑Dense BYOK (bring your own keys) is a free, open‑source AI research assistant for scientists in any field that runs on the researcher's own computer. The researcher supplies access to a model of their choice, hosted or running locally, and the application supplies everything else: a place for the work to run, a layer of scientific scaffolding, and a complete record. Each project is an ordinary folder, so the data, the code, the results, and the record stay on a machine the researcher administers and can be read years later without the application. Three things separate it from a chat assistant or a general‑purpose coding agent. It ships a library of written scientific procedures, guided workflow templates, catalogs of where research data can be found, and reviewer and writer roles the agent can hand work to. It keeps a Living Lab Notebook whose entries link into an argument and are added to but never erased. And it records what happened by watching what the agent does rather than by taking the agent's word for it, in a log the agent has no tool that can write to. That choice targets the most common failure, model overclaiming, in our earlier benchmark of nine frontier models, by making claims checkable rather than preventing them. On twenty interdisciplinary research prompts, scored under a rubric fixed in advance, K‑Dense BYOK led two managed platforms on both scientific quality and research execution. Its deliverables were the only ones that recorded the software they ran in, and the only ones that usually arrived with a command that regenerates the results. One of the managed platforms ran the same frontier model and supplied neither. Those environment records were files the agent wrote, not part of the observed log, which does not yet capture the software environment itself. The code is available under the MIT license at https://github.com/K‑Dense‑AI/k‑dense‑byok.
Authors:Changyi Li, Yu Xiao
Abstract:
Assembly actions are compositional: they combine a manipulation with a part or tool. In deployment, systems routinely encounter novel combinations of familiar components, yet an atomic action classifier assigns every unseen combination exactly zero probability by construction. The prevailing solution is verb‑‑noun decomposition, which predicts components separately and recombines them to reach unseen actions. While widely adopted, how decomposition generalizes under compositional shift remains poorly understood. We present a systematic analysis of verb‑‑noun decomposition across three assembly datasets (MECCANO, HAViD, and IMPACT). Although decomposition escapes the atomic ceiling, its generalization extends only partially beyond it. Unseen‑composition performance remains strongly tied to the co‑occurrence structure of the training data, indicating that much of the observed gain arises from interpolation within densely supported regions of the compositional space rather than from unconstrained recombination. Across datasets, failures consistently concentrate on the larger‑vocabulary component, and IMPACT's verb‑heavy vocabulary reverses the bottleneck from nouns to verbs. We further show that shared‑encoder training introduces component entanglement, encouraging reliance on co‑occurrence patterns that transfer poorly to unseen compositions and trailing independent recombination by up to 6.0× in harmonic mean. Taken together, these findings explain why decomposition achieves only partial compositional generalization in practice. By identifying primitive support, vocabulary asymmetry, and component entanglement as connected sources of error, we provide a portable diagnostic framework for studying compositional recognition beyond aggregate accuracy. Code: https://github.com/hisalaheiyo/assembly.
Authors:Amirhosein Azarpour, Seyyed Moein Kazemi
Abstract:
Kolmogorov‑Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial‑based KAN variants have addressed the computational overhead of original B‑spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real‑valued inputs and the bounded or semi‑infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes‑Wigert Kolmogorov‑Arnold Network (SW‑KAN), a novel architecture that employs Stieltjes‑Wigert q‑orthogonal polynomials defined on the semi‑infinite domain (0, infinity). We introduce a smooth exponential‑of‑tanh mapping that stably bridges the domain gap while preserving well‑conditioned gradients, and leverage a numerically stable three‑term recurrence that evaluates polynomial expansions in O(N) operations without special‑function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW‑KAN achieves superior accuracy‑efficiency trade‑offs across diverse tasks. The log‑normal weight structure and learnable q‑parameter of Stieltjes‑Wigert polynomials provide a distinct inductive bias that enables robust performance under resource‑constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource‑constrained settings.
Authors:Wooyoung Jung
Abstract:
Semantic querying of building knowledge graphs (KGs) underpins the integration of artificial intelligence into building operations, from natural‑language access to cross‑building analytics, but such KGs are rarely public owing to proprietary, security, and cost barriers. BuildGraph is a synthetic building KG dataset of 120 buildings in the Brick Schema ontology, grounded in U.S. Department of Energy prototype models and sensor‑placement patterns from real buildings. It spans eight commercial building types across three ASHRAE energy‑code vintages, with five realizations per archetype varying URI naming, sensor‑attachment predicates, and topology. A 75‑query SPARQL benchmark confirms structural completeness: BuildGraph reaches 91.5% Query Answerability Rate versus 35.8% for 59 real‑world Brick files. Independently, it reproduces real buildings' sensor‑type proportions on their shared vocabulary (cosine 0.86‑0.94), evidence of realistic instrumentation where measurable. A downstream text‑to‑SPARQL experiment with Gemma 4 (26B) reaches 27.3% Row‑Matching F1 (+17.5 pp over zero‑shot) on 12 held‑out buildings. BuildGraph gives facility managers and digital‑twin developers a testbed for portable analytics and natural‑language interfaces, and its dataset, generator, and benchmark are openly available at https://github.com/humanbuildingsynergy/BuildGraph.
Authors:Stefan G. Creadore
Abstract:
Large‑language‑model agents can propose and execute actions, but proposal, authority, dispatch, verified external effect, and serving promotion are different claims. We present Praxa, an agent harness that represents these states explicitly through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. We report four evidence lanes. First, an author‑run repository‑local audit at a pinned revision passed 1,027/1,027 unit tests and 89/89 Workerd tests, instrumented all 363 expected source files, and met four coverage floors; raw per‑test transcripts and independent reproduction are unavailable. Second, in a provider‑backed Terminal‑Bench Core 0.1.1 pilot across 12 curated tasks, baseline and reliability‑layer arms each passed 17/36 strict trials. The reliability layer used 37.49% more input and 50.73% more output tokens, so the pilot does not support superiority. Third, in a post‑debug, two‑order coordination‑proxy development comparison, baseline and a source‑authored candidate each completed 180/180 trials with equal measured accuracy, full hermetic crash recovery, and zero protected violations. The candidate used 37.11% fewer tokens, 33.84% lower estimated endpoint cost, and 11.63% fewer steps; this does not establish improved quality, latency, or production behavior. Fourth, deployed source/configuration evidence shows bounded reflection, recall accounting, memory compilation, and tool‑health paths, but no production outcome lift. Praxa's supported contribution is an evidence‑bound architecture that makes authority‑to‑effect transitions explicit and testable. Current evidence does not establish adversarial security, production safety, general specialist superiority, autonomous recursive optimization, or user benefit.
Authors:Athanasios Zeris
Abstract:
Frequency‑collapse attention [Zeris, 2026e] achieves large gains over standard dot‑product attention by replacing the Q/K dot product with a bandpass‑filtered inner product at a learned frequency. A natural follow‑up question is: which filter shape works best, and why? We test five hypotheses about filter properties ‑‑ DC suppression, Nyquist suppression, bandwidth, centre frequency, and multi‑scale coverage ‑‑ using a controlled ablation on character‑level language modelling (TinyShakespeare, 6‑layer GPT). Our main findings are: (1) DC and Nyquist components are actively harmful (val ~= 2.0, equivalent to phase randomisation), confirming that oscillatory bandpass structure is essential, not just any low‑dimensional spectral summary; (2) the optimal single‑scale bandwidth is sigma ~= 2 bins centred at paragraph scale (~70 tokens), giving a clean gain of Delta = +1.15 nats over BASE‑DOT; (3) admissible filters (zero‑mean, Mexican Hat DOG m = 2) outperform non‑admissible Gaussians at the same scale and provide partial protection against bilateral FFT leakage; (4) bilateral FFT leakage scales monotonically with spectral coverage ‑‑ narrowband filters (gap > +4) are clean, wideband filters (gap < +2) are leaky; and (5) causal time‑domain Morlet at character scale cannot beat BASE‑DOT (K=128 taps covers 50% of T=256 context), motivating word‑level experiments in the companion MorletQK paper [Zeris, 2026f]. Together, findings (1)‑(5) characterise FourierQK as effective in bidirectional attention settings (encoder‑style, e.g. BERT), where full‑sequence context is available at both training and inference time; autoregressive generation requires a causal spectral variant such as MorletQK [Zeris, 2026f] (decoder‑style, e.g. GPT). Code available at: https://github.com/AthanasiosZeris/energy‑gated‑attention