arXiv Papers with Code in Computer Science (June 2026)
Authors:Kartik Narayan, Vishal M. Patel
Abstract:
Low‑resolution face recognition (LR‑FR) remains a challenging task due to poor feature extraction and aggregation, as probe images often contain limited identity information resulting from extreme degradations such as blur, occlusion, and low contrast. Additionally, the domain gap between high‑resolution (HR) gallery images and low‑resolution (LR) probe images poses a significant challenge. A single feature encoder struggles to generalize effectively across both domains when fine‑tuned on an LR dataset, and this issue is further magnified by catastrophic forgetting. To address these challenges, we propose FaceMoE, an effective adaptation of Mixture of Experts (MoE) transfomer architecture for low‑resolution face‑recognition . Specifically, we introduce multiple specialized feed‑forward network (FFN) experts and incorporate a top‑k router, which dynamically assigns tokens to appropriate experts. This design emergently promotes specialization across experts for different semantic regions of the face, which enables FaceMoE to perform resolution‑aware feature extraction. Moreover, the top‑k router facilitates sparse expert activation, enabling the model to preserve pretrained knowledge when finetuned on a LR dataset, while increasing model capacity without proportional computational overhead. FaceMoE is trained with a combined face recognition loss, router z‑loss, and load balancing loss to ensure expert specialization and stable training. To the best of our knowledge, this is the first work leveraging MoE for LR‑FR. Extensive experiments across eleven datasets, spanning HR, mixed‑quality, and LR benchmarks, demonstrate that FaceMoE significantly outperforms state‑of‑the‑art methods. Code: https://github.com/Kartik‑3004/FaceMoE
Authors:Yujie Guo, Yudong Jin, Lingteng Qiu, Zehong Shen, Zhen Xu, Jing Zhang, Xianchao Shen, Hujun Bao, Sida Peng, Xiaowei Zhou
Abstract:
Producing 3D human representations from input views on the fly is essential for immersive live streaming systems, where representation compactness is as critical as high fidelity given limited computational power and transmission bandwidth. Although recent feed‑forward reconstruction methods achieve impressive quality through the view‑centric prediction of 3D representations, they repeatedly encode the same subject content across multiple views, leading to significant inter‑view redundancy. Our key insight is to perform predictions directly in 3D space, enabling the network to learn and produce a highly compact representation. To this end, we propose PointSplat, a novel human‑centric approach that directly infers Gaussian primitives from an input point set. The proposed method first estimates a coarse geometric proxy and performs ray casting to prune redundant points and establish explicit 2D‑‑3D correspondences. Subsequently, it employs a Point‑Image Transformer to fuse appearance and geometry features, predicting Gaussian attributes in a single forward pass. This design restricts predictions to foreground regions of interest, substantially reducing the total number of Gaussians while improving novel‑view rendering quality. Extensive experiments demonstrate that PointSplat achieves higher efficiency and quality while exhibiting strong robustness to variations in view count and image resolution across multiple datasets.
Authors:Or Hirschorn, Aaron Olender, Eli Alshan, Ianir Ideses, Lior Fritz, Sagie Benaim
Abstract:
We present a zero‑shot, training‑free and optimization‑free framework for generating 360 panoramic images and videos by directly injecting spherical priors into pre‑trained diffusion transformers. Existing methods either rely on costly fine‑tuning on scarce panoramic data that limits generalization, or leverage multi‑step optimization that incurs prohibitive inference latency. We observe that contemporary generative models natively exhibit some panoramic priors from large‑scale training. However, these emergent capabilities are insufficient, as the models fundamentally fail to satisfy the rigorous topological constraints imposed by equirectangular projection (ERP). We introduce a zero‑shot and optimization‑free approach that resolves these constraints at inference time. Spherical RoPE replaces standard rotary position embeddings: low‑frequency channels are re‑parameterized as 3D Cartesian coordinates to natively encode the spherical manifold, while high‑frequency channels are harmonically quantized to enforce exact periodicity. Coupled with complementary Semantic Distortion classifier‑free guidance (CFG) that explicitly steers geometry, we avoid retraining and inherit the full creative breadth of state‑of‑the‑art models. Our approach generalizes across diverse backbones and 360 generation modalities. We demonstrate this across text‑to‑panorama using Flux.1, Flux.2, and LTX‑Video backbones, achieving competitive performance against baselines, all while remaining training‑free. Project page: https://orhir.github.io/SpheRoPE
Authors:Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan
Abstract:
Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one's own cognitive processes. Yet LLMs exhibit systemic deficiencies in key metacognitive faculties: they hallucinate with high confidence, fail to recognize knowledge boundaries, and misrepresent their internal uncertainty‑‑undermining trustworthiness and reliability. Since monitoring task performance and adapting behavior accordingly are central to metacognition, we posit that models capable of accurately judging their own performance are better positioned to improve it. We operationalize this idea via two novel mechanisms: reinforcement learning with metacognitive feedback (RLMF), a paradigm to refine completion rankings during preference optimization based on the quality of a model's self‑judgments of performance, and metacognitive data selection, which uses similar self‑judgments to identify high‑value training examples, outperforming naive active learning. We apply these innovations to the problem of faithful calibration (FC), a task that is itself fundamentally metacognitive: the goal is to align expressed with intrinsic uncertainty, difficult even for frontier LLMs. We adopt a two‑stage, decoupled approach, first using these methods to calibrate the faithfulness of models' self‑reported confidence scores, then mapping to natural, context‑adaptable linguistic uncertainty via targeted output editing. Extensive experiments show RLMF achieves generalizable, state‑of‑the‑art FC on diverse tasks while preserving accuracy. Further, RLMF surpasses standard RL by up to 63% while enhancing models' ability to assess and express their own capability limits. This positions RLMF as a promising paradigm to enhance LLM metacognition toward improved abilities and alignment, and suggests metacognitive performance as an effective RL signal to overcome limits of prior intrinsic feedback methods.
Authors:Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn
Abstract:
Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long‑horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal. We introduce Freeform Preference Learning (FPL), a method for learning robot policies from freeform human preferences. Rather than asking annotators which of two trajectories is better overall, FPL lets them define natural‑language preference axes, such as speed, safety, quality of placement, or carefulness, and provide pairwise preferences along each axis. These annotations are used to learn a language‑conditioned reward model that maps a trajectory and preference label to an axis‑specific reward. We use this model to train a reward‑conditioned policy that optimizes across the multiple human‑specified dimensions. Across four real‑world and two simulated long‑horizon manipulation tasks, FPL improves over sparse‑reward and binary‑preference methods by 38 percentage points. Beyond improved performance, FPL learns dense progress signals without explicit subtask segmentation, shows compositionality of behavior not present in the data, and allows users to steer the policy towards different behaviors at test time without retraining. Blog post with videos available at https://freeform‑pl.github.io/fpl.website/
Authors:Sanghyuk Chun, William Yang, Amaya Dharmasiri, Olga Russakovsky
Abstract:
Uncertainty estimation has been a long‑standing challenge in AI models; it amounts to "knowing what you don't know," and metacognition is notoriously difficult even for humans (cf. the Dunning‑Kruger effect). Although it is still far from solved even in simpler classification systems, tackling it in multimodal large language models (MLLMs) is becoming increasingly important. Within MLLMs, uncertainty can stem from any of the diverse sources as well as from their relationships, and further can stem from the unbounded answers in the open‑ended setting. To tackle the issues, we propose CoMet, an MLLM uncertainty estimation method by decomposing uncertainty into a context‑specific term and a multiplicity‑specific term. The former captures ambiguity induced by the given context (e.g., task or prompt), while the latter captures how many plausible answers determined by the context remain compatible with the given input. We train a lightweight post‑hoc uncertainty module to estimate these quantities, which enables efficient uncertainty estimation without autoregressive answer generation or repeated sampling. Experiments on various open‑ended multimodal benchmarks, hallucination detection, and multiple‑choice visual question answering benchmarks show that CoMet consistently improves uncertainty estimation over existing baselines while remaining efficient in practice. Code is available at https://github.com/princetonvisualai/comet_uncertainty
Authors:Xiaopeng Lin, Ruoqi Yang, Shijie Lian, Zhaolong Shen, Bin Yu, Changti Wu, Haibao Liu, Yuxiang Zhang, Hong Li, Qiyuan Su, Haochen Liu, Xuguo He, Yukun Shi, Cong Huang, Zhirui Zhang, Bojun Cheng, Kai Chen
Abstract:
Vision‑language‑action (VLA) models across robot embodiments require high‑quality observation‑‑action supervision to learn deployable action distributions, yet scaling such robot data remains difficult, especially for high‑DoF humanoids. Teleoperation provides controller‑aligned supervision, while human egocentric videos capture diverse bimanual manipulation but do not directly provide executable robot actions. We introduce Human‑as‑Humanoid, a human‑to‑humanoid supervision framework that enables near‑real‑time human‑centric action generation, making human demonstrations usable for high‑DoF humanoid VLA training by jointly aligning the robot embodiment, the sensing setup, and the action‑label interface. Built on PrimeU, a human‑aligned 60‑DoF upper‑body humanoid, Human‑as‑Humanoid uses synchronized ego‑exo videos to pair deployment‑aligned egocentric observations with exocentric motion recovery, retargets the recovered human motion through staged Inverse Kinematics (IK) into controller‑aligned 60‑DoF action chunks, and trains the VLA model with Forward Kinematics (FK)‑aware supervision to preserve wrist and fingertip task‑space geometry. This converts large‑scale human demonstrations from visual observations into executable observation‑‑action supervision for the target humanoid. Experiments validate the conversion chain at the motion‑recovery, robot‑action‑space, and real‑robot deployment levels. Human‑as‑Humanoid yields a 4.8‑‑7.2x raw demonstration‑throughput gain over humanoid teleoperation in our data‑collection analysis, and on several downstream tasks, policies post‑trained only with the converted human labels generalize to real‑robot deployment without target‑task robot demonstrations. The official project website is available at https://zgc‑embodyai.github.io/Human‑as‑Humanoid.
Authors:Philippe Chlenski, Zachariah Carmichael, Ayush Warikoo, Chia-Tse Shao, Yingxiao Ye, Aobo Yang, Vivek Miglani, Nehal Bandi
Abstract:
Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log‑probabilities over output tokens. This creates a surrogate problem: when do measurements made on open models allow us to make claims about a closed model? We evaluate surrogate fidelity at the prediction, attribution, and representation levels. For binary classification tasks, log‑odds provide an API‑compatible scalar readout of the model's representation space, and leave‑one‑out attributions provide insight into model behavior. Across eleven models spanning four families (Llama, Qwen, GPT, and Gemini), we find that prediction fidelity substantially overstates attribution fidelity: models that agree on what the answer is often disagree on why. We document an access‑validity inversion: white‑box signals like attention patterns and perturbation magnitudes are highly stable across models but only weakly predictive of causal attributions, which black‑box input ablations capture by design. Mechanistic insight does not automatically transfer to closed targets, and prediction‑level agreement is insufficient to warrant such transfer. Code and results are available at https://github.com/facebookresearch/surrogate.
Authors:Lianyu Hu, Shengqian Qin, Zeqin Liao, Qing Guo, Liang Wan, Wei Feng, Yang Liu
Abstract:
Chain‑of‑thought (CoT) reasoning has enabled multi‑modal large language models (MLLMs) to tackle complex visual reasoning tasks by generating explicit intermediate reasoning steps in natural language. However, this text‑based reasoning paradigm is inherently slow at inference time with even thousands of tokens and fundamentally constrained by the expressiveness of natural language. In this paper, we propose CoLT, (Chain of Latent Thoughts), a novel framework that teaches multi‑modal models to reason through a chain of latent thought representations instead of verbose text tokens, which can perform thinking with as few as 3 steps. Naively forcing the model to think with latent states easily produces meaningless semantics and makes training unstable. To effectively regulate the latent reasoning process, we introduce a lightweight external decoder that provides step‑level supervision for each latent reasoning step in two complementary directions: a forward mode that decodes latent thoughts into the textual reasoning of the next step, and a backward mode that aligns decoder hidden states with the model's latent thoughts given preceding textual context. We further incorporate internal supervision that encourages coherent step‑by‑step latent transitions. The decoder and internal supervision are removed during inference to maintain high efficiency of latent reasoning. Extensive experiments on eight benchmarks demonstrate that CoLT not only outperforms existing latent reasoning methods such as CODI and SIM‑CoT, but also surpasses latent visual reasoning approaches that rely on auxiliary images with costly annotation requirements. Compared to text CoT methods, CoLT can notably reduce the inference time by 10.1× and text decoding time by 22.6×. Code is released at https://github.com/hulianyuyy/CoLT.
Authors:Yuhao Wang, Mu Qiao, Haiwen Diao, Yunzhi Zhuge, Pingping Zhang, Xindong Zhang, Lei Zhang, Huchuan Lu
Abstract:
Multimodal Large Language Models (MLLMs) incur prohibitive inference costs due to long visual token sequences. Training‑free visual token reduction provides an efficient solution. However, existing methods distort attention distributions, giving rise to a phenomenon we term Attention Logit Collapse. To address this issue, we propose ERA, an Entropy‑guided visual token pruning framework with Rectified Attention for efficient MLLMs. Specifically, ERA comprises three crucial components: Dual‑view Entropy Pruning (DEP), Bias‑aware Token Recycling (BTR), and Logit‑preserving Attention Rectification (LAR). First, DEP identifies representative anchor tokens by jointly modeling visual diversity and head‑wise saliency. BTR then recycles pruned tokens into their corresponding anchors while estimating a cluster‑level logit bias. Building upon this, LAR injects the estimated bias into attention logits, effectively rectifying the collapse induced by token reduction. Together, these components preserve visual evidence even under aggressive compression, enabling robust performance across single‑image, multi‑image, and video settings on a wide range of MLLMs. Beyond delivering practical acceleration, ERA establishes logit‑preserving visual token pruning as a principled framework for efficient MLLMs, unifying theoretical foundation, algorithmic design, and practical deployment. The code is at https://github.com/924973292/ERA.
Authors:Peng Li, Rawal Khirodkar, Junxuan Li, Yuan Dong, Chen Cao, Yuan Liu, Wenhan Luo, Yike Guo, Shunsuke Saito
Abstract:
Creating photorealistic, animatable 3D human avatars from monocular images still largely depends on Linear Blend Skinning (LBS) and parametric body models, which constrain expressivity and often introduce artifacts due to imperfect fitting. We propose LUNA, an LBS‑free universal neural animation model that directly maps multiple 2D controls like images, keypoints, sketches, and unseen characters into 3D Gaussian deformations, bypassing explicit body fitting. At its core, a transformer‑based motion regressor disentangles global rigid motion from fine‑grained local dynamics to capture both coherent movement and subtle non‑rigid effects. To resolve the inherent ambiguity of 2D‑to‑3D lifting while scaling beyond fitted datasets, we introduce hybrid supervision that distills soft structural priors from an LBS teacher and a loss that supports training on both limited fitted data and large in‑the‑wild unlabeled videos. Extensive experiments show LUNA achieves competitive visual fidelity compared to LBS‑based approaches, while delivering realistic human motion and zero‑shot cross‑identity generalization across diverse driving modalities. To the best of our knowledge, LUNA is the first end‑to‑end 3D animatable model that supports implicit 2D driving.
Authors:Qingyun Liu, Jiwen Zhang, Jingyi Hu, Siyuan Wang, Zhongyu Wei
Abstract:
Recent multimodal large language models (MLLMs) have strong potential as embodied agents, but their ability to collaborate in visually grounded environments remains underexplored. To address this gap, we introduce MECoBench, a multimodal embodied cooperation benchmark with an evaluation platform spanning diverse real‑world tasks, two cooperation structures, and three collaboration modes. Through extensive experiments across various MLLMs, we summarize three key findings: (i) Collaboration generally improves embodied task completion, but its benefits depend on balancing collaborative gains against coordination complexity. (ii) Communication is essential to collaboration gains, while the best collaboration mode depends on team size and model capability. (iii) Moreover, collaboration improves robustness under noisy priors and exploration conditions. Generally, MECoBench provides a systematic testbed for understanding the mechanisms and limits of multimodal embodied collaboration. Code and dataset are available at https://github.com/q‑i‑n‑g/MECoBench.
Authors:Xinyu Hou, Xiaoming Li, Zongsheng Yue, Chen Change Loy
Abstract:
Depth‑of‑field control is a fundamental tool in photography, yet post‑capture bokeh editing from a single image remains challenging. A practical editor should handle images captured under arbitrary focus and aperture settings. Existing methods typically assume an all‑in‑focus input, or first recover an all‑in‑focus image before rendering new bokeh. Such pipelines can discard useful blur cues from the source image and propagate reconstruction artifacts into the final edit. We introduce AnyBokeh, a physics‑guided framework for any‑to‑any bokeh editing. Instead of treating source blur merely as a degradation to be removed, AnyBokeh estimates the source blur state with a signed circle‑of‑confusion map and a disparity map. By modeling the linear relation between signed circle of confusion and disparity difference, AnyBokeh estimates a source‑specific optical fingerprint and transfers the source optical characteristics to the desired focus and aperture setting. A generative editor conditioned on both source and target circle‑of‑confusion maps then performs relative blur synthesis, enabling spatially adaptive deblurring, preservation, and defocus rendering. To support physically supervised learning, we further construct a high‑fidelity synthetic dataset with accurate depth, focus distance, and full EXIF metadata. Experiments on real‑world benchmarks show that AnyBokeh achieves faithful and controllable editing across any‑to‑any bokeh editing, all‑in‑focus‑to‑bokeh rendering, and defocus deblurring, while avoiding all‑in‑focus reconstruction and test‑time bokeh‑level calibration commonly required by existing approaches. The code and dataset will be available at https://github.com/itsmag11/AnyBokeh.
Authors:Felipe Tommaselli, Francisco Affonso, Arthur Pompeu, Gianluca Capezzuto, Arun Narenthiran Sivakumar, Girish Chowdhary, Marcelo Becker
Abstract:
Unstructured navigational features, such as irregular planting or discontinuities, remain the primary failure mode for under‑canopy agricultural robots. Existing geometric approaches often fail in these scenarios because they compress high‑dimensional visual data into deterministic spatial references, effectively discarding the uncertainty and semantic context required to navigate ambiguous terrain. To address this, we present LeCropFollow, a visual navigation framework that bypasses explicit geometric modeling in favor of a learned latent representation. By integrating a self‑supervised semantic heatmap extractor with TD‑MPC2, a Model‑Based Reinforcement Learning (MBRL) planner, our system optimizes trajectories directly within a latent manifold. The framework operates over the uncompressed heatmap signal, preserving the semantic context that geometric reductions discard. We demonstrate that this representational shift enables zero‑shot transfer from simplified simulation to the physical world without fine‑tuning. Extensive field experiments in late‑stage corn fields show that LeCropFollow matches state‑of‑the‑art baselines in unstructured rows but significantly outperforms them in plantation gaps, achieving a 2.4x reduction in semantic failures compared to keypoint‑based methods. These results suggest that latent planning offers a robust alternative to geometric estimation for operations in heterogeneous agricultural environments. Code, models, and data available: https://felipe‑tommaselli.github.io/lecropfollow .
Authors:Hubert Dymarkowski, Xingjian Fu, Rappy Saha, Jude Haris, José Cano
Abstract:
Deploying Vision Transformer (ViT) models on edge platforms remains challenging due to their high computational demands and the architectural heterogeneity of modern hybrid ViT models, which incorporate both fully connected and convolutional layers. This heterogeneity leads to significant variation in tensor shapes, requiring flexible and efficient FPGA‑based acceleration. In this paper, we present FlexViT, a reconfigurable FPGA accelerator for efficient ViT inference on resource‑constrained edge devices. Built on the SECDA‑TFLite framework, FlexViT employs a hardware‑software co‑design approach that maps both fully connected and convolutional layers onto a unified high‑throughput INT8 GEMM engine using a runtime im2col transformation. To efficiently support diverse layer configurations, we propose a dual‑mode dataflow that dynamically switches between input and weight reuse by reconfiguring the compute array at runtime. We further introduce a depth‑first tiling strategy that completes accumulation in a single pass, eliminating off‑chip partial‑sum transfers and reducing memory bandwidth requirements. We implement FlexViT on a PYNQ‑Z2 FPGA and evaluate it across a representative set of ViT models. FlexViT achieves up to 2.74x speedup on accelerator‑executed layers, translating into up to 1.40x end‑to‑end speedup compared to CPU‑only execution. The code is available at: https://github.com/gicLAB/FlexViT
Authors:Haojian Huang, Harold Haodong Chen, Meng Luo, Junjia Du, Shanqing Xu, Ziheng Chen, Yanxiang Huang, Yinchuan Li, Ying-Cong Chen
Abstract:
We introduce VidPair‑Halluc, a new benchmark for evaluating video hallucination in large video models (LVMs) under rigorous and controlled conditions. Unlike previous benchmarks that primarily rely on text‑based perturbations or adversarial questions while neglecting the consistency of visual backgrounds, VidPair‑Halluc features video pairs with highly similar backgrounds but distinctly different foreground semantics, enabling precise attribution of model errors to genuine hallucination rather than background variation. The benchmark is constructed through PairFlow, a pipeline that leverages recent advances in text‑to‑image and video generation to systematically compose stories, generate coherent video clips, and assemble them into adversarial pairs. Covering both spatial and temporal reasoning across ten semantic aspects, VidPair‑Halluc comprises 1K high‑quality adversarial video pairs and 11K spatio‑temporal QA pairs with control over background and foreground variations. Evaluations on mainstream LVMs show persistent difficulty with robust fine‑grained video understanding in adversarial settings, and code and data are available at the https://jethrojames.github.io/VidPair‑Halluc/.
Authors:Junzhe Jiang, Zipei Ma, Zijie Pan, Li Zhang
Abstract:
A pivotal step in autonomous driving simulation involves inserting foreground vehicles with predefined trajectories into simulated scenes. This process enhances scene diversity and facilitates the creation of various corner cases for testing and improving autonomous driving models. However, existing methods often rely on pre‑reconstructed 3D assets, which frequently lead to lighting inconsistencies between the inserted foreground and the background. Moreover, the reliance on limited, manually‑curated 3D assets hinders large‑scale deployment. To address these challenges, we propose DriveWeaver, a novel framework for controllable vehicle insertion in autonomous driving simulation. Specifically, for a masked target insertion area, DriveWeaver performs video inpainting conditioned on vehicle point clouds to generate high‑quality, temporally consistent vehicles. This video‑inpainting‑based approach ensures seamless blending between the foreground and background, while the readily available point cloud conditions enable superior generalization. To support long‑term generation, we further design a global‑to‑local hierarchical inpainting strategy, ensuring the consistent identity and appearance of the inserted vehicles. Meanwhile, we extract explicit 3D Gaussian representations of the inserted vehicles through an urban reconstruction pipeline to enable real‑time rendering for autonomous driving simulation. Extensive experiments across diverse datasets demonstrate that our method outperforms existing baselines in visual realism and geometric consistency, providing a robust tool for scalable autonomous driving scene augmentation.
Authors:Shaozu Ding, Linan Song, Marco De Vincenzi, Dajiang Suo
Abstract:
LiDAR has increasingly been integrated into traffic cameras to expand coverage and mitigate occlusion in roadside cooperative perception. However, how unimodal and camera‑LiDAR fusion architectures behave under variations in LiDAR point sparsity induced by sensor configurations and scene‑dependent sensing conditions remains underexplored. We introduce RESOLVE, a large‑scale real‑world benchmark dataset featuring multi‑resolution roadside LiDAR and synchronized camera‑LiDAR sensing for systematic evaluation of unimodal and fusion‑based architectures in roadside 3D detection and tracking. RESOLVE contains over 100k images and 26k point cloud frames with 220k manually annotated bounding boxes, captured at a real‑world urban intersection across diverse lighting and weather conditions and spanning 10 classes of traffic participants. In particular, RESOLVE enables controlled evaluation across three LiDAR resolution levels while keeping all other sensing and environmental factors fixed. This allows fair cross‑architecture comparisons under point cloud distribution shifts resulting from resolution variations, sensing distance, and training‑inference resolution mismatches. Results from extensive benchmark experiments reveal insights into how multimodal fusion can compensate for LiDAR point sparsity, offering clues for designing cost‑efficient roadside multimodal perception. The dataset and benchmark codes are available at https://github.com/ASU‑Suo‑Lab/RESOLVE.
Authors:Kyle Kramer
Abstract:
Residual connections add every sublayer's proposed update with a fixed coefficient of one; the network never evaluates whether an update is reliable before committing it. Drawing on the human‑factors principle of independent verification, we introduce Review Residuals, which scale each update by a learned, input‑dependent gate conditioned on both the current state and the proposed update: h_l = h_l‑1 + r_l u_l with r_l = sigmoid(W[RMSNorm(h_l‑1), RMSNorm(u_l)]). Conditioning the gate on the update is the property that distinguishes it from prior gated and scaled residuals. We report two findings. First, a depth‑stability result: a convex (Highway‑style) form of the gate reintroduces vanishing gradients and fails to train beyond ~20 layers, whereas the additive, identity‑preserving form trains stably at all depths we tested. Second, an emergence‑with‑scale result: trained from scratch across five sizes (60M‑1B parameters, multi‑seed), Review Residuals show no advantage at small scale but at 590M significantly outperform both a parameter‑matched Highway gate and a parameter‑matched standard residual (p<0.05), with a larger advantage at 1B. The benefit grows with model size rather than shrinking.
Authors:Sairam VCR, Varun Gopal, Poornima Jain, Vineeth N Balasubramanian, Muhammad Haris Khan
Abstract:
Real‑world detectors for autonomous driving, surveillance, and robotics must handle domain‑shifts under strict latency and memory constraints, yet existing source‑free object detection (SFOD) methods rely on heavyweight architectures that prioritize accuracy alone. We show this trade‑off is unnecessary: building on YOLOv10, an NMS‑free dual‑head detector, we achieve state‑of‑the‑art adaptation accuracy while being faster and more compact. We observe that directly applying vanilla mean‑teacher self‑training to dual‑head detectors leads to suboptimal adaptation performance due to two key factors. First, simple pseudo‑label generation strategies, such as using a single head or directly combining high‑confidence predictions from both heads, yield suboptimal supervision under domain‑shift. We propose DHF (Dual‑Head Pseudo‑Label Fusion) which selectively admits one‑to‑one (O2O) and one‑to‑many (O2M) head predictions, preserving precision and recovering missed objects. Second, we observe domain‑shift collapses multi‑scale feature discriminability. We propose the use of our MARD (Multi‑scale Adaptive Representation Diversification) loss which mitigates this by enforcing detection‑aware variance and covariance constraints on multi‑scale feature maps. Both modules are training‑time only, leaving inference unchanged. Across domain‑shift benchmarks, our method, RT‑SFOD yields 1.4 to 3.5% mAP gains, 1.3× higher throughput, with ～2× fewer parameters than prior state‑of‑the‑art SFOD methods, thus advancing the Pareto frontier of the speed‑accuracy‑model size trade‑off. We report main results with YOLOv10, and demonstrate generalizability with additional YOLO‑ and DETR‑based dual‑head detectors. Code is available here: https://github.com/Sairam13001/RT‑SFOD/
Authors:Kyuhwan Yeon, Benjamin Ramtoula, Daniele De Martini
Abstract:
Most end‑to‑end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street‑level visual context anchored to the intended driving route, providing visual‑spatial foresight independent of real‑time sensors. We propose a memory augmentation module featuring a dual‑memory architecture and an adaptive memory gate, which can be easily integrated into existing end‑to‑end approaches. This design pairs a contextual memory for retrieved priors with a persistent fallback memory, and dynamically regulates the influence of memories based on current state compatibility. Evaluated on the NAVSIM‑v2 benchmark, our approach consistently improves performance across diverse end‑to‑end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual‑memory design ensures safe fallback when the retrieved priors themselves become unreliable. Our project page is available at https://ori‑mrg.github.io/PriorEye.
Authors:Junha Jung, Minbyul Jeong, Suhyeon Lim, Sungwook Jung, Jaehoon Yun, Taeyun Roh, Mujeen Sung, Jaewoo Kang
Abstract:
Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post‑training pipelines remain predominantly outcome‑centric, relying on final answer correctness or sequence‑level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early‑stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning‑aware Policy Optimization (MRPO), an RL algorithm that incorporates step‑wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3‑VL‑8B‑Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT‑Vision‑34B by 2.79 points. Moreover, MRPO reduces early‑stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis‑lab/MRPO
Authors:Ruijia Zhang, Jiacheng Zhu, Hanqing Zhu, Laixi Shi
Abstract:
Low‑rank adaptation (LoRA) and its variants enable parameter‑efficient fine‑tuning of large language models under the supervised fine‑tuning (SFT) paradigm. However, their efficacy and behavior under Reinforcement learning with verifiable rewards (RLVR) are less well understood. In particular, two structurally initialized LoRA variants, PiSSA and MiLoRA, which outperform standard LoRA under SFT, can underperform standard LoRA under RLVR and may even exhibit training instability. These observations suggest that how to initialize the low‑rank matrices in RLVR remains unclear. In this work, we develop a theoretical analysis of LoRA in RLVR, showing that orthonormal initialization achieves the minimal gap between LoRA outcome and that of full fine‑tuning. Guided by this insight, we propose geometry‑preserving orthonormal initialization for low‑rank adaptation in RLVR, leading to two new variants, RLPO and RLMO. Experiments on mathematical reasoning benchmarks show that the proposed orthonormal initialization stabilizes RLVR training and outperforms standard LoRA, contrasting with PiSSA and MiLoRA. Finally, our unified analysis for LoRA initialization also explains why PiSSA and MiLoRA can underperform in RLVR, which may be of independent interest. Code and checkpoints are publicly available at https://github.com/Richard‑ZZZ/geometry‑preserving‑orthonormal‑init‑rlvr.
Authors:Parker Glenn, Alfy Samuel
Abstract:
Language model systems built around proprietary APIs often operate on a token‑based cost model. This becomes prohibitively expensive in the context of large databases, where LM‑enhanced relational operators can incur costs exceeding 10,000 for a single set of experiments, hindering thorough research and practical deployment. In this paper, we demonstrate that quantized, open‑weight models running locally on just 16GB of VRAM can match or exceed the accuracy of closed‑source counterparts at lower latency and a fraction of the price, challenging the prevailing assumption that closed‑source LM APIs are necessary for effective LM‑database integration. We present and analyze the key system optimizations required to efficiently deploy these open‑weight models within an LM‑DB system. By integrating these local models into the BlendSQL v0.1.0 framework, we demonstrate a 390x reduction in overall costs and 3.8x reduction in latency compared to a proprietary LM API. We make our code available at https://github.com/CapitalOne‑Research/play‑by‑the‑type‑rules/tree/main/sembench.
Authors:Xianda Zheng, Huan Gao, Meng-Fen Chiang, Michael Witbrock, Kaiqi Zhao, Shangyang Li
Abstract:
Despite recent progress, the reasoning capabilities of large multimodal language models (MLLMs) remain fundamentally constrained by static supervision, where fixed prompts, rules, or reward models provide non‑adaptive guidance throughout training. Such static signals are often sufficient to enforce output formats, but fail to shape the underlying reasoning process, leading to brittle generalization and performance saturation in complex decision‑making tasks. We propose Evo‑PI, a principle‑centric learning framework that treats reasoning principles as explicit, language‑based supervision signals that can be generated, evaluated, and iteratively evolved. Instead of relying on fixed rewards, Evo‑PI enables a co‑evolutionary loop in which principles guide model reasoning, while model behaviors in turn refine the principles that supervise them. This dynamic alignment mechanism allows supervision to progressively adapt to the model's reasoning deficiencies. We instantiate Evo‑PI in medical visual question answering as a high‑stakes testbed requiring structured visual‑textual reasoning. Across eight benchmarks and multiple model backbones, Evo‑PI consistently improves reasoning accuracy, achieving gains of up to 24.6%. Our results suggest that evolving principle‑guided supervision offers a scalable and general paradigm for training expert‑aligned reasoning in MLLMs. Code is available at https://github.com/zhengxianda/Evo_PI.
Authors:David Montalvo-García, Nicolás Gaggion, María J. Ledesma-Carbayo, Enzo Ferrante
Abstract:
Graph‑based cardiac segmentation with implicit anatomical correspondences provides topological guarantees and population‑level analysis capabilities, but models trained on independent frames of image sequences exhibit temporal discontinuities that affect reliable clinical measurements, particularly in cardiac ultrasound. In this work, we introduce self‑supervised temporal regularization as a post‑training refinement stage that exploits the temporal coherence in image sequences to enforce consistent cardiac segmentation and motion estimation over time, without requiring per‑frame annotations. By penalizing velocity and acceleration discontinuities across consecutive frames, our method achieves temporally consistent segmentations while maintaining the learned anatomical correspondences. We further leverage these correspondences to automatically map landmarks to the AHA 17‑segment clinical standard, enabling standardized regional assessment and detection of pathological myocardial motion patterns. Validation on CAMUS dataset demonstrates the clinical utility of combining temporal consistency with automatic regional mapping. The code is publicly available at https://github.com/david‑montalvoo/MaskHybridGNet‑TempReg
Authors:Ziyuan Liu, Ruifei Zhu, Ouqiao Ma, Yuantao Gu
Abstract:
Remote sensing change detection (CD) traditionally focuses on pixel‑level binary segmentation, which identifies where changes occur but neither what nor why. To bridge this semantic gap, we introduce JL1‑CC&QA, a multi‑task benchmark that extends the JL1‑CD dataset with two complementary annotation layers: change captioning (CC) and change question answering (QA). Built upon 5,000 bi‑temporal image pairs acquired by the Jilin‑1 satellite at 0.5‑0.75m ground sample distance, the benchmark comprises: (i) JL1‑CC, providing 17,021 quality‑verified captions that describe diverse land‑cover transformations; and (ii) JL1‑QA, offering 20,060 question‑answer pairs across eight question types, enabling fine‑grained, interactive interrogation of surface changes. All annotations are produced via a three‑stage pipeline consisting of multi‑modal large language model (LLM) generation, vision‑grounded LLM judging, and human expert verification. We hope that JL1‑CC&QA, as a benchmark unifying binary change masks, change captions, and change‑oriented QA over the same image set, will serve as a valuable resource for the community to advance multi‑task change understanding in remote sensing. The dataset is available at https://github.com/circleLZY/JL1‑CD.
Authors:Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek
Abstract:
Explainable AI (XAI) methods have demonstrated significant success in recent years at identifying relevant features in input data that drive deep learning model decisions, enhancing interpretability for users. However, the potential of XAI beyond providing model transparency has remained largely unexplored in adjacent machine learning domains. In this paper, we show for the first time how XAI can be utilized in the context of federated learning. Specifically, while federated learning enables collaborative model training without raw data sharing, it suffers from performance degradation when client data distributions exhibit statistical heterogeneity. We introduce FedXDS (Federated Learning via XAI‑guided Data Sharing), the first approach to utilize feature attribution techniques to identify precisely which data elements should be selectively shared between clients to mitigate heterogeneity. By employing propagation‑based attribution, our method identifies task‑relevant features through a single backward pass, enabling selective data sharing that aligns client contributions. To protect sensitive information, we incorporate metric privacy techniques that provide formal privacy guarantees while preserving utility. Experimental results demonstrate that our approach consistently achieves higher accuracy and faster convergence compared to existing methods across varying client numbers and heterogeneity settings. We provide theoretical privacy guarantees and empirically demonstrate robustness against both membership inference and feature inversion attacks. Code is available at https://github.com/MaxH1996/FedXDS.
Authors:Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti
Abstract:
While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open‑source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI‑text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open‑source the STEB code base at: https://github.com/rrivera1849/STEB.
Authors:Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo, Thanh-Ha Le
Abstract:
Remote photoplethysmography (rPPG) estimates physiological signals from facial videos by analyzing subtle pulse induced skin color variations. Despite recent progress, existing self‑supervised rPPG methods mainly reconstruct masked pixels or low‑level visual representations, which can bias the model toward facial appearance rather than latent physiological dy namics. Moreover, most recent Mamba‑based approaches scan facial video tokens only in chronological order, limiting their ability to exploit the cyclic structure of pulse signals. To ad dress these limitations, we propose RhythmJEPA, a rhythm structured joint‑embedding predictive learning framework for rPPG. Instead of reconstructing RGB frames, RhythmJEPA predicts latent teacher representations from masked facial videos, thereby encouraging physiology‑aware representation learning in the embedding space. To explicitly model pulse‑related tem poral structure, we introduce a Cyclic Rhythm‑State Plan ner (CRSP), which estimates frame‑wise latent physiological states and decodes the most plausible cyclic state path via dynamic programming with a constrained transition grammar. Guided by the decoded states, we further design a Dual Order Mamba Encoder (DOM), which combines conventional chronological scanning with state‑ordered scanning to capture both local temporal continuity and long‑range rhythm‑consistent dependencies. Finally, a lightweight Spatial Pulse Mixer (SPM) extracts compact pulse‑sensitive facial tokens with a favorable balance between complexity and performance. Experiments on PURE, UBFC‑rPPG, and MMPD show competitive performance over representative rPPG methods. The codes are available at https://github.com/deconasser/RhythmJEPA.
Authors:Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang, Quande Liu, Xintao Wang, Pengfei Wan, Kun Gai, Xihui Liu
Abstract:
Video World Models are interactive video generation models that predict future world states based on user actions and history video frames. A critical challenge in video world models is the lack of memory, causing inconsistent generated scenes over extended durations. Previous methods explored rule‑based context frame retrieval as memory, but they fail to generalize in scenarios with scene occlusions and dynamic objects. We propose MemLearner, a learning‑based adaptive context query method using query tokens to bridge context and predicted tokens. By leveraging the video generation model itself for context querying, MemLearner exploits pre‑trained visual priors without training additional modules from scratch, and incorporates efficient strategies for training and inference. We collect a dataset of long videos with scene occlusions and dynamic objects, paired with camera pose annotations, and propose a multi‑dataset training strategy leveraging both annotated rendered and unannotated real‑world videos. Extensive experiments demonstrate that MemLearner significantly outperforms prior video world models in terms of scene consistency and memory, particularly under challenging occlusion and dynamic scenarios.
Authors:Jingbo He, Michael Färber, Roberto Calandra
Abstract:
For robots manipulating open‑world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a robot‑collected touch‑vision‑language dataset with 29,279 tactile frames from full robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held‑out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame‑random splits can place near‑duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact‑sequence overlap reduces tactile‑to‑text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held‑out‑material Recall@1 at 25.1 +/‑ 6.1% averaged over three held‑out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw‑pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT‑trained embeddings improve category probes on unseen materials. RCT makes contact‑sequence‑aware, held‑out‑material evaluation reproducible and exposes novel‑material generalization as a central challenge for robotic tactile perception. The RCT dataset is open‑sourced at https://faerber‑lab.github.io/RCT/
Authors:Guanchen Lu, Yajuan Dun, Yi Zhou, Letian Tao, Jingliang Duan, Jie Li, Guofa Li
Abstract:
Scalable reinforcement learning has popularized high‑throughput sampling architectures, which significantly compresses the training time for off‑policy methods in robotic locomotion. However, the rapid increase of data volume and update frequency undermines the stability of value‑based methods and diminishes the plasticity of policy networks. To address these challenges, this work presents FastDSAC, a fast and high‑performance variant of the Distributional Actor‑Critic algorithm designed for parallel sampling scenarios. Specifically, we introduce a truncated Gaussian distribution to approximate the learned policy, which effectively excludes out‑of‑distribution actions that strain target value estimation while keeping necessary stochasticity for exploration. The proposed action constraint functions as an implicit regularization, which counteracts the plasticity loss typically caused by aggressive gradient updates. This preservation of network adaptability enhances sample efficiency, particularly in scenarios with a high update‑to‑data ratio, and accelerates the early training process. In contrast to prior fast reinforcement learning approaches that rely on discrete value distributions, our method utilizes a continuous Gaussian representation equipped with adaptive variance regulation, which improves value estimation accuracy by sampling confident and informative transitions. Extensive experiments on MuJoCo Playground and HumanoidBench demonstrate that FastDSAC not only stabilizes the overall training process but also achieves superior asymptotic performance and faster convergence compared to state‑of‑the‑art baselines.
Authors:Haoming Liu, Yuanhe Guo, Yijia Cao, Shenji Wan, Hongyi Wen
Abstract:
Diffusion models have emerged as a dominant paradigm in generative modeling, enabling high‑fidelity sampling from complex data distributions. Despite impressive capabilities, controlling diffusion models to produce outputs aligned with user intent remains an open challenge, especially when balancing global coherence with local precision. Existing control mechanisms vary in the granularity of their conditioning signals. For example, textual prompts guide generation globally through high‑level semantics, while ControlNet‑like approaches secure precise local structure via dense conditions. In this work, we introduce Histogram‑constrained Image Generation (HIG), a novel control mechanism that falls into the middle ground of control granularity. Our framework enforces user‑specified distributional constraints (e.g., color histograms or latent token distributions) during the generation process with exact precision. We model such control as an optimal transport (OT) problem and apply explicit guidance transformations during sampling, thereby driving the diffusion trajectory to align with the desired histogram. We demonstrate the versatility of HIG across diverse applications, including constrained generation via color/latent histograms and high‑capacity information embedding through histogram‑level encoding. Our findings underscore the promise of distributional control, a flexible and interpretable control scheme that is fully compatible with existing control mechanisms, diversifying the hybrid strategies for controllable image generation. Our project page is available at: https://maps‑research.github.io/hig/.
Authors:Ruiqi Xu, Daniel Aliaga
Abstract:
Despite advances in indoor scene generation, synthesizing coherent building exteriors consistent with generated interiors remains largely unexplored. Existing methods can generate floor plans and wall layouts but typically stop at a structural shell, lacking stylistically consistent facades and roofs. Completing these exteriors is challenging because the footprint, wall geometry, and opening semantics must remain fixed‑constraints that unconstrained generative models often violate. We introduce ShellMaker, a language‑guided exterior completion framework that operates under these structural constraints. Given a building scaffold and a text style prompt, ShellMaker generates a complete exterior mesh with PBR materials by combining parametric roof generation, LLM‑based part‑aware prompt refinement, joint wall‑roof material retrieval, and geometry‑aware assembly. Operating on a format agnostic scaffold representation, ShellMaker generalizes to indoor generators, CityGML, and CAD inputs, while maintaining structural consistency and improving architectural coherence over retrieval and unconstrained generative baselines. The project page is available at https://ruiqixu37.github.io/ShellMaker_web/
Authors:Ke Wang, Xiaoyi Pan, Zhaoyu Gu, Xiaofeng Ai, Zhiming Xu, Feng Zhao, Shunping Xiao
Abstract:
Synthetic aperture radar automatic target recognition (SAR ATR) is critical for Earth observation and defense, but its practical deployment is constrained by scarce annotated training data. Self‑supervised pre‑training alleviates this label bottleneck, yet prevailing Transformer architectures incur prohibitive quadratic computational complexity, and conventional universal masking neglects the unique electromagnetic scattering properties intrinsic to SAR imagery. To address these limitations, we propose SAMBA (Scattering‑Guided Bidirectional Mamba), an efficient self‑supervised pre‑training foundation model for SAR target interpretation. Our framework features three core innovations: (i) a linear‑complexity Mamba encoder with a mid‑sequence class token to mitigate computational bottlenecks; (ii) a three‑level hierarchical Scattering‑Guided Masked Autoencoder (SG‑MAE) masking strategy guided by SAR physical priors, aligning the pretext task with SAR's intrinsic imaging mechanism; (iii) a lightweight SpatialMix feature interaction module to enhance cross‑region feature fusion. We also design a two‑stage cross‑domain pre‑training pipeline to optimize the overall pre‑training process. Extensive evaluations demonstrate that SAMBA consistently delivers superior performance across all pre‑training configurations, with substantially fewer parameters than both CNN and Transformer baselines. Compared with the default masking strategy in standard MAE, the proposed SG‑MAE strategy further boosts the model's few‑shot transfer capability. Benchmarking on seven downstream datasets covering classification and detection tasks shows SAMBA achieves state‑of‑the‑art (SOTA) performance on most metrics, fully validating its robust generalizability across diverse SAR interpretation tasks. Source code and pre‑trained weights are publicly available at https://github.com/mynswkk/SAMBA.
Authors:Yuxiang Xie, Qi Lv, Jianming Xing, Zijian Hong, Xiang Deng, Weili Guan, Liqiang Nie
Abstract:
Vision‑language models achieve strong general perception but often struggle with the spatial reasoning required for embodied tasks. We present RoboSpatialBrain, our submission to the RoboSpatial Challenge at the Embodied Reasoning in Action Workshop, CVPR 2026, built on RoboBrain2.5‑8B‑NV. RoboSpatialBrain combines two training‑free, inference‑time mechanisms: a forced <think> prefix activation strategy paired with a task‑specific post‑prompt that elicits deliberate reasoning on context and compatibility tasks, and an explicit reference‑frame redirection pipeline that resolves camera‑centric and object‑centric ambiguity for context tasks. We additionally explore fine‑tuning RoboBrain2.5 on compatibility data and present a detailed analysis of its interaction with prompting. RoboSpatialBrain achieved first place in the RoboSpatial Challenge, with an overall success rate of 80.9% on RoboSpatial‑Home. Code is available at https://github.com/YuxiangXie2003/RoboSpatialBrain.
Authors:Md Raqib Khan, Santosh Kumar Vipparthi, Subrahmanyam Murala
Abstract:
Despite rapid progress in learning‑based stereo matching, high accuracy is often achieved at the cost of heavy backbones and computationally intensive 3D cost volume processing, resulting in substantial memory and runtime overhead. More critically, these methods frequently struggle to generalize across domains, limiting their practical deployment. We present LiteMatch, a lightweight stereo matching framework that achieves strong zero‑shot generalization through cost volume stabilization‑without expensive 3D convolutions. LiteMatch employs two complementary encoders: a Cross‑View Correspondence Encoder (CVCE) to capture global cross‑view interactions, and a High‑Frequency Encoder (HFE) that enhances fine structural details via FFT‑based frequency cues. To stabilize the cost volume, we introduce the Cost Volume Consistency Loss (CVC‑Loss), a voxel‑wise binary cross‑entropy objective applied to softmax‑normalized cost distributions. By encouraging sharp and unimodal disparity probabilities, CVC‑Loss promotes stable cost distributions and enables rapid convergence. A lightweight refinement module further produces sharp full‑resolution disparities with low‑iteration updates, avoiding heavy recurrent refinement. With a flexible design ranging from 3.36M to 9.58M parameters, LiteMatch achieves exceptional zero‑shot generalization, delivering competitive EPE and D1 performance across Scene Flow, KITTI, Middlebury, ETH3D, and DrivingStereo. Our results establish that lightweight architectures can indeed generalize across domains without sacrificing accuracy. \hrefhttps://mdraqibkhan.github.io/Litematch\textcolorblueCode
Authors:Nikolai Röhrich, Julian Gleißner, Ahmed H. A. Ibrahim, Silvan Mertes, Tobias Huber
Abstract:
Semantic segmentation models struggle with data sparsity and rare or visually diverse regions, e.g., dense regions or small objects in aerial or autonomous mobility data. While synthetic augmentation is an appealing solution, directly generating new labeled data risks misalignment of labels and generated pixels. Existing solutions to this problem often rely on external models, or employ coarse heuristics such as indiscriminately augmenting all foreground objects or entire backgrounds, which wastes capacity on uninformative pixels. To address this, we propose an uncertainty‑guided synthetic context augmentation strategy that strictly preserves label validity and efficiently maximizes pixel informativeness per synthetic sample ‑ no external guardrails required. Using a baseline segmenter's predictive entropy, we identify uncertain semantic regions and inpaint only the complementary visual context. When fine‑tuning the segmenter on this synthetic data, we compute the loss only over the original pixels, excluding inpainted regions. This focuses learning on the unmodified, uncertain regions while presenting them in novel contexts. We demonstrate substantial mIoU gains on Cityscapes, UAVID, and BDD100K with the largest gains on rare and difficult classes such as buses, trains, or (from the aerial perspective) cars. Our results demonstrate that uncertainty‑guided context augmentation is a highly effective lever to improve segmentation performance on complex datasets, with code provided at https://github.com/XITASO/Preserve‑the‑Hard‑Regenerate‑the‑Rest.
Authors:Asif Hanif, Mohammad Yaqub
Abstract:
Audio‑Language Models (ALMs) achieve strong zero‑shot performance by aligning audio with textual class descriptions. Although prompt learning improves accuracy on base classes through few‑shot supervised adaptation, we observe a critical trade‑off: it often degrades performance on novel classes, sometimes falling below zero‑shot accuracy. This exposes a base‑to‑novel generalization gap in prompt learning for ALMs. To address this issue, we propose ZEBRA (Zero‑shot Entropy‑Regularized Prompt Learning for Base‑to‑Novel Generalization), a plug‑and‑play framework that fuses zero‑shot logits with prompt‑learning logits, and employs self‑entropy regularization to reduce overfitting to base classes. Experiments across multiple audio classification datasets show that ZEBRA consistently improves novel‑class performance while maintaining strong base accuracy, significantly reducing the base‑to‑novel gap compared to standard prompt learning. The code is available at: https://github.com/asif‑hanif/zebra.
Authors:Yilin Liu, Pradeep Jayaraman, Chinthala Reddy, Xiang Xu, Hooman Shayani
Abstract:
Boundary Representation (B‑rep) is the most commonly used data format in Computer‑Aided Design (CAD) due to its analytical precision and direct support for parametric editing. However, its heterogeneous structure‑‑continuous parametric geometry combined with discrete topological graphs‑‑poses fundamental challenges for deep learning. Existing methods often predict the heterogeneous B‑rep graph directly, using fixed‑size padding or sequential tokenization to handle varying primitive counts. These approaches struggle with the combinatorial complexity of CAD models. Furthermore, the discrete, non‑differentiable nature of graph data prevents end‑to‑end optimization of geometry and watertightness. In this work, we introduce DualBrep, a novel continuous representation that unifies B‑rep geometry and topology within a fully structured Euclidean domain. DualBrep encodes a CAD model using dual scalar fields: a Signed Distance Function (SDF) representing global shape geometry, and an Unsigned Distance Field (UDF) implicitly encoding topological structure via a Voronoi partitioning of surface elements. Rather than processing these fields independently, we compress them into a single latent space. While the dual‑field formulation alone provides a flexible, primitive‑free segmentation signal that adapts to arbitrary face counts and surface types, the shared latent makes generation tractable. A Flow Matching model can sample geometry and topology jointly from a single code, avoiding the error accumulation that plagues sequential B‑rep predictors. Finally, a neural rebuilder extracts explicit B‑rep models‑‑comprising both prismatic and free‑form primitives‑‑directly from our continuous dual fields. We demonstrate that DualBrep is a robust backbone for CAD learning, achieving strong performance in point cloud reverse engineering and generative modeling via latent flow matching.
Authors:Clément Fuchs, Tim Bary, Benoît Macq
Abstract:
Conformal predictions have attracted significant attention in the field of uncertainty quantification, mainly because of their strong marginal coverage guarantees. Full conditional guarantee is not an attainable goal, a well known fact in conformal predictions literature. As a result, several approaches have tried to approximate this behavior by adapting the conformal sets of test‑time samples according to their similarity to calibration examples. Although the latter has gained traction and shown impressive performances for regression problems, its application to image classification remains under‑explored. We conduct an extensive benchmarking on natural image classification tasks with vision‑language models (VLMs), using our open source implementation of a recent localized conformal prediction algorithm. We show that straightforward usage of the cosine similarity between test‑time and calibration visual features, an intuitive choice for VLMs, is not sufficient to improve over the non‑local baselines. In response, we propose a simple non‑linear transformation of the cosine similarities, which conserves marginal coverage guarantees and achieves statistically significant mean set sizes reduction. Code is available at https://github.com/cfuchs2023/lcp‑vlm/.
Authors:Zikang Yan, Xiao Wang, Qingquan Yang, Zhendong Yang, Gaoting Chen, Zehua Chen, Bo Jiang, Jin Tang, Guosheng Xu
Abstract:
Accurate modeling of the divertor temperature field is essential for preventing material melting and damage and for extending the service life of fusion devices. However, conventional numerical methods, such as the Finite Element Method (FEM), are computationally expensive and therefore unsuitable for real‑time applications. Therefore, a fast and generalizable method is required for real‑time reconstruction of the divertor temperature field and subsequent real‑time control. To address the above issue, we propose a Physics‑aware Neural Operator Transformer (PNOT) to characterize the spatiotemporal evolution of the divertor temperature field. It models boundary heat‑flux relations as a structured graph and employs graph attention to explicitly capture spatial physical dependencies. Inspired by physics‑aware attention, we further develop a physics‑aware neural operator module to aggregate query points with similar physical conditions via slicing and model heat diffusion, while a gradient‑constrained Sobolev regularization loss enforces consistency between function values and their derivatives. Experimental results show that these physical constraints improve prediction accuracy while preserving physical consistency. The source code of this paper will be released on https://github.com/Event‑AHU/OpenFusion
Authors:Xu Yan, Huiqun Wang, Chen Wang, Lei Ren, Di Huang
Abstract:
Masked autoencoding has emerged as a prominent paradigm for self‑supervised learning on 3D point clouds, achieving competitive performance across downstream tasks. Unlike its 2D counterpart, 3D masked autoencoding directly reconstructs spatial coordinates, making it inherently susceptible to positional leakage. In this work, we identify that the decoder in existing 3D MAE frameworks tends to over‑rely on positional information, which weakens semantic representation learning and leads to suboptimal feature quality. To address this issue, we propose MPL‑MAE, a masked point learning framework that mitigates positional over‑reliance while enhancing the utilization of encoder features. Specifically, we introduce a recalibrated positional embedding module that suppresses metric‑dominant coordinate signals while preserving geometric topology, together with a gated positional interface module that dynamically regulates positional injection during reconstruction. These designs promote a more balanced interaction between spatial priors and semantic features, yielding robust and informative representations. Extensive experiments across downstream tasks demonstrate that MPL‑MAE consistently achieves competitive performance, validating its effectiveness. Code is available at https://github.com/yanx57/MPL‑MAE.
Authors:Johan Land
Abstract:
Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong ‑ making selection among candidates, not just generation, the central challenge. I present a solver for ARC‑AGI‑2, a few‑shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context‑preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long‑context prompt. Unlike self‑consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi‑private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task ‑ the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT‑5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.
Authors:Kartik Bali, Roland Aydin
Abstract:
Identifying and grounding precise geometric entities, such as edges, planar regions, and curved surfaces within 3D objects, is foundational to computer‑aided design (CAD), robotic manipulation, and scientific simulation. Although modern Vision Language Models (VLMs) have advanced referring segmentation (RIS) in the image domain, extending such language‑driven localization to structured 3D geometry is substantially harder. The 3D object appearance is highly sensitive to viewpoints; a single perspective may render a target entity clearly observable, while another may suffer from severe occlusion or foreshortening. In this work, we attempt to solve these challenges with MV‑GEL (Multi‑View Geometric Entity Localization), a framework for localizing fine‑grained geometric entities on polygon meshes from natural language queries. Our key insight is that reliable CAD entity (i.e., faces, edges or solids) localization depends on selecting views that make the queried entity maximally interpretable. We introduce GELviews, a prompt‑conditioned ranking module that prioritizes viewpoints based on language prompted observability of geometric CAD entities. Selected views are processed by a VLM‑based reasoning segmentation backbone, and predicted masks are lifted to the corresponding meshes via geometry‑aware ray casting. Our framework is completely CAD agnostic and relies only on 3D meshes. Experiments show up to a 1.7X improvement in face‑level IoU and over 4.5X gains in edge‑level F1 compared to vanilla baselines, substantially outperforming CLIP‑based and random view sampling, particularly for thin and view‑sensitive structures.The dataset, code and trained checkpoints are available at https://github.com/kbali1297/MV‑GEL.
Authors:Rafael Ayala
Abstract:
We report the exact value of the number of labeled partially ordered sets (equivalently, labeled T_0 topologies) on 19 points, P(19) = 646099441937791106493755218560442089979, a 39‑digit integer extending OEIS A001035, whose largest previously computed term was P(18) (Brinkmann and McKay). By the Stirling transform we also obtain the number of labeled topologies on 19 points, A000798(19) = 689054943207246404281592791142107048261. Our route is the Erné‑Stege moment reduction, which expresses P(19) through a few sums of antichain counts over the posets on at most 16 points. All of these are available from the posets on at most 15 points (whose number is catalogued, and which standard software generates on demand), except a single moment over the 16‑point posets. That moment is obtained not by enumerating the 16‑point posets but by inserting a single element into the 15‑point ones, with a per‑parent kernel that advances the sum at the cost of computing the parent's own antichain count. The result passes several independent checks, among them the residue predicted by the modular periodicity of A001035 and the recovery from the same sweep of the known count P(16) and the Erné‑Stege moments G(16,1) and G(16,2). We also report the moments G(16,3) and G(16,4), the latter an input to the analogous computation for 20 points.
Authors:Wenhao Li, Jinhao Dong, Hailin Zhang, Wenhang Shi, Wei Lu, Xiaoyong Du
Abstract:
Long‑context Large Language Model inference is severely bottlenecked by the massive Key‑Value (KV) cache, yet existing sparse attention methods often suffer from static fixed‑budget (Top‑k) retrieval or rely on proxy scores that are computationally expensive and biased. To address these limitations, we propose RaBitQCache, a novel sparse attention framework that utilizes randomized rotated binary quantization and high‑throughput binary‑INT4 arithmetic to efficiently estimate attention weights. Our proxy score serves as an unbiased estimator with a proven error bound, enabling adaptive Top‑p retrieval that dynamically adjusts the token budget based on actual attention sparsity. We further implement a hardware‑aware system with asynchronous pipelining and lazy updates to mask overhead. Evaluations demonstrate that RaBitQCache significantly accelerates inference and reduces memory I/O while preserving generation quality compared to state‑of‑the‑art baselines. Code is available at https://github.com/Sakuraaa0/RaBitQCache.git.
Authors:Jiawei Xu, Qiangqiang Zhou, Zhouping Li, Yanjiao Shi, Yugen Yi, Jiacong Yu
Abstract:
In recent years, most research on multimodal salient object detection (SOD) and camouflaged object detection (COD) typically aims to improve performance through complex cross‑modal feature fusion and decoding structures. However, this approach leads to an excessively large model parameter scale and often fails to deliver satisfactory detection performance due to structural redundancy. In contrast, the human visual process is able to efficiently perform salient and camouflaged object identification without such complex structures. This contrast raises an important question: Can we draw conceptual inspiration from the human visual process to achieve a simpler modeling strategy, and still realize accurate and efficient object detection? To answer this question, we propose HVPNet, a simple yet general bio‑inspired computational architecture. Drawing on the multi‑layered information integration of the retina as a conceptual metaphor, we designed a Retinal Integration Module (RIM), which effectively integrates multimodal features through a level‑specific multi‑stage integration strategy. To fully exploit these features, we further design a cortical decoder (CD) that breaks down the decoding process into low‑ and high‑level visual stages, abstracting the hierarchical processing in the human visual cortex. Benefiting from these designs, HVPNet can readily extend to seven tasks across four modalities. Without bells and whistles, it establishes an excellent accuracy‑efficiency trade‑off across 22 datasets spanning these seven tasks. Our code is available at https://github.com/jiaweiXu1029/HVPNet.
Authors:Dong Bi, Yongqi Zhao, Paul Kovacevic, Tomislav Mihalj, Ji Zhou, Jiayuan Gong, Arno Eichberger
Abstract:
Personalized driving can improve the user acceptance of automated driving systems. However, existing methods still provide limited support for translating natural‑language driving preferences, especially when such preferences are expressed implicitly, into executable and distinguishable driving behaviors. This paper proposes a large language model (LLM)‑supported personalized driving framework for highway lane‑change scenarios. The framework maps natural‑language driving commands to executable planning parameters in the open‑source Apollo automated driving stack according to three driving styles: aggressive, normal, and conservative. To establish this mapping, candidate planning parameters are evaluated based on the resulting lane‑change behaviors, and style‑specific parameter sets are constructed through clustering and style‑intensity ranking. For command interpretation, a retrieval dataset is constructed to support retrieval‑augmented generation (RAG), enabling LLM‑based interpretation of implicit user commands. Experimental results show that the derived parameter sets generate distinguishable personalized lane‑change behaviors, while RAG consistently improves preference interpretation, particularly for implicit commands. These results indicate the potential of integrating LLM‑based natural‑language interaction with Apollo to support personalized lane‑change behavior generation. The source code and the relevant datasets are available at: https://github.com/ftgTUGraz/LLM‑Personalized‑Driving.
Authors:Deniz Bickici, Michael Pabst, Shohei Mori, Dieter Schmalstieg
Abstract:
Open‑vocabulary 3D scene graph methods typically operate in two stages: first reconstruct, then enrich with vision‑language models, leaving the graph unqueryable during exploration. We argue that this sequential coupling is unnecessary and propose an asynchronous architecture in which lightweight online mapping runs concurrently with heavyweight semantic refinement. A probabilistic voxel‑based backbone maintains stable object identities incrementally, while background VLM agents progressively enrich the graph. This framework resolves duplicate object tracks through semantic loop closure, attaches fine‑grained visual attributes and derives spatial relations between objects. A multi‑target frame scheduler amortizes VLM cost by selecting a small set of informative frames that jointly cover multiple targets. The resulting scene graph is queryable during exploration and grows in semantic richness over time. Our method matches or outperforms existing open‑vocabulary 3D scene graph methods on semantic segmentation (ScanNet, Replica) and surpasses the prior state‑of‑the‑art across three visual grounding benchmarks (Sr3D+, Nr3D, ScanRefer) by 15.3 to 18.8 A@0.25. Project page: https://denizbickici.github.io/thinkgraphs/
Authors:Yao Shi, Kingfung Luo, Nan Tang, Yuyu Luo
Abstract:
Niche asset markets, such as Counter‑Strike 2 (CS2) weapon skins, are small, volatile, and heavily driven by community discussions and platform rules. These properties make them hard for traditional quantitative models, but provide an ideal testbed for studying how large language models (LLMs) turn unstructured text into trading actions. We present CSTrader, a multi‑agent framework for language‑grounded trading in the CS2 skin market. The system first integrates heterogeneous signals from various sources, then uses specialized agents for technical analysis, liquidity, events, and (reversed) sentiment, and finally applies risk control, transaction friction, and portfolio management agents to produce buy, sell, or hold decisions under realistic trading frictions. We build a live‑like evaluation environment with real CS2 data from a highly volatile period and evaluate several recent LLM backbones. Across models, CSTrader consistently outperforms both a falling market index (‑15.62%) and simple single‑prompt LLM baselines, achieving up to a 7.58% cumulative return with controlled risk. Ablation studies show that liquidity, reversed sentiment, and transaction friction agents are crucial for turning noisy language signals into stable profits, suggesting that niche, language‑driven markets are a useful benchmark for future language‑to‑action research. Code is available at: https://github.com/IatomicreactorI/CSGOTrading?tab=readme‑ov‑file#quick‑start
Authors:Seyed Alireza Molavi, Zhan Su, Yan Hu, Peyman Sheikholharam Mashhadi, Stefan Byttner, Prayag Tiwari
Abstract:
Composing independently trained LoRA adapters into a single large language model is useful for multi‑domain adaptation, especially when the original training data cannot be shared. A common approach is to use MoE‑style routing over LoRA experts, but for frozen pretrained adapters, soft weighted combinations can change the unit‑scale additive update under which each LoRA module was originally trained. We propose Hard‑Routed MoR‑LoRA, a two‑stage framework for composing frozen reasoning LoRA experts through unit‑scale hard selection. First, domain‑specific LoRA adapters are trained independently using reinforcement learning from verifiable feedback to obtain reasoning experts. Then, all experts are frozen, reasoning traces are distilled from them, and only a lightweight shared router together with a small attention LoRA is trained for integration. The router selects exactly one expert per token using hard top‑1 routing, while a straight‑through estimator enables gradient‑based training. Experiments across five benchmarks, multiple model scales, and additional model families show that Hard‑Routed MoR‑LoRA preserves expert behavior while requiring substantially fewer trainable parameters than soft‑routing mixture baselines. Our analysis further shows that normalized soft mixtures often concentrate most routing mass on a single expert, suggesting that hard unit‑scale routing provides a simple and efficient abstraction for frozen LoRA expert composition.
Authors:Jisung Park, Seohyeon Kang, Daeun Yoo, Eunsu Lee, Seoin Cho, Wooyeop Choi, Ian Choi, James R. Evan, Daesoo Kim, Sonia Gandhi, Minee L. Choi
Abstract:
Artificial intelligence is transforming our capability to solve biological challenges. In dimensionality bottleneck regimes exacerbated by high‑dimensional biological data, neural networks force distinct concepts into the lower dimensions known as superposition. Although this superposition is widely known to hinder interpretability, its impact on corrupting the geometry of latent spaces remains critically overlooked. Here, we utilized sparse autoencoders (SAEs) trained on over 100,000 multiplexed images of patient‑derived Parkinson's disease and healthy neurons to resolve superposition. This approach bypasses the mathematical non‑uniqueness of feature attribution by shifting to interpretable latent representation analysis. We theoretically and empirically demonstrate that superposition contaminates representational metric spaces, and thereby SAEs successfully recover geometric fidelity. By treating these geometrically purified representations as single‑cell state vectors, we adapted single‑cell RNA sequencing (scRNA‑seq) data analysis methodologies directly to the image domain. Finally, we introduce GW‑map, utilizing Gromov‑Wasserstein optimal transport to align these image representations with authentic scRNA‑seq data de novo. This coupling reconstructs hierarchical neuronal pathology pathways such as Calcium‑AIS scaffold, without reference spatial transcriptomics, establishing a scalable foundation for spatial biology. Code is available at https://github.com/jijihihi/Bio\_superposition
Authors:Binjie Zhang, Mike Zheng Shou
Abstract:
Tool‑augmented vision‑language models (VLMs) can solve multimodal, multi‑step tasks by calling external tools, yet they remain fragile in practice. Existing works have two common gaps. Supervised fine‑tuning (SFT) is built mostly on successful trajectories and offers little signal for recovery after tool failures, while sparse trajectory‑level RL rewards provide limited guidance on which step failed and how to repair it. We introduce ReGRPO (Reflection‑augmented Group Relative Policy Optimization), a framework that learns reflection‑guided correction in tool‑using agents. ReGRPO starts with a structured reflective data engine: we execute near‑miss actions to collect grounded failure observations, then build Reflection‑of‑Thought triplets (ErrorType, Evidence, FixPlan) paired with corrected actions for warm‑start SFT. We then optimize reflection tokens and corrective actions jointly within local trajectories using group‑relative advantages, and include a reflection‑cost term to reduce unnecessary reflection. Experiments on GTA and GAIA show that, under the same backbone and tool suite, ReGRPO consistently outperforms strong open‑source baselines and achieves the best results among the compared open‑source controllers. Code and RoT data are available at https://github.com/showlab/ReGRPO.
Authors:Stefanos-Iordanis Papadopoulos, Zacharias Chrysidis, Christos Koutlis, Symeon Papadopoulos, Panagiotis C. Petrantonakis
Abstract:
The proliferation of multimedia content on social platforms has fueled multimodal misinformation, where images are used to reinforce false claims. Consequently, Multimodal Fact‑Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated benchmarks that fail to capture the complexity of in‑the‑wild data. Furthermore, existing detection models rely on restricted intra‑modality consistency or unconstrained all‑to‑all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X‑POSE, a benchmark of real‑world, community‑annotated multimodal posts from X (formerly Twitter), augmented with full‑length news articles retrieved via VLM‑optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross‑attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state‑of‑the‑art specialized models and commercial VLMs. The code, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence‑triangulation
Authors:Dongyoon Hwang, Byungkun Lee, Dongjin Kim, Hyojin Jang, Hoiyeong Jin, Jueun Mun, Minho Park, Hojoon Lee, Hyunseung Kim, Jaegul Choo
Abstract:
Hierarchical Vision‑Language‑Action (VLA) models decouple high‑level planning from low‑level control to improve generalization in robot manipulation. Recent work in this paradigm uses 2D end‑effector trajectories predicted by a Vision‑Language Model (VLM) as explicit guidance for a downstream policy. However, state‑of‑the‑art low‑level policies operate in 3D metric space on point clouds, and feeding them 2D guidance that lacks depth forces each waypoint to be assigned the depth of whatever scene surface lies beneath it, producing geometrically distorted trajectories. We propose 3D HAMSTER, a hierarchical framework that closes this gap by having the planner directly output metrically reliable 3D trajectories. We augment a VLM with a dedicated depth encoder and a dense depth reconstruction objective to predict 3D waypoint sequences, which are directly integrated into a pointcloudbased low‑level policy. Across 3D trajectory prediction, simulation, and real‑world manipulation, 3D HAMSTER consistently outperforms proprietary VLMs and 2D‑guided baselines, with the largest gains under appearance‑altering shifts and unseen language, spatial, and visual conditions. The project page is available at https://davian‑robotics.github.io/3D_HAMSTER/.
Authors:Hyunsoo Lee, Inwoo Hwang, Young Min Kim
Abstract:
Generating diverse, coherent, and plausible content from partially given inputs remains a fundamental challenge for diffusion models. Existing approaches face clear limitations: training‑based approaches offer strong task‑specific results but require costly computation, and they generalize poorly across tasks. Training‑free approaches offer better efficiency, but they do not explicitly optimize over unobserved variables, leading to globally inconsistent results. To address these limitations, we introduce Accelerated Likelihood Maximization (ALM), a novel training‑free sampling strategy integrated into the reverse diffusion process that significantly extends the applicability of diffusion models beyond simple generation tasks. Unlike previous methods that implicitly influence missing regions through pre‑generated region constraints, we directly optimize the unobserved region during the sampling process, enabling globally coherent and plausible generation. Furthermore, we incorporate an acceleration strategy that significantly improves computational efficiency without sacrificing performance. Experimental results demonstrate that ALM consistently outperforms state‑of‑the‑art methods in various data domains and tasks, establishing a powerful paradigm for versatile content generation.
Authors:Yuan Wang, Wanxing Chang, Songtao Jiang, Shujian Gao, Xiaotian Zhang, Ruifeng Yuan, Weiwei Cao, Bowen Shi, Ling Zhang, Zuozhu Liu, Jianpeng Zhang
Abstract:
Traditional metrics for Medical Report Generation (MRG) predominantly rely on surface‑level n‑gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing AtomiMed, a universal, modality‑agnostic evaluation framework that decomposes complex medical narratives into a standardized, multi‑level hierarchy of Atomic Clinical Facts, encompassing Disease‑level entities and Attribute‑level descriptors, including location, morphology, and severity. By implementing an Agentic Cross‑Verification loop between ground‑truth and predicted reports, AtomiMed simulates a multi‑radiologist peer‑review process to verify clinical consistency, thus enabling the decoupled assessment of diagnostic detection and descriptive accuracy. To facilitate standardized evaluation, we introduce MRGEvalKit, an open‑source toolkit for automated hierarchical extraction, and curate OmniMRG‑Bench, a comprehensive multi‑modal benchmark covering X‑ray, CT, MRI, and Ultrasound. Extensive experiments on multiple expert‑annotated reader studies demonstrate that AtomiMed achieves significantly higher correlation with human radiologist judgment compared to traditional and model‑based metrics. Our code are release at https://github.com/Venn2336/MRGEvalkit
Authors:Hongmin Li
Abstract:
AI‑assisted research has entered a stage in which the central question is not only whether systems can generate hypotheses, run experiments, or produce manuscripts, but whether their scientific claims are calibrated to the evidence that supports them. This Perspective‑style paper develops a conceptual and methodological framework for evidence‑licensed claims in AI‑assisted research. Motivated by representative routes including specialized scientific foundation models, LLM research assistants, multi‑agent co‑scientists, AI Scientist pipelines, mathematical discovery agents, and self‑driving laboratories, it represents AI‑assisted research as five operators: hypothesis generation, model‑mediated consequence derivation, external validation, belief update, and claim calibration. The central claim is that calibration is not merely cautious wording but a mechanism for managing scientific assertion rights: evidence licenses some forms of speech and withholds others. The paper distinguishes linguistic, consequence‑based, interventional, and evidence‑licensed semantics; defines the claim‑evidence gap and epistemic debt; and treats minimal structural reconstruction across heterogeneous outputs as an upward form of claim calibration. AISim‑Cal is included as an illustrative synthetic dynamics exercise, not as an empirical forecast or benchmark. The resulting principles are: no claim without license, validation does not determine claim level, and automation amplifies the need for calibration. Reliable AI‑assisted research is therefore evaluated as a loop that generates hypotheses, derives testable consequences, accepts independent adjudication, updates beliefs, and outputs only evidence‑licensed claims.
Authors:Yibing Zhang, Xunpeng Yi, Qinglong Yan, Yeda Wang, Han Xu, Jiayi Ma
Abstract:
With the advancement of imaging technology, ultra‑high‑definition images have become increasingly essential in modern visual applications. However, existing multi‑focus image fusion remains largely confined to low‑resolution images and faces three major barriers in UHD scenarios, namely data availability, model adaptability, and deployment feasibility, which severely hinder its practical application. To shatter these barriers, first, we propose the UHD‑MFF dataset, the first large‑scale ultra‑high‑resolution multi‑focus fusion dataset. Second, we propose a scale‑specialized lookup‑table framework tailored for ultra‑high‑resolution images, termed as UMF‑LUT. It consists of Coarse‑Region Lookup Table (C‑LUT) and Detail‑Edge Lookup Table (D‑LUT). Specifically, C‑LUT performs joint queries of multiple gradient cues and semantic cues at low‑resolution scales to enable region‑level decision‑making. Also, D‑LUT operates at high‑resolution scales, leveraging efficient Laplacian cues to provide complementary edge‑level decision information. Such a design makes the model particularly well‑suited for ultra‑high‑resolution multi‑focus image fusion. Finally, it offers strong deployability with minimal computational overhead, enabling real‑time 4K multi‑focus fusion and showing promising potential for smartphone. Extensive experiments demonstrate that it outperforms SOTA methods in both visual fidelity and quantitative metrics. It effectively advances the development of multi‑focus image fusion toward ultra‑high‑resolution imaging scenarios. The code is available at https://github.com/zyb5/UHD‑MFF.
Authors:Xutian Li, Yifeng Zhu, Xianlin Zhao, Yanzhen Zou, Lu Zhang, Bing Xie
Abstract:
Large language models (LLMs) are increasingly used for software evolution, yet most interaction paradigms remain code‑centric and require manual context management and prompt iteration. We present FeatX, a feature‑oriented tool for editing software by editing features. Given an existing repository, FeatX extracts a hierarchical epic‑feature structure with explicit feature‑to‑code mappings, then invokes a three‑stage Evolution Agent to translate feature edits into code patches. The workflow is exposed through four coordinated panels. Across a controlled user study and replay experiments on 38 real‑world feature‑editing commits, FeatX significantly reduces cognitive load and improves usability compared with vanilla ChatGPT. It also achieves a 42.6% relative improvement in function‑level modification localization F1 over strong LLM baselines, at substantially lower cost (\0.07 in total). The tool and collected dataset are available at https://github.com/a496263365/FeatX/tree/demo, with a demonstration video at https://youtu.be/OZqKZ4Ii‑yM.
Authors:Dong Yeong Kim, JunGyu Lee, Jaewon Choi, June Young Seo, Myeongseop Kim, Jinwook Choi, Taek Min Kim, Young-Gon Kim
Abstract:
Real‑time video segmentation of the prostate in Transrectal Ultrasound (TRUS) is essential for image‑guided interventions. While conventional 2D methods suffer from inter‑frame inconsistencies by disregarding temporal context, 3D architectures incur prohibitive latency. To resolve this dilemma, we present a Temporally Consistent Learning Framework that distills temporal coherence into a 2D network during training, preserving single‑frame inference efficiency. Our design is driven by a key clinical observation: the prostate exhibits geometric stability, whereas the surrounding acoustic environment fluctuates due to physiological motion and transducer pressure. Because conventional temporal constraints propagate erroneous gradients from these unstable regions, we introduce a Confidence‑Weighted Temporal Consistency objective derived from optical flow warping residuals, selectively attenuating contributions from unreliable regions. Complementing this pixel‑wise constraint, a Dual‑scale Prototype Alignment Module enforces semantic coherence through contrastive optimization of local boundary and global semantic features. Furthermore, to eliminate the need for dense per‑frame video annotations, we employ geometric equivariance‑based pseudo‑labeling with knowledge distillation from a pretrained teacher. Extensive experiments on SUN‑SEG and our newly introduced TRUS‑V benchmark (2,679 frames) demonstrate state‑of‑the‑art accuracy and temporal consistency at real‑time speed. Code and dataset are available at https://github.com/DYDevelop/DTC‑TRUS.
Authors:Prakhar Dixit, Tim Oates
Abstract:
We propose Intelligent Schema Memory (ISM), a self‑evolving memory‑augmented system that improves mathematical reasoning for a frozen LLM under continual learning with hard episodic resets. ISM maintains a compact, self‑refined bank of strategy schemas learned from both successful and failed episodes, with symbolic tools that check intermediate steps and certify answers. Without updating model parameters, ISM outperforms passive, retrieval, and reflection baselines on MATH‑Hard and OlympiadBench, using 64% and 86% fewer schemas respectively than the strongest passive baseline. These results show that small, actively maintained, and verified strategy memories can support reliable continual mathematical reasoning under strict episodic isolation. The codebase is available at https://github.com/pdx97/ISM .
Authors:Raiyaan Abdullah, Shehreen Azad, Yogesh Singh Rawat
Abstract:
Multimodal large language models (MLLMs) have rapidly advanced video understanding, achieving strong zero‑shot and few‑shot recognition across standard benchmarks. Yet their ability to deny an action by recognizing when an activity is not happening despite strong contextual cues remains largely unexplored. We introduce UCF101‑AD, a large‑scale benchmark consisting of paired Action‑Presence and Action‑Denial clips, designed to evaluate this capacity for denial. Each negative video in UCF101‑AD preserves the same contextual and motion cues, including persons, objects, and locations, as its positive counterpart, but the defining action itself is explicitly absent. Evaluating 20 state‑of‑the‑art MLLMs reveals a consistent failure: models that exceed 85% accuracy on the positive action classes collapse below 50% on their action‑denial counterparts, indicating a strong inclination to affirm plausible actions rather than verify that they truly occur. This exposes a critical blind spot in modern video understanding: the inability to reason causally about whether a motion actually happens. To probe this issue, we explore a causal graph formulation, CausalAct, which expresses scene structure through natural‑language prompts linking context, interaction, and motion. Incorporating such causal cues substantially reduces false positives, demonstrating that denial is a learnable reasoning skill. UCF101‑AD provides a new lens for diagnosing and improving causal reasoning in multimodal models. Dataset and relevant code: https://github.com/raiyaan‑abdullah/Learn‑to‑Deny.
Authors:Jinyu Li, Xiao Wei, Bin Wen, Kai Li, Yuqin Lin, Xiaobao Wang, Longbiao Wang, Jianwu Dang
Abstract:
Spontaneous speech is a vital non‑invasive biomarker for Alzheimer's Disease (AD), yet many systems overlook non‑linear structural disruptions and clinical heterogeneity in pathological language. We propose a Multi‑View Gated Graph Attention Network that transcribes audio via Automatic Speech Recognition (ASR) to construct semantic, dependency, and co‑occurrence graphs, characterizing speech through a "content‑structure‑flow" framework. Notably, the co‑occurrence graph leverages Pointwise Mutual Information (PMI) from a normative corpus to quantify narrative logic and linguistic deviation. To address symptomatic diversity, an adaptive gated fusion mechanism dynamically integrates these views. Evaluated on the ADReSSo dataset, our model achieves 90.00% accuracy. Ablation results confirm that the PMI‑based graph and heterogeneity‑aware gating are essential for robust classification across diverse clinical populations. Our source code is publicly available at https://github.com/opeacc/AD.
Authors:Qianchu Liu, Sheng Zhang, Guanghui Qin, Jeya Maria Jose Valanarasu, Maximilian Rokuss, Mingyu Lu, Timothy Ossowski, Juan Manuel Zambrano Chaves, Cliff Wong, Peniel Argaw, Yashna Hasija, Mu Wei, Wen-wai Yim, Qin Liu, Zilin Jing, Jason Entenmann, Naoto Usuyama, Tristan Naumann, Hoifung Poon
Abstract:
As AI agents become increasingly capable of complex, long‑horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real‑world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment. The benchmark suite spans diverse workflows throughout the patient journey and a broad range of modalities. Each task is designed to replicate an end‑to‑end clinical workflow: given minimal instructions, an agent must explore raw healthcare data, operate within a complex environment, and execute multi‑step solutions that go beyond naive prompting. A final task success rate is reported to provide a single, interpretable metric for HealthAgentBench overall performance for each agent. Evaluating frontier agents on HealthAgentBench, we find that overall task success rate remains low, underscoring the difficulty of the suite. The strongest and the most cost effective agent, Codex GPT‑5.5, achieves only approximately 42% success rate. Beyond aggregate performance, HealthAgentBench reveals nuanced strengths and weaknesses across task categories. Frontier agents show promise in automatically developing research modeling pipelines over EHR data, but medical imaging remains especially challenging, particularly for Claude Code models, while Codex GPT‑5.5 shows emerging capability. Tasks that combine large search spaces with compositional reasoning requirements remain difficult for all current agents. Together, these results suggest that HealthAgentBench provides a challenging and realistic benchmark with substantial room for future progress. We release our benchmark at https://github.com/microsoft/HealthAgentBench.
Authors:Kaiwen Xiong, Haonian Ji, Shi Qiu, Zeyu Zheng, Cihang Xie, Xinyu Ye, Huaxiu Yao
Abstract:
Production large language‑model (LLM) agents are increasingly deployed not as lone problem‑solvers but as managers: a main model creates specialized subagents, delegates work, and orchestrates their parallel, asynchronous returns through dynamic workflows. Whether one model can actually run such a team is largely unmeasured: existing benchmarks score a policy's own task‑solving or a fixed multi‑agent system's emergent behavior, but none isolate the management ability of the single LLM acting as leader. We introduce ClawArena‑Team, a benchmark of 41 multi‑turn, multimodal, multi‑directory scenarios spanning 258 evaluation rounds and 72 staged updates that measures this management ability. The main agent is deliberately constrained: it natively perceives only text and directly accesses only part of the workspace. It commands a fixed, locally served subagent pool, so score differences reflect management skill, not raw capability. All scoring is execution‑based with no LLM judge: an overall score ‑‑ the Subagent‑Management Score (SMS) ‑‑ multiplies task correctness by a least‑privilege and modality‑routing factor. Across twelve proprietary, community‑hosted, and self‑hosted models, experiments show that the management bottleneck is privilege granting rather than perception (no model exceeds 50% workspace‑permission precision); that cost and management quality are decoupled (API cost spans over 100 times while the overall score spans under 4 times, with the cheapest open models on the Pareto frontier); and that most leaderboard scores cluster within a 9.9‑point band while orchestration behaviors diverge by more than an order of magnitude. Code is available at https://github.com/aiming‑lab/ClawArena.
Authors:Jiyong Boo, Byeongin Joung, Hyemin Yang, Kuk-Jin Yoon
Abstract:
Monocular 3D lane detection plays a critical role in autonomous driving, yet recovering reliable 3D geometry from a single image remains challenging due to inherent depth ambiguity. Prior methods project image features into Bird's‑Eye‑View (BEV) space under a flat‑ground assumption, causing geometric distortion on real‑world roads. Recent methods instead predict explicit height maps to capture non‑planar surfaces, but still rely on sparse anchor‑based regression and exploit the recovered geometry merely for spatial transformation rather than semantic understanding. To overcome these limitations, we propose HSDF‑Lane, which implicitly models the road surface as a Height‑aligned Signed Distance Field (HSDF) over a densely sampled 3D feature volume. Through differentiable rendering, the HSDF jointly produces an accurate height map and surface‑aligned features. We further introduce Lane‑aware Semantic Positional Encoding (LSPE), which injects a lane‑existence prior derived from the surface‑aligned features into the transformer queries, coupling geometric structure with semantic guidance. Extensive experiments on the OpenLane benchmark show that HSDF‑Lane achieves state‑of‑the‑art performance in both 3D lane detection and height map estimation.
Authors:Hao Sun, Yu Song, Shiyu Teng, Ziwei Niu, Yen-Wei Chen
Abstract:
VLA models have emerged as a powerful paradigm for transferring semantic knowledge from web‑scale data to physical robotic control. However, current single‑frame architectures suffer from intrinsic limitations: temporal myopia that discards historical dynamics, reasoning gaps between high‑level instructions and low‑level motor commands, and inference inefficiency due to autoregressive scalar decoding. In this work, we propose MIRTH, a unified framework designed to address these challenges. MIRTH augments a pretrained VLA backbone with three key innovations: (1) dual‑scale temporal memory hubs that compress long‑term scene evolution and short‑term motion trends into compact embeddings; (2) latent reasoning tokens optimized via a mutual‑information objective carving out a semantic plan space to align multimodal context with action trajectories; and (3) a parallel action decoding scheme that replaces autoregressive generation with vector‑wise prediction to maximize control throughput. Extensive evaluations on the LIBERO simulation benchmark and a real‑world LeRobot platform demonstrate that MIRTH achieves state‑of‑the‑art performance and exhibiting emergent error recovery capabilities. The codes and collected datasets are released at http://github.com/kiva12138/mirth.
Authors:Oleksii Nasypanyi, Jaemin Cho, Utku Ozbulak, Byungkon Kang, Francois Rameau
Abstract:
Scene Coordinate Regression (SCR) methods are increasingly adopted for visual localization. In these approaches, the scene is implicitly encoded within a neural network that regresses a 3D world coordinate for each image pixel. Because the scene is represented only through the network parameters and not stored explicitly as images or maps, such methods are often assumed to be privacy‑preserving. In this work, we show that this assumption is incorrect in practice. Specifically, we introduce a query‑based attack that reconstructs the 3D geometry of the training environment from an SCR model under different levels of model access. To do so, we repeatedly query the model with batches of proxy images unrelated to the target scene to obtain dense pixel‑wise 3D coordinates. Reliable points are identified through their stability under small input perturbations and can be further refined in a white‑box setting. These stable points are accumulated across independent query batches to recover the scene geometry. From the recovered 3D representation, we also invert the network features to synthesize images from arbitrary viewpoints, revealing additional appearance information. Experiments on indoor and outdoor datasets demonstrate that substantial portions of training environments can be reconstructed with high geometric fidelity. Beyond geometry, we also recover an approximate color appearance, which exposes recognizable layout and potentially sensitive scene elements. This directly contradicts claims in the literature that SCR representations are privacy‑preserving by design, and reveals a real risk when such systems are deployed in private or security‑critical spaces. The project page is available at https://jaeminch0.github.io/seeing‑through‑the‑weights‑privacy‑leakage‑in‑scene‑coordinate‑regression.
Authors:Apurva Gandhi, Vishwas Suryanarayanan, Raja Hasnain Anwar, Firoz Shaik, Shubhang Desai, Thong Q. Nguyen, Muhammad Taqi Raza, Vishal Chowdhary, Graham Neubig
Abstract:
Creating and editing slides is a rich, multimodal activity that is ubiquitous in professional and educational settings, making it an ideal testbed for real‑world computer‑use agents. Microsoft PowerPoint is among the most widely adopted and feature‑rich environments for presentation creation. We introduce PPT‑Eval, a benchmark of 120 PowerPoint tasks across 12 files that cover both content creation and presentation editing scenarios, organized by difficulty. A central challenge in this domain is evaluation: tasks are complex, multimodal, and often admit many valid solutions. Moreover, today's agents frequently make only partial progress, which binary success metrics fail to capture. To address this, we design a robust evaluation framework to help create task‑specific rubrics for PowerPoint tasks, taking inspiration from and building on past works for rubric‑based evaluation. These rubrics award partial credit for intermediate steps, penalize unnecessary changes and poor aesthetics, and provide natural language feedback. This nuanced approach proves highly effective, achieving a Kendall's τ‑b correlation of 0.77 with human judgments. We find that existing frontier agents still struggle with solving PowerPoint tasks, with strong models like Claude‑4.5‑Opus achieving only a 45% success rate and an average partial score of 57%. The benchmark is located at: https://microsoft.github.io/ppteval.
Authors:Duc Cao Dinh, Khai Le-Duc, Florent Draye, Chris Ngo, Terry Jingchen Zhang, Bernhard Schölkopf, Zhijing Jin
Abstract:
3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene, leading to ambiguous predictions and high computational cost, especially in cluttered environments. We observe that many referential expressions rely on local spatial context and often correspond to restricted spatial regions rather than the full scene. Motivated by this insight, we propose PruneGround, an effective plug‑and‑play framework for 3DVG built upon three key components. First, we introduce Language‑Guided Spatial Pruning (LGSP), which leverages a frozen Vision Language Model (VLM) to identify language‑relevant regions, thereby reducing spatial computation and grounding candidates in the narrower search space. Second, we propose MultiView‑Conditioned Description Reformulation (MCDR), which decomposes complex expressions into simplified target‑anchor relations and augments missing spatial cues through multi‑view reasoning. Finally, we propose LLM‑Grounder, which repurposes a detection‑pretrained spatial LLM into a language‑conditioned grounding model by aligning point cloud and linguistic representations within the pruned region. Extensive experiments on the three most popular point cloud benchmarks demonstrate that our method achieves state‑of‑the‑art results on all three ScanRefer settings and on 9 out of 10 Nr3D/Sr3D settings. Code and models are publicly available: https://github.com/leduckhai/PruneGround
Authors:Amirhossein Abaskohi, Giuseppe Carenini, Peter West, Yuhang He
Abstract:
Large language models increasingly operate over long contexts, where the KV cache becomes a dominant memory bottleneck: its size grows linearly with sequence length and must be retained throughout decoding, making full GPU caching prohibitively expensive without compression. Existing KV cache compression methods struggle to balance efficiency with faithful context preservation. Token eviction discards information, while semantic grouping fixes compression decisions at prefill time; neither can recover token‑level detail from a compressed span once it becomes relevant during generation. As a solution, we propose SeKV, a resolution‑adaptive semantic KV cache that organizes context into entropy‑guided semantic spans and stores them across a GPU‑CPU memory hierarchy without discarding information. Each span keeps a lightweight summary vector on GPU for coarse routing and a low‑rank SVD basis on CPU for on‑demand token‑level reconstruction. A trained zoom‑in mechanism selectively expands query‑relevant spans during decoding, enabling precise retrieval without materializing the full KV cache on GPU. SeKV enables adaptive token‑level reconstruction while keeping the base LLM fully frozen and adding fewer than 0.05% trainable parameters. Across four benchmarks, SeKV improves over the strongest semantic compression baseline by 5.9% on average while reducing GPU memory by 53.3% versus full KV caching at 128K context. Code is available on https://github.com/AmirAbaskohi/SeKV.
Authors:Junghwan Park
Abstract:
Few‑shot segmentation asks a model to delineate a target class in a query image from only a handful of annotated examples, a setting most acute in remote sensing, where labels are scarce and the imagery departs sharply from the natural images on which vision backbones are pretrained. Prevailing approaches either train a segmenter on labelled episodes, which raises accuracy within the training distribution but binds the model to it, or reduce each class to a lossy summary of frozen features, a single prototype, a few cluster prototypes, or a discrete clustering, none of which preserves the internal structure of a multimodal class. We argue that a class is better described by a distribution than by a point, and that frozen self‑supervised features already carry enough structure to estimate that distribution directly. We introduce FROST, a training‑free few‑shot segmenter that treats the reference foreground and background as two point clouds on the unit sphere of frozen DINOv3 features and labels each query token by a nonparametric density ratio, with a threshold the Bayes rule fixes at zero under equal priors. Because the variance of a density estimate shrinks as its sample grows, the decision sharpens as references accumulate, and every remaining quantity from the kernel bandwidth to the spatial gate is read from the support set rather than tuned. We develop FROST for overhead imagery, where a class is typically a scatter of many small and dissimilar instances that a density tracks but a lossy summary blurs. Across seventeen remote‑sensing benchmarks FROST surpasses both training‑free and learning‑based methods, leading by 5.6 mIoU from a single annotated example and widening its lead as the support set grows, all while remaining among the smallest models compared. Code is available at https://github.com/jhpark‑ai/FROST.
Authors:Arshia Soltani Moakhar, Iman Gholami, Max Springer, Mahdi JafariRaviz, MohammadTaghi Hajiaghayi
Abstract:
While Large Language Models (LLMs) have demonstrated exceptional capabilities in mathematical reasoning, they frequently produce subtle errors that evade human detection. Formal mathematical languages like Lean 4 offer mechanical proof checking, strongly motivating the need for autoformalization: the automatic translation of natural language mathematics into verifiable code. Recent trends indicate that general‑purpose LLMs, heavily optimized for standard programming, now outperform smaller models explicitly fine‑tuned for Lean. Leveraging this shift, we introduce an agentic autoformalization framework powered by general coding LLMs. At the core of our system is an orchestrator that manages a multi‑agent pipeline tailored for research‑level mathematics. Because cutting‑edge research frequently relies on concepts outside the scope of existing libraries like Mathlib, our system dynamically extends necessary type definitions and validates them via a novel Auxiliary Lemma technique before formalizing the primary theorems. We applied our approach to PutnamBench, producing machine‑checked Lean proofs for a random sample of 32 problems. Furthermore, we evaluate our system on five papers from the ACM Symposium on Theory of Computing (STOC) spanning combinatorics, communication complexity, mechanism design, and learning theory, successfully formalizing their main theorems and validating the generated formalizations with human experts; for all five we also formalize the proofs alongside the statements, and notably two of them are proved with no axioms beyond Lean's kernel. All of our formalizations are available at https://beyondthelibrary.github.io/formal_arxiv .
Authors:Anjali Parashar, Chuchu Fan
Abstract:
To ensure safe on‑road behavior, pre‑deployment testing and failure discovery of Autonomous Driving Systems (ADS) is crucial. Present day simulation based testing methods focus largely on mathematical models for efficient search of optimal scenarios, assuming a fixed scenario representation. On the other hand, real‑world testing involves substantial manual effort to design scenario templates for testing. These templates represent distinct failure scenarios consisting of pre‑deployment vehicle movements, map types, etc. Historical failure records for ADS are a reliable source of real‑world failure conditions, which can be used for scenario generation. In this work, we propose a scenario generation pipeline using categorical and contextual information available from historical records in natural language format. Our approach consists of modular LLM based synthetic scenario generation, compatible with the testing constraints of a given system. We successfully apply our method to generate a diverse set of scenarios for testing autonomous navigation on Metadrive simulator using the NHTSA ADS crash records. Our approach results in accurate and diverse scenario generation with a combination of 4 road types, 3 non ego vehicle movement types, including on road anomalies in the form of working zones. Generated scenarios align with the provided testing conditions, and reveals interesting failures of the system within a limited testing budget of 20 scenarios. Code is available at https://github.com/anjaliParashar/crash2scenario.
Authors:Björn Braun, Christian Holz
Abstract:
To enable personalized, real‑time coaching using Augmented Reality glasses or fixed camera setups in domains such as sports, cooking, or music, a system must understand not just what a person does, but how well they execute an activity. In an ego‑exo video setting, this requires simultaneously detecting individual skilled actions and classifying each as correct or needing improvement, which Ego‑Exo4D's proficiency demonstration benchmark formalized. We first adapt seven state‑of‑the‑art temporal action detection architectures to this task, extend the evaluation protocol to disentangle detection from grading, and show that existing methods grade near‑randomly. We then introduce SkillSpotter, a pose‑aware multi‑view architecture that jointly detects and grades skilled actions through three task‑specific modules: (1) adaptive temporal suppression to handle the varying density of skilled actions across diverse activities, (2) gated 3D body pose fusion to leverage body kinematics as a complementary signal to visual features, and (3) bidirectional cross‑view attention to combine ego and exo views effectively. SkillSpotter improves class‑specific mAP from 12.40 to 21.82 (+76%) and balanced accuracy from 55.99% to 60.40% over the best baseline. SkillSpotter's modules transfer to other temporal action detection models with consistent gains, and our method generalizes beyond Ego‑Exo4D to HoloAssist. Code: https://github.com/eth‑siplab/SkillSpotter
Authors:Wen-Chin Huang, Tomoki Toda
Abstract:
UTMOS has become one of the most commonly used deep neural network‑based speech quality assessment (SQA) metrics in speech processing research. In this paper, we attack UTMOS to probe its robustness. Starting from high‑quality speech samples, we optimize the input in two directions: a score‑preserving attack, which degrades perceived quality while maintaining the predicted score, and a quality‑preserving attack, which lowers the predicted score while maintaining perceived quality. We consider three input spaces: raw waveform, mel spectrogram with a HiFi‑GAN vocoder, and the latent space of EnCodec, a neural audio codec. Experimental results show that score‑preserving attacks are effective against UTMOS. Although perfect quality‑preserving attacks are more difficult, optimization in the EnCodec latent space provides the best chance of success. These results reveal failure modes of UTMOS and highlight the importance of robustness analysis for DNN‑based SQA metrics.
Authors:Chaeyeon Lee, Khang Nguyen Quoc, Jinsol Song, Yosep Chong, Kwangil Yim, Jin Tae Kwak
Abstract:
Whole slide image (WSI) analysis is central to computational pathology, with multiple instance learning (MIL) emerging as the standard pipeline for slide‑level diagnosis. However, conventional approaches formulate WSI diagnosis as a flat classification task over discrete labels, contradicting the inherently hierarchical, coarse‑to‑fine nature of clinical reasoning. Although recent hierarchical classifiers and vision‑language models (VLMs) have sought to address this structural gap, they either fail to capture semantic continuity between related diagnoses or suffer from unconstrained text generation that produces taxonomic hallucinations and parent‑child label violations. To address these limitations, we propose TaxoMIL, a taxonomy‑constrained framework that reformulates WSI diagnosis as a multi‑granularity text generation task. TaxoMIL utilizes a dual‑head Transformer decoder to generate coarse‑ and fine‑level diagnostic text, and introduces taxonomy‑guided objectives that explicitly structure the label embedding space and strictly ground slide‑level visual representations within the clinical taxonomy. Extensive experiments across three diverse WSI datasets demonstrate that TaxoMIL consistently outperforms state‑of‑the‑art MIL classifiers and VLM‑based generative methods, yielding accurate and hierarchy‑aware diagnostic predictions. The code is released at https://github.com/QuIIL/TaxoMIL
Authors:Geonho Bang, Geunju Baek, Dongyoung Lee, Wonjun Jeong, Jun Won Choi
Abstract:
Long‑range 3D object detection is critical for safe autonomous driving at highway speeds, yet existing radar‑camera fusion methods remain limited at extended ranges. BEV‑based methods capture scene‑level context but incur rapidly growing computation and often lose fine‑grained object detail, while query‑based methods are efficient but provide limited scene‑level context. Temporal fusion further requires both multi‑frame accumulation for sparse distant observations and object‑level motion modeling for fast‑moving objects. We propose Horizon3D, a sparse radar‑camera fusion framework for long‑range 3D object detection that combines Gaussian primitives with sparse BEV features. Horizon3D initializes Gaussian primitives at radar‑ and camera‑estimated object keypoints using Keypoint‑Guided Gaussian Initialization, refines them through Object‑Centric Sparse Fusion, and splats them onto the BEV plane to fuse object‑level detail with sparse radar BEV context. It further introduces Dual‑Path Temporal Fusion, which aggregates temporal cues through a BEV path for scene‑level accumulation and a Gaussian path for object‑level motion propagation. Experiments on TruckScenes show that Horizon3D achieves state‑of‑the‑art radar‑camera 3D detection performance. On the validation set, it outperforms the previous best method by +3.0 NDS and +1.6 mAP while maintaining competitive inference speed.
Authors:Rui Zhou, Tianci Xie
Abstract:
Full fine‑tuning adapts large language models to new tasks but can erode capabilities they already possess. Existing remedies protect through proxies such as parameter distances, importance penalties, output matching, or dominant singular directions of the weights, but none directly asks which activation directions the preserved capability relies on. We argue that a capability is characterized more faithfully by the activation subspace it induces than by the singular geometry of the weight matrix, and develop function‑space protection, instantiated as FORA (Function‑space Orthogonal Residual Adaptation). From label‑free calibration inputs, FORA estimates, per layer, the principal directions Q of the input‑activation covariance and forms a right projector P_Q = I ‑ QQ^T. Paired with a left projector P_U from the weight SVD, the update is ΔW = P_U M P_Q + U_2 D_δ V_2^T: a high‑capacity branch structurally barred from reading capability‑relevant function directions, plus a narrow spectral channel for controlled plasticity. The construction extends to parameter‑efficient adaptation via M \to (α/r) BA. Across three settings on Qwen3‑1.7B, including COGS and GSM8K learned while preserving translation and translation learned while preserving math, FORA consistently improves preservation over weight‑space projection and standard regularization, with only a small new‑task trade‑off in the math‑preservation setting. A controlled ablation isolating the projection source shows that the advantage comes not from projection itself, but from projecting onto capability‑derived rather than weight‑derived directions. Code is available at https://github.com/zrui239/FORA.
Authors:Jiaan Wang, Sirui Liu, Yu Li, Kaiyuan Yang, Juan Cao, Sheng Tang
Abstract:
AI‑generated image (AIGI) detection is undergoing a critical transition from laboratory benchmarks to open‑world adversarial defense. The prevalent paradigm focuses on finding static feature spaces, assuming that some invariant artifacts learned from historical data can achieve universal zero‑shot generalization. While achieving saturation on several AIGI benchmarks, this static hypothesis suffers a severe performance drop against rapidly evolving generators (e.g., SD3, Nano Banana Pro). To address these limitations, we propose that the field should expand beyond "static generalization" to a new paradigm of "dynamic adaptation". We introduce Fleet, a framework that pioneers a dynamic paradigm of continuous few‑shot evolution, enabling rapid alignment with emerging generative threats. Fleet improves few‑shot adaptation by replacing unconstrained feature updates with constrained routing correction, where avoidance routing redirects novel AI samples away from Non‑AI‑dominated routes within decoupled subspaces. To validate this, we present Treasure, a benchmark spanning 64 models and 360k images, featuring diverse architectures and 20 closed‑source commercial engines. Experiments reveal that while static SOTA methods fail catastrophically on modern generators, Fleet restores performance from 20.4% to 73.1% with only 10‑shot adaptation on "Doubao Seedream 4.0". Code and data are available at https://github.com/ICTMCG/Fleet .
Authors:Guangsheng Bao, Lihua Rong, Yanbin Zhao, Xiao Yu, Qiji Zhou, Yue Zhang
Abstract:
Existing AI‑generated text detectors are vulnerable to attacks that manipulate textual characteristics. In this study, we propose a novel Triospect Detection Framework by using additional perspectives of content (core ideas) and expression (stylistic elements) within a given text. Experiments on two benchmarks involving 17 attacks, 12 domains, and 17 source models demonstrate that Triospect is robust against these attacks. It improves the strong baseline by a significant margin of 22.3% (AUROC) and 13% (TPR01) on the Humanize‑16K after‑attack subset, and by 9.1% (AUROC) and 22% (TPR01) on the adversarial RAID. This framework marks a pioneering effort in statistical methods to enhance detection reliability against attacks. We release our data and code at https://github.com/baoguangsheng/triospect.
Authors:Jingwang Ling, Lifan Wu, Feng Xu, Shuang Zhao
Abstract:
Reconstructing physics‑based 3D assets ‑‑ geometry, materials, and illumination ‑‑ from multi‑view images is a core problem in computer graphics and vision, and a prerequisite for realistic relighting and editing. Physics‑based inverse rendering offers an accurate image‑formation model, but is severely underconstrained: without strong priors, illumination is baked into materials, and reconstructions generalize poorly to novel views and lighting. Data‑driven diffusion models, in contrast, predict visually plausible materials, yet their predictions rarely satisfy the rendering equation and are not directly usable for physics‑based rendering. We bridge these two paradigms rather than replacing either. Our key idea is to treat the predictions of a state‑of‑the‑art diffusion model not as target material values but as a similarity kernel for optimization: we introduce a regularization loss that penalizes deviations in the optimized material over surface regions where the diffusion predictions are near‑constant, while leaving the optimization free to match the input images. Built on this regularizer, our end‑to‑end pipeline jointly reconstructs geometry, materials, and illumination, yielding high‑quality assets that drop into standard rendering pipelines and relight faithfully. On the Synthetic4Relight, Stanford‑ORB, and DTC‑Synthetic datasets, our method significantly outperforms state‑of‑the‑art baselines in both reconstruction accuracy and relighting quality.
Authors:Hiroki Takeda, Yuto Miyatake, Daisuke Furihata
Abstract:
Tensor Train (TT) decomposition is a powerful technique for analyzing high‑dimensional data. Existing algorithms for computing TT decompositions can be categorized into two main types: conventional batch‑based approaches and recursive online methods. In the context of streaming data, batch methods typically achieve higher reconstruction accuracy but often suffer from memory exhaustion, while online methods provide greater computational efficiency. In this work, we introduce Online TT‑ALS (Alternating Least Squares), an algorithm that sequentially enforces orthogonality constraints. This approach allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy. Theoretically, we prove that enforcing these orthogonal gauge constraints guarantees monotonic decrease of the local objective function and temporal smoothness. Computationally, our deterministic single‑sweep update reduces the rank dependence from quadratic to linear, achieving an overall complexity of \mathcalO(I^n‑1 r). Experimental results demonstrate that the proposed method outperforms existing online techniques not only in terms of mathematical approximation accuracy but also in human perception‑based video quality metrics. Furthermore, compared to recent deep learning‑based paradigms, our algebraic approach achieves speedups of several orders of magnitude. Consequently, our method exhibits high computational efficiency and is suitable for low‑latency real‑time processing applications.
Authors:Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu, Zhendong Mao
Abstract:
Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image. In this paper, we identify an internal signature of hallucination: progressive degradation of text‑to‑image cross‑attention during generation, leading to specific failure patterns like unfocused or biased attention. Existing mitigation strategies are largely outcome‑driven and do not explicitly target this failure mode. To address this problem, we propose ADAPT (Attention Dynamics Alignment with Preference Tuning), an attention‑based framework that intervenes directly on text‑to‑image cross‑attention dynamics. We propose ADAPT with three key contributions: a cross‑attention visual anchor refined from early decoding to provide stable spatial grounding, an attention‑supervised inference mechanism that detects and corrects attention drift online, and a Visual Attention Guidance DPO that aligns preferences toward visually grounded responses. Experiments show that each component of ADAPT contributes to hallucination reduction, and the full framework achieves new best results across multiple hallucination benchmarks, reducing hallucination rates by 40%‑60% across mainstream backbones while preserving general multimodal capabilities. Our work provides an attention‑based perspective on mitigating hallucinations by exploring the model's internal text‑to‑image cross‑attention behaviors. Code is available at https://github.com/yao‑ustc/ADAPT
Authors:Gaurab Baral, Aaditya Khanal, Yangyang Tao, Junxiu Zhou
Abstract:
This paper investigates knowledge distillation from a large reasoning model (DeepSeek‑R1) to a compact student model (Qwen2.5‑7B). Using historical problems from the John O'Bryan Mathematics Competition at Northern Kentucky University (2011‑2025), we build a Chain‑of‑Thought (CoT) training corpus through a dual‑agent framework. The dataset is used to fine‑tune the student model with Low‑Rank Adaptation (LoRA) on Apple Silicon hardware using the MLX framework. The base Qwen2.5‑7B model achieves 64.67% accuracy on competition problems, while the DeepSeek‑R1 teacher achieves 91.40%. An initial 1,000‑iteration training run revealed severe overfitting, with validation loss reaching a minimum at iteration 200 before rising steadily. Based on this finding, we ran five independent training runs each limited to 200 iterations with varied random seeds to assess result stability. Across these five runs, the fine‑tuned student model achieves a mean accuracy of 69.43% (std dev 0.17%) on the competition dataset, a 4.76 percentage‑point improvement over the base model, and generalizes to 73.1% (std dev 0.18%) on the MATH‑500 benchmark. We further study how response length affects answer quality across six reasoning levels (R1‑R6): accuracy declines consistently from 69.43% at R1 (mean 220 words) to 41.9% at R6 (mean 31.2 words), with the two‑person speed section most sensitive to token reduction. These results demonstrate that CoT distillation improves compact student models and that response length is a critical factor in mathematical reasoning quality.
Authors:Brian Wei, Srikumar Sastry, Daniel Cher, Eric Xing, Nathan Jacobs
Abstract:
Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging. Unlike natural imagery, satellite scenes are structured by spatially complex and semantically distinct geometries. Prior work addresses this complexity by adapting natural image frameworks using dense rasters or sparse prompts, trading off annotation cost and fidelity while breaking compatibility with vector primitives commonly used to represent geographic information. We introduce TerraDiT‑Ω, a unified spatial control framework that generates satellite imagery directly from any native geospatial primitive. By jointly leveraging precise annotations (polygons, polylines) and coarser ones (bounding boxes, points), the model supports controllable layouts across varying annotation budgets, broadening applicability to design tasks such as urban planning while remaining naturally compatible with end‑to‑end GeoAI workflows. To effectively leverage these primitives during generation, we propose Geometry‑Aware Local Attention, a conditioning mechanism that injects explicit geometric cues into the attention space. Across all conditioning formats, our approach consistently outperforms both dense‑control and sparse‑control baselines. Furthermore, this flexibility enables controllable synthetic data augmentation using a single generative model, improving downstream performance on land‑cover segmentation, object detection, road graph extraction, and scene classification. Code, data, and weights are available at https://github.com/mvrl/TerraDiT.
Authors:Huanlin Gao, Fang Zhao, Qiang Hui, Fuyuan Shi, Shaoan Zhao, Yantao Li, Chao Tan, Ting Lu, Yuren You, Kai Wang, Shiguo Lian
Abstract:
We propose OTCache, a training‑free framework for accelerating diffusion sampling via caching schedule prediction. Existing graph‑based caching methods reduce redundant computation by optimizing shortest‑path objectives, but rely on an additive independence assumption, which often breaks down in the low NFE regime. To address this issue, OTCache models caching schedules across inference budgets as a smooth evolution in policy space, inspired by Optimal Transport (OT). The framework consists of three stages: (1) obtaining a high‑fidelity reference schedule using a graph‑based caching method under a conservative budget; (2) performing a lightweight anchor search under an extreme low‑budget setting via Optuna optimization with an end‑to‑end perceptual objective; and (3) predicting schedules for target budgets via quantile interpolation between the reference and anchor policies using continuous warping representations. Experiments on FLUX.1 [dev], Qwen‑Image, and HunyuanVideo show that OTCache achieves 4.5x, 4.7x, and 3.66x acceleration, respectively, while consistently improving generation fidelity over state‑of‑the‑art caching baselines. This work provides a new perspective on accelerating diffusion models through Optimal‑Transport‑inspired schedule modeling. Code:https://github.com/UnicomAI/OTCache
Authors:Shen Zheng, Anurag Ghosh, Gaurav Parmar, Srinivasa Narasimhan
Abstract:
Image‑to‑image (I2I) translation has achieved strong results in tasks like human relighting and driving scene translation using latent diffusion models (LDMs). However, compact LDMs often struggle to preserve fine‑grained structures because the encoder compresses high‑resolution inputs into a spatially downsampled latent space. To address this issue, we propose a simple saliency‑guided warp‑unwarp framework that reallocates spatial representation toward salient regions before encoding, enabling better preservation of structural details without increasing latent resolution. The warped image is processed by the original diffusion model and then mapped back via an inverse warp. In addition, we propose a simple and efficient outpainting‑based synthetic data generation pipeline to produce high‑quality paired data for image relighting. Our method is model‑agnostic, requires no architectural modification, and introduces negligible computational overhead. Experiments on human relighting, driving scene relighting, and translation demonstrate improved structural preservation, lighting faithfulness, and image quality, with our framework extending naturally to video via frame‑by‑frame application with good temporal stability. Project Webpage: https://shenzheng2000.github.io/WarpI2I.github.io
Authors:Wencong Wu, Xiuwei Zhang, Hanlin Yin, Hongxi Zhang, Yanning Zhang
Abstract:
Transformer‑based approaches have obtained excellent performance in multispectral object detection tasks due to their ability to model long‑range dependencies and capture complementary information. However, previous transformer‑based multispectral detection methods tend to use all available tokens for similarity calculation, which results in redundant information interaction from irrelevant areas, leading to degraded detection performance. To overcome this challenge, we propose a novel Dual Sparse Aggregation Transformer (DSAFormer) for multispectral object detection, which consists of a Dual Sparse Transformer (DSFormer) and a Learnable Addition Fusion Block (LAFB). Specifically, the DSFormer is designed to exploit and boost cross‑modal complementary information, thereby improving detection performance. It incorporates three key components: A Spatial Sparse Multi‑Head Cross‑Attention (SSMHCA) mechanism selectively captures cross‑modal relationships at the spatial level by reserving only the high query‑key similarity scores, eliminating irrelevant interactions. A Channel Sparse Multi‑Head Cross‑Attention (CSMHCA) mechanism performs similar sparse calculations at the channel level to enhance feature representation and filter out low matching query‑key. A Multi‑Scale Feature Refinement Layer (MSFRL) is developed to aggregate hierarchical features and suppress redundant information. To effectively fuse multimodal features, the LAFB is introduced to aggregate intramodal and intermodal feature information by feature reweighting. Extensive experimental results have demonstrated that our proposed DSAFormer achieves better detection performance against state‑of‑the‑art methods on four public datasets, including the MFAD, FLIR, M^3FD, and LLVIP. The source code of our DSAFormer will be released at https://github.com/WenCongWu/DSAFormer.
Authors:Mert Onur Cakiroglu, Zhihe Lu, Mehmet Dalkilic, Hasan Kurban
Abstract:
AI‑generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three‑feature clip‑length classifier reaches a leave‑one‑generator‑out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20‑paper survey finds none applying all six standard controls that would catch this, so we combine them into an audited protocol and apply it to six representative feature sources (three published detectors and three repurposed signal sources), re‑running it cross‑dataset on AIGVDBench. The audit both debunks and certifies: the trivial classifier collapses to near chance (0.529), a CLIP baseline is caught carrying dataset identity, and the 2025 forensic detector WaveRep clears the floor at out‑of‑distribution LOGO AUC 0.996 with chance‑level real‑vs‑real coherence. At a deployable FPR of 0.1%, multiple high‑AUC methods fall to single‑digit recall and the leaderboard order changes, so we recommend an audited tuple (AUC, above‑floor margin, operating‑point recall, and calibration) over a single number. As a white‑box positive control, we add TemporalSpec (codec motion vectors); via cross‑substrate feature fusion (XSFF), a second substrate adds genuine complementarity that survives the audit. We release VidAudit, to our knowledge the largest unified and audited detector collection for this task, providing 14 detectors behind one plugin API, a leaderboard, and Croissant metadata, available at https://github.com/KurbanIntelligenceLab/vidaudit. Together, the protocol and toolkit move evaluation from leaderboard rank toward whether a result measures what it claims.
Authors:Jaden Clark, Changhao Wang, Yihuai Gao, Seongheon Hong, Hojung Choi, Mark Cutkosky, Yifan Hou, Shuran Song
Abstract:
Robot manipulation often relies on sensory feedback beyond vision, particularly in contact‑rich settings where force, tactile, or audio signals reveal interaction states that are not directly observable from images. However, these modalities are often hardware‑ and task‑specific, and large‑scale multisensory robot datasets remain scarce. As a result, it is impractical to pretrain policies with every sensor they may encounter. We study multisensory continual learning: adapting a pretrained robot policy to new tasks with newly introduced modalities while preserving performance under the original sensor suite. We propose MuSe, which incorporates limited multisensory data into pretrained vision‑only policies through multi‑stage fusion, multisensory future prediction, and experience replay over pretraining data. We instantiate MuSe by augmenting a pretrained vision‑only policy with force‑torque sensing and evaluate it on real‑world manipulation tasks. Our experiments show that MuSe performs strongly on contact‑rich finetuning tasks while preserving, and in some cases improving, performance on the original pretraining tasks. These results suggest that a modest multisensory dataset can improve general robot capabilities beyond the finetuning distribution. Project website: https://jadenvc.github.io/multisensory‑continual‑learning/
Authors:Koorosh Roohi, Javad Rajabi, Andrew Fleet, Babak Taati
Abstract:
Photomosaics are large images whose local regions are seen as independent tiles while their overall arrangement forms a coherent scene. Generating them at high resolution, with every tile convincing in its own right, is computationally expensive, since the canvas must hold many detailed tiles at once. We present PhotoQuilt, a training‑free framework that generates photomosaics at arbitrary resolution. Diffusion models struggle to satisfy both scales at once, as direct high‑resolution generation is costly and tends toward one smooth image rather than a mosaic, while patch‑based tiling keeps local detail but loses global structure. PhotoQuilt resolves this with a bootstrapped tiled denoising procedure. We first produce a global composition at low resolution to fix the layout, then upscale it in latent space and re‑inject noise to restore generative capacity. Denoising proceeds within fixed tiles, so each forms its own image while the shared global structure holds them in one layout. Because tile generation is handled separately, PhotoQuilt scales to large canvases without quadratic attention cost. Experiments show that PhotoQuilt outperforms current baselines on both global structure and local realism.
Authors:Mohammad Mahdi Abootorabi, Sina Namazi, Armin Saadat, Lyuyang Wang, Obed Dzikunu, Paul F. R. Wilson, Zhuoxin Guo, Brian Wodlinger, Parvin Mousavi, Purang Abolmaesumi
Abstract:
Micro‑ultrasound (μUS) is a new, emerging, and promising imaging modality for prostate cancer (PCa) detection, but accurate identification of suspicious tissue remains highly dependent on clinical experience, leading to substantial inter‑observer variability. Machine‑learning assistance can reduce this variability; however, training reliable deep models is challenging because supervision is sparse and noisy ‑‑ typically limited to core‑level histopathology outcomes (e.g., cancer grade and its percentage in a biopsy core) without pixel‑level lesion annotations and under severe class imbalance. We introduce Prost‑RL, which reframes μUS PCa detection as a spatially aware, policy‑driven inference problem by learning where to look before decoding. Prost‑RL integrates a lightweight reinforcement‑learning policy into a foundation‑model encoder‑decoder to generate interpretable spatial attention maps that act as soft prompts for both cancer‑likelihood heatmap prediction and image‑level classification. We further propose Adaptive Policy Optimization (APO) to stabilize hybrid supervised‑RL training and a noise‑robust objective combining symmetric cross‑entropy with negative‑entropy regularization to mitigate weak‑label noise and encourage sharp localization. On a cohort of 6,607 biopsy cores from 693 patients across five clinical sites, Prost‑RL achieves 79.0\pm3.5 AUROC with 64.6\pm6.3% sensitivity at 80% specificity for core‑level detection (+2.1 AUROC and +4.5 sensitivity points over the strongest baseline), and 79.3\pm5.8 AUROC for clinically significant cancer classification. The learned policy highlights biopsy‑aligned regions, providing transparent, spatially grounded evidence alongside quantitative risk predictions. Code is available at: https://github.com/DeepRCL/Prost‑RL.
Authors:Peilin Chen, Xiaoxuan Yang
Abstract:
Spiking neural networks (SNNs) have emerged as a promising paradigm for energy‑efficient brain‑inspired computing. However, existing online unsupervised SNN learning suffers from low training accuracy and poor scalability. Although current online supervised learning algorithms perform well on large‑scale datasets and networks, the non‑hardware‑friendly operations hinder efficient edge deployment. In this work, we propose SpikON, the first algorithm‑hardware co‑design framework for efficient and scalable end‑to‑end online supervised SNN learning. We first propose the learnable threshold through time and scaled weight centralization through time techniques to address the inefficiency of traditional algorithms. Moreover, to reduce latency and energy consumption, we introduce the novel training dataflow and cascade computation reuse scheme for SNNs that allows concurrent forward‑backward computation and temporal reuse across timesteps. We further design the dedicated SNN accelerator with a dual‑parallel engine and customized SIMD‑based SNN core for efficient end‑to‑end online learning. Experiments show that the SpikON algorithm achieves 32.2% and 35.0% reductions in training latency and energy consumption over the baseline, without sacrificing accuracy. Moreover, the SpikON co‑design achieves 7.2x (11.5x) and 26.8x (15.8x) training throughput (energy efficiency) compared with the edge Apple M4 GPU and TPU‑like accelerator, respectively. The code is available at https://github.com/peilin‑chen/SpikON.
Authors:Wei Geng, Nitinder Mohan, Jörg Ott
Abstract:
Edge‑cloud inference collaborations are often designed with a routing estimator that decides whether to offload each frame from weak models at the edge to stronger models in the cloud. Existing systems place the routing estimator after the weak detector, so the weak forward pass still runs even on frames that are later offloaded. In this paper, we argue that this weak‑conditioned design can be suboptimal when the offload budget varies. First, we present a competitive weak‑skipping estimator (0.153 GFLOPs, about 29x lighter than the weak detector at 4.49 GFLOPs) that extracts routing signal from raw pixels, outperforming the common after‑weak placement weak‑conditioned baselines. Second, we show that neither weak‑skipping nor weak‑conditioned placement dominates across the full operating curve, and we propose budget‑adaptive routing, which selects between them by offload budget via two offline‑tuned thresholds. On PASCAL VOC, our budget‑adaptive router traces the upper accuracy envelope of both fixed placements across the operating range. Our method reduces per‑frame latency by up to 19.1 ms (about 30% lower at rho = 0.9). Besides outperforming SOTA methods, it is surprisingly stronger than the strong model (+1.7 pp over the strong model's peak mAP) at some operating points with far less compute. Artifacts are available at https://github.com/ViGeng/bgt‑ada
Authors:Rasul Khanbayov, Erchin Serpedin, Hasan Kurban
Abstract:
Prototype‑based medical image classifiers present three clinical limitations: they treat findings as independent, silently amplify unsafe physician feedback, and require full retraining whenever a new finding is needed. We present GRAPE (Graph‑Augmented Prototype Explanations), a unified architecture that addresses all three challenges. First, a Graph Attention Task Head models anatomical concept co‑occurrence, boosting macro‑F1 by +13.8,pp over the prototype baseline on TBX11K. Second, a Concept‑Mismatch Safety Check ‑ the first such mechanism in prototype‑based medical classifiers ‑ warns when the model's dominant finding inside a doctor‑drawn region conflicts with the claimed label, catching 85% of erroneous annotations versus 51% for MC‑Dropout with no extra inference cost. Third, Open‑Vocabulary Prototype Anchoring aligns visual prototypes to clinical text, allowing a new finding to be added from a single labeled image without modifying any other component. On NIH ChestX‑ray14, one Effusion example recovers full‑supervision localization accuracy; on TBX11K, prototype maps achieve 2.6x better lesion localization than end‑to‑end baselines. All three capabilities add only +1~ms latency at interactive batch size. The project page is https://github.com/KurbanIntelligenceLab/GRAPE.
Authors:Mizanur Rahman, Abeer Badawi, Elahe Rahimi, Laleh Seyyed-Kalantari, Frank Rudzicz, Enamul Hoque, Elham Dolatabadi
Abstract:
Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric. We introduce a framework that formulates therapeutic response generation as a decision‑refinement problem driven by multi‑dimensional, human‑aligned evaluation. In Stage I, we introduce TheraJudge, an open‑source therapeutic evaluator trained via preference‑based optimization on human‑annotated data to produce reliable judgments across 7 psychological dimensions. In Stage II, we introduce TheraAgent, which operationalizes TheraJudge's evaluations through a coordinated refinement process with specialized Critic, Coach, and Therapist roles that translate evaluative signals into targeted response revisions. Empirically, TheraJudge achieves strong agreement with clinician ratings, with intraclass correlation coefficients (ICC = 0.87‑0.95), surpassing supervised baselines and strong closed‑source judges, particularly on critical dimensions such as Safety, Relevance, and Empathy. Acting on these evaluations, TheraAgent yields a +0.43 improvement in human‑rated therapeutic quality (on a 5‑point scale) under blind evaluation, with 96% clinician inter‑rater reliability. Low‑quality responses (\leq 3) improve by +2.45 points with a 94% recovery rate, demonstrating targeted correction of unsafe outputs. Overall, our results indicate that effective alignment of mental‑health LLMs stems from acting on human‑aligned evaluation, rather than relying solely on stronger generation. We release code at https://github.com/vis‑nlp/TheraAlign.
Authors:Brent A. Griffin, Jason J. Corso
Abstract:
Foundation model pseudo‑labeling ‑ labeling data strictly via zero‑shot inference ‑ enables massive scale, but performance is undermined by hallucinations that evade standard thresholds. To eliminate these errors, we introduce the Turing‑inspired Label Imitation Game (LIG), a framework that formalizes pseudo‑label pruning as an adversarial interrogation. Rather than filtering labels via isolated thresholds, we use the LIG to train a Turing Test Network (TTN), a task‑agnostic "judge" that evaluates candidate pseudo‑labels within a dataset‑wide context. Experiments across four diverse datasets demonstrate the TTN's robustness, consistently enhancing label accuracy for three state‑of‑the‑art vision‑language models without costly supervision or retraining. Crucially, we demonstrate that learned semantic‑contextual logic is a robust alternative to spatial‑geometric verification, enabling a unique zero‑shot task transfer capability ‑ a TTN trained strictly on image classification datasets can effectively prune complex object detection pseudo‑labels. This pruning yields F1‑score gains of 28% for the worst‑performing baseline categories and 44% with task‑specific fine‑tuning. Significantly, we also observe Category Revival, where the TTN pruning "detoxifies" the training signal for downstream models and enables them to recover from zero recall on transfer‑vulnerable classes. The pre‑trained TTN models and code are available at https://github.com/voxel51/ttn.
Authors:Yujin Kim, Nidhi Soma, Sarah Dean
Abstract:
Probabilistic downscaling is the task of modeling the conditional distribution of high‑resolution fields given coarse inputs, and is a central challenge to atmospheric science, climate modeling, and other multiscale physical systems. A widely used paradigm decomposes the problem into a deterministic mean predictor followed by a stochastic residual generator. While effective in idealized settings, this mean‑‑residual approach frequently produces biased and under‑dispersive ensembles in real‑world applications. Is this merely generic predictive uncertainty miscalibration? We show that the root cause is more fundamental: residual target misspecification, the residual distribution induced during training differs systematically from the one required at test time due to downscaling bias. To close this gap, we introduce ReMatch (Residual Distribution Matching). ReMatch aligns the training residual distribution toward the test‑time regime via optimal transport in a low‑dimensional PCA space. This preserves the statistical benefits of the mean‑‑residual framework while reducing the train‑‑test mismatch in the residual targets seen by the stochastic generator. On a controlled synthetic benchmark with varying bias levels and a real‑world HRRR‑‑ERA5 wind field downscaling task, ReMatch substantially reduces under‑dispersion, improves calibration (SSR and CRPS), and outperforms strong baselines, including the standard mean‑‑residual model and its variants, as well as state‑of‑the‑art super‑resolution models. Our code is available at https://github.com/sdean‑group/ReMatch.git.
Authors:Octavian Pascu, Dan Oneata, Horia Cucu, Nicolas M. Muller
Abstract:
Audio deepfakes are a growing challenge for the general public, as well as for journalists and fact‑checkers. The latter need reliable tools to verify the authenticity of their sources, while at the same time keeping their information private. Commercial deepfake detection solutions rely on cloud‑based processing, which raises privacy concerns. To solve this problem, we propose an on‑device audio deepfake detection model. We show that a truncated self‑supervised backbone with a simple logistic classifier is both very fast and often more accurate than existing solutions. Our solution outperforms the baseline AASIST by 10% and improves inference speed by 40%. We integrate this model into a browser plug‑in, which allows journalists and fact‑checkers to detect deepfakes easily and securely. Code for the plugin is available at https://github.com/OctavianPascu97/Audio‑Deepfakes‑Browser‑Plugin.
Authors:Bartłomiej Cupiał, Jan Łojek, Mikołaj Garstecki, Szymon Pobłocki, Alicja Ziarko, Piotr Miłoś
Abstract:
We study when natural‑language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi‑turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test‑time computation. To separate these effects, we introduce a controlled student‑teacher protocol across Omni‑MATH, Codeforces, BBEH Linguini, and ARC‑AGI1, evaluating thirteen open‑weight models in both student and teacher roles. We compare external feedback, self‑feedback, and unguided self‑refinement, while varying interaction history, task difficulty, and teacher access to privileged task information. Across settings, we find that multi‑turn improvement is often not evidence of feedback use: self‑generated feedback adds little beyond unguided self‑refinement, whereas the strongest external teachers produce substantially larger feedback‑specific gains, suggesting that useful feedback must provide guidance beyond generic retry. Dense student‑teacher interaction matrices further show that interactive gains are driven more by the student's ability to use feedback than by the teacher's identity, although teacher choice remains important for a fixed student. These results suggest that feedback‑based agents should be evaluated against repeated‑attempt baselines, and that ability to act on feedback, not merely feedback availability, is a central bottleneck for interactive improvement. We release our controlled student‑teacher evaluation framework at https://j‑lojek.github.io/feedback‑generation‑is‑a‑bottleneck/.
Authors:Peizhi Niu, Wenjie Qu, Shangding Gu, Tianneng Shi, Yuankai Li, Ahmad Tawaha, Hend Alzahrani, Vincent Siu, Boyi Li, Chenguang Wang, Jiaheng Zhang, Basel Alomair, Ming Jin, Muhao Chen, Chi Wang, Costas Spanos, Dawn Song
Abstract:
Claw‑like AI agents (e.g., OpenClaw) are always‑on processes with persistent access to credentials, files, tools, and external services. They take on system‑level responsibilities ‑‑ installing packages, maintaining state, scheduling subtasks, and mediating I/O ‑‑ making security failures far more severe than in other agents. Yet existing benchmarks focus on model responses and tool calls, leaving cross‑component failure modes largely unmeasured. We adopt a computer‑system analogy: treating a Claw‑like agent as an agentic computer system whose gateway runtime plays an OS‑like mediation role, whose Skills resemble user‑installed applications, and whose Plugins resemble loadable extensions with runtime privileges. Each component has a classical counterpart whose protection mechanisms ‑‑ refined over decades of cybersecurity research ‑‑ are absent on the agent side. From this perspective, we develop SafeClawArena, a benchmark of 406 adversarial tasks across four attack surfaces (Skill Supply‑Chain Integrity, Persistent State Exploitation, Cross‑Boundary Data Flow, and Indirect Prompt Injection), executed in containerized replicas of real agent platforms with canary‑marked credentials and evaluated via automated taint tracking across nine output channels. We evaluate three platforms (OpenClaw, NemoClaw, SeClaw) and five frontier LLMs. The highest attack success rate reaches 70%; malicious Plugins succeed in 100% of cases regardless of the LLM. SeClaw cuts GPT‑5.4's attack success rate from 70% to 22%, partly through utility‑security tradeoffs rather than active defenses, while Claude‑Opus‑4.6 already sits near a 22% floor on every platform. These results expose the inadequacy of current defenses and suggest directions for future hardening. Code and data: https://github.com/sunblaze‑ucb/SafeClawArena.
Authors:Ying Yuan, Xinyu Liu, Sriram Krishna, David Held
Abstract:
Large‑scale dexterous grasp datasets encode rich priors over hand‑object interaction, but their use has largely been confined to grasp generation and pick‑and‑place manipulation. We study whether such data can instead support functional dexterity in articulated tool use, where a robot must acquire a tool, maintain contact, and operate its functional moving parts. We adapt a hierarchical imitation learning framework that combines high‑level hand sub‑goal prediction with a low‑level goal‑conditioned controller. We construct a 355k‑trajectory grasp‑pretraining dataset from large‑scale dexterous grasp annotations and use it to pretrain the low‑level controller. The controller is then fine‑tuned on downstream task demonstrations. To evaluate this setting, we introduce DexCraft, a simulation benchmark with six articulated tool‑use tasks requiring coordinated finger motion. Across simulation and real‑world experiments, our approach outperforms end‑to‑end diffusion policy baselines and hierarchical policies trained from scratch. In the real world, it improves full‑task success by 33.3 percentage points over DP3. These results show that grasp datasets can serve not only as resources for grasp synthesis, but also as scalable pretraining data for contact‑rich dexterous manipulation. Videos are shown on https://yingyuan0414.github.io/grasp2dexterity/ .
Authors:Federico Felizzi
Abstract:
Structured tabular data dominates clinical medicine, yet existing benchmarks fail to reflect real‑world properties like complex survey sampling, demographic oversampling, and subgroup fairness. We introduce the NHANES Accelerometry Cardiometabolic Benchmark, derived from NHANES 2003‑2006, comprising 1,381 adults with hip‑worn accelerometry, fasting laboratory biomarkers, dietary intake, and anthropometrics. We evaluate three tabular learning methods ‑‑ ridge regression, XGBoost, and the foundation model TabPFN v2 ‑‑ to predict glycated haemoglobin (HbA1c), fasting triglycerides, and C‑reactive protein (CRP) from activity phenotypes and lifestyle covariates. TabPFN v2 achieves the best overall performance (HbA1c R^2=0.156, CRP R^2=0.383), while triglycerides remain largely unpredictable (R^2 < 0.05), consistent with known genetic dominance. We apply split conformal prediction to generate distribution‑free 90% prediction intervals and evaluate demographic coverage equity across sex and race/ethnicity subgroups. Marginal coverage aligns with the 90% target for CRP and HbA1c but falls below for triglycerides. At the subgroup level, we observe localized undercoverage (e.g., HbA1c for Mexican American participants), illustrating the gap between marginal guarantees and the conditional coverage required for clinical fairness. Code and data are at https://github.com/felizzi/nhanes‑accel‑cardiometabolic‑benchmark.
Authors:Yogeswar Reddy Thota
Abstract:
Current operating systems expose interfaces optimized for human users but not for AI agents. Humans benefit from pixels, icons, windows, visual grouping, mouse movement, and keyboard shortcuts; AI agents instead need compact semantic state, grounded actions, and reliable feedback. As a result, many computer‑use agents are forced to interpret screenshots, OCR output, and visual crops, introducing high token costs, visual ambiguity, latency, and coordinate uncertainty. This paper introduces LUMOS (Language Model Unified Machine‑Readable Operating‑System Semantics), a semantic interaction layer between AI agents and operating systems. LUMOS converts native accessibility metadata and browser UI structures into machine readable semantic blueprints with stable identifiers, roles, names, values, bounds, and action affordances. It also supports live semantic pointer grounding by querying the UI element under or near the cursor through operating‑system automation APIs. An LLM then acts through an accessibility grounded observe act loop using constrained visible‑UI primitives rather than application‑specific scripts. LUMOS does not claim to replace visual agents; instead, it reduces dependence on screenshots when operating systems already provide semantic structure. These results suggest a path toward AI‑native operating systems and machine‑readable interaction layers.
Authors:Mohamed el Amine Boudjoghra, Ivan Laptev, Angela Dai
Abstract:
Articulated 3D objects are essential for interactive environments in embodied AI, robotics, and virtual reality, but reconstructing their structure and motion from sparse observations remains challenging. Existing approaches remain largely constrained by lack of supervised data or lack the priors needed to reliably recover articulation, hidden geometry, and internal object structure. We present the first debate‑driven agentic approach to articulated 3D object reconstruction from text or image inputs that both grounds articulation reasoning in concrete motion and exposes the occluded geometry revealed under articulation. High‑level agents reason about object semantics and motion using knowledge from vision‑language and video models, while low‑level agents estimate articulation parameters and interaction points; together, they engage in a two‑round structured debate that first exploits global‑‑local disagreement and then grounds the agents in freely generated video. The same video prior, conditioned on the agreed articulation, then drives each part through its motion to expose occluded interiors and geometry that cannot be inferred from a single static view. By combining agentic reasoning with a video generative prior, our approach jointly infers articulation and reconstructs complete 3D articulated objects, producing high‑fidelity geometry, internal structure, and motion‑consistent states beyond directly observed surfaces.
Authors:Sen Liang, Cong Wang, Zhentao Yu, Fengbin Guan, Zhengguang Zhou, Teng Hu, Youliang Zhang, Yuan Zhou, Xin Li, Qinglin Lu, Zhibo Chen
Abstract:
Existing instruction‑based video editing datasets commonly focus on single‑task appearance editing, failing to meet the complex creative demands of real‑world scenarios. To bridge this gap, we present Goku, a large‑scale dataset featuring 2 million high‑quality, instruction‑aligned video editing pairs, which is the first to extend task boundaries from basic appearance editing to multi‑task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesis challenges inherent in these complex tasks, we design an efficient data synthesis pipeline that decomposes complex edits into controllable sub‑problems and introduce a progressive filtering system for data reliability throughout the whole process. Furthermore, we explore the optimal network structures on Goku, and propose Goku‑Edit. To deeply comprehend complex editing instructions, Goku‑Edit leverages an MLLM as its text encoder and adopts a decoupled dual‑branch design: a dedicated mask branch handles structural control, freeing the main branch for appearance rendering. A comprehensive video editing benchmark, Goku‑Bench, is also proposed with 1,000 human‑verified test cases and 7 novel editing‑specific metrics. Evaluated on Goku‑Bench, Goku‑Edit obtains up to +8% improvement on other open‑source models in terms of instruction following.
Authors:Juntao Jiang, Jinsheng Bai, Linxuan Fan, Yali Bi, Jiangning Zhang, Yong Liu
Abstract:
We present APRIL‑MedSeg, a YAML‑driven modular framework for 2D medical image segmentation. It provides a unified and extensible ecosystem that decomposes segmentation networks into reusable components. Also, the framework integrates a broad spectrum of advanced paradigms, including semi‑supervised learning, domain adaptation, knowledge distillation, weakly supervised learning, and text‑guided segmentation as well as foundation model support. A registry‑based configuration system with inheritance enables flexible and reproducible experiment management, supporting seamless switching across models, datasets, and training strategies. In addition, the framework provides a unified interface for medical datasets, augmentation pipelines, deployment utilities and model ensembling. Overall, APRIL‑MedSeg is designed as a general‑purpose research and development platform that bridges algorithmic innovation and practical deployment, while also serving as a structured ecosystem for systematically organizing and reproducing advances in medical image segmentation. The code is available at https://github.com/juntaoJianggavin/APRIL‑MedSeg under an Apache 2.0 license.
Authors:Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci
Abstract:
Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day‑to‑day coding‑agent usage across multiple agents and model families for serving‑system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding‑agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day‑to‑day use of Claude Code and Codex. Our analysis shows that coding‑agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily‑tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower‑overhead tool calling, append‑length‑aware prefill, semantic‑aware tool‑latency prediction, and improved KV‑cache management around human‑paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw‑syfi/TraceLab.git the project website is https://tracelab.cs.washington.edu.
Authors:Haoyang Li, Guanlin Li, Youhe Feng, Chen Zhao, Zhuoran Wang, Yang Li, Qizhe Wei, Shifeng Bao, Haitao Shen, Yihan Zhao, Tong Yang, Jing Zhang
Abstract:
Cross‑embodiment transfer in vision‑language‑action (VLA) models remains challenging because low‑level state and action spaces differ fundamentally across robot platforms. We observe that the high‑level cognitive process underlying manipulation, including scene perception, object identification, task planning, and sub‑task decomposition, is largely shared across embodiments. Based on this observation, we present ZR‑0, a 2.6 billion parameter end‑to‑end VLA model that uses dense Embodied Chain‑of‑Thought (ECoT) supervision to align cross‑embodiment representations within the vision‑language model (VLM). ZR‑0 adopts a dual‑stream architecture: a pre‑trained VLM (System 2) generates structured ECoT reasoning during training, while a Diffusion Transformer‑based action expert (System 1) produces continuous action chunks via flow matching. The two components are coupled through cross‑attention, with an attention mask that restricts the action expert to input prompt features only, enabling ECoT generation to be entirely skipped at inference without any performance loss. ZR‑0 is pre‑trained on ProcCorpus‑60M, a large‑scale dataset comprising approximately 60 million frames (approximately 1,000 hours) from over 400K trajectories, with dense ECoT annotations covering 96.8% of all frames. We evaluate ZR‑0 on three simulation benchmarks spanning single‑arm (LIBERO), bimanual (RoboTwin 2.0), and humanoid (RoboCasa GR‑1 Tabletop) embodiments, as well as real‑world experiments on the xArm platform, demonstrating strong performance across all settings. Code and model checkpoints are available at https://github.com/RUCKBReasoning/ZR‑0.
Authors:Deyin Liu, Jicheng Xu, Lin Yuanbo Wu, Xiaowei Zhao, Xiatian Zhu, Zhe Jin, Anjan Dutta
Abstract:
Human image animation, which aims to generate a video of a reference subject following a provided action sequence, has received increasing research interest. With the development of diffusion‑based/flow‑based video foundation models, existing animation works have began to upgrade the guidance information from 2D skeleton/pose to 3D modeling conditions. Despite achieving reasonable results, these approaches face challenges in synthesizing trajectory‑controllable human motion within natural scene under changed camera views. In this work, we present a scene‑adaptive human image animation framework that controls both human motion and camera trajectories within a reconstructed 3D environment for video generation. To achieve this, we first develop a ground‑adaptive 3D motion retargeting approach to enable user‑friendly motion trajectory control adapting to the changes of elevations of ground and orientations automatically. Then we design a viewpoint‑adaptive latent fusion mechanism to inject point‑cloud geometric priors through scene‑visibility masking into the generative process, providing precise guidance of viewpoint changes under camera control. Experiments on two standard human image animation benchmark datasets demonstrate remarkable improvements of our method over the state of the arts in related video generation metics. Project page: https://robinhood256100.github.io/web‑disp
Authors:Marek Polewczyk, Maximilian Schambach, Marco Spinaci, Sam Thelin, Johannes Höhne
Abstract:
We introduce FlexTab, a flexible encoder‑decoder architecture for in‑context learning on tabular data that pairs a single, task‑agnostic encoder with a suite of task‑specific decoders. Unlike existing tabular in‑context learners, which entangle feature representations with a specific prediction target, our design produces target‑agnostic row embeddings that can be leveraged across a wide range of downstream tasks within a table‑native in‑context learning setup. We demonstrate this flexibility on six distinct problems: classification, regression, anomaly detection, clustering, entity matching, and entity classification in relational databases. Both the encoder and the task‑specific decoders are trained on a large corpus of real‑world, unlabeled tables. FlexTab achieves state‑of‑the‑art performance on classification, regression, anomaly detection and entity matching, while remaining competitive with specialized models on entity classification in a relational setting. These results demonstrate that a single shared encoder, paired with task‑specific decoders, can serve as an effective general‑purpose backbone for diverse tabular prediction problems. The inference code and checkpoints will be made publicly available at https://github.com/SAP‑samples/flextab.
Authors:Wenjia Jiang, Zongyuan Cai, Yuanhang Shao, Chenru Wang, Boyan Han, Zhixue Song, Keyu Chen, Shengwei An, Xu Yang, Zhou Yang
Abstract:
Multi‑round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode: lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code‑generation task: from a scientific paper section, the agent writes Python in the open‑source Manim library to render a mathematical animation. We present ManimAgent, a self‑evolving multimodal agent that carries reflection experience across tasks through a dual‑channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision‑language model scores the rendered keyframes; the resulting signals populate a positive channel M+ that stores success rationales as soft Reference Examples, and a negative channel M‑ that stores validated failure patterns as hard Known Pitfalls. On a fixed‑probe evaluation against no‑memory, matched‑budget retrieval‑augmented generation, and shuffled‑memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows. We will release the code, frozen memory snapshots, and the task stream.
Authors:Ambrosio-Cestero, Gregorio, Galindo Andrades, Cipriano, Gonzalez-Jimenez, Javier, Ruiz-Sarmiento, Jose-Raul
Abstract:
Robotic applications increasingly rely on distributed computational infrastructures that combine embedded devices, edge servers, and cloud resources. This evolution, together with the collaborative nature of robotics projects, has made the development, integration, deployment, and long‑term operation of robotic systems significantly more complex. In practice, multi‑user robotics software teams face persistent challenges related to dependency isolation, compatibility, reproducibility, efficient sharing of specialized hardware, and deployment across heterogeneous environments. In this paper, we present CSAR (Containerized System Architecture for Robotics), a container‑centric architectural framework designed specifically for robotics teams and the edge‑cloud continuum. CSAR combines LXC/LXD‑based system containerization, ROS 2/DDS‑based communication, and a three‑layer edge infrastructure to organize computation into hardware‑affine, persistent execution environments that remain decoupled from the volatility of experimental workloads. Through its Infrastructure Core, Platform and Multi‑User Orchestration, and Compute and Acceleration layers, CSAR provides strong isolation, controlled resource sharing, and topology‑aware networking for distributed robotic applications. To demonstrate its validity, we describe a real deployment of CSAR in an academic robotics laboratory and evaluate it through representative use cases involving edge‑offloaded 3D SLAM and GPU‑accelerated semantic mapping. The results indicate that CSAR simplifies software integration, improves the utilization of shared computational resources, and facilitates safe prototyping, as well as reproducible and collaborative experimentation in robotics teams. The implementation described in this paper, including deployment templates, configuration files, and documentation, is available at https://github.com/goyoambrosio/CSAR.
Authors:Chengzeng You, Binbin Xu, Soteris Demetriou
Abstract:
The structural vulnerabilities of point cloud‑based 3D object detectors remain poorly understood. Prior work has studied adversarial robustness primarily on isolated 3D object models, while recent LiDAR spoofing attacks target richer and more realistic driving scenes but focus mainly on physical realizability rather than understanding detector behavior or attack efficiency. In this work, we investigate how LiDAR‑based detectors rely on spatial evidence in complex scenes and whether these reliance patterns can be exploited to induce failures more efficiently. To this end, we propose an explainability‑guided adversarial analysis methodology. We introduce the Saliency‑LiDAR (SALL) method, which aggregates Integrated Gradient attributions across scenes to produce universal saliency maps for LiDAR‑based 3D object detectors. Guided by these maps, we design the Explainability‑aware Frustum Attack (EFA), which selectively perturbs only the most influential frustums rather than uniformly attacking entire object regions. Experiments on KITTI and nuScenes, across detectors such as PointPillars and SECOND, show that EFA reduces detection recall by more than 15 percentage points while requiring 25‑50% fewer perturbed frustums than the state‑of‑the‑art non‑saliency‑aware baseline. These findings reveal that modern 3D detectors concentrate discriminative evidence in a small subset of spatial regions, exposing a structural robustness vulnerability in current LiDAR perception systems. Our code is released at https://github.com/SecMindLab/Saliency_LiDAR.
Authors:Tianyu Jin, Shuo Chen, Yida Wang, Liuyu Xiang, Yingzhuo Liu, Zhiyao Jiang, Yexin Li, Zhaofeng He
Abstract:
Long‑horizon strategic planning in complex strategy games demands concurrent reasoning across multiple decision domains under imperfect information and sparse reward. Existing LLM‑based agents suffer from three systematic failures: scene blindness from raw tile coordinates, context overflow and domain coupling from monolithic state dumps, and shallow cross‑game learning that treats each episode in isolation. We present SAGA, an LLM multi‑agent framework with three mechanisms each directly targeting one class of failure: (i) a Map‑Semantic Scene Graph that encodes typed spatial relations among game entities into per‑unit natural‑language context, resolving spatial blindness without global token inflation; (ii) a Tool‑Augmented Planner that pulls fine‑grained domain state on demand and dispatches per‑domain directives to dedicated specialist controllers, eliminating context overflow, domain coupling, and mechanical constraint violations; and (iii) a Dual‑Horizon Feedback Loop that combines periodic within‑game goal generation with structured cross‑game causal post‑mortem, enabling principled strategic evolution without manual reward engineering. Evaluated on FreeCiv, SAGA attains the highest mean civilization score ‑‑ the environment's sole sparse objective reward ‑‑ with lower variance than the two strongest baselines, and is the only method that significantly surpasses every baseline on infrastructure construction, the resource axis most readily sacrificed under multi‑objective conflict. It outscores the two strongest baselines in most head‑to‑head games while cutting output tokens (the dominant decoding cost) by 27%. Equipped with the cross‑game evolution module, SAGA reaches the highest end‑of‑chain score across five successive episodes. Ablation studies confirm that each architectural component contributes independently to this advantage.
Authors:Yuan Li, Youyuan Lin, Zitang Sun, Yung-Hao Yang, Kiyofumi Miyoshi, Chenhui Chu, Shin'ya Nishida
Abstract:
Blind image quality assessment (BIQA) is commonly built on two basic learning paradigms: regression and ranking. Regression calibrates absolute scores, whereas ranking recovers quality structure from ordinal relations. Although joint regression‑ranking supervision often improves BIQA, the relation between the two paradigms remains largely empirical and underexplored. In this work, we revisit what underlies regression and ranking and identify pairwise relational distance, termed quality margin, as their common bridge. Our derivation shows that, at the objective‑optimization level, both paradigms fit quality margins: regression fits margins induced by score endpoints, while ranking fits transformed or sign‑level margins through preference probabilities. Motivated by this insight, we propose MR‑IQA, a direct quality‑margin optimization framework for reinforcement learning (RL)‑based BIQA. MR‑IQA samples quality scores and optimizes pairwise margin errors as policy rewards, thereby modeling quality structure more explicitly. Experiments on six BIQA benchmarks show competitive general performance, and controlled comparisons demonstrate that MR‑IQA achieves the strongest average PLCC/SRCC over regression‑ or ranking‑based RL methods. Our findings provide a new insight into unifying regression and ranking, offering a theoretical basis for understanding quality‑structure modeling in BIQA and beyond. Code is available at https://github.com/RobinY99/MR‑IQA.
Authors:Vladimir Beskorovainyi
Abstract:
Organizations that cannot send data to a cloud API increasingly ask: how good is Text‑to‑SQL if the model must run on‑premises on open weights, and which popular accuracy "recipes" are worth their compute? We answer with an honest, fully reproducible benchmark on the BIRD development split (n=1534, Execution Accuracy), evaluating three open model families across two generations ‑‑ Qwen2.5‑Coder (7B/14B/32B), CodeLlama‑Instruct (7B/13B/34B), and Llama‑3.x (8B, 70B) ‑‑ under one matched protocol, ablating a model‑agnostic recipe (schema linking, self‑correction, self‑consistency) component by component, with every difference tested by the paired McNemar test. Four findings stand out. (i) Generation matters more than raw size, and the recipe is family‑robust: Qwen2.5‑Coder dominates the older CodeLlama at matched size (39.1 vs 20.9 at 7B), but a modern non‑Qwen model (Llama‑3.3‑70B, 49.2 on a matched serving) is competitive, so CodeLlama's weakness reflects its 2023 generation, not "non‑Qwen = weak". (ii) Self‑correction is a robust, near‑free win, significant on all three families where there is room to improve. (iii) Schema linking does not help, and a stronger linker does not rescue it: a retrieval/embedding linker with 96.5% gold‑table recall is statistically indistinguishable from no linking, ruling out the "weak lexical strawman" objection across three families. (iv) Self‑consistency is poor value (+0.13 pp for ~5x tokens, not significant). We report real per‑stage cost (/1k queries) and release all code, predictions, and summaries; archived code and data: https://doi.org/10.5281/zenodo.20952794
Authors:Ziyang Lian, Qingya Zhang, Hao Wang, Huiwen Xiong, Qi Yang, Lingyi Meng, Xiaoyi Gu, Rui Wang
Abstract:
Professional translation is often a team‑based process: translators, reviewers, and project managers must coordinate terminology, legal force, and accountability across documents. Yet many LLM‑based translation tools treat human corrections as isolated edits. Expert decisions made in one segment or by one member are rarely captured as reusable knowledge for the rest of the team. We present DeepTrans Studio, a collaborative translation workspace that lets professionals intercept selected nodes in an agentic translation workflow, review evidence, revise AI outputs, and save approved decisions to a shared team memory. During the demo, attendees will role‑play translators and reviewers, resolve preset terminology and legal‑modal risks, and see how their decisions are propagated to downstream segments and surfaced in a teammate's workspace as reusable precedents. The demo illustrates how human interventions in AI‑mediated work can become shared, traceable knowledge rather than one‑off corrections.
Authors:Guang-Xing Li
Abstract:
Continuous physical fields represent a large fraction of data under scientific investigation. Their multiscale structures are central to discovery, yet useful coordinates are not known in advance. Standard self‑supervised methods define context and targets in fixed image coordinates, posing a predictive task misaligned with fields organized across a continuous scale hierarchy. We introduce ScaleAware‑JEPA, a framework that constructs dense, label‑free latent coordinates for continuous scalar fields. Constrained Diffusion Decomposition (CDD) separates each field into pixel‑registered scale components and provides the scale coordinates that define the masking geometry. The resulting JEPA objective predicts hidden structure with a context footprint tied to the diffusion scale of each component rather than to an arbitrary patch size. Across MHD turbulence, interstellar molecular gas and urban nighttime‑light structure, the learned geometry maps back to coherent morphology, forming dense structural atlases without labels or predefined segmentation rules. By tying latent prediction to the scale hierarchy of a field, ScaleAware‑JEPA constructs latent coordinates through which complex physical patterns can be inspected before their relevant structures have been prescribed. Code is available at https://github.com/gxli/SA‑JEPA.
Authors:Youngseok Hwang, Sungho Bae, Dohun Lee, Jaeeun Seo, Jeehong Kim, Wonhee Lee, Hyunwoo Park
Abstract:
Maritime anomaly detection is essential for ensuring maritime safety, security, and efficient traffic management at sea, with Automatic Identification System (AIS) data serving as a primary data source. Despite its importance, most publicly available AIS datasets lack predefined anomaly labels, forcing prior studies to rely on either distribution‑based rarity or domain rule/expert‑assisted labeling. These approaches, however, face fundamental limitations: statistical rarity often fails to reflect practically critical events, while expert‑based labeling is costly, subjective, and difficult to scale. Moreover, both paradigms tend to overlook interaction‑driven hazards such as near‑miss approaches between vessels. To address these challenges, we propose an equation‑grounded anomaly taxonomy that is implementable under a limited AIS observation schema and extensible to other AIS datasets. Specifically, the taxonomy defines three anomaly types: unexpected AIS activity (A1), route deviation (A2), and close approach (A3), covering both single‑vessel and inter‑vessel anomalies. Building on this taxonomy, we introduce a unified score‑synthesize‑label pipeline that produces LLM‑guided plausibility scores, uses them to synthesize anomalies, and assigns timestamp‑level labels. To rigorously assess detection performance, we further design benchmark evaluation settings that account for variations in temporal‑window length and anomaly‑type composition, and evaluate a broad range of time‑series models and anomaly detection models. Together, these contributions provide a systematic basis for evaluating maritime anomaly detection methods across different anomaly types. Our code is available at https://github.com/snudial/open‑maritime‑anomaly‑detection.
Authors:Zhongqiang Song, Guanying Chen, Yuqi Zhang, Yin Zou, Chuanyu Fu, Zhiyuan Yuan, Chuan Huang, Shuguang Cui, Xiaochun Cao
Abstract:
This paper addresses the problem of monocular metric depth estimation in aerial UAV imagery. Although recent data‑driven methods have achieved remarkable progress in ground‑level scenarios, models trained primarily on street‑view and indoor datasets exhibit significant domain gaps when applied to aerial viewpoints. To tackle these challenges, we introduce AerialMetric, a benchmark dataset designed to evaluate and facilitate the adaptation of monocular metric depth estimation under UAV aerial viewpoints. The dataset consists of four complementary subsets collected from different sources, jointly covering real‑world photogrammetry data, controlled aerial acquisition settings, photorealistic synthetic scenes, and in‑the‑wild Internet imagery. Totally, AerialMetric provides 52K real‑world and 16K synthetic image‑depth pairs with reliable metric ground truth. Based on this dataset, we conduct systematic evaluations of existing state‑of‑the‑art models under aerial settings and investigate the impact of viewpoint, altitude, and camera parameters on metric depth prediction. In addition, by fine‑tuning representative metric depth model on our dataset, we establish a comprehensive aerial benchmark and achieve state‑of‑the‑art performance across diverse aerial imagery. Our dataset, code, and model weight are publicly available at https://kuieless.github.io/AerialMetric‑ECCV2026‑page/.
Authors:Aojie Yuan, Yi Nian, Haiyue Zhang, Zijian Su, Yue Zhao
Abstract:
Hallucination is the reliability bottleneck for LLM‑based agents, and fact attribution verifiers are the last line of defense ‑‑ yet today's verifiers emit only opaque binary labels, leaving agents unable to self‑correct and operators unable to audit. We present SEVA, a structured verification agent that emits evidence alignments, step‑by‑step reasoning chains, calibrated confidence, and a six‑category error diagnosis with actionable fixes. Training such an agent with RL is non‑trivial: standard binary reward on multi‑component output triggers advantage collapse ‑‑ within‑group reward variance vanishes and the GRPO gradient disappears. We resolve this with a process reward that decomposes verification quality into five independent components weighted 70/30 toward process signals, restoring the gradient and inducing an implicit curriculum ‑‑ the agent first masters verification behavior (alignment 0.917 ‑> 0.997, format 72% ‑> 100%), then outcomes (F1 64.9 ‑> 69.0). Structured output further enables a Verify ‑> Reflect ‑> Probe ‑> Refine self‑evolution loop, which over four rounds on a 7B model surfaces an unexpected structural finding: each round produces a benchmark‑specialist, not a generalist (+15 pp on HaluEval, ‑10 to ‑14 pp on TruthfulQA in the same model, persistent at 4x data). On ClearFacts, SEVA‑3B matches GPT‑4o‑mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output ‑‑ confirming a principle that should generalize: for any RL task with multi‑component generation, reward granularity must match output granularity.
Authors:Heshan Fernando, Quan Xiao, Yan Xin, Tianyi Chen
Abstract:
Telecom question answering (QA) is a challenging setting for retrieval‑augmented generation (RAG): evidence is fragmented across standards, papers, encyclopedic resources, and web documents, and answers often hinge on technical tables, equations, and specialized protocol language. In low‑resource subdomains, generator fine‑tuning can over‑specialize and degrade general capability, making query‑side retriever adaptation an attractive alternative. To this end, we ask whether a fixed‑generator, query‑adapted RAG system can outperform generator‑side adaptation, and which retriever objectives best support that setting. We motivate retrieval, rather than generator fine‑tuning, as the adaptation target through a capacity comparison: under bounded‑parameter and soft‑retrieval assumptions, query‑encoder tuning can have a smaller estimation term than supervised fine‑tuning when its effective dimension is smaller. We identify two particularly relevant objectives ‑‑ the latent‑document RAG likelihood, which optimizes generation utility, and the InfoNCE contrastive objective, which improves semantic retrieval geometry ‑‑ and leverage them jointly through a retriever optimization method targeting downstream QA performance in the telecom domain. Specifically, we introduce ARMOR, Adaptive Regularized Mixture Optimization for Retrievers, which learns separate temperatures for the RAG retrieval distribution and InfoNCE softmax and regularizes the adapted query encoder toward the frozen base query encoder. Across telecom‑specific retrieval and generative QA benchmarks, we show that ARMOR improves evidence retrieval and answer generation in several in‑domain settings. Code is available at https://github.com/heshandevaka/ARMOR.git.
Authors:Sunqi Fan, Lingshan Chen, Runqi Yin, Qingle Liu, Yongming Rao, Meng-Hao Guo, Shi-Min Hu
Abstract:
Data, as the fundamental substrate of modern intelligence, has greatly driven the development of current foundation models. Naturally, researchers aim to extend this paradigm to the domain of GUI agents, hoping to build strong GUI agents through a similar paradigm. However, GUI agent data cannot be directly harvested from the internet, making it costly and difficult to collect at scale. As a result, current GUI agents suffer from poor cross‑device generalization and limited visual grounding ability for fine‑grained GUI elements. As an attempt to address data challenge in GUI agents, we propose GUICrafter, a weakly‑supervised GUI agent leveraging massive unannotated screenshots to substantially reduce the reliance on expensive human annotations. GUICrafter explores a curriculum learning framework for training GUI agents through two progressive stages. First, the model learns visual grounding from large‑scale unannotated screenshots and webpages, leveraging the rich contextual signals inherent in GUI interactions without human annotations. Then, in Stage 2, we leverage a small amount of high‑quality data to calibrate the model via reinforcement learning. Experiments show that GUICrafter achieves competitive, or even superior, performance to advanced systems like UI‑TARS while using only 0.1% of its data. Furthermore, under the same amount of annotated data, GUICrafter surpasses all previous methods such as GUI‑R1. Code, data, and models are available at https://github.com/fansunqi/GUICrafter.
Authors:Jiamei Jiang, Jiajing Zhang, Feifei Mo, Linjing Li, Daniel Zeng
Abstract:
Planning often requires symbolic specifications that are both executable and verifiable. For large language models deployed in autonomous or decision‑support systems, failures in such formalization may lead to unverifiable decisions, execution failures, or unsafe downstream behavior. We present NL‑PDDL‑Bench, a multi‑domain benchmark for natural‑language‑to‑PDDL specification construction with planner‑verified executability and controlled difficulty scaling by object count. We further propose a planner‑in‑the‑loop framework that uses validator and planner diagnostics to revise non‑executable specifications through localized edits. Building on this infrastructure, we develop a planner‑grounded optimization recipe that combines parameter‑efficient Low‑Rank Adaptation supervised fine‑tuning, offline planner‑derived preference pairs for Direct Preference Optimization, and inference‑time planner‑in‑the‑loop repair, without requiring online planner calls during training. We also provide a unified evaluation suite for parseability, solvability, specification similarity, and outcome‑aware plan‑level consistency against planner references. Experiments on representative model families show substantial gains in planner success and plan‑level agreement, with improved robustness under difficulty scaling and cross‑domain variation. These results highlight the value of externally verifiable formalization for reliable deployment of LLMs in safety‑ or security‑sensitive planning systems. Code and data are available at: https://github.com/ibasicplan/NL‑PDDL‑Bench
Authors:Hairui Chen, Yanwu Yang, Jianfeng Cao, Hanyang Peng, Chenfei Ye, Ting Ma
Abstract:
Brain networks exhibit a modular community structure that varies across individuals and neurological conditions. However, existing self‑supervised learning (SSL) methods often overlook this heterogeneity, relying on generic masking strategies that fail to capture subject‑specific functional organization. We propose BrainPICM, a self‑supervised framework for brain network analysis via progressive individualized community aware masking. BrainPICM formulates ROI‑to‑community mapping as a progressive unbalanced optimal transport process, yielding soft assignments and per‑ROI confidence scores. Guided by these confidence estimates, a curriculum‑style masking strategy gradually incorporates low‑confidence, potentially pathological regions into training, enabling the model to learn both stable modular structures and individual variations. Additionally, a deviation‑aware aggregation module quantifies functional reorganization by measuring mass redistribution relative to a population template, enhancing interpretability and downstream prediction. Experiments on three fMRI datasets (ABIDE‑I, ADHD‑200, ADNI) show that BrainPICM consistently outperforms state‑of‑the‑art supervised and SSL methods in diagnostic accuracy, indicating that explicitly injecting modular community structure into masked modeling yields more functionally consistent and generalizable representations. The source code for this approach will be released at https://github.com/Hrychen7/BrainPICM.
Authors:Duc Anh Nguyen
Abstract:
We ask a simple question about decoder‑only transformers: between which two layers is the probability of a predicted token actually produced? Existing layer‑wise readout tools answer only approximately. The logit lens and its trained variant report a per‑layer level of probability but give no additive decomposition; their estimates are biased and non‑monotone across depth. Direct Logit Attribution and related residual‑stream methods are additive, but only in logit space, the softmax nonlinearity breaks additivity in probability space, precisely the quantity one usually cares about. Layer Conductance integrates gradients per layer, but attributes each to its own baseline and so does not sum to the total change in prediction. We introduce IG‑Lens, a telescoping application of Integrated Gradients along a single path through the hidden states from a baseline to the final layer. Crediting each segment to the layer it terminates at yields a layer‑wise attribution whose sum is exactly the change in target probability, with the softmax inside the integration path rather than linearized away. Our default estimator credits each integration step its observed change in target probability (a prediction‑aware reweighting in the spirit of IDGI) rather than its raw gradient. Because the readout is a one‑dimensional probability, this collapses each segment to a telescoping sum of endpoint values, so completeness holds exactly (to floating point) at any step count, removing Riemann discretization error while suppressing steps that show gradient sensitivity without a change in output. We give the telescoping identity and its proof, verify completeness to floating point, and describe a single‑pass batched implementation computing the full token‑by‑layer map without any backward call. Code: https://github.com/anhnda/IGLens.
Authors:Zhengyuan Li, Zeyun Deng, Yifan Shen, Liangyan Gui, Miaolan Xie, Joseph Campbell, Xifeng Gao, Kui Wu, Zherong Pan, Aniket Bera
Abstract:
Self‑collision remains a persistent challenge in SMPL‑based human pose estimation and motion generation. Under extreme articulations or stochastic motion synthesis, generated meshes frequently exhibit self‑penetrations, leading to physically implausible results. We propose PoseShield, a neural collision constraint defined directly in SMPL pose space. We formulate collision correction as a constrained optimization problem and connect the learned constraint with the Eikonal equation. Enforcing Eikonal regularization ensures non‑vanishing gradients near the collision boundary, improving numerical stability and robustness of the optimization process. Unlike prior methods that operate in the mesh space or rely on heuristic penalties, our approach operates directly in the low‑dimensional space of human poses and is theoretically grounded. The same learned constraint extends to human motion sequences, providing a generator‑agnostic post‑hoc collision corrector without retraining the underlying motion model. Experiments on a newly constructed SMPL pose benchmark show that our method achieves a 95.8% success rate and outperforms state‑of‑the‑art baselines.
Authors:Piyush Arora, Navlika Singh, Umberto Cappellazzo, Stavros Petridis, Maja Pantic
Abstract:
Audio‑Visual Speech Recognition takes two input modalities, acoustic and visual streams, where visual information from lip movements aids recognition when audio is noisy. Recently, LLM‑based AVSR models have emerged as a promising paradigm by connecting pre‑trained audio‑visual encoders to an LLM, achieving strong results in clean conditions. However, these models are predominantly optimized for clean acoustic conditions, with limited attention to making the LLM backbone robust to noise. No explicit mechanism is employed to produce stable representations under corrupted audio, leading to performance degradation in noisy environments. To address this, we propose VIB‑AVSR, which integrates Variational Information Bottleneck layers at targeted positions within the LLM backbone to regularize representations. VIB‑AVSR reduces degradation under noisy conditions across multiple SNR levels and noise types, without requiring architectural modifications or additional training data.
Authors:Yeji Kim, Housam Babiker, Mi-Young Kim, Randy Goebel
Abstract:
Mixture‑of‑Experts (MoE) architectures have recently been extended with role‑based mechanisms for interpretability. This is typically done by assigning semantic roles to individual expert components, for example roles like synergy, redundancy, and uniqueness in multimodal settings. However, whether such structural role decomposition preserves explanation faithfulness of the overall architecture remains largely underexplored. We hypothesize that inter‑expert representation overlap weakens effective role separation and degrades attribution‑based faithfulness, even when semantic roles are explicitly defined. To address this limitation, we introduce representation‑level decorrelation regularization to explicitly reduce inter‑expert similarity in latent space. Using representation decorrelation objectives, we encourage clearer specialization among experts by minimizing representation overlap. Our experiments show that across multiple multimodal benchmarks, this separation consistently improves explanation faithfulness, as measured by comprehensiveness, sufficiency, and their Area Over the Perturbation Curve (AOPC) summaries, while preserving task performance. We further show that these improvements are not limited to role‑based architectures such as Interpretable Multimodal Interaction‑aware MoE (I2MoE). Similar trends are observed in a standard sparse MoE baseline, suggesting that representation‑level separation may provide a more general mechanism for enhancing explanation faithfulness in MoE systems. Overall, our findings suggest that structural role decomposition alone may be insufficient to guarantee faithful explanations and that representation‑level separation helps improve explanation faithfulness. To support reproducibility, the source code and supplementary material are publicly available at https://github.com/dut0817/FL‑I2MoE_Decor.
Authors:Hang Su, Chao Sun, Zhaofan Li, Wei Hu, Juhua Liu, Bo Du
Abstract:
Vision‑language foundation models have shown strong potential in medical image analysis. Although foundation models for ultrasound imaging have recently emerged, the domain remains particularly challenging due to severe speckle noise, acquisition variability, and subtle anatomical boundaries, leading to high inter‑observer variability. Existing CLIP‑based models rely primarily on global image‑text alignment, limiting their sensitivity to clinically decisive local structures. We propose SonoCLIP, the first million‑scale region‑controllable fetal ultrasound vision‑language foundation model that integrates segmentation masks as mask‑channel visual prompts within the vision encoder, enabling joint global‑local contrastive representation learning. To support scalable region‑text alignment, we introduce a sigmoid‑based pairwise contrastive loss that improves stability under large‑scale supervision. We further curate a 1.44M‑image multimodal fetal ultrasound dataset spanning 24 standard planes for large‑scale pretraining. Extensive cross‑center evaluations demonstrate that SonoCLIP achieves superior zero‑shot transfer performance under both global and mask‑guided inference, establishing a controllable and clinically oriented foundation model for fetal ultrasound analysis. Our code and data are available at https://github.com/Harrison‑one/SonoCLIP.
Authors:Xiaomeng Fan, Wei Wu, Yuwei Wu, Zhi Gao, Shiyu Luo, Mingyang Gao, Haoyu Zhao, Zhenxin Diao, Yuxuan Ba, Lijia Feng, Yunde Jia, Mehrtash Harandi
Abstract:
Multimodal large language models (MLLMs) are increasingly expected to generate fine‑grained descriptions of visual content. However, we observe and theoretically show that generating fine‑grained responses poses a reliability challenge, i.e., fine‑grained generation is more error‑prone than coarse‑grained generation. This phenomenon suggests that models should generate the finest description that remains reliable rather than simply produce more specific outputs. To investigate this problem, we develop \textscGranFact, a granularity‑aware benchmark consisting of expert‑verified multi‑object images with coarse‑to‑fine category annotations. Then, we design a hierarchy‑aware evaluation algorithm, which assesses both whether model predictions are visually correct and how specific the correct predictions are. We also propose a reliability‑prioritized preference optimization method based on Direct Preference Optimization, which penalizes unreliable fine‑grained claims while rewarding reliable specificity. Experiments on \textscGranFact show that our method improves fine‑grained generation while preserving reliability. Code and data are available \hrefhttps://github.com/WeiWu2025/GranFacthere.
Authors:Sherwin Vishesh Jathanna
Abstract:
LLM‑based assistants transmit user queries verbatim to third‑party API endpoints that lie outside the user's audit or control. When those queries contain personally identifiable information (PII), the data persists on remote infrastructure subject to breach, subpoena, or policy change. Placeholder redaction (the prevailing mitigation) suppresses PII at the cost of semantic coherence, producing structurally degraded queries and correspondingly degraded responses. We present SurrogateShield, a client‑side proxy that substitutes detected PII with locally generated, type‑consistent surrogate values prior to transmission and restores originals in the response. No real PII crosses the network boundary. Detection runs through a three‑stage cascade (PatternScan, EntityTrace, and ContextGuard) covering 22 PII types and quasi‑identifier combinations grounded in Sweeney's k‑anonymity framework. Surrogate‑to‑original mappings are sealed in an AES‑256‑GCM encrypted per‑conversation ShadowMap that never leaves the device. Evaluations on a 1,124‑query corpus demonstrate that the cascade reliably detects PII, achieving an overall F1 score of 98.87%. Surrogate substitution substantially outperforms placeholder redaction in semantic utility, yielding a 13.26 pp improvement in BERTScore (roberta‑large), from 81.59% to 94.85%. Within this corpus, the local pipeline restricted real PII transmission across all tested query types; in a 100‑query adversarial trial, a prompted LLM adversary recovered no original values from surrogate‑substituted messages.
Authors:Weisong Liu, Haochen Wang, Kuan Gao, Yuhao Wang, Yikang Zhou, Zhongwei Ren, Jacky Mai, Anna Wang, Yanwei Li, Jason Li, Zhaoxiang Zhang
Abstract:
We propose MotionAtlas, a system for detailed captioning of motion‑centric videos, comprising (1) a dedicated human‑annotated benchmark, (2) a scalable, high‑quality pipeline to construct training samples, and (3) a family of powerful Video‑MLLMs. Unlike conventional global motion captioning datasets, we focus on region‑aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas‑Bench, a comprehensive benchmark comprising 2,073 multiple‑choice questions, meticulously annotated for a curated set of high‑quality, motion‑centric videos, to evaluate fine‑grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self‑bootstrap refinement to suppress fine‑grained hallucinations, yielding 159k high‑quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video‑MLLMs, including Molmo2 and Qwen3‑VL. For instance, MotionAtlas‑4B surpasses Qwen3‑VL‑4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.
Authors:Mijin Yoo, In Cho, Subin Jeon, Jiwoo Lee, Eunbyung Park, Seon Joo Kim
Abstract:
A 3D scene is understood through its objects, not the primitives that compose them. Yet feed‑forward reconstruction methods output dense, unstructured sets of points or Gaussians, leaving object‑level structure to be recovered after the fact. We propose a feed‑forward framework that decomposes a scene into instance‑structured 3D token groups directly from unposed multi‑view images ‑‑ compact object‑centric units from which reconstruction, segmentation, and manipulation all follow. Each token group pairs an instance token capturing entity‑level identity with anchor tokens that encode local geometry and appearance, which are decoded into a set of 3D Gaussians. This two‑level factorization decouples object identity from local appearance, making object instances a native interface of the representation rather than a derived product. The token groups are learned through differentiable rendering with joint reconstruction and segmentation supervision, requiring no 3D annotations. Our feed‑forward model surpasses per‑scene optimization baselines in class‑agnostic instance segmentation while remaining competitive in novel view synthesis. Beyond these metrics, the same token groups directly unlock instance‑level scene editing ‑‑ removing, translating, or inserting objects by operating on their groups ‑‑ as well as efficient open‑vocabulary 3D instance retrieval, where retrieval complexity scales with the number of instances rather than primitives.
Authors:Minghui Ma, Bin Guo, Han Wang, Mengqi Chen, Jingqi Liu, Yan Liu, Zhiwen Yu
Abstract:
Social influence dialogue changes user behavior by altering internal cognitive states. The central evaluation question is whether the user's beliefs, desires, intentions, and emotions measurably change over the course of conversation, a process‑oriented criterion that neither surface‑level text metrics (BLEU/ROUGE) nor single‑score LLM judgments can capture. We propose the Cognitive World Model (CogWM), an LLM‑based user model that reframes multi‑turn dialogue evaluation from ``what did the user say'' to ``how did the user's internal cognitive state evolves.'' CogWM jointly predicts BDI/E cognitive states and user utterances and serves as both a user simulator and an evaluation platform, using a three‑tier evaluation framework that covers turn‑level fidelity, trajectory‑level state dynamics, and task‑level composite scoring. Trained via our Summarize‑and‑Allocate (SaA) annotation pipeline on 150,454 user‑turn samples across four social influence scenarios, CogWM achieves 77.6% emotion accuracy (2.1× over GPT‑5.5). In 3600 multi‑agent discrimination trials, it distinguishes six commercial agents by their cognitive influence, with Llama‑4‑Scout ranking first (CTS +0.233). CogWM moves social influence dialogue evaluation from terminal judgment to process tracking. We have released our code\footnote\scriptsize Code: https://github.com/lucianma05‑create/CogWM and models\footnoteModel: https://www.modelscope.cn/models/LucianMa/CogWM‑14B.
Authors:Pawan Sasanka Ammanamanchi, Siddharth Bhat, Stella Biderman
Abstract:
Benchmarks for LLM‑assisted theorem proving in Lean are often treated as intrinsically reliable because every solved instance comes with a machine‑checked proof. However, the kernel only checks that a proof establishes a \emphformal statement; it does not verify that the statement faithfully encodes the intended informal problem, nor that evaluation harnesses are robust to trivial or adversarial solutions. We audit five widely used Lean theorem‑proving benchmarks and their forks, using corpus‑scale static checkers to surface 4,833 findings, including 398 mechanically certified issues such as counterexamples, vacuous theorems, and unsound axioms. We also document semantic defects such as missing hypotheses, problem simplification, incomplete or incorrect translations, and Lean‑specific specification hazards. Beyond dataset construction, we survey evaluation‑time failure modes and show, on corrected subsets, that defects can both inflate and deflate reported prover scores. We propose a fault taxonomy, a suite of automated checkers and recall‑oriented semantic audit prompts, and release standards to guide the creation of formal math datasets and to make evaluation more reproducible and trustworthy. Our checkers, audit prompts, and corrected dataset snapshots are available at https://github.com/Shashi456/atp‑checkers.
Authors:TaeYoung Kim, Gilhwan Kang, Tae Ihn Kim, Seungwon Song, Hun Keon Ko
Abstract:
While robust map maintenance has advanced significantly, existing studies have focused on specific tasks, especially dynamic object removal or change detection. In this paper, we take a holistic view of the map maintenance problem and propose MTD‑Map, a single‑stage framework that handles both dynamic object removal and change detection without separate task‑specific modules. MTD‑Map employs an explicit representation that compactly encodes the direction and duration of occupancy transitions through Mixture Transition Distribution (MTD) modeling. We develop a recursive MTD formulation that encodes historical occupancy patterns into an augmented state to capture high‑order temporal dependencies. Furthermore, a stability‑driven adaptive strategy balances noise suppression with the preservation of quasi‑static structures. Extensive experiments verify that MTD‑Map robustly removes dynamic objects and achieves competitive change detection performance, subsequently reducing computational costs. Our project page is available at: https://taeyoung96.github.io/mtd_map/.
Authors:Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon
Abstract:
Vision‑language dataset distillation (VLDD) compresses a large image‑text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision‑language models under strict data and compute budgets. Most existing methods match expert trajectories or cross‑modal statistics, yet still enforce full‑dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank‑deficient image‑‑text correlation, with shared semantics concentrated in a low‑dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low‑rank factorization, but does not explicitly control dominant alignment capacity and structure in the representation space. We thus propose a rank‑aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment‑capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes distilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality‑private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross‑modal retrieval and improved transfer indicators under fixed budgets.
Authors:Zain Naboulsi
Abstract:
When two companies bid to buy the same target, no one knows exactly what the target is worth. Each bidder pays for due diligence: costly, imperfect homework that sharpens its own private estimate before it bids. How much of that homework is worth buying? We build a simple computer model of the bidding contest and let it teach itself to bid well by playing against itself, the way a game engine learns chess. The economic question, how much diligence pays for itself, and the computational question, when the contest becomes too complex to solve exactly, are both controlled by a single thing: how many pieces of private information a bidder carries. Our main finding is that the right amount of diligence is modest and finite. It falls as diligence gets more expensive, and it falls further when both sides are doing their homework, because competition erodes the value of knowing more. We also test a recent claim from AI research: that simple, general self‑play methods can rival the specialized, expensive algorithms usually built for games like these. Running on an ordinary laptop with no costly frontier AI, we find the simple methods are the best of the self‑learning approaches, though purpose‑built exact methods still win whenever the game is small enough to solve outright. The simple methods earn their keep only once the game grows too large to solve exactly, which is the regime real deals live in, and there we show they still find strong bidding strategies. The contribution is threefold: a cheap, reproducible way to study deal‑making under uncertainty; a concrete, model‑based answer to how much due diligence is worth buying; and evidence about when lightweight, general‑purpose AI is good enough to replace specialized methods. We release all the games, code, and experiments.
Authors:Tuo Chen, Minjing Dong, Benlei Cui, Jian Liu, Jie Gui
Abstract:
Self‑supervised learning (SSL) pretrained models have become a dominant paradigm for visual representation learning, but they are vulnerable to backdoor attacks. Existing defenses struggle to defend against such attacks in a fully black‑box setting because they often require access to labels, attack patterns, or training data. To tackle this issue, we propose a new attack‑agnostic, model‑agnostic, and modality‑agnostic black‑box test‑time defense paradigm, called \emphPlatonic Representation Defense. It is inspired by the Platonic Representation Hypothesis, which suggests that large‑scale independently trained encoders converge toward compatible projections of the same underlying reality. We formalize this idea as a conditional energy function defined over source representations and a set of reference representations. The energy function is trained for detection through noise‑contrastive estimation and for representation purification through denoising score matching. Theoretically, the energy gap between matched and mismatched samples is lower bounded by the mutual information between source and reference representations. We demonstrate the effectiveness of our method on multiple self‑supervised encoders and more than 10 attacks. The method can perform both representation detection and purification, and achieves substantial performance gains across multiple attacks. Code is available \hrefhttps://github.com/jsrdcht/Platonic‑Representation‑Defensehere.
Authors:Sunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang
Abstract:
Video understanding is a fundamental capability for multimodal intelligence, and recent Multimodal Large Language Models (MLLMs) have achieved remarkable performance on Video Question Answering (VideoQA) benchmarks. However, existing benchmarks primarily evaluate whether models can perceive shallow visual cues, while rarely examining whether MLLMs can learn deeper knowledge or procedural skills from video tutorials and generalize them to downstream long‑horizon agentic tasks. To address this gap, we introduce VG‑GUIBench (Video‑Guided GUI Benchmark), a new benchmark designed to evaluate whether MLLM‑based GUI agents can follow video tutorials to complete corresponding GUI interactive tasks. Furthermore, we observe that the performance of models on both VideoQA and video‑guided agentic tasks critically depends on effective keyframe extraction. Based on this observation, we propose TASKER (Task‑driven And Scene‑aware Keyframe searchER), a keyframe extraction algorithm that jointly considers task relevance and scene dynamics to identify informative frames. Experimental results demonstrate that TASKER achieves significant performance improvements on both VideoQA and video‑guided agentic task benchmarks, outperforming the best baseline by 2.0% on the EgoSchema fullset and 1.8% on the NExT‑QA dataset, respectively. These results further highlight the potential of generalized keyframe extraction methods for video understanding tasks. Our code and data are available at https://github.com/VG‑GUI‑TASKER/VG‑GUI‑TASKER.
Authors:Subhadip Mitra
Abstract:
Inference‑time safety methods for large language models have proliferated, yet no systematic comparison exists. We evaluate five defense paradigms (no defense, static steering, CAST, AlphaSteer, probe‑gated) across seven instruction‑tuned models (7‑31B) and five attack types (GCG, AutoDAN, DeepInception, prefilling, intent laundering). Our central finding: prompt‑time activation defenses are structurally blind to prefilling attacks. AlphaSteer achieves 0% attack success on GCG, AutoDAN, and intent laundering but 50% on prefilling. We prove a corollary: any defense that gates intervention on a single layer's activation alignment with a benign reference (cone, subspace, or null‑space) is blind to attacks that craft activations to lie inside that reference, whether checked at prompt time or per token. As its constructive contrapositive we introduce response‑time probing: a linear probe on the model's hidden state at the first generated tokens, with AUROC 0.97‑1.00 across all seven models. Combined with a halt, it cuts prefilling attack success to 0/40 on every model with 0% benign false positives, outperforming Llama Guard 3. Cross‑template generalisation depends on probe depth, so we scope the claim to the canonical prefilling‑template family. Composing the response‑halt with AlphaSteer's null‑space steering gives an orthogonal split (the halt catches prefilling, AlphaSteer catches semantic attacks), reaching defense success 0.983 on Mistral and 0.994 on Llama and dominating both components. We further show MMLU fails to capture steering's true utility cost, which appears as behavioral hedging rather than factual loss, and that diverse negative training sets cut probe false positives from 80‑100% to near zero. Code, attacks, per‑sample results, and the judge prompt are released.
Authors:Dayong Liang, Kaisong Gong, Yi Cai, Changmeng Zheng, Xiao-Yong Wei
Abstract:
Existing multi‑agent debate frameworks suffer from two critical limitations: they rely on static architectures where agent roles and coordination patterns are fixed at design time, and they require instantiating multiple model copies, incurring substantial computational overhead. We propose Mixture of Debaters (MoD), a unified framework that enables dynamic self‑debate within a single model by leveraging the Mixture‑of‑Experts paradigm. We address three key challenges in adapting MoE for dialectical reasoning: (1) dual‑routing that decouples role allocation from process flow, dynamically determining when to debate versus when to synthesize; (2) momentum switching that smooths token‑level routing with local context, reducing expert‑switch jitter; and (3) unified self‑debate that encapsulates diverse debating personas into lightweight expert modules, eliminating inter‑agent communication while preserving behavioral diversity. Extensive experiments on multimodal benchmarks demonstrate that MoD outperforms both single‑model baselines and conventional multi‑agent systems, achieving superior accuracy with 3.7x lower latency and 87% reduction in token consumption.The source code can be accessed at https://github.com/YongLD/MoD.
Authors:Soroush Hashemifar, Havva Alizadeh Noughabi, Fattane Zarrinkalam, Ali Dehghantanha
Abstract:
Retrieval‑Augmented Generation (RAG) improves the factuality of large language models by grounding responses in external evidence, yet real‑world deployments remain fragile. Failures often stem from missing or weakly relevant evidence, as well as from generation that does not faithfully reflect the retrieved context. Many existing approaches rely on fine‑tuning, privileged access to internal model signals, or resource‑insensitive escalation strategies, which limits their practicality in black‑box and budget‑constrained settings. We propose D2R‑RAG (Diagnose‑to‑Repair RAG), a model‑agnostic and resource‑aware framework that combines lightweight failure diagnosis with adaptive repair. D2R‑RAG derives interpretable failure signatures from observable signals in the query, retrieved evidence, and generated response, and then selects from a small set of corrective actions under explicit latency and VRAM constraints. Experiments on FEVER and HotpotQA show that D2R‑RAG improves reliability over recent baselines and achieves better accuracy‑‑efficiency trade‑offs across multiple compute budgets. The code is available at https://github.com/CyberScienceLab/D2R‑RAG/.
Authors:Xuanhua Yin, Yuxuan Jia, Chuanzhi Xu, Weidong Cai
Abstract:
High‑resolution Diffusion Transformer (DiT) inference contains substantial spatial redundancy, but many spatially adaptive implementations encode regional computation as attention masks, which can inadvertently move scaled dot‑product attention (SDPA) away from FlashAttention fast paths. We identify this avoidable systems bottleneck as Mask‑Induced Dispatch Tax (MIDT) and show that it grows with latent sequence length. We introduce SAFE‑DiT, a training‑free Semantics‑Aware Fast‑path Execution framework that separates exact mask elision from approximation‑based spatial scheduling. SAFE‑DiT removes only provenance‑certified image self‑attention masks that induce a row‑wise constant shift in attention logits, preserves semantics‑bearing masks such as text‑padding masks, and realizes spatial adaptation through prompt‑conditioned token partitioning, selective state updates with global context, and periodic context refresh. We call this acceleration‑only configuration SAFE‑Core and report sensitivity‑weighted classifier‑free guidance separately as SAFE‑DiT+SW. On the evaluated PyTorch SDPA stack, redundant masks make long‑sequence attention 4.1× to 5.8× slower than the mask‑free path. On Lumina‑Next, SAFE‑DiT achieves 2.69× end‑to‑end acceleration at 1024^2 resolution and 5.09× at 2560^2, reduces peak memory at 2560^2 from 94.1 to 27.9 GB, and enables 3072^2 generation when dense inference runs out of memory. Paired metrics, component ablations, and a blinded human study support visual non‑inferiority of SAFE‑Core to the dense fast‑path baseline, while SAFE‑DiT+SW provides a separate prompt‑alignment operating point without reintroducing spatial self‑attention masks. Code is available at https://github.com/xuanhuayin/SAFE‑DiT.
Authors:Xiao Wang, Liye Jin, Dan Xu, Yuehang Li, Lan Chen, Yaowei Wang, Yonghong Tian, Jin Tang
Abstract:
Vision‑language tracking guided by natural language specifications leverages high‑level semantic cues of target objects to substantially boost tracking accuracy and robustness. Existing studies have verified that adaptively optimizing textual descriptions throughout the tracking process can effectively mitigate the semantic‑visual mismatch induced by dynamic variations in target appearance, position, and other inherent attributes. Nevertheless, mainstream methods that directly generate textual information via sequence models or large language models inevitably suffer from inherent defects, including erroneous target updating, excessive background distraction, and pervasive hallucination artifacts. To address the aforementioned limitations, this paper proposes a novel language dependency parsing mechanism to precisely distill core tracking principal components, encompassing target objects, semantic concepts, and background contextual information. On this basis, we perform component‑aware adaptive textual description updates by exploiting the powerful cross‑modal understanding capability of the pre‑trained vision‑language model Qwen‑VL. By integrating the proposed elaborately designed modules into the baseline framework, our method achieves consistent and superior tracking performance on multiple large‑scale vision‑language tracking benchmarks, including TNL2K, LaSOT, TNLLT, and OTB‑LANG. The source code and pre‑trained models will be released at https://github.com/Event‑AHU/Open_VLTrack.
Authors:Zhengqi Pei, Qingming Huang, Shuhui Wang
Abstract:
Chain‑of‑Thought (CoT) improves large language models (LLMs) on difficult reasoning tasks, but it often incurs long natural‑language rationales that are poorly aligned with efficient machine reasoning. We propose Communicative Language Symbolism Routing (CLSR), a test‑time framework in which multiple LLM agents autonomously invent, evolve, and share compact Language Symbolism Frameworks (LSFs), while a latent‑free router adaptively selects and composes these languages per query to optimize the accuracy‑token trade‑off. Unlike prompt optimization that refines surface instructions, CLSR treats each LSF as a reusable symbolic protocol with compact symbols, usage rules, and a message‑passing contract, and improves it through an evolutionary loop driven by correctness and token cost. At inference time, the router may invoke a single low‑cost LSF call, ensemble multiple LSFs, or execute a multi‑round LSF composition protocol on harder queries. Across challenging benchmarks, CLSR reduces latency‑oriented generated token completion by 3～ 6× compared to standard CoT while maintaining accuracy. We further derive an information‑theoretic lower bound on token cost under arbitrary symbolism and show that, under an interpreter‑realizability premise, multi‑round LSF protocols conditionally subsume program‑execution pipelines. Code is publicly available (https://github.com/pzqpzq/LSF_MDia).
Authors:Dishan Sarkar
Abstract:
Adaptive Financial Transformer (AFT) is proposed for stock return prediction under non‑stationary financial markets. The model incorporates a Market Regime Encoder, an Adaptive Gate Network, and an Adaptive Financial Context module to dynamically bias self‑attention based on semantic relationships between financial indicators. Unlike conventional Transformer architectures that treat all input features uniformly, the proposed approach groups 95 engineered financial features into 11 semantic categories and adapts attention according to latent market regimes. The study also identifies and corrects sequence alignment and backtesting issues that can inflate reported trading performance, and introduces a financially‑aware composite objective that jointly optimizes prediction error, directional accuracy, and non‑overlapping Sharpe ratio. Extensive experiments compare the proposed architecture against classical machine learning models, recurrent neural networks, and Transformer baselines using chronological evaluation, five random seeds, ablation studies, hyperparameter optimization, explainability analysis, and multi‑stock validation. Results demonstrate competitive predictive performance while reducing model complexity by 15.2% and improving parameter efficiency through feature selection, providing an interpretable Transformer architecture for financial time‑series forecasting.
Authors:Isao Kurosawa
Abstract:
Reliable event detection underpins induced‑seismicity monitoring for Carbon dioxide Capture and Storage (CCS) and geothermal operations, distributed acoustic sensing (DAS), and industrial condition monitoring. In each setting a detector must stay reliable both when sensors fail and when the signal is buried in noise. These two failure modes are routinely conflated, and architectural complexity is often credited with robustness it may not deserve. We assemble a unified binary event‑detection benchmark from three physically distinct real sources ‑‑ Hi‑net seismic waveforms, Utah FORGE 2024 borehole DAS, and MAFAULDA industrial vibration ‑‑ each mapped to a common 8‑channel, 256‑sample representation, and evaluate a fault‑tolerant detector (CEPHALON) trained with per‑sample sensor‑dropout against standard detectors (a 1D convolutional network, a temporal convolutional network, and a compact Transformer) trained with an identical recipe. On clean data every model is near‑perfect (AUC ~ 0.99). Under progressive sensor loss, simple models with sensor‑dropout are already robust and CEPHALON holds no advantage. Under additive noise, however, CEPHALON degrades far more gracefully: at ‑2.5 dB its overall AUC is 0.939 versus 0.532‑0.572 for the convolutional baselines. Same‑architecture ablations isolate the cause: disabling internal redundancy at inference reduces the low‑SNR advantage only modestly, whereas removing sensor‑dropout training collapses it (0.899 to 0.603 at ‑5 dB). The training recipe is therefore the dominant cause and parallel redundancy only secondary. We release a complete, numbered, reproducible pipeline so that every figure can be regenerated.
Authors:Yiming Jiang, Hanzhang Tu, Wenfeng Song, Siyou Lin, Liang An, Shuai Li, Aimin Hao, Yebin Liu
Abstract:
Uncalibrated volumetric video streaming for human reconstruction is essential for holographic communication and AR/VR, yet remains challenging due to the need for temporal consistency and computational efficiency from sparse‑view inputs. Existing methods rely on per‑scene optimization or calibrated cameras, while recent feed‑forward models are limited to low‑resolution (0.5K) single‑frame synthesis. We present HiReFF, a feed‑forward method for 2K‑resolution 360° human video reconstruction from uncalibrated sparse‑view videos. Our framework decomposes the problem into two key tasks: foreground 3D Gaussian reconstruction from sparse‑view videos (four views separated by 90°) and computationally efficient high‑resolution synthesis. To enable the former, we propose Scale‑synchronized Camera Calibration to resolve scale ambiguity for multi‑view supervision, and Gaussian‑wise Foreground Masking to reconstruct clean foregrounds by modulating Gaussian parameters. For efficient high‑resolution synthesis, our High‑resolution Side‑tuning achieves 2K rendering by augmenting the Gaussian head with supplementary features while keeping the backbone at 0.5K, drastically reducing computational overhead. Experiments demonstrate that HiReFF significantly outperforms existing methods in high‑resolution streaming volumetric video reconstruction. https://iridescentjiang.github.io/HiReFF
Authors:Aymen Mir, Riza Alp Guler, Jian Wang, Peter Wonka, Bing Zhou, Gerard Pons-Moll
Abstract:
We study the problem of physically plausible shadow casting when animating 3D Gaussian Splatting (3DGS) avatars, either individually or in multi‑avatar and object‑interaction scenarios, within existing 3DGS scenes. In contrast to prior methods that rely on binary hit tests and mesh‑based shadow casters, our method performs shadow computation entirely in Gaussian space, without requiring any mesh reconstruction. We introduce RAGA, a Ray‑Traced Gaussian Shadow Casting formulation based on exact ray‑Gaussian line integrals. For each occluding Gaussian, we integrate the opacity profile along the shadow ray and normalize by the theoretical maximum integral, producing a weight that captures how the ray traverses the occluder rather than merely whether an intersection occurred. To reduce temporal variance from clothing deformations in animated avatars, we further introduce an avatar proxy representation that stabilizes shadow casting while preserving visual fidelity. We implement RAGA using custom CUDA kernels integrated with the NVIDIA OptiX framework; as such, our shadow tracer runs at rates of about 50 FPS. We evaluate on single‑avatar, multi‑avatar, and avatar‑object interaction scenarios across multiple datasets, demonstrating substantially improved shadow realism, temporal stability, and scene coherence. Our project page is available at https://miraymen.github.io/raga/.
Authors:Zhihong Liu, Zheng Li, Jiachun Jin, Siqi Kou, Yitao Jian, Fengpei Yu, Zhijie Deng
Abstract:
While text‑guided image editing has made remarkable progress, it remains limited in structural portrait retouching. Textual descriptions struggle to convey fine‑grained changes to facial features and body proportions. To address this gap, we introduce Exemplar‑Based Portrait Photo Retouching, where the model is given an exemplar pair and tasked with inferring and applying the same retouching operations to a new query image. Existing exemplar‑based editing methods primarily focus on tasks with pronounced visual transformations. In contrast, structural portrait retouching involves extremely delicate and localized modifications, making accurate extraction and transfer of these edits challenging. To tackle this, we propose MirrorPPR, a novel framework designed to capture and transfer subtle structural retouching operations. Our method uses a Retouching Operation Extractor to capture the subtle differences from the exemplar pair. The extracted representations are then injected into a pre‑trained Diffusion Transformer (DiT) through a connector and Low‑Rank Adaptation (LoRA) modules. Furthermore, constructing perfectly aligned cross‑identity training pairs is severely hindered by operation misalignment. To overcome this, we propose an advanced data self‑augmentation paradigm that ensures strictly aligned retouching operations. To alleviate data scarcity and support this novel task, we introduce MirrorPPR47M, a large‑scale dataset with over 47 million retouched pairs. By structuring the dataset into simulated and professional subsets, we enable progressive curriculum learning to smoothly optimize the network. Extensive experiments demonstrate that MirrorPPR significantly outperforms existing baselines in both retouching quality and identity preservation. The project page is available at https://sjtu‑deng‑lab.github.io/MirrorPPR.
Authors:Dacheng Qi, Chenyu Wang, Jingwei Xu, Yi Ma, Shenghua Gao
Abstract:
Computer‑aided design (CAD) plays a fundamental role in modern manufacturing by providing the high precision required for industrial production. Recent large language model based approaches formulate CAD generation as a sequence prediction problem and have achieved promising results. However, existing methods and evaluation protocols primarily emphasize visual similarity, while overlooking precise geometric parameters and correct metric scale. Small numerical deviations that are negligible at the shape‑level may still violate industrial tolerance requirements, a problem further compounded by current autoregressive paradigms that utilize command sequence representations, aggressively quantize numerical parameters to ease LLM prediction. In this work, we present Pointer‑CAD v2. Compared with v1 (arXiv:2603.04337), this version directly predicts continuous values, bypassing the need for quantized numerical parameters and thereby eliminating quantization errors. Specifically, we propose a unified framework that decouples parameter reasoning from geometric construction through a Plan‑Then‑Construct paradigm. Our method first produces a structured design plan with explicit metric scale parameters. These parameters are organized into a dictionary and directly referenced during sequence generation via a pointer mechanism, eliminating discretization errors and ensuring dimensionally consistent execution. In addition, we construct a new large‑scale dataset with plan‑level annotation and introduce three hierarchical geometry accuracy metrics to evaluate parametric fidelity at the vertex, edge, and face levels. Extensive experiments demonstrate that Pointer‑CAD v2 consistently outperforms existing baselines and achieves substantial improvements in geometric accuracy, enabling reliable CAD generation for precision‑critical engineering applications.
Authors:Dingyi Yao, Xinqi Zhang, Lihui Peng, Jianming Hu, Danya Yao, Yi Zhang
Abstract:
Synthetic data mitigates the data scarcity problem in autonomous driving perception. However, the synthetic‑to‑real gap leads to performance degradation, hindering real‑world model generalization. Although current methods leverage diffusion models for photorealistic style transfer to bridge this gap, they critically ignore a practical asymmetry: while synthetic data possesses perfect pixel‑level annotations, real‑world style reference images generally lack corresponding labels. Consequently, existing methods relying on symmetric semantic guidance suffer from either prohibitive annotation costs or severe semantic misalignment. To address this dilemma, we formally propose a novel task: Asymmetric Style Transfer for Autonomous Driving (ASTAD), which requires semantically consistent transfer using only labeled synthetic content and unlabeled real‑world references. We further introduce the ASTModel, a training‑free two‑stage framework designed to bridge this domain gap under asymmetric constraints. ASTModel first extracts a coarse semantic prior from the unlabeled target, followed by dynamic prior refinement and class‑consistent style injection during the denoising process. Extensive experiments demonstrate that ASTModel significantly outperforms existing methods in downstream perception utility and structural fidelity, while offering a 3.2× inference speedup. This work aligns synthetic‑to‑real adaptation with practical constraints, holding the potential to accelerate the scalable deployment of robust autonomous driving systems. Code: https://github.com/Dingyi‑Yao/ASTAD.
Authors:Cong Wang, Haiyu Wu, Zhiwei Jiang, Zifeng Cheng, Fei Shen, Yafeng Yin, Qing Gu
Abstract:
Concept erasure aims to prevent image generative models from producing unsafe content while preserving their general generative capability. Meanwhile, next‑scale autoregressive (AR) image generation has recently emerged as a new generative paradigm characterized by next‑scale prediction, for which concept erasure remains largely unexplored. In this paradigm, semantic information is highly compressed at early scales, leading to severe entanglement between unsafe and unrelated semantics. In this paper, we propose ScaleErasure, an inference‑time concept erasure method that performs minimal intervention. ScaleErasure precisely selects and guides predicted logits that are most relevant to the unsafe concept, thereby enabling effective erasure under severe semantic entanglement. Specifically, ScaleErasure performs two additional forward passes conditioned on the unsafe concept and the corresponding safe concept, and leverages their outputs to guide the target logits away from unsafe concepts toward safe concepts. To enable precise and minimal intervention, logits selection and guidance are conducted across three dimensions: scales, tokens, and bit channels. Experiments demonstrate that ScaleErasure outperforms adapted baselines in the next‑scale AR paradigm, achieving more precise concept erasure while largely preserving general generative capability. The code is available at https://github.com/coziiizz/ScaleErasure.
Authors:Alex Kwon
Abstract:
LLM agents carry conclusions across steps and sessions in compressed memory, and memory products (e.g., mem0, LangMem) rewrite conversation into stored "facts" that later steps trust. We show this rewriting manufactures confidence: across our constructed agent settings, a casual, hedged remark becomes a confident, dated assertion the agent then obeys like a verified fact, granting every above‑clearance request it faces. No attacker is needed: a role that was true once and never corrected is stored as a flat fact and acted on like a deliberate injection. We then isolate what the agent responds to. It is not the source: attributed, unattributed, and even forged "system of record" claims all grant alike. It is the confidence of the phrasing. A hedge is discounted, a flat assertion is obeyed, and this holds with no special keyword. Not all hedges are equal, though: the evidential register is the least‑discounted, with "reportedly" obeyed like a flat assertion on most models. The obvious fixes fail. A passive "unverified" tag is ignored, and an active "do not trust this" instruction escalates even correct memory, so it is safe only by refusing to decide. The real fix lives in the store: keep the tentative phrasing rather than upgrade it. But that is hygiene, not a defense against an attacker who can simply write a confident lie. The deployable lesson is narrower and constructive: a single load‑bearing memory is the hazard, and one redundant source restores correct decisions. We release the harness and demonstrations.
Authors:Jiashuo Sun, Yue He, Wenxuan Liu, Tao Mao, Jiazheng Wang, Xiang Chen, Min Liu
Abstract:
Vision‑Language‑Action (VLA) models represent a promising direction for embodied intelligence in surgical robotics. Despite the prevalence of VLA benchmarks for general robotics, standardized evaluation platforms specifically designed for surgical contexts remain absent. To address this limitation, we present SurgVLA‑Bench, the first comprehensive benchmark for evaluating VLA models in laparoscopic surgical robotics. Leveraging the SurRoL simulation platform, we construct a hierarchical task taxonomy ranging from atomic actions to complete surgical procedures, complemented by a multi‑dimensional evaluation framework assessing action accuracy and semantic consistency. We then systematically evaluate two representative paradigms, including autoregressive models such as OpenVLA, and flow matching models such as π_0, π_0.5, and SmolVLA. Our experiments show that autoregressive models tend to excel in semantic understanding, while flow matching models often achieve higher task precision but may face generalization trade‑offs. However, even the best‑performing models remain far from satisfactory, as the constrained endoscopic field of view, restricted viewing angles, and frequent occlusions persist as fundamental physical bottlenecks. The code and data are available at https://github.com/VCL‑HNU/SurgVLA
Authors:Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter
Abstract:
Data‑generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated‑task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data‑level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM‑Playground/tree/prior‑dev.
Authors:Jintao Kong, Zhihao Zhang, Weihuang Chen, Liming Chen, Zhongyu Guo, Shuaiyu Liu, Hongbin Sun
Abstract:
Autonomous navigation in unknown environments requires a robot to efficiently reach a predefined goal while exploring without prior maps. Although progress has been made in this area, most existing works still rely on traditional planning methods with hand‑crafted rules, while learning‑based methods often suffer from limited environmental memory and challenges in simulation‑to‑real (sim‑to‑real) transfer. To overcome these limitations, we propose a Contextual‑memory Oriented Reinforcement‑learning (CORE) planner for robot navigation in unknown environments. The proposed CORE planner effectively combines the core advantages of traditional and learning‑based methods. Specifically, our method uses a sparse visibility graph for structured environment representation, reducing the computational overhead of dense grid maps, and employs a Transformer network to achieve a holistic environmental understanding, thereby significantly improving navigation efficiency. Moreover, we introduce a visibility graph‑based graph sparsification method and a contextual memory mechanism, which alleviates local optima and enhances computational performance in large‑scale scenes. Finally, our approach achieves zero‑shot sim‑to‑real transfer after training solely on image‑based environments, requiring no fine‑tuning. Experimental results show that CORE Planner consistently outperforms state‑of‑the‑art methods, including the traditional FAR Planner and all learning‑based baselines, across representative environments, reducing travel distance by 13% over traditional FAR Planner and by up to 48% relative to learning‑based baselines, with larger gains observed in more complex environments. In real‑world scenarios, CORE successfully navigates without human intervention, showcasing zero‑shot sim‑to‑real transfer. Code is available at https://github.com/BBD00/core_planner.
Authors:Seungryeol Baek, Wooseok Sim, Hogun Park
Abstract:
A Knowledge Graph (KG) represents facts as structured triples and is widely used to organize relational knowledge across diverse domains. Just as textual information ranges from words and sentences to complete documents, KG information can be interpreted at multiple levels, from entities, relations, and triples to subgraphs and entire KGs. However, existing KG embedding methods mainly focus on entities, relations, and triples, leaving graph‑level semantics largely unaddressed. Conventional graph‑level methods, which typically compare graphs based on structural patterns, are also insufficient because structural similarity alone cannot guarantee semantic similarity between KGs. To evaluate how well different methods capture such graph‑level semantic information, we study graph‑to‑graph semantic similarity, which determines whether a pair of KGs represents semantically corresponding underlying information. To obtain reliable ground‑truth correspondences, we construct a semantic matching dataset by modifying text documents, extracting KGs from both original and modified documents, and transferring their known correspondences to KG pairs. We compare text‑based, structure‑based, and KG embedding‑based approaches on each dataset. For the KG embedding‑based approach, we introduce two scoring functions: EmbPairSim, which uses maximal pairwise entity similarity, and AvgEmbSim, which uses a frequency‑weighted centroid. Experiments on WikiText‑2 and CC‑News show that EmbPairSim achieves up to 5.3 pp higher MRR than Sentence‑BERT while using substantially fewer parameters. These results suggest that KGE representations can serve as compact and effective signals for graph‑to‑graph semantic similarity in KGs. Our code is available at https://github.com/SeungRyeolBaek/KG‑to‑KG‑Semantic‑Similarity.
Authors:Rui-Xi Wang, Runzhong Wang, Connor W. Coley
Abstract:
Predicting tandem mass spectra (MS/MS) from molecular structures represents a central task in analytical chemistry with direct relevance to clinical metabolomics, systems biology, and adjacent disciplines. In this work, we revisit the problem through the lens of object detection on molecular graphs. Molecular fragmentation, a central step in MS/MS prediction, can be approximated as detecting a set of subgraphs (i.e., fragments) and their associated spectral contributions. Existing fragment‑based models follow a two‑stage paradigm ‑‑ first generating candidate fragments and then scoring them ‑‑ analogous to two‑stage R‑CNNs in computer vision. Towards higher accuracy and faster inference, we introduce GLACIER, a single‑stage transformer‑based fragment detection neural network for molecular graphs. This unified formulation eliminates the need for candidate enumeration, enabling scalable and globally consistent modeling of molecular fragmentation. GLACIER is faster and more accurate than existing state‑of‑the‑art by a significant margin, achieving 70.0% and 69.7% Top‑1 retrieval accuracy with and without contrastive finetuning on the MassSpecGym dataset (from the previous SOTA of 64.0%) and 52.5% and 38.5% respectively on the NIST'20 dataset (from 33.2%). Furthermore, GLACIER provides nearly 8‑fold inference speedup over our prior two‑stage model. Code is available at https://github.com/coleygroup/ms‑pred
Authors:Qiuyu Guo, Jianye Yang, Wenjie Zhang, Hanchen Wang, Ying Zhang, Xuemin Lin
Abstract:
Subgraph isomorphism counting is a fundamental problem in graph analytics, which aims to find the number of subgraph isomorphisms of a query graph in a data graph. The candidate tree‑based framework provides a promising foundation for subgraph counting tasks, offering a unified counting paradigm that can be extended beyond tree patterns. However, supporting subgraph isomorphism within this framework remains challenging, as it requires handling both the non‑tree edge constraint and the injective mapping constraint. Although existing solutions employ sampling or learning techniques to address these constraints in this framework, they still either suffer from inherent sampling failures or rely heavily on supervision. In this paper, we propose ASC, an algebraic subgraph counting approach built on the candidate tree‑based counting framework. In our method, the non‑tree edge constraint is directly incorporated into the candidate tree‑based counting process through a matrix‑based computation method, enabling subgraph homomorphism counting with high accuracy in polynomial time. Based on the resulting subgraph homomorphism count, we further apply a local sampling method to address the injective mapping constraint, thereby obtaining the final subgraph isomorphism count. Extensive experiments show that ASC can achieve substantially better and more stable performance over the baselines across various datasets, while scaling to billion‑edge graphs. Most impressively, as a non‑learning method, ASC can even achieve more than an order of magnitude higher average accuracy than the state‑of‑the‑art learning‑based method FlowSC with similar efficiency. This paper is the full version of the work accepted at SIGMOD 2027. The code is available at https://github.com/EricaGuoQiuyu/AlgebraicSubgraphCounting.
Authors:Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed, Sk Imran Hossain, Engelbert Mephu Nguifo
Abstract:
Jointly learning to segment and classify medical images demands cross‑task synergy, yet encoder‑sharing architectures limit decoder reconstruction to task‑private representations, permanently discarding the boundary cues and semantic priors each branch could supply to the other. This work introduces BTI‑Net, which establishes bidirectional communication at every decoder level through two parallel pathways via Task Interaction Modules (TIM). Spatial boundary context is gated into the classification branch, while global semantic priors multiplicatively modulate the decoder, with refined features propagating progressively from coarse semantics to fine boundary detail across all four decoder resolutions. Since cross‑task interaction is not equally reliable for every input, Uncertainty Proxy Attention (UPA) gates each TIM output per instance and per level using three signals that capture cross‑task alignment, scene complexity, and prediction confidence, without external annotations or additional inference passes. Experiments on three medical benchmarks spanning ultrasound, dermoscopy, and brain MRI demonstrate consistent improvements in segmentation IoU and classification accuracy over both encoder‑sharing and decoder‑interaction baselines. Ablation confirms adaptive gating contributes +2.36 IoU over fixed bidirectional interaction, and classification accuracy improves by up to +2.26 points over the strongest multi‑task baseline. UPA's uncertainty proxies serve as reliable single‑pass task‑failure signals without the overhead of stochastic sampling. Code: https://github.com/C‑loud‑Nine/BTI‑Net_MTL
Authors:Stephan Goerttler, Min Wu, Fei He
Abstract:
A growing number of techniques leverage the spatial structures that underlie many real‑world datasets. Despite these advances, the complementary task of estimating spatial structures and understanding their role within these techniques has often been overlooked. In neurophysiological data analysis specifically, numerous methods exist to estimate brain connectivity, but most are not explicitly model‑based, dynamic, multivariate, or directed. To address these limitations, we previously introduced noise‑driven heat modelling on graphs for neurophysiological connectivity estimation. In this study, we extend this framework by relaxing earlier noise assumptions and adding regularisation to improve robustness. We also develop a simulation procedure to characterise and evaluate our technique in a controlled setting. Finally, we demonstrate that the technique is able to capture meaningful spatial structure across two experiments, each using two real‑world datasets. The explicit model formulation of our connectivity estimator has the potential to improve the interpretability of graph‑based techniques across a wide range of applications. The code implementing our method is available at https://github.com/sgoerttler/Heat_Connectivity.
Authors:Young-Jun Lee, Seungone Kim, Minki Kang, Alistair Cheong Liang Chuen, Zerui Chen, Seungho Han, Taehee Jung, Dongyeop Kang
Abstract:
Would experience designing faster GPU kernels also help close in on a long‑standing open mathematical conjecture? Large Language Models (LLMs) integrated into evolutionary search have recently produced state‑of‑the‑art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce Evolution Fine‑Tuning (EFT), a mid‑training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K‑trajectory dataset spanning 10 domains and 371 optimization tasks, and fine‑tune open‑source LLMs from 2B to 9B parameters. Empirically, EFT confers cross‑task generalization: across 22 held‑out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test‑time RL, our model matches state‑of‑the‑art performance on two circle‑packing tasks and outperforms its base‑model counterpart on the Erdős minimum‑overlap problem. EFT thus serves as a "practice phase" for general‑purpose discovery agents that do not solve new problems from scratch.
Authors:Joy Bose
Abstract:
Computational knowledge graphs assign philosophical concepts to traditions based on corpus frequency: the school that mentions a concept most becomes its attributed tradition. We argue this conflates three measurements: textual power, historical priority, and philosophical significance, demonstrated using the darshana‑graph, a knowledge graph of 28,322 relationships across Hindu, Buddhist, and Jain traditions. Seven of the top 25 concepts by betweenness centrality predate their attributed school by 288 to 2,288 years. Moksha, attributed to Advaita Vedanta, appears first in Jain sources over 1,200 years earlier. The most reliable snapshot, at 300 BCE using only explicitly dated sources, shows a genuinely pluralistic structure: 59% Vedic, 24% Jain, 18% Buddhist. We also quantify a critical distortion in the temporal method: between 300 CE and 800 CE the network grows from 18 to 1,028 nodes, with 97.4% carrying Advaita proxy dates, revealing that apparent dominance reflects textual survival, not philosophical history. Beyond correcting attribution bias, the temporally grounded graph enables structural homology analysis across traditions. Ego‑network feature vectors applied to 48 temporally labelled concepts across eight traditions identify cross‑tradition concept pairs with high structural similarity. The method recovers known correspondences including purusha‑jiva (Samkhya/Jain, sim 0.990) and prakriti‑maya (Samkhya/Vedic, sim 0.972), and surfaces novel homologies. Nibbana and samsara score 0.954 despite being doctrinal opposites: both function as the ultimate reference concept in their tradition's soteriology. Cetana (Buddhist intention) and ajiva (Jain non‑living matter) score 0.923, a pairing absent from the literature. These are not claims of doctrinal equivalence but of measurable structural homology: different philosophical vocabularies navigating a shared conceptual space.
Authors:Darian Fernández-Gutiérrez, Rafael Bello, Marilyn Bello, Natalia Díaz-Rodríguez
Abstract:
Concept‑based Explainable AI (C‑XAI) seeks human‑understandable explanations grounded in semantic concepts, yet validation is limited by the scarcity of fine‑grained concept annotations. We evaluate whether mid‑scale Multimodal Large Language Models (MLLMs) can perform localized concept naming under strict zero‑shot conditions by assigning labels to bounding‑box regions at both object and part levels. We propose a reproducible zero‑shot evaluation protocol for Concept Naming (CoNa) with (i) closed‑set, category‑constrained prompting for moderate vocabularies and (ii) Open‑CoNa, an embedding‑similarity‑based strategy for large label spaces. Experiments with four MLLMs (7B‑32B) show consistent performance trends across datasets, reaching 62%‑88% object‑level exact‑match accuracy, highlighting the potential of training‑free concept annotation from localized regions. We discuss limitations and failure modes and release a reproducible framework to support future low‑cost C‑XAI research.
Authors:Yang Guo, Zihan Yang, Feifei Kou, Yulan Hu, Ran Zhang, Siyuan Yao
Abstract:
Small Object Detection (SOD) is a fundamental yet challenging problem in computer vision due to its limited spatial resolution and weak visual cues. Although recent approaches have achieved remarkable advances, the background distractors in different frequency spectra still degrade the performance. In this paper, we propose a novel small object detection framework termed SFDNet, which is capable of detecting small objects via efficient spectrum‑aware feature disentanglement. Specifically, we propose an Adaptive Spectrum Disentanglement (ASD) module that decomposes backbone features into multiple complementary spectral components, aiming to construct discriminative object‑relevant representations by discarding the background distractors for each component. Afterwards, to strengthen the semantic consistency of the similar objects in the same class, we propose a Class‑Wise Prototype Distillation (CPD) procedure, which establishes class prototypes for the object instances and enforces the compact representation by efficient prototype distillation. Extensive experiments on multiple challenging benchmarks show that SFDNet outperforms existing state‑of‑the‑art methods by a large margin. Our code is available at https://github.com/ManOfStory/SFDNet.
Authors:Yupu Lu, Hang Xu, Yizhou Chen, Jia Pan
Abstract:
Keypose‑based manipulation decomposes tasks into critical waypoints to simplify policy learning for long‑horizon tasks, but existing approaches rely on task‑specific heuristics or manual annotation to extract keyposes from demonstrations. We present an automatic trajectory labelling pipeline for grasp‑related tasks. This pipeline combines vision‑language models (VLMs) for semantic event detection with classical trajectory analysis for precise temporal alignment, requiring VLM inference only on one single demo among repeating ones per task. Using the labelled data, we train a keypose‑guided Diffusion Policy (DP) that exploits keypose conditioning to intervene demonstration distributions. We explore the possibility to apply this property for cross‑embodiment transfer: candidate keyposes are sampled and filtered via a reachability map, steering the policy toward kinematically feasible keyposes for the target robot. As a preliminary feasibility study, experiments on two robomimic tasks show that the labelled data produces policies matching a standard DP baseline, and that reachability‑filtered keypose conditioning may benefit zero‑shot transfer on the multimodal insertion task when feasible candidates are available.
Authors:Aditya Pratap Singh
Abstract:
IndicTrans2 is the strongest open English to Indic translation system, but like most systems it is trained on general text and tends to sound stiff on casual, conversational input. We adapt IndicTrans2‑1B to conversational register across all 21 Indic languages using only public data (OpenSubtitles, BPCC‑H‑Daily, Tatoeba). Plain fine‑tuning improves conversational chrF but forgets the general domain (it drops 3.9 chrF on FLORES for Hindi). Mixing general data back into training (experience replay) and then averaging the fine‑tuned weights with the base (model souping) removes that trade‑off: the resulting model beats IndicTrans2‑1B on conversational chrF in every one of the 21 languages (mean +6.2) while matching it on FLORES (mean change ‑0.17, all within 0.7 chrF). Paired bootstrap tests confirm the conversational gains are significant (p <= 0.004) and that FLORES is not significantly degraded. We are deliberate about scope: these are chrF gains, and a blind human plus multi‑model LLM check does not confirm them as a perceived quality improvement, so we treat the conversational gain as largely a register match to the references rather than proof of better translation. The techniques are not new; the contribution is the honest, end‑to‑end study in the Indic conversational setting.
Authors:Chenghao Qian, Nedko Savov, Lingdong Kong, Yeying Jin, Rui Song, Wenjing Li, Zhun Zhong, Jiaqi Ma, Gustav Markkula, Luc Van Gool
Abstract:
Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion. The key limitation of existing methods is the lack of diversity in weather appearance and effective control over weather dynamics (e.g., temporal evolution and particle motion). Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics. Additionally, general‑purpose video editors optimized for clean and aesthetic outputs tend to suppress heavy weather phenomena, making dense particle effects difficult to generate. To address these, we propose a Semantic‑Aware, Physics‑Informed, and Geometry‑Grounded framework that steers an off‑the‑shelf video editor to synthesize diverse global appearances and detailed particle dynamics. We factorize the synthesis into three conditional signals, so that each provides a distinct and stable source of guidance: semantics specifies what the weather should look like, dynamics governs how it evolves over time, and geometry determines where it should appear in the scene. Specifically, we introduce (1) semantic‑aware appearance anchoring to establish the target appearance from scene semantics and user input; (2) physics‑informed dynamic simulation to generate particle effects by simulating a Gaussian‑represented particle field under gravity, wind, and turbulence; and (3) geometry‑grounded video synthesis to align the simulated particles with target scene geometry and synthesize the final video. Experiments demonstrate that our method produces diverse, physically and visually realistic weather effects. Furthermore, we show that our synthesized data significantly improves the robustness of autonomous driving semantic segmentation under adverse weather conditions. Project page: https://jumponthemoon.github.io/w‑crafter/.
Authors:Satyasa Khadka, Sandhya Baral, Sudip Tiwari, Sharad Kumar Ghimire
Abstract:
This paper presents a robust Automatic Number Plate Recognition (ANPR) system tailored for Nepali license plates written in Devanagari script. In this paper, a pipelined model was used that integrates YOLO‑based models for license plate and character detection, followed by a CNN classifier trained on 34 Devanagari characters. Two publicly available data sets were used that incorporate diverse lighting, fonts, and structural variations. Data augmentation and additional training on embossed plates enhanced the generalizability of the model. The system achieved a recognition accuracy of up to 93%, demonstrating strong performance under real‑world conditions and providing a scalable solution for traffic management in Nepal. Code: https://github.com/Satyasakhadka/Nepali‑NumberPlate‑Character‑Recognition
Authors:Dong Wang, Wenwu Tang, Yun Cheng, Olga Saukh
Abstract:
Large language models have driven recent progress in language and multimodal AI, yet pre‑training them at scale is prohibitively expensive. Low‑rank pre‑training, which factorizes each weight matrix into a rank‑r product to reduce both parameters and FLOPs, is a promising response but typically lags full‑rank training in quality. We propose Duplicated Latent Residual (DLR), a training‑only, parameter‑free, foldable plug‑in for low‑rank pre‑training. DLR augments the standard low‑rank output Bz with a fixed structured residual alpha/sqrt(K) Expand_K(z) that replicates each latent coordinate K = ceil(d_out/r) times across the output. With alpha fixed, DLR adds zero learnable parameters per layer; after training, it is absorbed into the up‑projection in closed form, B = B + alpha/sqrt(K) R^T, so deployment parameter count, FLOPs and memory match the underlying low‑rank backbone exactly. Across LLaMA models from 60M to 7B parameters, DLR strengthens low‑rank pre‑training on C4 validation perplexity in most settings, with the clearest gains at 130M and above; folded checkpoints transfer cleanly to supervised fine‑tuning on standard benchmarks.
Authors:Deepayan Das, Davide Talon, Yiming Wang, Massimiliano Mancini, Elisa Ricci
Abstract:
Personalizing Multimodal Large Language Models (MLLMs) aims to recognize users' unique concepts from visual data and provide personalized responses. Although prior work has shown the benefit of concept descriptions and reasoning for this task, MLLM descriptions often include information, such as state and context, that does not help and may in fact hinder the unique identification of the target concept among other visually similar items. Effective descriptions of personal concepts should instead be accurate, discriminative, and free of distracting details. To achieve such descriptions, we introduce Reinforced Reference Game (RRG), a learning framework that promotes discriminative descriptions through a novel reinforced multimodal reference game. The MLLM plays both the roles of speaker and listener in a contrastive game setting, whose goal is to effectively communicate discriminative information about a target concept. Our approach formulates a verifiable contrastive reward over hard positives (dissimilar views of the same concept) and hard negatives (visually similar but different concepts). Empirically, RRG achieves state‑of‑the‑art across multiple tasks on three personalization benchmarks. RRG generalizes to unseen domains and outperforms existing methods based on concept descriptions and personalization‑specific RL frameworks. We will release code and models in the project page.
Authors:Zhihui Ke, Yuyang Liu, Xiaobo Zhou, Tie Qiu
Abstract:
3D Gaussian Splatting~(3DGS) has emerged as a promising paradigm for reconstructing streamable free‑viewpoint video~(FVV) from multi‑view videos. However, 3DGS‑based FVVs typically lack user interaction and editing capabilities, which diminishes the immersive experience. Recent research has integrated language features from CLIP into 3DGS via distillation, enabling open‑vocabulary queries and supporting many downstream applications. Nevertheless, the stringent requirements of FVV, low frame size and high FPS, make current language Gaussian representations unsuitable for language‑embedded FVV. In this paper, we propose DLGStream, a novel language‑embedded FVV representation that streams time‑varying language features alongside Gaussian attributes to support 4D environment interaction, scene editing, and spatial intelligence. Specifically, we propose a dual‑opacity dynamic language Gaussian representation, which maintains two opacity attributes for color and language features to deal with performance degradation that occurs when colors and features are jointly optimized. Furthermore, we introduce an interpolation‑based deformation field to reduce temporal redundancy. This deformation field can also be used for 4D frame interpolation, boosting FVV sequences from low to high FPS. Experimental results demonstrate that DLGStream achieves superior performance in both on open‑vocabulary segmentation and reconstruction quality with an average frame size of merely 43 KB. The code is available on \hrefhttps://github.com/kkkzh/DLGStreamhttps://github.com/kkkzh/DLGStream.
Authors:Yuxuan Yang, Feiyang Ren, Bowen Zeng, Dalin Zhang, Jinpeng Chen, Gang Chen, Huan Li
Abstract:
Long‑context LLM inference faces a fundamental conflict: head‑adaptive compression algorithms (e.g., Top‑p nucleus sampling) offer superior accuracy by dynamically fluctuating memory budgets, yet modern inference engines (e.g., vLLM) demand rigid, static memory patterns to leverage CUDA Graphs and PagedAttention. We resolve this ``Static‑Dynamic'' mismatch with HARD‑KV, a unified framework that that bridges dynamic selection with rigid system constraints. HARD‑KV introduces a Cascade Cache hierarchy, managing the token lifecycle across dense, sparse, and condensed tiers. Crucially, we propose a Logits Calibration mechanism that normalizes diverse importance metrics into a unified probability space, enabling consistent Top‑p budgeting across heterogeneous heads. To bridge the efficiency gap, we offer a system‑level solution, which rewrites fragmented, dynamic indices into contiguous physical layouts compatible with high‑performance inference engine. Extensive experiments on math‑reasoning benchmarks (AIME, U‑Math) verify that HARD‑KV achieves up to 2× throughput improvement over static baselines while maintaining high‑fidelity generation in 10k+ token scenarios. Code is available at https://github.com/SuDIS‑ZJU/HARDInfer.
Authors:Shuo Cheng, Chuye Zhang, Alfred Cueva, Caelan Garrett, Ajay Mandlekar, Danfei Xu
Abstract:
Human videos are a scalable source of supervision for robot manipulation, as they are abundant and naturally capture rich object interactions. However, transferring human demonstrations to robots remains challenging due to embodiment mismatch, scene variation, and robot‑specific feasibility constraints. We present Human2Any, a framework for learning reusable object‑centric interaction priors from human videos without requiring real‑world robot demonstrations in the target task contexts. Human2Any represents manipulation through object‑object interaction motion, capturing task‑relevant scene changes while abstracting away embodiment‑specific details. It composes learned interaction priors with robot‑side feasibility reasoning and motion planning, allowing the same human‑derived knowledge to adapt to different embodiments, scene geometries, and task contexts. We validate Human2Any across diverse manipulation settings, including real‑world experiments on a Franka tabletop setup and an RBY‑1 humanoid mobile robot, demonstrating robust interaction‑centric manipulation without real‑world robot training data. Project website: https://human2any.github.io/.
Authors:Dong Li, Yujie Zheng, Chengdeng Cao, Siyu Teng, Yuchen Li, Yang Gao, Long Chen
Abstract:
Lunar helium‑3 is a highly valuable strategic resource, pivotal to the advancement of both deep‑space exploration and space mining. Existing lunar helium‑3 exploration methodologies rely primarily on indirect measurements via remote sensing, which are often characterized by limited precision, low reliability, and insufficient spatial resolution. In this paper, we introduce He3‑Seeker, an active robotic exploration method for helium‑3 distribution mapping. First, we provide a formal definition of the active helium‑3 exploration problem. Subsequently, we developed the He3‑Seeker framework, which is conceptually based on multi‑point drilling, sampling, and in situ analysis. In particular, we use robotic information planning (RIP) to guide autonomous robot navigation and active sensing. Additionally, to thoroughly evaluate the proposed algorithm, we introduce a reliable method for generating reference data of lunar helium‑3 distribution based on low‑resolution orbital remote sensing measurements. Simulation experiments verify that He3‑Seeker achieves both rapid and high‑fidelity mapping of helium‑3 distribution, providing a reliable solution for resource exploration tasks. Our code and simulation environment will be publicly accessible at https://github.com/OpenSpace‑Lab/He3‑Seeker.
Authors:Minh Son Hoang, Dinh Phu Tran, Quyen Nguyen Duc, Dam Hoang Phuong, Daeyoung Kim
Abstract:
Diffusion prior‑based methods have shown impressive results in real‑world image super‑resolution (ISR), yet two key challenges persist: balancing pixel‑level fidelity with semantic quality, and adapting to diverse degradations. Existing dual‑branch approaches freeze the pixel module during semantic training, but the semantic branch can still expand capacity within the pixel subspace, precluding genuine perceptual improvement. Moreover, using a single static adapter cannot generalize across heterogeneous real‑world corruptions. To address both issues, we propose FreqOrtho‑SR, which comprises: Frequency‑guided Mixture of LoRA Experts (FreqMoE), it routes inputs to specialized experts via a non‑parametric FFT‑based degradation‑feature extractor that encodes frequency‑domain signatures, enabling stable and interpretable specialization across corruption types; and Orthogonal Gradient Projection (OGP), which reframes the dual‑objective optimization as a subspace‑constrained problem: by extracting the pixel‑fidelity subspace via SVD on combined expert weight deltas and projecting semantic gradients onto its null space, OGP guarantees orthogonality between the two objectives, enabling genuinely complementary learning without mutual interference. Experiments show that FreqOrtho‑SR achieves competitive overall performance and a strong fidelity‑perception trade‑off across multiple benchmarks with efficient single‑step inference. The source code of our method can be found at \hrefhttps://github.com/sonhm3029/FreqOrtho‑SR\textttsonhm3029/FreqOrtho‑SR.
Authors:Han Luo, Bingbing Wen, Lucy Lu Wang
Abstract:
LLM agents are expected to act over multiple turns, using search, browsing interfaces, and terminal tools to complete user goals. Yet not every goal is well specified or achievable in the available environment. In such cases, a reliable agent should recognize that further interaction is unlikely to help and abstain from additional tool calls. We define Agentic Abstention, the problem of deciding when an agent should stop acting under uncertainty. Unlike standard LLM abstention, which is usually evaluated as a single‑turn answer‑or‑abstain decision, agentic abstention is a sequential decision problem: an agent can answer, abstain, or gather more information at each turn, and the need to abstain may only become clear after interacting with the environment. We study this problem across web shopping, terminal environments, and question answering, evaluating 13 LLM‑as‑agent systems and 2 agent scaffolds on more than 28,000 tasks. Our results show that the main challenge is not only whether agents can abstain, but also when they abstain. Some agents never abstain when they should, while others do so only after many unnecessary interactions. This gap is especially large on tasks where the instruction appears feasible until the environment reveals otherwise (e.g., no valid result matches the instruction). We further find that model scale, reasoning, and agent scaffolding affect abstention in different ways, where larger or more capable models sometimes perform worse at timely abstention. Finally, we introduce CONVOLVE, a context engineering method for improving agentic abstention that distills full interaction trajectories into reusable stopping rules. On WebShop, CONVOLVE substantially improves timely abstention without updating model parameters, raising Llama‑3.3‑70B's timely recall rate from 26.7 to 57.4. Our dataset and code are available at https://lhannnn.github.io/agentic‑abstention
Authors:Darrell Lewis-Sandy
Abstract:
We ask under what conditions an agent with a harm‑minimizing policy can displace an approval‑seeking (RLHF) agent in a competitive market, and when that policy is sufficient to prevent community harm. We use evolutionary game theory (finite‑population Moran‑Fermi pairwise comparison) to formalize this subject to assumptions of wisher hindsight, peer testimony, a monotone harm ledger, sufficient information density of community feedback, and a finite, depleting resource pool, in a negative‑sum environment. We show that adoption is favored when the prior distributions on how readily wishers attune to community sentiment are monotone, exhibit endpoint inversion, and have a centro‑symmetric pairing property, and demonstrate this with several long‑tailed priors (Hill, Pareto, Lomax, Frechet). Where it is favored, a critical adoption level separates communities that drift back to the approval‑seeking agent from those for which the audited agent fixes; above that level fixation is the overwhelmingly likely outcome. We derive when fixation is attainable as a bound on the effective (informational) size N_c of the community, which must be small enough to allow fixation before depletion. We present these as Theorems 5.4 and 5.5; the algebraic and finite‑grid backbone is machine‑checked in Lean 4, with the barrier‑crossing asymptotics retained as explicit hypotheses. We show that a self‑audited agent with a community ledger is not, in general, sufficient to prevent community harm. Sufficiency depends both upon the alignment of the agent's audit with community values and the timeframe over which harm is evaluated. Regardless of alignment, once adoption reaches dominance, the state is absorbing. The same policy that reduced harm under alignment becomes a trap, welfare‑negative under misalignment and, even under alignment, one that locks in harm deferred past the adoption horizon.
Authors:Dawon Ahn, Auder Der, Evangelos E. Papalexakis
Abstract:
Accurately explaining hidden patterns in multi‑aspect data has typically been done by leveraging labels and/or accompanying auxiliary metadata. However, labels and auxiliary data may be inaccurate (e.g. nonstandard, inconsistent), insufficient (e.g. static tabular metadata for time‑dependent recordings), or unavailable. % We propose \fullmethod (\method), which leverages the knowledge of large language models (LLMs) to explain the hidden patterns in human narratives. \method uses task‑agnostic and task‑specific prompts to explain extracted co‑clustered latent patterns from tensor decomposition. To evaluate these explanations, we test the LLMs on forward and backward inference tasks. % Our demo system is available at https://github.com/dawonahn/ECML_PKDD_AnTenA.
Authors:Shanghua Gao, Ayush Noori, Richard Zhu, Curtis Ginder, Zhenglun Kong, Xiaorui Su, Justin Kauffman, Benjamin S. Glicksberg, Joshua Lampert, Ankit Sakhuja, Ashwin Sawant, ATHENA-R1 Evaluation Consortium, David A. Clifton, Noa Dagan, Ran Balicer, Marinka Zitnik
Abstract:
Treatment reasoning underpins every therapeutic decision, integrating disease context, comorbidities, medications, contraindications, and evolving biomedical knowledge to select an appropriate therapy. It is inherently iterative: candidates are weighed against many constraints, revised as evidence emerges, and grounded in verifiable sources. Here we introduce ATHENA‑R1, an AI agent for treatment reasoning across all FDA approved drugs since 1939, trained by reinforcement learning over a universe of 212 biomedical tools. At each step it identifies missing information, selects and runs relevant tools, and incorporates the evidence. To train it without human‑annotated traces, we build a two‑level self‑learning framework: multi‑agent systems construct the tools, tasks, and reasoning trajectories for supervised fine‑tuning, then reinforcement learning with scientific feedback rewards reasoning quality (evidence gathering, grounded tool use, logical non‑redundancy). Across five benchmarks of 3,168 drug reasoning tasks and 456 patient treatment cases, ATHENA‑R1 outperforms language models and tool‑use systems, reaching 94.7% accuracy on open‑ended drug reasoning and 82.9% on treatment reasoning, 17.8 and 10.7 points above GPT‑5. In blinded evaluations by experts from 28 rare disease organizations, it is preferred over reference models on all criteria, and physicians rated it favorably on complex hospitalized cardiovascular and infectious‑disease cases. Adverse‑event hypotheses it generated, tested in electronic health records from 5.4 million patients, reached adjusted odds ratios of 1.48‑1.84, with no elevation among negative controls. Because it requires knowing what evidence to seek before concluding, treatment reasoning has long been hard for AI; we show it can be reframed as a learnable process of iterative evidence gathering that reinforcement learning can train AI to perform.
Authors:Ximiao Zhang, Min Xu, Xiuzhuang Zhou
Abstract:
Current anomaly detection methods primarily focus on structural anomalies, while paying insufficient attention to anomalies that violate logical constraints. Conversely, top‑performing logical anomaly detection approaches address this by modeling global semantic consistency, but perform poorly on subtle structural anomalies due to inadequate detection granularity. In this paper, we propose LogiCo, a unified framework for Logical and structural anomaly detection via Component‑level feature reconstruction. Unlike existing methods that rely on explicit global semantic modeling, LogiCo employs a novel component‑level feature reconstruction technique to capture inter‑component logical constraints. Specifically, LogiCo maps pre‑trained image features into a discrete component‑level feature space and performs collaborative feature reconstruction at both component and patch levels, enabling it to effectively detect both logical and structural anomalies. Furthermore, to address the specific challenge of count‑related logical anomalies, we integrate a segmentation‑map discriminator that extends the model's capability to identify quantitative inconsistencies. LogiCo achieves state‑of‑the‑art performance on both logical and structural anomaly detection across four benchmarks, including MVTec‑LOCO, MVTec‑AD, VisA, and Real‑IAD, demonstrating its superiority and practical feasibility. The code is available at https://github.com/cnulab/LogiCo.
Authors:Eryn Libert-Scott, Emma A. M. Stanley, Vibujithan Vigneshwaran, Matthias Wilms, Erik Y. Ohara, Nils D. Forkert
Abstract:
Causally linking disease‑related factors to image‑derived biomarkers provides a powerful pathway to understanding disease mechanisms. Despite growing interest in applying causal artificial intelligence (AI) approaches for this task, these methods still need to be adapted for complex medical images, and especially, neuroimaging. However, the lack of ground‑truth data presents a barrier to development. To bridge this gap, we developed and tested a method for generating synthetic neuroimages, which adhere to a user‑specified causal structure describing the non‑image to image variable relationships, permitting the creation of ground‑truth neuroimaging datasets. In the simulated T1‑weighted magnetic resonance images, anatomical variability is modeled by sampling from a subspace estimated from real data and deforming a template image to create unique simulated subjects. Causal relationships are encoded via precise volumetric changes of any region‑of‑interest without unwanted global artifacts. We achieved relative volume errors of 0.3‑2.66% for the targeted regions‑of‑interest and demonstrate their statistically significant causal relationships, while maintaining mean absolute errors for non‑target brain regions between 0.034‑0.397ml. An initial evaluation of causal discovery methods exposes their limited ability to suppress spurious connections, highlighting the need for image‑appropriate methods. Our framework is the first to enable the generation of realistic synthetic 3D neuroimages with explicit causal control that can serve as the missing ground‑truth data necessary for the objective benchmarking and development of causal AI methods.
Authors:Ioannis Tzachristas, John Pavlopoulos
Abstract:
Large Language Models (LLMs) often face ethical tradeoffs in which several responses may be defensible but express different priorities, such as fairness, honesty, courage, or restraint. We introduce VirtueMap, a framework for describing these patterns through an Aristotelian virtue‑ethics lens. Instead of asking for a single correct answer, VirtueMap asks humans or LLMs to rank all five responses to each of seven general, non‑lethal, non‑political, and non‑religious ethical dilemmas. To define the reference orderings used for scoring, we first proposed, for each dilemma and virtue, an ordering of the five responses from most to least expressive of that virtue. We then collected more than 100 respondent evaluations per ordering and retained it as operational ground truth only when at least 95% confirmed it. Rankings are scored against these retained orderings using normalized Borda alignment, yielding profiles over Practical Wisdom, Justice, Truthfulness, Courage, and Temperance. We apply VirtueMap to nine LLM families in a repeated‑run evaluation and find high mean rank consistency (90.3%), with the largest differences appearing on Courage, Temperance, and Justice. We also release an interactive website that computes profiles locally in the browser and compares respondents with measured LLM profiles.
Authors:Yong Yi Bay, Kathleen A. Yearick
Abstract:
People overthink; language models over‑sample, and the extra effort can talk both into a worse answer. Reasoning systems answer a hard question by sampling it many times (test‑time scaling), and the more they draw, the more often a correct answer turns up somewhere, so coverage, the fraction of problems with at least one correct try, climbs and appears to be progress. But a deployed system must return one answer, and choosing it, not knowing which try is right, is selection; selection is capped, and past a point extra samples only make the model surer of a confident mistake, even as every draw adds cost. The gap between climbing coverage and stalled selection, the identifiability gap, is the answer a model can produce but not pick. So the real question is not whether to sample but how far, and the answer is: not far. For picking an answer, the vote has already settled within a few dozen draws, the modal ceiling; for scoring a benchmark, sooner still, the correlation ceiling. Beyond that, extra draws cost compute and add nothing, and can even make the answer worse. This paper turns the cutoff into a single number, the effective number of samples, that any sampling run already reveals. The bottleneck is recognizing a right answer, not generating one.
Authors:Ruitao Chen, Mozhang Guo, Jinge Li
Abstract:
Deformable 3D Gaussian Splatting (3DGS) has emerged as an efficient approach for rendering dynamic scenes in a wide range of 3D applications. However, existing deformation field‑based approaches largely lack explicit object‑level modeling, often resulting in inconsistent Gaussian deformations within individual objects and unwanted coupling between different objects. To address this limitation, we introduce a semantics‑guided framework that enforces dynamic regularization at the object level, aiming to achieve spatially consistent object‑wise deformation. Specifically, we first extract segmentation masks using the Segment Anything Model (SAM) and derive semantic features from input images. An object‑ID map is then constructed via feature relevance matching with a predefined object dictionary. Guided by this object‑ID map, we identify the pixel‑wise top‑k contributing Gaussians for each object and impose consistency regularization on their deformation parameters, including position, scale, and rotation. Unlike prior methods that learn deformation fields without explicit object‑level constraints, our approach incorporates semantic cues to guide deformation behavior at the object level. Experimental results demonstrate that our semantics‑aware regularization improves object‑level deformation consistency and outperforms baseline methods in rendering quality, achieving higher PSNR and SSIM and lower LPIPS in dynamic 3DGS rendering. Our project page is available at https://dyn‑reg‑3dgs.github.io/.
Authors:Mohamed Shawky Sabae, Philipp Langsteiner, Jan-Niklas Dihlmann, Hendrik Lensch
Abstract:
Inverse rendering requires separating illumination from surface materials, which is highly ambiguous due to their tight coupling in observed images. While Gaussian Splatting is efficient for novel view synthesis, existing relightable methods approximate scene lighting using discrete point lights, global environment maps, or implicit representations. By ignoring the physical spatial extent of real‑world emitters, these approaches produce incorrect light attenuation and unrealistic shadows. We present AEGIR (Area Emitters for Gaussian Inverse Rendering), a framework that explicitly models local area emitters within a relightable Gaussian Splatting representation. Joint optimization of emitters, materials, and geometry is challenging due to flexible emitter parameterization, which increases both the number of parameters and the ambiguity between illumination and materials. We address this by introducing a differentiable deferred rendering pipeline that integrates multiple importance sampling with targeted regularization. As a result, AEGIR accurately simulates local light transport and achieves more consistent decomposition. Experiments show that explicit area emitters improve illumination reconstruction and enhance downstream tasks, including novel view synthesis, controlled relighting, and virtual object insertion, particularly in scenes with complex local lighting.
Authors:Jun Wang, Peirong Liu
Abstract:
Forecasting longitudinal brain lesion evolution is critical for disease monitoring and treatment planning. Existing approaches typically learn a direct mapping from a baseline image to a future observation, without explicitly modeling the physical mechanisms underlying the lesion progression. Such an entangled modeling of structural deformation and image intensity variation limits physical plausibility, model generalization, and interpretability. To address this, we propose PDF, a Physics‑grounded Disentangled Flow matching framework for longitudinal brain disease forecasting. We explicitly decompose the longitudinal modeling of lesion growth into two processes, each learned by a dedicated flow matching network: morphology evolution, which captures lesion growth and structural deformation; and intensity evolution, which models signal changes driven by variations in lesion concentration. To enforce physics‑grounded constraints, we introduce a PDE‑regularized loss based on lesion growth dynamics, that enforces a diffusion‑reaction‑advection formulation for morphological evolution. Experiments on three public longitudinal datasets spanning diverse brain diseases demonstrate state‑of‑the‑art performance, validating the effectiveness of the disentangled modeling framework and physics‑grounded learning design. Code is publicly available at https://github.com/jhuldr/PDF.
Authors:Md Mozaharul Mottalib, Rahmatollah Beheshti
Abstract:
Effective sub‑typing (also known as grouping or clustering) of patients using their electronic health record (EHR) data can greatly inform precision medicine efforts. However, subtyping temporal EHR datasets is known to be challenging due to inherent EHR issues, including complexity and irregularity. In this study, we propose a self‑supervised Mamba‑based model that learns effective EHR representations and enables enhanced patient subtyping. We evaluate the proposed model on public and private real‑world EHR datasets to classify the data based on the available labels and subtype patients based on the representations learned from the model. Through an extensive set of experiments, we demonstrate that our model's design choices lead to better performance compared to competitive baseline models for prediction. Moreover, we evaluate several clustering techniques to demonstrate that our findings offer valuable insights into subtyping patients based on temporal records from EHR models\footnoteOur implementations are available at https://github.com/healthylaife/triplet_mamba.
Authors:David Charatan, Daniel Xu, Richard Szeliski, George Kopanas, Vincent Sitzmann
Abstract:
Differentiable rendering has emerged as a powerful approach for 3D reconstruction and novel view synthesis. State‑of‑the‑art differentiable rendering methods combine a variety of custom representations of 3D geometry and appearance with specialized renderers. However, most downstream tasks in computer graphics rely on 3D meshes. While prior work has attempted differentiable rendering with mesh representations, these approaches are limited to object‑centric scenes and fail to reconstruct large‑scale, unbounded scenes. In this work, we introduce Meshtryoshka, a novel mesh differentiable rendering framework that combines an off‑the‑shelf triangle rasterizer with a 3D representation that consists of nested mesh shells which resemble a matryoshka doll. In every forward pass, the mesh shells are extracted anew from a 3D signed distance function via iso‑surface extraction, and the opacities for each vertex are computed as a function of signed distance. Each mesh shell is then rasterized independently, and the final image is created via alpha compositing. Crucially, mesh vertex positions are updated only indirectly via gradients that flow through the opacity values into the signed distance function, and hence, our method is compatible with off‑the‑shelf mesh renderers that need not be differentiable with respect to vertex positions. On object‑centric scenes, our method performs competitively with surface‑based differentiable rendering techniques. Our differentiable mesh rendering method scales to unbounded, real‑world 3D scenes, where it yields high‑quality novel view synthesis results approaching those of state‑of‑the‑art, non‑mesh methods. Our method suggests that it may be possible to solve the differentiable rendering problem without relying on specialized renderers, only using conventional tools from the computer graphics toolbox.
Authors:Oscar Miró López-Feliu, Daimy van Loo, Xanthos Kekkos, Mikel Blom, Clara Rus
Abstract:
Fayyazi et al. (2025) recently proposed FACTER, a model‑agnostic framework designed to jointly enforce fairness and statistical coverage in LLM‑based recommendation through conformal thresholding and iterative prompt repair. In this work, we conduct a reproducibility study of the FACTER framework across diverse architectures and dataset sparsity levels, evaluating both the original open‑ended generation task and a constrained re‑ranking extension. Under the strict reproduction, we observe a divergence in recommendation utility, which we trace to underspecified target‑set evaluation in the original study. We then use the constrained re‑ranking setting to evaluate FACTER when the candidate set is fixed, and introduce a static Fair Zero‑Shot baseline to isolate the contribution of the iterative prompt repair loop. Our analysis shows that FACTER consistently reduces adaptive‑threshold violation counts, but that these reductions are not consistently reflected under the fixed threshold or in global fairness metrics. In the constrained ranking setting, static fairness instructions achieve comparable semantic‑parity outcomes to FACTER's dynamic repair loop, suggesting that the additional online repair mechanism provides limited benefit in this formulation. All code and reproduction artifacts are available at https://github.com/oscar‑omlf/facter‑repr.
Authors:Nhi Nguyen, Shauli Ravfogel, Rajesh Ranganath
Abstract:
Large language models (LLMs) are increasingly deployed in high‑stakes domains, where free‑text explanations such as chain‑of‑thought and post‑hoc rationales are used to justify model outputs. Yet it remains unclear whether these explanations are sufficient, i.e., if they contain enough information to explain the model's output‑generating process. We generalize classical sufficiency from feature attributions to arbitrary explanations and prove that explanation sufficiency can change depending on the input distribution, which must be explicitly defined for LLM explanations. We propose using the LLM itself to generate alternative inputs conditioned on an explanation, capturing its beliefs about possible inputs. We formalize self‑consistent sufficiency as a goal for free‑text explanations and introduce an information‑theoretic metric, SCSuff, that enables evaluation of free‑text explanations without relying on predefined biases or shortcuts. Our experiments show that SCSuff agrees with targeted perturbation tests where applicable and demonstrate that explanation sufficiency can vary with the input distribution. We find LLM explanations are generally insufficient and weakly correlated with model size, accuracy, or output entropy. Analysis of final‑token hidden states shows that top and bottom SCSuff scores can be predicted from internal representations, suggesting that SCSuff can guide detection and improvement of sufficient LLM explanations. The code for this paper is available at https://github.com/rajesh‑lab/self‑consistent‑sufficiency .
Authors:Jingwen Liu, Weibin Liao, Xin Gao, Junfeng Zhao, Yasha Wang
Abstract:
Recent progress in Text‑to‑SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.0 and BIRD remains far below that on classical academic datasets. We argue that the main bottleneck is no longer reasoning, but database representation. Real databases contain repeated audit columns, large groups of similar tables, opaque identifiers whose meanings are stored only in documentation, and extensive data dictionaries with little query‑relevant information. Existing query‑aware methods, including schema linking and retrieval‑based schema selection, filter this raw context but still operate on redundant and verbose representations. We reformulate the problem as database context compression, a query‑agnostic transformation that rewrites schemas, semantic descriptions, and external documentation into a compact representation. We formalize this transformation with the SGCF (Support‑Gain Component Factorization) principle, which unifies repeated column extraction, isomorphic table templating, semantic componentization, and evidence purification under a single coverage objective. Based on SGCF, we propose DBCC, a database‑side middleware that performs offline structural and semantic compression together with lightweight online evidence purification. DBCC is model‑agnostic and can be integrated into existing Text‑to‑SQL pipelines. On Spider 2.0‑Snow and BIRD, DBCC reduces input context by up to two orders of magnitude (from 2.6M to 34.7K tokens on the largest Spider 2.0‑Snow subset), improves schema‑linking strict recall from 0% to 56.5% under DeepSeek‑V3.2 (63.1% under Claude Opus 4.7), and consistently increases end‑to‑end execution accuracy by 1.8‑1.9% over three recent Text‑to‑SQL systems. Our code is open‑sourced at https://github.com/MrBlankness/SchemaCompression.
Authors:Anya Ji, Abhijith Varma Mudunuri, David M. Chan, Alane Suhr
Abstract:
While recent vision‑language models (VLMs) have achieved significant improvements on static visual‑to‑code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a benchmark for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. Animation2Code consists of 1,069 web animation videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human‑aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal alignment when comparing rendered animations against ground‑truth samples. Benchmarking state‑of‑the‑art VLMs on this dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya‑ji.github.io/animation2code‑website .
Authors:Tianlong Wang, Yuhang Wang, Weibin Liao, Xin Gao, Xinyu Ma, Yang Lin, Yasha Wang, Liantao Ma
Abstract:
Current approaches to enhance Large Language Model (LLM) reasoning, such as Chain‑of‑Thought and "Wait" prompts, primarily encourage models to think more, yet often fail to guide them toward Truth. While Representation Editing (RepE) offers a intrinsic control, its application to dynamic reasoning trajectories remains underexplored. In this work, we bridge this gap by investigating the geometry of truth within unfolding reasoning chains. We uncover three critical insights: (1) Truth is encoded at the sentence level and is entangled with latent reasoning patterns; (2) Effective intervention follows an Uncertainty Principle and a Decay Effect, requiring localization to early, high‑entropy forks; (3) Naive steering vectors suffer from noise, risking collateral damage to correct trajectories. Based on these findings, we propose DynaSteer, a dynamic RepE framework. DynaSteer employs pattern clustering to disentangle reasoning manifolds and utilizes Fisher‑LDA to project purified truth. By dynamically monitoring lookahead entropy, it selectively steers and rolls back trajectories only when necessary. Comprehensive experimental results on several MATH benchmark verify the effectiveness of DynaSteer, and experiments on out‑of‑domain coding tasks further confirm its generalization ability. Our code is publicly available at https://github.com/tianlwang/DynaSteer.
Authors:Shuang Song, Jiyong Kim, Rongjun Qin
Abstract:
High‑resolution satellite imagery demands 3D reconstruction methods that deliver both speed and geometric accuracy. Recent adaptations of 3D Gaussian Splatting (3DGS) to satellite imagery demonstrate strong efficiency, but reconstruction quality often degrades under diverse illumination across multi‑date, high‑altitude acquisitions (with small intersection angles), limiting applicability to remote sensing and vision tasks. We present SatSplat, the first framework to adapt 2D Gaussian Splatting (2DGS) to satellite photogrammetry, with online camera adjustment. We approximate satellite cameras with an affine model and learn a minimal delta parameterization for in‑splat camera refinement from dense observations. The formulation is implemented with a 2DGS scene representation. To handle time‑varying shadows and illumination changes, we integrate geometric shadow mapping and per‑camera color correction during training. Across the evaluated DFC2019 and IARPA2016 benchmark sites, SatSplat achieves strong geometric accuracy while significantly outperforming prior 3DGS‑based baselines. On our processed DFC2019 benchmark, SatSplat reduces mean absolute error by 11.93% and peak video memory by 31% relative to the previous state of the art. Our approach enables large‑scale digital surface modeling with practical computational efficiency. The project page is available at https://gdaosu.github.io/satsplat/.
Authors:Yuexi Du, Leya Barrientos, Laura Sheiman, John Lewin, Hemant D. Tagare, Nicha C. Dvornek
Abstract:
Multiview mammography relies on paired craniocaudal (CC) and mediolateral oblique (MLO) views to provide complementary projections of a 3D breast volume, enabling precise anomaly localization. However, acquiring high‑quality, balanced datasets remains challenging for deep learning applications. We propose a novel method to synthesize multiview mammograms by leveraging the inherent geometric relationship between CC and MLO views. To enforce an implicit 3D consistency prior during generation, we develop an alignment module that searches a 2D affine transformation subspace to establish optimal anatomical correspondence. Leveraging this alignment, we introduce a pixel‑space self‑consistency loss based on the Earth Mover's Distance (EMD) between the 1D anteroposterior (AP) axis tissue distributions of the generated images. Integrated into a pretrained flow matching model, MammoFlow forces synthesized pairs to share physically plausible tissue distributions from the chest wall to the nipple. To our knowledge, this is the first work to guide multiview mammogram generation using implicit geometric tissue correspondence. Our method demonstrates superior image quality, passes expert radiologist evaluation, and generates physically consistent pairs that improve downstream classification AUC by 5%. Code is available at https://github.com/XYPB/MammoFlow
Authors:Xiao Song, Haonan Qin, Zhaoxu Zhang, Jiong Zhang, Yuqi Fang, Caifeng Shan
Abstract:
Large vision‑language models (LVLMs) are increasingly used for clinical image understanding, yet they remain vulnerable to \emphhallucinations‑‑producing textual findings or attributes not supported by the image. We present a vision‑traceable hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding, requiring neither modification nor internal access to the hidden states of LVLMs. Given an LVLM response, we extract visually verifiable entities and use a medical‑domain‑adapted Qwen‑VL grounding verifier to localize each entity on the input image. To enhance the robustness of our detection method, we introduce a counterfactual entity perturbation method and estimate visual evidence uncertainty by contrasting factual and counterfactual grounding results. Specifically, we compute an entity‑level uncertainty score from the positive confidence, counterfactual confidence, and their grounding overlap for binary hallucination decision‑making. Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves hallucination detection performance over recent baselines, while providing interpretable localization evidence and strong cross‑model transferability. Code and dataset are available at https://github.com/Agentic‑CliniAI/CounterVHD.
Authors:Yang Liu, Xuxin Tang, Jiahao Xu, Chris North
Abstract:
Dimension reduction and semantic interaction support image clustering by making similarity structure visible and manipulable. Existing semantic interaction methods encode users' clustering criterion (a user‑interpretable semantic dimension, e.g., action, location, or mood) from direct manipulation to steer reprojection, giving users direct control over the resulting layout. Yet they typically depend on learned embeddings or a predefined criterion. In practice, users' clustering criterion often emerges gradually and becomes refined through interaction rather than being fully clear at the outset. In this work, we present CriterionSI (Criterion‑guided Semantic Interaction), a method that translates incremental drag interactions into criterion‑guided reprojection. CriterionSI uses large language models to infer and refine the clustering criterion from sequential user drags, while grounding semantic interpretation in human‑provided feedback rather than fixed prior assumptions. CriterionSI combines the inferred criterion with local drags to guide global reprojection. The simulation‑based evaluation and usage scenario demonstrate that CriterionSI can discover and refine the target criterion from sequential interactions and progressively produce criterion‑aligned clustering layouts. Our code and data are available at: https://github.com/4C79/CriterionSI.
Authors:Angchen Xie, Nikhil Sobanbabu, Ishayu Shikhare, Alan Wang, Max Simchowitz, Guanya Shi
Abstract:
High‑precision humanoid control is limited by target‑domain dynamics mismatch, where the same control objective can induce different realized motions under changes in terrain, payload, or actuator response. Existing methods either pursue zero‑shot transfer through domain randomization or in‑context adaptation without target‑domain specialization, or require heavy adaptation pipelines that leverage target‑domain data, such as model calibration, residual learning, or policy retraining. In this paper, we present FADA (Few‑Shot Domain Adaptation via Dynamics Alignment), a three‑stage Planner‑Inverse Dynamics Model (Planner‑IDM) framework for few‑shot adaptation in humanoid control. FADA first trains an oracle policy with privileged information and then distills the oracle behavior into a deployable Planner‑IDM student through DAgger. At deployment, FADA freezes the planner and finetunes only the IDM using approximately 2 minutes of target‑domain rollouts with standard supervised learning. Rather than requiring optimal demonstrations or rewards, FADA uses the paired actions and observations that are observed during these rollouts as supervision, aligning the IDM's action generation with target‑domain dynamics. Experiments show that FADA outperforms both in‑context and end‑to‑end adaptation baselines, improving task performance under dynamics shifts and enabling real humanoid robots to execute diverse high‑precision whole‑body tasks. Implementation details and qualitative hardware rollout videos are available at https://lecar‑lab.github.io/FADA‑humanoid/.
Authors:In Kyu Lee, Sumin Seo, Jaesik Min
Abstract:
Accurate correspondence matching across multiple angiographic views is the prerequisite for 3D coronary reconstruction and interventional guidance. However, the development of robust deep learning models for this task has been stifled by a fundamental data bottleneck. Obtaining ground truth for matching tasks in angiography pairs is prohibitively expensive and hard to scale. To overcome this barrier, we introduce a physically‑grounded data generation framework that synthesizes high‑fidelity Digital Reconstructed Radiographs (DRRs) from 3D Coronary CT Angiography (CCTA) volumes. Our framework generates dense, highly accurate 3D‑to‑2D projection labels by simulating realistic C‑arm acquisition geometry on patient anatomy at zero human cost. Leveraging this dense supervision, we propose a Geometry‑Informed Matching Module (GIMM) that integrates global feature and anatomical structure into correspondence learning. Unlike real angiography where assessment relies on subjective human annotation, our dataset provides 2D correspondence labels with paired images, allowing human‑free evaluation. We comprehensively evaluate our method on the proposed CT‑derived DRR dataset and demonstrate improvements over other matching baseline models.
Authors:Jia-Wei Liao, Li-Xuan Peng, Mei-Heng Yueh, Min Sun, Cheng-Fu Chou, Jun-Cheng Chen
Abstract:
Recently, diffusion models have been widely adopted in generative modeling and have served as foundational models for many image generation tasks. To control the generation without costly re‑training or fine‑tuning, many works seek inference‑time guidance methods to steer the latent via a differentiable objective at inference time. However, these methods cannot effectively preserve the original Gaussian distribution because they introduce distributional drift, thereby degrading the sample quality. To address this gap, we propose DiffRGD, a distribution‑aware guidance framework that explicitly preserves the latent Gaussian structure. DiffRGD formulates each sampling step as a constrained optimization problem on a spherical manifold induced by the latent Gaussian distribution, and solves it efficiently via Riemannian Gradient Descent (RGD). DiffRGD is a plug‑and‑play method that can be seamlessly integrated into any pre‑trained diffusion model. Extensive experiments demonstrate that DiffRGD outperforms previous methods in most image restoration and conditional generation tasks. Our project page is available at https://diffrgd.github.io/.
Authors:Shanwen Wang, Xin Sun, Sirui Wang, Xiao Xiang Zhu
Abstract:
Open‑vocabulary semantic segmentation (OVSS) enables text‑guided segmentation of unseen objects, breaking fixed‑class limitations to achieve open‑world understanding. However, existing OVSS methods primarily focus on modifying the CLIP attention mechanism, which still suffers from unstable local segmentation for remote sensing (RS) domain. To address these limitations, we propose RSGPNet, a training‑free geometric prompting framework for RS OVSS that refines segmentation by leveraging object geometric areas and consistency constraints. Specifically, RSGPNet comprises three core modules: a Text‑guided Coarse Mask module (TCM), a Geometric Re‑prompting Module (GRP), and a Coarse‑to‑fine Consistency Verification Mechanism (CVM). TCM utilizes text prompts and the input image to construct initial coarse segmentation masks. GRP then converts these coarse masks into geometric box prompts, feeding them back into the segmentation model to generate refined masks. Finally, CVM employs consistency computation to prevent prompting from reinforcing erroneous regions. They allow the model to improve segmentation accuracy in complex areas, such as category boundaries. Extensive experiments on RS datasets demonstrate that RSGPNet significantly outperforms state‑of‑the‑art methods across both quantitative and qualitative metrics while exhibiting excellent interpretability. The code is released at \hrefhttps://github.com/wangshanwen001/RSGPNethttps://github.com/wangshanwen001/RSGPNet.
Authors:Jianlong Xiong, ChuanBo Xie, Le Yu, Quansong He, Tao He
Abstract:
Recent advances in network architecture design have introduced layer attention to enhance inter‑layer interactions. In such frameworks, each layer queries all preceding layers to establish cross‑layer connections. However, layer attention results in quadratic computational complexity with respect to network depth. To mitigate this issue, prior works have proposed Recurrent Layer Attention (RLA) and linear attention mechanisms, which suffer from static information updates and limited long‑range cross‑layer dependency modeling. To overcome these limitations, we propose Key‑Correlated Layer Attention (KCLA), inspired by our observation that Key representations in layer attention exhibit high cosine similarity. KCLA achieves linear computational complexity while preserving dynamic information updates, directly derived from the foundational definition of layer attention. Furthermore, KCLA maintains long‑range cross‑layer connections and features a fixed spatial complexity, independent of network depth. Empirical evaluations demonstrate that KCLA delivers good performance across diverse tasks, including image recognition, object detection, and medical image segmentation. The code is publicly available at https://github.com/bgx666/KCLA.
Authors:Yunhun Nam, Jongheon Jeong
Abstract:
Vision‑Language Models (VLMs) have shown strong performance in visual understanding, yet they still suffer from hallucinations, generating content that is not grounded in the image. Preference alignment is a promising approach to improve visual faithfulness, but its success depends heavily on how preference pairs are constructed. Existing methods exhibit two key limitations; (a) intervention‑based methods often introduce significant deviation from the policy distribution, and (b) sampling‑based methods often underuse visual information during the construction. In this paper, we propose ViPSy (Vision‑driven Preference Synthesis), a framework for constructing preference data that are both policy‑aligned and visually grounded. Our framework consists of two stages; in the first stage, ViPSy derives a visual cue from recurring object‑level content across semantically aligned image variants, so preference construction can rely on visual information rather than language priors. In the second stage, ViPSy conditions the policy's own rollouts on this cue, allowing candidates to be guided by visually grounded content while staying close to the policy's response distribution. The resulting candidates remain close to the policy's response distribution while better leveraging visual information from the image. Experiments show that the resulting VLM, preference‑aligned with ViPSy‑constructed preference pairs, achieves a new state‑of‑the‑art in hallucination mitigation. Compared with the previous state‑of‑the‑art method, it reduces hallucination rates on AMBER and Object HalBench by 35.7% and 24.5%, respectively. The resulting model further improves on general visual grounding benchmarks, e.g., MMStar, MMVP, and CV‑Bench, while also yielding gains in semantic segmentation and ImageNet linear probing, underscoring the effectiveness of our framework in enhancing the model's visual capabilities.
Authors:Jiasheng Wang, Tanun Jitwatcharakomol, Piyawadee Jongpradubgiat, Simeng Zhu
Abstract:
Accurate lesion segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT‑PET, a reasoning‑augmented framework that couples a high‑sensitivity voxel‑level segmentation model with lesion‑level large language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion‑level reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, rewarding correct lesion classification and anatomically concordant site assignment. Across AutoPET and an OSU test cohort, RADIANT‑PET consistently outperforms strong image‑only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM‑based lesion‑level reasoning adds a novel reasoning layer beyond conventional segmentation, suppressing physiologic false positives and aligning voxel‑level predictions with clinical interpretation. The project repository is available at: https://github.com/jwang‑580/RADIANT‑PET.
Authors:Sifei Meng, Dmitry Ilvovsky
Abstract:
Multi‑turn retrieval‑augmented generation (RAG) is challenging due to evolving user intent, conversational noise, and strict context limits. We propose a training‑free hybrid retrieval pipeline for SemEval‑2026 Task 8 that combines dense and sparse retrieval with controlled query rewriting and cross‑encoder reranking. On the official test set of Task A, our system achieves 0.5453 nDCG@5, ranking third among 38 teams and outperforming the strongest baseline score of 0.4795. For Task C, we reuse the documents retrieved for Task A and apply a lightweight generation pipeline guided by the official prompt, achieving 0.5312 as the harmonic mean of relevance and faithfulness and ranking 15th among 29 teams. All retrieval components are open‑source, while query rewriting and answer generation rely on LLM APIs.
Authors:Roberto Cruz, David Rey-Blanco
Abstract:
Synthetic healthcare data is increasingly important for research, education, and machine learning development where access to real patient data is limited by privacy and governance constraints. While Synthea provides a widely adopted framework for generating realistic longitudinal electronic health record data, its current implementation presents adoption barriers for many researchers and data scientists due to deployment complexity and limited integration with modern Python‑based workflows. This paper introduces PySynthea, a Python‑native reimplementation of Synthea designed to improve accessibility, extensibility, and interoperability within the scientific Python ecosystem. The framework provides modular synthetic patient generation, configurable healthcare simulation pipelines, and support for standard healthcare data formats while integrating naturally with tools such as pandas and machine learning workflows. By reducing operational complexity and aligning synthetic data generation with the dominant data science ecosystem, PySynthea aims to accelerate experimentation and broaden the use of synthetic healthcare data in research and applied AI development. The code in this github repository https://github.com/TIET‑AI/tietai‑synthea.
Authors:Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, Sewon Min
Abstract:
Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems depend on complex parsing pipelines that linearize HTML and discard layout, visual structure, and formatting. We introduce PixelRAG, a new retrieval‑augmented method that represents websites in their native visual form and performs retrieval and reading entirely in pixel space, enabling an end‑to‑end architecture that eliminates text abstraction. PixelRAG is, to our knowledge, the first pipeline to operate over a full Wikipedia corpus in this form, scaling to a datastore of 30 million screenshot images with an efficient visual retrieval index. Built on an existing visual embedding model (i.e., Qwen3‑VL‑Embedding), PixelRAG further fine‑tunes this model on screenshot data with carefully curated contrastive training data. Retrieved screenshots are then fed directly as pixel inputs to a VLM, without intermediate text conversion. PixelRAG consistently outperforms both no‑retrieval and text‑based RAG baselines, most surprisingly on widely studied text‑centric tasks such as NQ and SimpleQA. It also achieves strong gains on multimodal open‑domain QA (e.g., MMSearch), benchmarks over noisy news corpora (e.g., LiveVQA), and agentic benchmarks (e.g., MoNaCo), improving accuracy by up to 18.1% over text‑based baselines. Finally, pixel representations enable a new efficiency lever for RAG through image compression, achieving up to 3x token cost reduction at lower resolutions while maintaining accuracy. Our results challenge the necessity of text representations in web retrieval, suggesting that web RAG can operate directly in the web's native visual form while improving both performance and efficiency.
Authors:Chunyi Peng, Haidong Xin, Xuanshuo Sheng, Xin Dai, Zhenghao Liu, Shuo Wang, Yukun Yan, Zulong Chen, Yu Gu, Ge Yu
Abstract:
Large Language Models (LLMs) have demonstrated strong capabilities in general conversation, instruction following, and complex reasoning. However, in long‑term dialogue settings, they often struggle to locate and utilize historical information most relevant to the current query. Existing approaches address this issue by constructing structured text‑centered memory units through compressing and reorganizing user interaction history. However, these systems often rely on brute‑force extraction of crucial evidence to associate episodes across dialogue sessions, causing substantial computational overhead and weakening structural cues such as speaker transitions, turn boundaries, and local contextual relationships. To avoid fragile text‑based memory representations, we propose MemShot, which leverages dialogue structuring for long‑term dialogue modeling and relies on the model's internal visual reasoning capabilities to associate key episodes. Specifically, MemShot renders local contiguous dialogue spans into structured visual memory units, preserving meta‑information and chronological dialogue turns while avoiding heavy‑weight textual memory construction. Experimental results show that MemShot achieves stable and competitive performance on both LoCoMo and LongMemEval, while substantially shortening the memory construction pipeline and delivering 70× speedup. Further analysis reveals that MemShot enhances the localization and utilization of historical evidence by directing memory processing toward structured local dialogue cues rather than surface‑level lexical matching in a flat text stream. All codes are released on https://github.com/NEUIR/MemShot.
Authors:Adib Sakhawat, Syed Rifat Raiyan, Tahsin Islam, Takia Farhin, Hasan Mahmud, Md Kamrul Hasan
Abstract:
We argue, with systematic empirical evidence, that a large language model's political ideology is not a fixed point, but a conditional distribution \mathbbP(position\midcontext) over a real political space. We evaluate nine current LLMs using a unified measurement framework anchored by VAA‑CHES projection models, which map responses onto three validated dimensions (lrgen, lrecon, galtan) across six contextual axes. Our findings reveal high sensitivity to context: persuasive framing and under‑represented languages displace coordinates by up to 0.57 and 0.52 units, respectively, while chain‑of‑thought reasoning often amplifies rather than dampens paraphrase instability. Despite this local plasticity, the model cohort occupies a remarkably narrow Overton envelope overall, occupying roughly one‑third the spread of major European parties. Supported by a multi‑trait multi‑method (MTMM) analysis, we conclude that a single point cannot summarize LLM political behavior; it must be characterized as a shape. Our code and data are publicly available at https://github.com/sakhadib/LLM‑Ideoplasticity.
Authors:Daoming Wan, Yizheng Huang, Jimmy X. Huang
Abstract:
In recent years, text clustering has become a critical technique for applications including intent discovery, topic mining, and recommendation systems. However, evaluating text clustering algorithms remains challenging since many real‑world textual datasets are not suitable for clustering assessment due to ambiguous semantic boundaries, the high dimensionality of embeddings, and inconsistent cluster structure. Current clustering dataset generators are designed for numerical data, providing limited support for text‑specific benchmarking. This paper introduces TextClusterLab, a comprehensive framework for text clustering research. TextClusterLab offers a Large Language Model (LLM)‑driven text clustering dataset generator to produce synthetic text datasets for evaluating clustering algorithms. This generator supports setting various clustering attributes, such as class imbalance, intra‑cluster compactness, and inter‑cluster diversity. These generated datasets can serve as practical benchmarks for testing the robustness and versatility of text clustering algorithms in diverse scenarios. Moreover, we introduce a benchmark to verify whether a text dataset is suitable for clustering evaluation. Therefore, TextClusterLab provides an integrated framework for reproducible and comprehensive text‑specific clustering research. Our TextClusterLab is publicly available at https://github.com/research‑paper‑code/TextClusterLab, and some synthetic example datasets with various attributes are publicly available at https://huggingface.co/datasets/DW‑irlab/TextClusterLab.
Authors:Dihong Huang, Zhenyu Wei, Zhuxiu Xu, Yunchao Yao, Sikai Li, Mingyu Ding
Abstract:
Dexterous manipulation policies can solve individual skills, but composing them to perform multiple tasks with a single hand remains challenging. Adding a new task on top of an existing manipulation skill often imposes conflicting demands on overlapping fingers and contact modes, causing destructive interference between preserving an existing manipulation outcome and executing a new one. We propose DexCompose, a role‑aware residual composition framework that reuses pretrained dexterous policies for multi‑task manipulation through explicit finger‑level action ownership. Given two pretrained full‑hand policies, DexCompose first collects successful post‑task states from the first skill and performs release tests over candidate finger masks to identify which fingers are necessary for maintaining the established skill state. It then trains two asymmetric residual modules: a bounded residual stabilizer for task preservation, and a context‑aware residual that adapts the frozen downstream policy only within the action subspace assigned to the new task. We evaluate the framework on 16 composite dexterous manipulation tasks spanning four object‑retention skills and four downstream interactions. DexCompose achieves a 77.4% average composite success rate, demonstrating that structural action ownership with dual residuals offers a promising direction for composing dexterous skills beyond conventional policy chaining.
Authors:Yana Wei, Hongbo Peng, Yanlin Lai, Liang Zhao, Kangheng Lin, En Yu, Keyu Lv, Han Zhou, Yin Tang, Haodong Li, Mitt Huang, Hangyu Guo, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Daxin Jiang, Vishal M. Patel
Abstract:
We introduce PerceptionRubrics, a rubric‑based evaluation framework that addresses the gap between saturated benchmark scores and real‑world brittleness. Shifting evaluation from holistic semantic matching to rigorous atomic auditing, PerceptionRubrics pairs 1,038 information‑dense images with over 10,000 instance‑specific rubrics. These criteria are derived from golden captions constructed via a novel Circular Peer‑Review consensus pipeline and then distilled into a dual‑stream system of Must‑Right (essential facts) and Easy‑Wrong (fine‑grained details) rubrics. Crucially, PerceptionRubrics implements a Gated Scoring mechanism: unlike linear averages, failure on mandatory visual facts triggers sharp binary penalties. Extensive evaluation yields critical insights: (1) The Reliability Gap: models often verify fragmented elements correctly yet fail strict conjunctive constraints, exposing brittleness in dense domains; (2) Open‑Closed Stratification: contrary to reasoning trends, we reveal a persistent 8% perception deficit between open‑source and proprietary frontiers; and (3) Human‑Aligned Rigor: our gated metrics substantially out‑align conventional benchmarks, validating that strict perceptual fidelity is the prerequisite for reliable generation.
Authors:Jia-Chen Zhao, Beiqi Chen, Xinyang Chen, Guangcong Wang, Liqiang Nie
Abstract:
We present StructSplat, a feed‑forward and generalizable 3D Gaussian reconstruction framework that operates directly on uncalibrated images without requiring camera parameters. Existing methods either rely on per‑scene optimization or assume known camera poses, and often entangle geometry and appearance within a unified backbone, limiting reconstruction fidelity and generalization. Our key idea is to adopt a structured representation that organizes geometry, semantic, and texture cues with explicit roles in the reconstruction process. Specifically, we introduce a pixel‑aligned feature injection mechanism to enable accurate texture modeling from 2D observations, incorporate semantic‑aware priors to improve global consistency, and design a camera alignment strategy to prevent information leakage and improve generalization. Experiments show that our method significantly outperforms prior approaches on challenging benchmarks. On DL3DV, our method achieves 28.045 PSNR, surpassing AnySplat (22.377) by +5.67 dB. In cross‑dataset evaluation, our method achieves +1.94 dB over AnySplat on ACID and +1.72 dB on RealEstate10K. Project page: https://structsplat.github.io Code: https://github.com/J‑C‑Zhao/StructSplat
Authors:Yelin Wang, Zijia Song, Shuo Ye, Chuanguang Yang, Miaoyu Wang, Yong Xu, Zhulin An, Yongjun Xu, Zitong Yu
Abstract:
Remote Sensing Image Change Captioning (RSICC) aims to describe changes between bi‑temporal remote sensing images and holds significant research and application value. However, most existing methods rely on conventional deep learning architectures, and the limited model capacity constrains performance. Although large‑model post‑training techniques have achieved great success in general domains, their direct transfer to RSICC remains challenging due to data scarcity and the need for fine‑grained change understanding. To address this, we propose RSICCLLM, the first post‑training framework for large vision‑language models in RSICC. Specifically, we design a data generation paradigm, release the instruction dataset RSICI, and establish a task‑specific RSICC benchmark. We further introduce Difference‑aware Supervised Fine‑tuning to explicitly extract change representations and guide the model in perceiving and understanding temporal differences. In addition, we propose Dual‑Negative Preference Optimization (DNPO), which employs two complementary negative‑sample construction strategies to construct the preference dataset RSICP and further refine model performance. Extensive experiments validate the superior capability of RSICCLLM, which achieves outstanding results with only 7B parameters, surpassing models of substantially larger scales. The code and dataset will be made publicly available at https://github.com/keaill/RSICCLLM.
Authors:Sihang Nie, Xiaofen Xing, Rui Xing, Haoming Li, Ruitong Xiao, Jingyuan Xing, Baiji Liu, Xiangmin Xu
Abstract:
Recently, Large Language Model (LLM)‑based Text‑to‑Speech (TTS) models have achieved remarkable naturalness. However, the standard Supervised Fine‑Tuning paradigm often converges to statistically averaged prosody, limiting emotional expressiveness. While preference‑driven optimization offers a promising alternative, existing approaches suffer from two structural mismatches: information conflict, where content and emotion in a shared latent space produce conflicting gradients, leading to reward hacking and semantic degradation; and scale gap, where sparse sentence‑level rewards struggle to guide dense frame‑level generation. To overcome these challenges, we propose HPRO, a hierarchical progressive reward optimization framework. Within HPRO, we introduce the HD‑Emo codec as a novel differentiable reward model to resolve the information conflict. It extracts speech into distinct content and style preference tokens, structurally isolating emotional optimization from semantic content. Building upon this structured preference space, HPRO bridges the scale gap by progressively aligning frame‑, word‑ and sentence‑level objectives. Experiments demonstrate that HPRO significantly enhances emotional expressiveness, while effectively preserving linguistic intelligibility. The code and audio samples are publicly available at https://xxh333.github.io/hpro‑demo/.
Authors:Youzhi Liu, Li Gao, Yifei Qian, Liu Liu, Yang Cai, Ziqiao Li
Abstract:
Quadruped robots have achieved remarkable locomotion, yet their behavioral repertoire remains confined to a few gaits‑‑far from the expressive, companion‑like presence long envisioned for them. Attempts to import the humanoid recipe of large‑scale motion data have inherited one tacit assumption: that robot motion must first pass through an animal body, making data collection dependent on cooperative animals, reconstruction fragile across species, and retargeting ill‑posed across incompatible morphologies. We propose Uni‑Mo, a fully automated pipeline that removes the animal from the loop by reframing data scarcity as a generation problem: an LLM proposes motion prompts, a video diffusion model synthesizes the corresponding robot behaviors, and the generated videos are lifted into 3D reference trajectories used to train tracking policies deployed on a real Unitree Go2. To make naively‑drifting generations reliably extractable, we introduce an Identity Consistency Loss that enforces appearance coherence across frames. We release Quad‑Imaginarium at https://github.com/GaoLii/Quad‑Imaginarium.git, the resulting open‑source dataset of 7,488 language‑annotated quadruped motions (18.5 hours) spanning acrobatic and performative behaviors. We validate 392 randomly sampled motions on a real Unitree Go2 with a 96.7% deployment success rate, complemented by a 97.6% success rate across the full dataset in simulation.
Authors:Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao, Su Linxiang, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li
Abstract:
Extracting dynamic 4D object interactions from massive, in‑the‑wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs. However, existing monocular 4D reconstruction methods primarily focus on isolated objects, often failing under the severe occlusions and complex dynamics inherent in multi‑object interactions. To bridge this gap, we propose HAT‑4D, the first agentic framework designed to reconstruct the 3D geometry, temporal dynamics, and physical interactions of multiple objects from a single video. By integrating VLMs with a multi‑level human‑in‑the‑loop feedback mechanism, HAT‑4D efficiently resolves depth ambiguities and interaction‑induced occlusions during 3D generation and 4D propagation, yielding physically plausible assets without relying on expensive multicamera rigs. As a scalable data engine, HAT‑4D facilitates the creation of MVOIK‑4D, an open‑world benchmark for monocular 4D interaction reconstruction, accompanied by a novel multi‑dimensional evaluation protocol focused on physical plausibility and temporal consistency. Extensive experiments demonstrate that HAT‑4D achieves SOTA performance on most evaluation metrics, while maintaining competitive semantic alignment. Ablation studies show that introducing a small amount of human feedback improves interaction reconstruction. Moreover, the data produced by HAT‑4D effectively improves baseline performance when used for fine‑tuning. Our data and code are available at https://lijiaxin0111.github.io/HAT4D/
Authors:Xiaocheng Yang, Abdulrahman Alrabah, Dilek Hakkani-Tür, Gokhan Tur
Abstract:
Multi‑agent systems (MAS) built on large language models (LLMs) provide a promising framework for solving complex tasks through role specialization and structured interaction. However, their performance is often limited by miscoordination and, more fundamentally, the lack of fine‑grained credit assignment across agents. Existing approaches typically rely on coarse‑grained feedback, making it difficult to identify which agents or interaction steps are responsible for errors. We propose Gradient‑Based Connections (GBC), an approach for fine‑grained attribution and optimization of multi‑agent systems. GBC models a MAS as a computational graph and introduces gradient‑based connection weights to quantify the influence of each agent's output on downstream agents at the token level. By constructing an attribution graph and propagating task‑specific loss signals backward, our method enables precise identification of error sources and targeted prompt optimization. We further develop AgentChord, an efficient implementation that leverages prefix‑based gradient computation. Experiments on MultiWOZ and τ‑bench show that GBC improves multi‑agent performance and outperforms strong single‑agent and multi‑agent baselines, and higher attribution quality is associated with greater optimization effectiveness. Code is available at: https://github.com/yxc‑cyber/AgentChord.
Authors:Hong Li, Minqi Meng, Yanjun Liang, Chongjie Ye, Houyuan Chen, Weiqing Xiao, Xianda Guo, Guojun Lei, Xuhui Liu, Chaojie Yang, Yanlun Peng, Hao Zhao, Baochang Zhang
Abstract:
Reconstructing high‑fidelity, relightable 3D avatars from a single in‑the‑wild image is a challenging ill‑posed problem, primarily hindered by the scarcity of high‑quality PBR data and the complexity of disentangling illumination from intrinsic materials. In this paper, we present a data‑efficient framework that leverages the robust priors of a unified pre‑trained diffusion backbone to sequentially address texture completion, delighting, and material decomposition. Unlike existing methods that rely on fragmented pipelines or extensive proprietary datasets, we utilize cascaded Low‑Rank Adaptations (LoRAs) to adapt the strong generative prior of the diffusion model for each sub‑task in UV space. Specifically, we first employ an Inpainting LoRA to complete missing UV textures caused by occlusion, leveraging the model's semantic understanding to generate semantically and photometrically coherent details. Subsequently, a Light‑Homogenization LoRA and a novel Cross‑Intrinsic Attention mechanism are introduced to remove baked‑in lighting and collaboratively synthesize pixel‑aligned PBR maps (Albedo, Normal, Roughness, Specular, and Displacement). To ensure physical plausibility, we impose a UV‑space differentiable BRDF shading loss during the decomposition stage, forcing the generative process to adhere to the rendering equation without the artifacts typical of rasterization‑based supervision. Extensive experiments demonstrate that our method, trained on fewer than 100 real 3D scans, generates comprehensive, 4K‑resolution PBR assets with superior realism and generalization compared to state‑of‑the‑art methods, and all training code and model weights will be released upon acceptance.
Authors:Mansoo Jung, Youngwook Kim, Jungwoo Lee
Abstract:
Test‑Time Adaptation (TTA) methods commonly update the affine parameters of normalization layers to adapt deployed models under distribution shifts. However, per‑channel affine parameters perform axis‑aligned scaling and shifting, making them geometrically incapable of correcting cross‑channel structural changes induced by distribution shift. To address this limitation, we propose MixTTA, a lightweight plug‑in module that equips normalization layers with a low‑rank cross‑channel transformation, enabling inter‑channel mixing at each layer. To ensure that the low‑rank branch captures only cross‑channel interactions, we also propose Decoupling Projection that enforces strict separation from the diagonal affine path, along with Spectral Projection that prevents rank‑1 collapse under non‑stationary test streams. MixTTA can be seamlessly integrated into any existing normalization‑based TTA method. Experiments in both standard and wild TTA settings show consistent improvements over strong baselines while mitigating adaptation failure under challenging conditions. The source code is publicly available at https://github.com/delta6189/MixTTA.
Authors:Xiaotian Ma, Qiong Feng, Yongqiang Tian, Wei Song, Peng Liang
Abstract:
Modern JVM software increasingly integrates multiple programming languages, such as Java, Kotlin, Groovy, and Scala, within a single application. Supporting such interoperability requires JVM compilers to perform cross‑language compilation while reconciling subtle semantic differences across language boundaries. Errors in this process can lead to critical miscompilations, yet existing compiler testing techniques focus exclusively on isolated, singlelanguage compilation. To address this gap, we present CrossLangFuzzer, the first differential testing framework for cross‑language JVM compilation. CrossLangFuzzer leverages the Kotlin compiler's unified intermediate representation (IR) to synthesize cross‑language test programs. It further applies seven mutation operators to diversify generated test programs and improve bug‑finding capability. Evaluated on the latest versions of five major JVM compilers, CrossLangFuzzer uncovered 32 confirmed bugs, including 15 in Kotlin, 4 in Groovy, 7 in Scala 3, 2 in Scala 2, and 4 in Java. CrossLangFuzzer is open‑source at https://github.com/XYZboom/CrossLangFuzzer
Authors:Peiwen Zhang, Yufan Deng, Shangkun Sun, Juncheng Ma, Duomin Wang, Jonas Du, Zilin Pan, Ye Huang, Hao Liang, Songyan Huang, Ruihua Zhang, Enze Xie, Ming-Yu Liu, Daquan Zhou
Abstract:
Video generation models have emerged as a promising paradigm for embodied world simulation. However, both general‑domain video generators and robot‑specific data fine‑tuned models can still produce physically implausible manipulations, including discontinuous motion trajectories and inconsistent robot‑object interactions, which limits their reliability as world simulators. Through extensive experiments, we find that such physical instability mainly arises from two factors: deformation of moving objects and implausible spatio‑temporal correlations among interacting entities, particularly during contact. Building on this observation, we propose PhysisForcing, a scalable training framework that strengthens physical consistency by focusing supervision on physics‑informative regions through joint optimization of pixel‑level and semantic‑level features. The framework consists of a pixel‑level trajectory alignment loss, which supervises DiT features using reference point trajectories, and a semantic‑level relational alignment loss, which aligns DiT features with inter‑region relations extracted from a frozen video understanding encoder. Extensive experiments on R‑Bench, PAI‑Bench, and EZS‑Bench show that PhysisForcing consistently improves embodied video generation over strong baselines, improving the Wan2.2‑I2V‑A14B and Cosmos3‑Nano base models on R‑Bench by 22.3% and 9.2% (7.1% and 3.7% over vanilla finetuning), with the Cosmos3‑Nano variant attaining the best overall score. Beyond generation, as a world model under the WorldArena action‑planner protocol it raises the closed‑loop success rate from 16.0% to 24.0% and further improves downstream policy success, indicating that physically aligned video models yield stronger representations for robotic manipulation.
Authors:Alex Colagrande, Paul Caillon, Eva Feillet, Alexandre Allauzen
Abstract:
Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral convolution relies on low‑dimensional Fourier‑domain representations and can handle inputs at different resolutions. This design aligns well with settings where the Fourier basis diagonalizes the underlying operator, such as linear, constant‑coefficient PDEs on periodic domains, in which Fourier modes evolve independently. However, nonlinear PDEs may benefit from an additional inductive bias, as they exhibit structured interactions between modes, governed by polynomial nonlinearities. To capture this inductive bias, we introduce the Higher‑Order Spectral Convolution, a spectral mixer that extends FNO from diagonal modulation to explicit n‑linear mode mixing, aligned with the dynamics of nonlinear PDEs. Our experiments on standard benchmarks show that the proposed Higher‑Order FNO (HO‑FNO) retains the efficiency of FNO‑based architectures and consistently improves over other spectral neural operators. HO‑FNO also performs on par with or better than state‑of‑the‑art transformers and state‑space models on several datasets, with stronger gains in highly nonlinear regimes, such as the Poisson equation with polynomial forcing, where a single HO‑FNO layer outperforms FNO models with up to 16 layers. We open‑source our code for reproducibility at: https://github.com/AlexColagrande/HO‑FNO.
Authors:Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang
Abstract:
Real‑world object removal is challenging due to two key difficulties: the target object's non‑local effects, such as shadows and reflections, which are difficult to model, and the fact that user‑provided masks are often inaccurate or incomplete. With billions of parameters and tens of denoising steps, diffusion‑based models achieve strong removal performance at the expense of substantial computational cost, limiting their use in interactive applications and on edge devices. To address these challenges, we present OSOR (One‑Step Object Removal), which simultaneously achieves efficient, effect‑aware, and mask‑robust object removal. Concretely, OSOR introduces: (1) an occupancy‑guided discriminator for precise boundary supervision, enabling stable single‑step diffusion training; (2) an alpha head that leverages knowledge from pretrained diffusion models to predict appropriate removal regions with minimal overhead, thereby handling imperfect masks; and (3) a semantic‑anchored verification pipeline (SAVP) that filters noisy instruction‑based triplets to produce effect‑aware supervision at scale. Using SAVP, we curate CORNE, which contains 280K verified removal pairs, and further annotate AnimeEraseBench and TextEraseBench to evaluate performance on more complex removal tasks. Experiments show that OSOR surpasses strong multi‑step diffusion baselines in perceptual quality while achieving 4× to 30× faster inference.
Authors:Pragati Shuddhodhan Meshram, Varun Chandrasekaran
Abstract:
Attributing a generated image to its source diffusion model is a fundamental challenge in provenance verification and intellectual property protection. This problem is particularly difficult because diffusion models trained on different datasets can converge to similar score functions and thus similar output distributions, making the generated images themselves unreliable as attribution evidence. Existing non‑invasive methods either fail on architecturally similar variants or rely on signals that vanish when models share the same autoencoder. We propose Spectral Denoising Signatures (SDS), a non‑invasive attribution method that identifies the source model by fingerprinting each candidate model's denoising behavior. Our key insight is that a model's denoising score function exhibits a distinctive spectral geometry, reflected in how it redistributes energy across spatial frequency bands during denoising. By probing this behavior with frequency‑controlled perturbations, SDS extracts a stable signature that is intrinsic to the model, requiring only standard forward passes with no inversion, optimization, or generation‑time enrollment. Our results demonstrate that SDS achieves approximately 99.9% accuracy across eight diverse diffusion models and 96.2% under cross‑domain prompt shift, outperforming non‑invasive baselines across variations in training data, architecture, and training procedure, establishing spectral geometry as a principled and practical basis for diffusion model attribution. Code is available at: https://github.com/Pragati‑Meshram/SGS
Authors:Jiyao Wang, Qingyong Hu, Duoxun Tang, Xiao Yang, Kaishun Wu, Jiangbo Yu
Abstract:
Video‑based remote physiological measurement (RPM) is highly accessible but remains fragile under varying illumination, skin tones, and motion. Radio frequency (RF) radar is largely invariant to illumination and appearance, providing complementary cardio‑respiratory micro‑motion cues; however, requiring radar at inference is often impractical due to its limited ubiquity and deployment overhead. We propose RPM‑Distill, a physiology‑guided cross‑modal distillation framework that leverages synchronized radar only during training while retaining video‑only inference. Our key observation is that although RGB and RF waveforms differ in sensing physics and time‑domain morphology, they share similar latent periodic rhythm in the frequency domain. We thus distill physiology‑structured spectral evidence to improve robustness, via losses that (i) anchor the fundamental peak, (ii) match the off‑peak background distribution, and (iii) preserve spectral morphology and sharpness. To avoid negative transfer under sample‑level teacher quality and alignment uncertainty, a spectral policy network predicts sample‑level distillation gates and component weights from the student‑‑teacher spectral relation map, learned with a meta bilevel objective on a small labeled validation split. Through extensive experiments in challenging conditions and cross‑dataset settings, RPM‑Distill brings 81% MAE and 21% correlation improvement over unimodal baselines. Code is at https://github.com/WJULYW/RPM‑Distill.
Authors:Boyuan Chen, Zichen Dang, Chuang Yang, Lap-Pui Chau, Yi Wang
Abstract:
In real‑world deployments, scene text detectors inevitably face distribution shifts beyond the training distribution. Prior work often depends on large‑scale scene‑text pretraining, yet evaluation under cross‑domain changes and real‑world imaging degradations remains limited. We propose TextDS, an efficient framework for scene text detection under distribution shifts. First, we propose a data‑efficient dual‑encoder design with visual foundation models, eliminating the reliance on large‑scale scene‑text pretraining. Second, we introduce Step‑wise LoRA adaptation (SWLoRA), which performs progressive low‑rank refinement with a dynamic early‑exit mechanism for effective feature adaptation. Third, we propose Common Subspace Fusion (CSF) to align and fuse the two branches in a shared subspace while retaining complementary, shift‑robust information. Finally, we construct adverse‑condition scene text detection datasets to address the gap in evaluating under imaging degradation. Experiments show that TextDS achieves competitive performance in scene text detection, demonstrating robustness across domains and adverse imaging conditions with only 4.9M trainable parameters.
Authors:Dongbin Zhang, Hao Liu, Binquan Dai, Kangjie Chen, Chuming Wang, Chen Li, Jing Lyu, Haoqian Wang
Abstract:
High‑fidelity and expressive controllable human animation is essential for content creation and digital avatar applications. However, existing methods face a dilemma between expressiveness and disentanglement. Mainstream 2D pose‑conditioned approaches suffer from "motion‑shape entanglement", leading to the leakage of the driving subject's body shape. Conversely, methods relying on 3D priors (e.g., SMPL) achieve geometric disentanglement but struggle to capture facial expressions and complex gestures, resulting in rigid animations. To this end, we propose EMOSH, a novel framework for high‑fidelity controllable human video generation. First, an Expressive Human Model (EHM) is introduced as the core control representation. By explicitly disentangling shape and pose parameters, we fundamentally resolve the body shape leakage issue. Alongside this, a robust motion tracker is designed to accurately estimate EHM parameters from video. Second, we propose a Coarse‑to‑Fine Hybrid Motion Injection strategy, enabling more fine‑grained control over expressions and gestures. Furthermore, we introduce a Spatially‑Aligned Conditioning mechanism to bridge the domain gap between training and inference, improving identity consistency. Extensive experiments demonstrate that EMOSH outperforms previous methods in both self‑driven and cross‑driven scenarios, producing high‑fidelity videos with vivid expressions while maintaining shape disentanglement.
Authors:Xirui Teng, Nan Xi, Junsong Yuan
Abstract:
Analyzing fine‑grained skill activities (e.g., sports, surgery) requires not only recognizing visual patterns but also performing step‑by‑step visual reasoning that leads to the final judgment. While recent advances in action quality assessment have achieved remarkable progress in evaluating performance, existing models remain black boxes, where they lack the ability to explicitly reveal the reasoning processes underlying their judgments. To address this limitation, we propose Latent Visual Diffusion Reasoning (LVDR), a novel framework that integrates keypoint‑guided Monte Carlo Tree Search (MCTS) to model and visualize the latent visual reasoning process. LVDR not only produces more accurate skill assessments but also uncovers the critical visual reasoning sequences that contribute to the final evaluation. Extensive experiments across four datasets spanning diverse sports and surgical domains demonstrate that LVDR achieves competitive quantitative performance while providing interpretable visual reasoning trajectories leading to the final predictions. Source codes and models can be found through the following link: https://github.com/XiruiTeng/LVDR_Official.git.
Authors:Stefan F. Schouten, Ilia Markov, Piek Vossen
Abstract:
We introduce a new, contextual, multilingual dataset called ToxiREX: Toxic REasoning in ConteXt. The dataset consists of threads of Reddit comments and structured characterizations of what the comments imply, following a systematic toxic reasoning schema developed in a previous paper. Using the schema allows us to capture and explain implicit and context‑dependent toxicity, while supporting mappings to existing toxicity taxonomies. The dataset includes comments in six languages (English, Arabic, Turkish, Spanish, German, and Dutch), collected from posts connected to specific major events (e.g. the 2023 Turkey earthquakes; the Russian invasion of Ukraine). We describe the context‑preserving preprocessing of the threads. We create a training set of 125 thousand comments which is annotated by a commercially available LLM, and a test set of just under three thousand comments that is annotated by native speakers. We show that apparent disagreements in the test set annotations often reflect defensible alternative interpretations rather than noise. Finally, we provide baseline results by prompting and fine‑tuning language models. To produce these results, we develop evaluation strategies for our hierarchical, schema‑based predictions. While models perform better than random, there remains a lot of room for improvement, showing the task to be challenging. ToxiREX is the first dataset to simultaneously incorporate multiple languages, conversational context, and implicit toxicity, while using the toxic reasoning schema for rich, structured annotations. Dataset available at: https://github.com/cltl/toxirex
Authors:Yiming Gao, Hieu Nguyen, Jun Li, Shahram Ghandeharizadeh
Abstract:
A stash is a storage medium such as Dynamic Random Access Memory (DRAM), Solid State Disk (SSD), Hard Disk Drive (HDD), or Non‑Volatile Memory (NVM). This paper presents a disaggregated transactional key‑value (KV) store, DiStash, that governs KVs cross pools of stash types. It enables an application to use a single transaction to read and write different copies of one or more key‑value pair across the different pools of stashes. It simplifies the application logic by (a) preventing undesirable race conditions that may cause copies of data across different stash pools to reflect different values and/or (b) failures that may result in loss of key‑value pairs. A configuration of DiStash may use a pool of stashes as either ephemeral or durable storage. The application dictates whether the content of its participating stashes are inclusive (replicated) or exclusive (tiered). We implement a DiStash by extending FoundationDB. We quantify the tradeoffs with its design decisions using microbenchmarks and eBay's production workload. We open source our implementation at https://github.com/ebay‑USC/DiStash.
Authors:ZhengXian Wu, Hangrui Xu, Kai Shi, Zhuohong Chen, Yunyao Yu, Chuanrui Zhang, Zirui Liao, Jun Yang, Zhenyu Yang, Haonan Lu, Haoqian Wang
Abstract:
Knowledge‑based Visual Question Answering (KB‑VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fixed retrieve‑then‑generate pipeline with a pre‑selected retriever and a static top‑k setting, which is not adaptive during reasoning. We propose ProMSA, a progressive multimodal search agent for KB‑VQA. Given an image‑question pair, the agent iteratively chooses image search, text search, or stop, under explicit tool‑call budgets and with deduplication to avoid redundant retrieval. For training, we first use rejection‑sampling SFT to learn valid tool‑use formats, then optimize the agent with TN‑GSPO, a sequence‑level RL objective that normalizes updates by both generation length and tool‑interaction depth. Experiments on E‑VQA and InfoSeek show consistent gains over strong RAG and agent baselines, and improved retrieval and end‑to‑end accuracy. The code is available at https://github.com/DingWu1021/Promsa.
Authors:Haoyuan Wang, Yabo Chen, Haibin Huang, Chi Zhang, Xuelong Li
Abstract:
Building interactive world models requires generating realistic videos while maintaining controllable dynamics over long horizons. Autoregressive video generation offers a scalable foundation, but suffers from error accumulation and temporal degradation during extended rollouts. This issue is further amplified under heterogeneous controls such as human motion and camera trajectories, which may interfere and destabilize a pretrained video prior, while existing methods often trade off controllability and visual quality. We propose "Directing the World", a fast autoregressive framework for controllable world‑model video generation with compositional human‑motion and camera‑trajectory control. Our key idea is to decouple control learning while preserving a unified autoregressive video prior. We introduce a Fast‑Slow Memory training strategy to stabilize long‑horizon rollout learning and improve convergence. For human motion control, we design a t‑guided Dynamic Projection mechanism and a refined Motion‑CFG strategy, enabling temporally smooth and accurate motion alignment without degrading visual fidelity, and supporting multi‑person control.After learning a robust motion prior, we introduce a second‑stage camera‑trajectory control module to compose human dynamics with viewpoint changes for coherent world exploration. We further construct a large‑scale dataset with synchronized video, text, human‑motion, and camera‑trajectory annotations, organized into motion‑centric and camera‑centric subsets for decoupled training. Extensive experiments show stable long‑horizon generation with precise controllability and high visual quality. See more at https://whydahuzi.github.io/Directing‑the‑World.github.io/.
Authors:Nicola Fanelli, Pasquale De Marinis, Raffaele Scaringi, Eva Cetinic, Gennaro Vessio, Giovanna Castellano
Abstract:
Multimodal Large Language Models (MLLMs) describe artworks with remarkable fluency, yet the visual reasoning behind their outputs remains opaque. When an MLLM names a style, identifies a subject, or recognizes an iconographic symbol, does it ground each claim in the relevant region of the canvas, draw on an undifferentiated visual signal, or rely primarily on textual priors? We study this using the Token Activation Map (TAM), which produces, for each generated token, a heatmap isolating the visual evidence specific to that token from prior‑context interference. Applying TAM to a curated set of paintings spanning multiple periods and genres, we analyze grounding patterns across five semantically distinct token categories: common visual objects, style descriptors, metadata, iconographic tokens, and affective expressions. We find that visual grounding varies substantially with token semantics. We further show that MLLMs attempt to identify artworks and artists, achieving higher accuracy in artist attribution than in title prediction, where hallucinations are more frequent. Finally, we compare TAM with SAM~3 open‑vocabulary segmentation. To ensure reproducibility, we release our code, experimental configurations, prompts, and qualitative results on the project page at https://nicolafan.github.io/tamart/.
Authors:Yuheng Qiu, Jingyi Luo, Chenfei Ye, Ting Ma, Jianfeng Cao
Abstract:
Deep learning has demonstrated remarkable success in high‑throughput histopathology image analysis. However, the performance of learning‑based models critically depends on the quality and size of annotations by expert pathologists, which is a resource‑intensive and time‑consuming process. To address the limitations of data scarcity and annotation burden, several methods have been proposed to synthesize paired histopathology data. Nevertheless, these frameworks typically still require annotation data, albeit in reduced quantities, to impose structural constraints during training. In this work, we present CHIS, a plug‑in framework that guides the sampling trajectory of a pretrained diffusion model through two key stages: structural initialization at the start and textural modulation during generation. The initial noise state is refined by fusing the phase information from a prior mask with the amplitude of Gaussian noise in the frequency domain, yielding a structurally informed starting point. During the reverse diffusion process, we adaptively modulate both coarse‑grained and fine‑grained textures at different wavelet decomposition levels. This enables a diffusion model pretrained solely on unlabeled images to generate outputs that align with prior structural masks while preserving the reference tissue style. We conducted extensive experiments demonstrating the superiority of CHIS in generation fidelity and its substantial benefits for downstream segmentation tasks. Code is available at https://github.com/IBIL‑Code/CHIS.
Authors:Qiaoyue Yang, Sven Heutger, Christopher Niemann, Magnus Jung, Ayoub Al-Hamadi, Sven Wachsmuth
Abstract:
Human motion describes the three‑dimensional full‑body movement of a person. Anticipating such motion holds significant relevance across a wide range of application domains such as human‑robot interaction, autonomous driving, animation, and healthcare. In recent research, spatial and temporal dependencies are modeled by bidirectional attention mechanisms. These typically anticipate human motion in an autoregressive manner which could cause an accumulation of errors over time. As a consequence, they solely focus on local pose forecasting. To address these limitations, we propose a non‑autoregressive transformer based on spatio‑temporal attention, and train it not only for local pose anticipation, but also for global motion prediction in space. Furthermore, to enhance its applicability in real‑world scenarios, our model is also trained to recover missing joints due to occlusions, and is capable of processing varying lengths of history observations. Our code is publicly available at https://github.com/Q‑Y‑Yang/Prediction‑of‑Local‑and‑Global‑Human‑Motion.
Authors:Andrin Rehmann, Heiko Zimmermann, Dion Häfner
Abstract:
Differentiable partial differential equation (PDE) solvers underpin solver‑in‑the‑loop ML training, gradient‑based optimal control, and inverse problems, yet the practical cost of obtaining correct, usable gradients from a given solver on a given problem is largely undocumented. Integration effort, computational cost, gradient accuracy, and numerical conditioning vary widely across solvers and are discoverable only by trial and error. We introduce Mosaic, an extensible benchmarking framework for differentiable PDE solvers that standardizes access to solver gradients. Each solver is packaged as a containerized component (Tesseract) exposing a uniform gradient API regardless of language or automatic differentiation (AD) strategy, enabling researchers to evaluate, compare, and build on non‑trivial physical solvers. Our evaluation of 14 solvers across fluid dynamics, structural mechanics, and heat transfer demonstrates that the benchmark surfaces practically relevant differences: order‑of‑magnitude variation in computational cost and Jacobian conditioning, alongside structural incompatibilities that eliminate solvers from realistic tasks entirely. Despite this variation, all solvers that produce gradients converge to similar optima, indicating that the practical barriers are memory limits, numerical stability, and setup compatibility rather than gradient accuracy alone. Mosaic is open‑source and available at https://github.com/pasteurlabs/mosaic.
Authors:Zhaotong Yang, Ying Tai, Jiahui Zhan, Yu Zheng, Jianjun Qian, Jian Yang
Abstract:
Unified fashion generation integrates tasks like virtual try‑on and garment reconstruction into a single model to reduce task‑specific adaptation costs. However, naive parameter sharing across semantically distinct tasks induces negative transfer through severe inter‑task gradient conflict. We propose OrthoTryOn, a unified framework mitigating this interference within a shared Low‑Rank Adaptation (LoRA) module. Its Orthogonal Subspace Projection (OSP) applies task‑specific orthogonal rotations to bottleneck features, mapping them into decorrelated coordinate frames. To address residual semantic coupling at inference time, we further propose Fisher‑guided Negative Guidance (FNG), a parameter‑free strategy that utilizes diagonal Fisher information to quantify inter‑task sensitivity overlap and explicitly repels generation trajectories from the most confusable task via Classifier‑Free Guidance. Extensive experiments demonstrate that OrthoTryOn avoids the severe performance degradation typical of naive unified training and even surpasses independently trained task‑specific models, achieving state‑of‑the‑art results across multiple benchmarks while generalizing robustly across diverse diffusion backbones. Code is available at https://github.com/NJU‑PCALab/OrthoTryOn.
Authors:Haoyu Zhang, Meng Liu, Qianlong Xiang, Kun Wang, Yaowei Wang, Liqiang Nie
Abstract:
Spatial intelligence is essential for low‑altitude unmanned aerial vehicle (UAV) perception, collaboration, and navigation. However, existing UAV benchmarks often emphasize image‑level recognition, single‑view understanding, or narrow answer formats, leaving 3D spatial inference, multi‑view collaboration, scene dynamics, and diverse task formulations insufficiently evaluated. To address these gaps, we introduce SpatialUAV, a real low‑altitude UAV benchmark comprising 4,331 curated instances across 14 fine‑grained task types, covering semantic discrimination, spatial relation, aerial‑‑aerial collaboration, aerial‑‑ground collaboration, and motion understanding. SpatialUAV organizes all samples into a unified visual‑input‑‑question‑‑answer schema, while supporting seven input configurations and nine answer formats, including option labels, region identifiers, geometric values, cross‑view correspondences, and free‑form motion descriptions. To ensure reliable and grounded evaluation, our data construction pipeline integrates detector‑assisted regions, depth supervision, metadata‑derived rules, extensive manual annotation, blind filtering, and multi‑turn human validation, together with task‑specific metrics for heterogeneous outputs. Evaluating representative vision‑language models across three categories, we show that current models remain far from human‑level performance, with pronounced bottlenecks in cross‑view association, structured grounding, geometric reasoning, and temporal viewpoint understanding. These results offer empirical guidance for advancing low‑altitude UAV spatial intelligence. Code and data are available at https://github.com/Hyu‑Zhang/SpatialUAV.
Authors:Zhaoning Shi, Bo Ma, Hao Xu, Zepeng Yang, Bo Liang
Abstract:
This paper addresses the lack of explicit memory mechanisms in current object detection models and proposes Hippocampus‑DETR, a novel detection framework based on biological hippocampal memory modeling. This framework integrates a hippocampal memory network module, HipNet, into the DETR architecture and systematically simulates the anatomical structure and functional organization of hippocampal subregions, including the entorhinal cortex, dentate gyrus, CA3, CA1, and subiculum. Through this design, Hippocampus‑DETR realizes pattern separation, pattern completion, importance filtering, and information integration of visual encoding features. During training, different memory submodules are optimized using a layer‑wise training strategy, ultimately forming a memory system with memory retrieval and completion capabilities. Experimental results demonstrate that Hippocampus‑DETR achieves higher detection accuracy than current mainstream models. More importantly, models equipped with this framework also exhibit excellent generalization ability and data efficiency in tasks such as few‑shot image classification, multimodal feature construction, and image restoration. Subsequent experiments further validate the functional necessity and internal interpretability of each memory submodule. This study not only provides a novel object detection framework, but also offers a feasible technical pathway for integrating neurocognitive mechanisms with deep learning models, highlighting its significant value in improving model learning efficiency and task robustness. The project is available at https://github.com/2186cloud/hipnet.
Authors:Mingcheng Wang, Junbo Qiao, Yunchen Li, Lingfu Jiang, Wei Li, Jie Hu, Jiao Xie, Zhou Yu, Xinghao Chen, Guixu Zhang, Shaohui Lin
Abstract:
Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content‑aware speculative decoding algorithm, termed CSD, which integrates an entropy‑based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate tokens, increasing the acceptance rate in low‑detail areas to accelerate generation. Moreover, a distribution alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina‑mGPT and Janus‑Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.
Authors:Zhong Wang, Lin Zhang, Linfei Li, Ying Shen, Shaoming Zhang, Pengcheng Shi, Shengjie Zhao
Abstract:
Simultaneous Localization and Mapping (SLAM) is essential for autonomous systems, yet achieving reliable, globally consistent pose estimation and dense mapping in complex environments remains challenging due to geometric degeneracy and sensor drift. While multi‑sensor fusion addresses these issues, existing systems often lack the modularity to adapt to diverse platforms and rely on mathematically inconsistent fusion or suboptimal map representations. To address these limitations, we propose LXD‑SLAM (LiDAR+X Dense SLAM), a highly versatile and unified multi‑sensor fusion framework. Centered around 3D LiDAR, our system allows for the plug‑and‑play integration of LiDAR, Camera, IMU, Wheel Encoder, and GNSS, supporting up to 32 distinct sensor combinations. We employ a mathematically unified Iterative Error‑Sate Kalman Filter with an adaptive hierarchical prediction strategy and an update step that minimizes point‑to‑mesh distances and visual reprojection errors. To support this, the environment is modeled using continuous multi‑layered Gaussian Process (GP) sub‑meshes, which enables efficient ray‑to‑mesh depth recovery for visual features. For global consistency, we introduce an Extended Scan Context (ESC) descriptor derived from the GP sub‑meshes alongside a Bidirectional PnP optimization for robust multi‑modal loop closure within a hybrid pose graph. Extensive evaluations on public datasets and real‑world experiments demonstrate that LXD‑SLAM matches or exceeds state‑of‑the‑art specialized odometry solutions across various configurations while generating high‑fidelity, globally consistent dense meshes in real‑time. The relevant codes and data will be made available at https://github.com/peterWon/LXD‑SLAM upon publication.
Authors:Jian Shi, Cheng Zhen, Pingping Zhang, Rui Xu, Yanan Lv, Yili Ma, Huan Bi, Haojie Li, Huchuan Lu
Abstract:
Language‑guided Medical Image Segmentation (LMIS) has shown great potential to improve the delineation of anatomical structures and lesions by integrating clinical textual information. Existing methods generally rely on either implicit interaction between textual and visual features or auxiliary coarse‑grained supervision for cross‑modal alignment. However, these methods lack explicit and fine‑grained constraints to ensure semantic consistency, causing a mismatch between language and the segmentation outputs. To address this issue, we propose Text‑as‑Illumination Retinex Network (TIRNet), a novel Retinex‑inspired framework that treats text embeddings as semantic illumination for feature modulation, thereby improving semantic consistency in LMIS. TIRNet introduces two key blocks integrated at each decoder stage: (1) the Retinex‑inspired Text Modulation Block (RTMB), which employs positive and negative illumination maps to enhance text‑relevant foreground features and suppress background interference; and (2) the Consistent Detail Compensation Block (CDCB), which selectively recovers high‑frequency details via a consistency‑gated mechanism conditioned on illumination reliability. Furthermore, we propose a Multi‑Scale Illumination Supervision Loss (MSIS‑Loss), comprising a Region‑Grounded Contrastive Loss (RGC‑Loss) that enforces cross‑modal similarity to be concentrated in text‑relevant foreground regions and suppressed in background regions, and a Background Suppression Loss (BS‑Loss) that provides pixel‑level supervision for negative illumination maps, jointly ensuring a precise cross‑modal alignment at each decoder stage. Extensive experiments on the MosMedData+ and QaTa‑COV19 datasets demonstrate that TIRNet achieves state‑of‑the‑art performance in LMIS. The code is available at: https://github.com/anaanaa/TIRNet.
Authors:Ruochang Li, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun
Abstract:
Retrieval‑augmented generation (RAG) enhances LLMs by incorporating external knowledge to support response generation. However, conflicts between retrieved context and parametric knowledge have emerged as a critical challenge in RAG systems. To mitigate such conflicts, numerous studies have attempted to identify and edit knowledge‑related internal neurons, aiming to improve the ability of LLMs to rely on contextual evidence during generation. However, these neuron‑level approaches may introduce unintended cascading effects that compromise the general capabilities of LLMs, as the modified neurons are often entangled with broader model behaviors and functionalities. In this paper, we introduce SHIFT, a novel framework that reformulates neuron‑level modification as learnable gate modulation, allowing LLMs to adaptively regulate internal activations for knowledge conflict resolution. Technically, our SHIFT equips LLMs with a lightweight gate module and optimizes fewer than 0.01% trainable parameters while keeping the backbone model frozen. During generation, the gate module adjusts the model's internal representations to adaptively leverage contextual and parametric knowledge. Extensive experiments on six datasets validate the effectiveness of our SHIFT in comparison with various competing baselines. All datasets and code are available at https://github.com/OpenBMB/SHIFT.
Authors:Taïga Gonçalves, Yongsong Huang, Tomo Miyazaki, Shinichiro Omachi
Abstract:
The existence of adversarial attacks is often attributed to the presence of non‑robust features in neural networks. While prior defenses reduce their impact via pruning, masking, or feature recalibration, we instead propose to jointly learn to amplify and attenuate these signals through a simple activation scaling mechanism. To this end, we introduce Activation Amplification and Attenuation (A3), a lightweight plug‑in module that enhances adversarial robustness with minimal modifications of the activations. A3 dynamically rescales the activations using a learnable mask and a scaling factor derived from the original activation magnitudes. The influence of adversarial perturbations can be amplified or attenuated using the same learnable parameters by simply flipping the sign of the scaling operation. The amplified signals serve as negative references to construct novel contrastive and ranking loss functions. Experimental analysis shows that learning to degrade the predictions in amplification mode simultaneously improves adversarial robustness in attenuation mode. Moreover, A3 relies on only a small number of learnable parameters, with most of its behavior being determined by the scaling mechanism rather than additional network capacity. Extensive experiments demonstrate that integrating A3 into different backbones, datasets, and training methods consistently improves adversarial robustness while introducing negligible computational and memory overhead compared to existing plug‑in modules. Code is available at: https://github.com/tgoncalv/A3.
Authors:Sichao He, Aiersi Tuerhong, Shangjun She, Tianhao Chu, Yuling Wu, Junfeng Zuo, Si Wu
Abstract:
Continuous attractor neural networks (CANNs) are the canonical computational framework for how the brain encodes continuous variables such as spatial position, head direction, and movement direction, and explain the activity of hippocampal place cells, entorhinal grid cells, and head‑direction cells. CANN research, however, is fragmented: most results rest on lab‑specific implementations, general‑purpose simulators lack CANN‑specific abstractions, and the path from spike trains to attractor geometry in real recordings lacks a standardized toolkit. Here, we present a comprehensive open‑source toolkit that unifies the full CANN research workflow. It combines three tightly integrated components: 1) canns, a Python library on BrainPy/JAX that provides standardized 1D/2D CANNs, spike‑frequency‑adaptation variants, grid cell networks, hierarchical path‑integration models, and brain‑inspired attractor architectures, together with curated datasets, task generators, an analyzer module and trainer modules for biologically plausible plasticity; 2) canns‑lib, a Rust acceleration backend delivering hundreds‑of‑times speedups for spatial‑navigation workloads and modest gains for Ripser‑based persistent homology; 3) ASA (Attractor Structure Analyzer), a PySide6 pipeline applying persistent homology and cohomology to experimental neural recordings to detect ring‑like and toroidal attractor signatures in real data. The toolkit ships with full‑detail reproducible pipelines that recover recent CANN results including SFA‑driven anticipative tracking, theta sweeps in head‑direction/place/grid systems, and hierarchical path integration.
Authors:Anuki Pasqual, Dulan Lokugeegana, Manimohan Thiriloganathan, Nuthya Rathnayake, Kithsiri Samarasinghe, Udaya S. K. P. Miriya Thanthrige
Abstract:
Vehicle license plate recognition is an integral component of intelligent transportation systems. In this work, we present an embedded real‑time license plate recognition system customized for developing countries. We address the challenge of handling complex, unstructured traffic scenes with diverse vehicle types while implementing the system on an embedded platform for low‑cost deployment. Our method consists of license plate detection on a multi‑vehicle image, followed by character recognition on the detected license plates. Both steps use lightweight convolutional neural networks to balance accuracy and efficiency. We also introduce the SL‑LPR dataset of Sri Lankan road images, which contains a variety of vehicle types and traffic conditions typically seen in developing countries. On this dataset, the license plate detection and character recognition models achieved 93.6% mAP and 87.88% accuracy, respectively, and were competitive against larger models on several public datasets. To achieve real‑time performance in a resource‑constrained embedded environment, we applied low‑bitwidth quantization using the Brevitas library and implemented FPGA acceleration for the models using the FINN framework. The end‑to‑end system can operate at 11.5~FPS when implemented on the Xilinx Kria KV260 platform. These results demonstrate that our system is effective for real‑time license plate recognition on an embedded device, even in complex traffic scenarios. The SL‑LPR dataset is available for research use at: https://github.com/sl‑lpr‑uom/SL‑LPR.git.
Authors:Dikshit Chauhan, Anupam Trivedi
Abstract:
Constrained single‑objective numerical optimization requires a careful balance among feasibility, objective convergence, and computational efficiency under a fixed function‑evaluation budget. This paper proposes DE‑2LS, a late‑stage, locally search‑enhanced variant of differential evolution built on the RDEx framework. The proposed method preserves the original RDEx components, including mutation and crossover operators, success‑history adaptation, archive mechanism, population‑size reduction, and ε‑based constraint handling. A lightweight coordinate‑pattern local search is added as a guarded polishing component around the current best solution. It is activated only in the late stage of the run, uses a small evaluation budget, and accepts candidates through a feasibility‑aware comparison rule. Ablation results show that the finalized DE‑2LS configuration achieves the best U‑score among all tested variants, confirming that controlled late‑stage refinement is more effective than aggressive or premature local search. In the direct comparison with RDEx, DE‑2LS achieves a 5.58% gain in U‑score. In the four‑algorithm comparison, DE‑2LS obtains the highest overall U‑score of 80968 and the best total rank of 48 among RDEx, CL‑SRDE, and UDE‑III. These results indicate that DE‑2LS improves the exploitation capability of the RDEx‑based search framework while preserving its speed advantage under the combined speed‑accuracy scoring criterion. The source code of DE‑2LS is available at https://github.com/ChauhanDikshit?tab=repositories.
Authors:Dikshit Chauhan
Abstract:
Unconstrained single‑objective numerical optimization requires a careful balance among global exploration, late‑stage exploitation, and function‑evaluation efficiency. This paper presents DE‑2LS, a late‑stage, local‑search‑enhanced differential evolution framework built on RDEx for unconstrained single‑objective optimization with variable bounds. The proposed method preserves the original RDEx evolutionary search engine and introduces two conservative refinements: a smoothed exploitation‑biased branch‑rate update in the late search stage and a guarded coordinate‑pattern local‑search that serves as a budget‑aware refinement mechanism. Since the considered setting is unconstrained apart from variable bounds, all selection and local‑search acceptance decisions are based solely on objective values. To determine the final algorithm configuration, we conduct a staged ablation study by testing multiple settings of the EB‑rate smoothing mechanism, the initial EB‑rate, the standard‑branch Gaussian sampling scale, the selection‑pressure parameters, and the local‑search coefficients. The final configuration is selected using a U‑score‑based evaluation that jointly reflects solution quality and convergence speed. Experimental results show that DE‑2LS consistently improves the original RDEx in direct head‑to‑head comparison. In particular, DE‑2LS increases the U‑score from 33602.0 to 37448.0, corresponding to an improvement of 11.45%. Moreover, compared with several competitive and IEEE CEC‑winning algorithms, DE‑2LS achieves the best overall U‑score of 178966.5, outperforming the others by 34.43%. These results show that a carefully designed late‑stage local‑search strategy can improve both convergence speed and the final objective quality of the algorithm. The source code of DE‑2LS is available at https://github.com/ChauhanDikshit?tab=repositories.
Authors:Guoheng Sun, Kaixi Feng, Shwai He, Xiaochuan Gong, Yexiao He, Ziyao Wang, Zheyu Shen, Wanghao Ye, Ramana Rao Kompella, Gaowen Liu, Ang Li
Abstract:
Vision‑Language‑Action (VLA) models enable instruction‑driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed‑loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce Drop‑Then‑Recovery (DTR), an analysis protocol that removes selected blocks from a pretrained VLA model and then fine‑tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose GateProbe, a one‑shot virtual‑gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real‑robot industrial scenarios, we find a strong asymmetry in post‑removal recoverability: \ullanguage backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal. On LIBERO, removing half of the LLM blocks even improves OpenVLA‑OFT from 95.0% to 98.3% under the same downstream fine‑tuning budget, and retaining only two language blocks still recovers baseline‑level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.
Authors:Qinfeng Zhu, Lei Fan
Abstract:
Panoramic images capture the complete visual sphere in a single frame, providing spatial context unattainable by conventional cameras. Yet this completeness comes at a geometric cost: the 2‑sphere cannot be faithfully mapped to the plane, and every planar representation introduces distortions that violate the assumptions underlying standard vision architectures. This survey traces the evolution of panoramic scene analysis along a methodological trajectory, from projection‑based adaptation, through distortion‑aware engineering, to sphere‑native modeling and geometry‑aware tokenization for foundation models, and argues that this evolution reflects a progressive deepening of geometric commitment rather than a simple accumulation of techniques. We organize the literature along two orthogonal dimensions: architectural design (how operators interact with spherical geometry) and training paradigm (how knowledge is transferred across domains). Covering dense prediction (semantic segmentation, depth estimation, and room layout estimation), unified multi‑task understanding, open‑world perception, vision‑language reasoning, and dynamic video analysis, we identify a central unresolved tension: among the methods surveyed, none simultaneously delivers strict spherical equivariance and full reuse of perspective‑pretrained foundation‑model weights, and we argue that this is a structural rather than incidental gap. We further expose five systematic gaps in current evaluation protocols, namely the absence of spherical‑area‑weighted metrics, seam‑consistency testing, polar‑robustness stratification, cross‑projection generalization, and open‑world protocol standardization, and propose a six‑point research roadmap toward general‑purpose panoramic intelligence. The corresponding repository is publicly available at: https://github.com/zhuqinfeng1999/Awesome‑Panoramic‑Scene‑Analysis.
Authors:Zhuo Zuo, Li Yue, Wenhao Zheng, Chenpeng Wang, Xianggen Liu
Abstract:
Despite their strong general capabilities, large language models (LLMs) often remain unreliable when outputs must be numerically precise. A key reason is the training objective: standard cross‑entropy treats numeric tokens as unstructured categories and ignores the metric structure of their values. We address this mismatch with Smooth Maximum Mean Discrepancy (SMMD), which builds on the classic MMD by incorporating value‑distance kernels over numeric tokens and graph‑based smoothness. With this kernel defined over a numeric sub‑vocabulary, SMMD aligns the predicted numeric distribution to the target via kernel matching and smooths the prediction‑target residual over the induced kernel graph to encourage local consistency. We evaluate SMMD on four numeric‑target tasks: mathematical reasoning, arithmetic calculation, clock‑time recognition, and chart question answering, across multiple open‑weight LLM and VLM backbones. SMMD consistently improves accuracy over both cross‑entropy and recent numeric‑target losses; analyses show complementary effects between MMD and smoothness and underscore the importance of distance‑based kernel design. Code is available at https://github.com/Zuozhuo/smmd‑loss.
Authors:Chang Liu, Yimeng Bai, Xiaoyan Zhao, Yang Zhang, Qifan Wang, Fuli Feng, Wenge Rong
Abstract:
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities in complex problem‑solving tasks, motivating their use for preference reasoning in recommender systems. Latent reasoning, which operates in continuous hidden spaces rather than discrete tokens, has recently emerged as a promising paradigm for LLM‑based recommendation. However, existing methods often start from unconstrained reasoning points, where hidden representations are misaligned with target item embeddings, leading to suboptimal reasoning trajectories. Inspired by cognitive neuroscience, which suggests that human multi‑step reasoning is guided by intuition as a latent prior, we propose \emphIntuRec, a two‑stage framework that anchors latent reasoning with \emphrecommendation intuition. In the extraction stage, the LLM‑based recommender generates a top‑K candidate set based on users' histories as the source of intuition. In the injection stage, the candidate set is transformed into a preference‑aligned intuition embedding using self‑ and cross‑attention mechanisms, which initializes the reasoning start point and guides subsequent latent reasoning. By providing a semantically grounded starting point, IntuRec efficiently explores the preference space along more accurate reasoning trajectories. Extensive experiments on multiple real‑world datasets demonstrate that IntuRec consistently outperforms state‑of‑the‑art baselines. We release our code at https://github.com/Ten‑Mao/IntuRec.
Authors:Zhiqiang Xie, Yijing Lin, Zhipeng Gao, Dong In Kim
Abstract:
Edge devices increasingly invoke large language models (LLMs) through API services for context aware edge intelligence, while edge generated data may be collected to improve LLMs and may introduce sensitive, copyrighted, harmful, or outdated information into model behavior. Machine unlearning offers a practical way to remove the influence of undesired data without retraining LLMs. However, existing methods still face two gaps. The first is API only black box access, where target model parameters and internal logits are unavailable. The second is how to preserve retained utility when unlearning target data and retained data share highly similar prompt structures or semantic patterns. To address these challenges, we propose Controlled Behavioral Divergence (CBD), an API only black box unlearning framework. CBD uses two auxiliary models to create controlled behavioral divergence between retained inputs and unlearning target inputs, converts this divergence into an unlearning relevance score, and routes unlearning related prompts away from the target LLM. To improve discrimination accuracy under high similarity between target and retained data, CBD constructs a gradient statistics based discriminative basis by estimating empirical Fisher matrices and solving a regularized generalized eigenvalue problem, guiding the unlearning signal toward target specific information rather than shared prompt structures. Compared with eleven white box and gray box unlearning baselines, CBD achieves a better unlearning utility trade off and its performance varies little across settings. On ToFU forget10, CBD approaches the retrained reference on the forget set while raising model utility to 74.90, about 15% above the second best baseline. On WMDP, it lowers hazardous knowledge accuracy to 25.68, near random guessing, while preserving MMLU accuracy of 52.67. Code is at https://github.com/DGL‑codes/CBD.
Authors:Ponhvoan Srey, Xiaobao Wu, Cong-Duy Nguyen, Quang Minh Nguyen, Duc Anh Vu, Anh Tuan Luu
Abstract:
Probe‑based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe‑based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in‑domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in‑domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these best‑practice findings, we train benchmark‑based pretrained probes that transfer reasonably well to open‑ended factual generation, providing a stable off‑the‑shelf baseline. Our work encourages more deployment‑oriented evaluation of probe‑based uncertainty estimators. The code repository is available at https://github.com/ponhvoan/ProbeUE.
Authors:Kai Wang, Zhaopeng Gu, Yixiang Chen, Yuan Xu, Qisen Ma, Peng Su, Zhaowen Li, Yan Huang, Liang Wang
Abstract:
World‑action models have shown promising robot‑manipulation performance by jointly predicting future visual states and actions. However, existing methods mainly rely on short‑term history and short‑horizon future prediction, which is insufficient for long‑horizon tasks whose correct execution depends on earlier observations and task progress. Such temporally dependent tasks require effective use of complementary temporal information, including recent local context, cross‑stage historical events, immediate future dynamics, and global task progress. To address long‑term forgetting and poor awareness of the global task state, we introduce DiM‑WAM, a memory‑augmented world‑action model that integrates multi‑scale historical context, local future dynamics, and global task progress. The memory extracts compact visual event information from real observations, updates multiple memory banks through independent similarity‑based merging, and then reads the bank‑identity‑ and time‑embedded long‑term context to condition video and action denoising. A progress‑supervision objective further encourages memory tokens to encode not only completed historical events but also the current task stage and its implications for the remaining task. On RMBench, DiM‑WAM raises average success from 28.4% with LingBot‑VA to 69.8%, exceeding the explicit‑memory Mem‑0 baseline at 42.0%. On four real‑world Franka tasks, it improves average stage success from 70.7% to 91.5% and full‑task success from 52.5% to 80.0%. Project page: https://wangkai‑casia.github.io/dim‑wam/\texttthttps://wangkai‑casia.github.io/dim‑wam/.
Authors:Wenqi Ge, Junde Guo, Zhen Fu, Shunpeng Yang, Jiayu Chen, Hua Chen
Abstract:
Achieving everyday tasks with humanoid robots requires coordinating stable locomotion with versatile manipulation. However, existing whole‑body controllers still face significant challenges. Methods trained solely via command sampling, without motion‑capture (MoCap) data, often struggle with sparse rewards and require carefully tuned curricula to converge. This is especially problematic for upper‑body control, where the resulting motions deviate from human‑like statistics and degrade whole‑body coordination. Conversely, approaches that imitate full‑body MoCap data suffer from dataset imbalance, as many locomotion trajectories are overly aggressive for stable‑locomotion scenarios, necessitating extensive data filtering and augmentation. To address this, we present Composite Whole‑Body Imitation (CWI), a framework that decouples the use of MoCap data for upper‑body manipulation and lower‑body locomotion. This decoupling allows us to exploit the full MoCap dataset of diverse manipulation references, while stable, command‑conditioned lower‑body locomotion is guided by dual discriminators trained on curated expert‑quality walking and squatting clips via an Adversarial Motion Prior (AMP). A multi‑critic architecture reduces conflicts among locomotion, manipulation, and motion‑style objectives, and a teacher‑‑student distillation stage yields a whole‑body policy conditioned only on bimanual hand poses and velocity/height commands. We evaluate CWI through simulation experiments and real‑world deployment on a full‑size LimX Oli humanoid. The results show competitive loco‑manipulation performance, robust whole‑body coordination, and practical teleoperation without full‑body motion‑capture equipment. A project page with supplementary material can be found at https://cwi‑ral.github.io/CWI‑RAL‑Webpage.
Authors:Yu Peng, Matloob Khushi, Josiah Poon
Abstract:
Cryptocurrency price prediction is a significant challenge in quantitative investment. In recent years, time series models have made significant progress in financial forecasting tasks, especially in the stock market. Despite the growing performance over the past few years, we question the validity of this line of research in cryptocurrency prediction. Specifically, time series models (e.g., LSTM, GRU, and Transformers) are effective at extracting temporal relationships in stock market data. However, in pure price‑based cryptocurrency prediction, facing data with extreme volatility and wild swings, time series models have difficulty learning effective information. To validate our claim, we propose CryptoGAT, a lightweight Graph Attention Network that recasts cryptocurrency pure price prediction as a cross‑asset graph problem rather than a temporal modeling task. Extensive experiments on real cryptocurrency benchmarks demonstrate that our proposed CryptoGAT outperforms various state‑of‑the‑art forecasting methods with a notable margin. Moreover, we conduct comprehensive empirical studies to explore the fundamental differences exposed by time series models in stock and cryptocurrency prediction: differences in predictability of the signal and cross‑asset dependencies. This finding opens up new research directions for the cryptocurrency pure price prediction task and inspires further graph‑based exploration in the field. The source code is available at https://github.com/FanBroWell/CryptoGAT
Authors:Gen Li, Jieyuan Lan, Pengcheng Xu, Zongyuan Wu, Masaki Ogura, Tao Feng
Abstract:
LLM‑agent simulation faces a joint grounding and scaling problem: agents should act in environments that reflect real urban constraints, yet direct online LLM calls for city‑scale populations are computationally prohibitive. We present GenWorld, an empirically grounded urban simulation infrastructure that combines a building‑level synthetic city, a structured agent‑environment interface, and offline compilation of LLM‑derived decision signals into lookup policies for scalable rollout. In a reference instantiation for Higashihiroshima, Japan, GenWorld grounds 196,608 synthetic residents in census and geospatial data, validates demographic consistency against census tabulations, and uses YJMob100K mobile‑phone data as a commuting‑distance diagnostic. We demonstrate the infrastructure through three reproducible cases: a full‑city weekday rollout, a weekday‑weekend behavioral contrast, and a warning‑response perturbation with auditable replanning traces. These cases support GenWorld as a reproducible platform for grounded and scalable LLM‑agent studies, while calibrated forecasting for traffic, evacuation, or policy outcomes remains future work.
Authors:Liu Yu, Can Chen, Ping Kuang, Zhikun Feng, Fan Zhou, Gillian Dobbie
Abstract:
Large Vision‑Language Models (LVLMs) exhibit sophisticated reasoning but remain susceptible to object hallucination. Deviating from the prevailing attention intensity assumption, we reveal a deeper dynamic structural misalignment: hallucination is triggered at decision‑critical steps where specific attention heads, acting as risky mediators, decouple from visual evidence to lock onto language priors. This establishes a pathological shortcut that bypasses visual grounding. To dismantle this, we propose Fox (Faithfulness and Observational‑flow via eXpression‑rectification), a training‑free inference‑time framework. Fox diagnoses structural misalignment using a visual attention entropy probe to localize risky mediators unsupervisedly. We then execute a targeted causal intervention via numerical logit saturation to physically sever the shortcut path. Finally, a conflict‑gated cooperative decoding strategy reconciles interventional faithfulness with observational fluency. Extensive experiments demonstrate that Fox achieves SOTA performance, outperforming SID by 29.1% while preserving linguistic richness. Code is available at https://github.com/Cc2021start/Fox.
Authors:Sirui Chen, Shibo Zhao, Zhen Wu, Jiaman Li, Guanya Shi, C. Karen Liu
Abstract:
Current humanoid reinforcement‑learning policies excel at free‑space motions but struggle with contact‑rich tasks, as pure kinematic tracking cannot resolve the physical ambiguities of interacting with objects and uneven terrain. To address this, we introduce SceneBot, a unified motion‑tracking framework capable of handling freespace locomotion, terrain traversal, and whole‑body manipulation. SceneBot conditions a single policy on both reference motions and per‑link contact labels, explicitly defining expected environmental interactions. To overcome the lack of annotated interaction data, we propose a hindsight scene reconstruction approach that infers scene‑interaction graphs from retargeted human motion. Trained on 7.5 hours of this reconstructed, contact‑rich data, SceneBot successfully generalizes to unseen motions and environments. Our results demonstrate that SceneBot is the first general framework to seamlessly unify free‑space and contact‑rich behaviors executing complex, long‑horizon tasks like carrying a box upstairs and establishing contact conditioning as a powerful interface for humanoid control. All code and data will be open‑sourced. More demos and information are available at: https://ericcsr.github.io/scenebot/
Authors:Arnav Raj
Abstract:
Reinforcement learning from human feedback (RLHF) in production does not always have a synchronous reward signal. Code‑execution verifiers, slow judge ensembles, and queued human review can return several gradient steps after the rollout that produced them, breaking the synchronous‑reward assumption underlying standard PPO. We address this gap with Retroactive Advantage Correction (RAC): each pending slow completion is queued, aged through a non‑negative kernel, and reinjected as a clipped residual into the next optimiser step's advantage. We prove that under an unbiased clipped importance ratio, the cumulative RAC correction is exactly unbiased when the effective delay kernel reinjects all of its mass, and carries a bias linear in the unreinjected fraction otherwise; at the no‑delay identity kernel it reduces to V‑trace. On a tabular Markov decision process (MDP) proof‑of‑concept, RAC reduces the closed‑form policy bias by up to 47.9x at the two‑slow‑channel configuration, beating wait‑for‑slow at lower wall‑clock cost. RAC integrates with PPO and GRPO through a two‑line reward‑manager patch.
Authors:Arnav Raj
Abstract:
Reward models for Reinforcement Learning from Human Feedback (RLHF) pool preferences across thousands of annotators and fit one global affine calibrator, collapsing raters with systematically different rating‑scale offsets and slopes into a single average‑rater fit that does not match any individual annotator. PEBS is a per‑rater empirical‑Bayes shrinkage estimator: it fits per‑rater affine calibrators on a held‑out slice of each annotator's ratings and applies Morris‑James‑Stein empirical‑Bayes shrinkage toward the population mean, in closed form and without retraining the reward model. On PRISM, PEBS reduces within‑user held‑out RMSE by 8.58% over the pooled population‑slope baseline. The procedure replicates on PluriHarms harm ratings (Qwen‑2.5 base, in‑family) with a +9.66% RMSE reduction over the same population‑slope baseline. PEBS is a closed‑form post‑hoc estimator for annotator‑specific affine calibration in RLHF reward modeling; it leaves the reward base model unchanged and estimates only the rater‑level map used at inference time for new ratings.
Authors:Tim Alexander Bader, Tim Dieter Eberhardt, Maximilian Dillitzer, Wilhelm Stork
Abstract:
Camera‑based perception systems for autonomous driving are typically developed and evaluated using fixed sensor rigs, while real‑world vehicle fleets exhibit substantial variation in camera placement, orientation, field of view, and camera count. This mismatch introduces a cross‑rig domain gap in which only the geometric observation process changes. To study this effect under controlled conditions, we introduce Plentiful CARLA Camera Rigs, a benchmark that renders identical driving scenes under 14 systematically designed camera rigs. This setup enables direct analysis of cross‑rig generalization without confounding changes in scene content or appearance. Using the benchmark, we analyze cross‑rig transfer behavior of representative multi‑view perception architectures and observe substantial performance shifts induced by geometric rig variation. To facilitate structured analysis, we further introduce two calibration‑based descriptors derived from rig metadata: Rig Variance, capturing internal rig diversity, and Rig Contrastive Distance, measuring geometric discrepancy between rigs. Our experiments show that geometric rig differences strongly correlate with relative cross‑rig performance shifts and that Rig Contrastive Distance provides a reliable proxy for ranking transfer difficulty between sensor rigs.
Authors:Thomas Shih-Chao Liang, Zhuoran Yu, Yong Jae Lee
Abstract:
Large Language Models (LLMs) possess broad conceptual knowledge acquired through large‑scale text pretraining, yet their potential to supervise models in other modalities remains underexplored. In this work, we propose LaViD‑‑Language‑to‑Visual Knowledge Distillation‑‑a simple and effective framework for transferring high‑level semantic knowledge from a language‑only teacher to a vision‑only student model. Instead of relying on paired multimodal data, LaViD elicits conceptual signals from an LLM by prompting it to generate multiple‑choice questions (MCQs) that probe semantic distinctions between visual classes. Each class is mapped to a soft label distribution over these MCQs, forming a rich conceptual signature that guides the student through an auxiliary distillation loss. Notably, despite using a language‑only teacher without access to image data, LaViD consistently outperforms recent methods like MaKD that distill from vision‑language models across multiple fine‑grained benchmarks. It also achieves competitive or superior performance compared to state‑of‑the‑art visual distillation methods such as DKD and MLKD, with further gains when combined with logit standardization. On the Waterbirds dataset, LaViD substantially improves worst‑group accuracy, demonstrating enhanced robustness to spurious correlations with distillation. Code is available at https://github.com/lliangthomas/lavid.
Authors:Advay Ranade
Abstract:
The 2024 Department of Justice antitrust complaint against RealPage, Inc. named five major residential REITs for coordinating algorithmic rent pricing across hundreds of thousands of apartment units in major US metropolitan areas. This paper studies whether census‑tract‑level corporate landlord concentration (CLC), measured from SEC EDGAR 10‑K property filings geocoded to census tracts, the first such application in the literature, is associated with rent growth 2019‑2023, and whether that association is larger in majority‑minority neighborhoods. Rent outcomes are measured using the Zillow Observed Rent Index (ZORI). To account for the possibility that corporate landlords preferentially locate in neighborhoods already seeing rent appreciation, all regressions control for a fully novel Algorithmic Housing Burden Index (AHBI), a composite of pre‑existing rent burden and market tightness from ACS data. Across 665 census tracts in ten US metropolitan areas, doubling REIT concentration is associated with 2.8 percentage points higher rent growth (p = 0.086, p = 0.030, HC1 robust). This association is significantly stronger in majority‑minority tracts. Within the same metro, high‑CLC majority‑minority tracts are associated with 5.9 percentage points higher rent growth than comparable white tracts (p = 0.039). An XGBoost model predicts 44 percent of out‑of‑sample rent growth variance, with SHAP analysis independently confirming that CLC's contribution is positive in minority tracts and negative in white tracts. Taken all together, these findings provide the first tract‑level evidence consistent with corporate landlord concentration being associated with disproportionately higher rent growth in communities of color.
Authors:Daniel Cher, Hamza Iqbal, Eric Xing, Brian Wei, Nathan Jacobs
Abstract:
Geolocation encoders, which map geographic coordinates to learned representations, are emerging as an effective means of capturing visual and non‑visual characteristics from a latitude‑longitude pair alone. However, existing approaches project coordinates onto fixed bases (e.g., spherical harmonics), allocating representational capacity uniformly and devoting equal resources to the open ocean and to a developing city. We introduce Tessellating the Earth (TTE), a location encoder built from learnable Spherical Voronoi partitions that concentrates representational capacity where it is needed in a fully differentiable, end‑to‑end manner. Each Voronoi site carries its own embedding and migrates during training toward discriminative areas. To bridge the gap between local spatial structure and global semantic understanding, we introduce \emphglobal semantic tokens: a set of shared learnable concept tokens that distill semantic knowledge from the satellite imagery into a compact vocabulary the location encoder can reference at inference, enabling geographically distant sites covering similar environments to share semantics. TTE sets a new state of the art for location encoders across a suite of geospatial classification and regression tasks, and achieves the strongest results when used as a geographic prior for fine‑grained species classification on iNaturalist‑2018. Code, and weights are available at https://github.com/mvrl/TTE.
Authors:Yujin Tang, Chenming Shang, Ruize Xu, Nikhil Singh
Abstract:
Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when an agent genuinely needs to remember what it saw rather than what it could write down. We introduce DMV‑Bench (Code: https://github.com/yyyujintang/DMV‑Bench), the first interactive benchmark for multimodal‑agent visual memory. DMV‑Bench is built on a controlled home‑furnishing e‑commerce catalogue of 1,000 product variants in which a text‑leakage contract keeps the discriminative signal of each task in the pixels alone. Across a chain of autonomous shopping sessions, every visited product image carries a unique, pre‑rendered incidental cue, and the agent is later asked to recall a particular cued product and navigate to its URL. Inspired by dual‑coding theory, we propose DualMem, a memory architecture that maintains a visual and a verbal code in parallel. On DMV‑Bench, DualMem outperforms a caption baseline and three recent multimodal agent‑memory systems at every chain length J in 5, 10, 15, 50 on both Gemini 2.5 Flash and Qwen2.5‑VL‑7B, with the lead surviving controls for memory‑bank size and encoding‑position bias, and an asymmetric dual‑coding regime in which vision carries the cue end‑to‑end while the verbal channel plays a smaller query‑grounding role.
Authors:Trung Thanh Nguyen, Daniel Lusk, Kilian Gerberding, Janusch Vajna-Jehle, Tuan-Anh Vu, Duc Viet Le, Tu Vo, Phi Le Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide, Julian Frey, Teja Kattenborn
Abstract:
Automated instance segmentation of forest LiDAR point clouds is increasingly critical as forest monitoring moves toward scalable, detailed, 3D measurement. Yet, progress is constrained by label scarcity for tree instances; a single hectare can hold millions of points and hundreds of overlapping, complex crowns, making manual annotation from scratch with raw data laborious and error‑prone. Annotations are often corrected from automatic pre‑segmentations, but remain costly as these provide no interactive or AI‑assisted refinement. Inspired by the promptable paradigm of foundation segmentation models, we propose SelectAnyTree, a promptable instance segmentation model that delineates any individual tree in a 3D forest point cloud from a few clicks. It introduces two key components: Click‑to‑query prompt encoder and Canopy Height Model (CHM)‑guided first prompt. The former turns each click into a single content query, encoding its 3D position and positive/negative polarity together with a pooled local backbone feature. The latter provides treetops as a geometry‑ and ecologically guided first prompt without any user input. The resulting prompt query is then decoded into one tree mask by a state‑space query decoder to efficiently capture long‑range context in large‑scale forest scenes with linear‑time complexity. We evaluate SelectAnyTree in interactive and instance‑level settings across seven diverse forest regions and an independent held‑out test dataset, demonstrating strong generalization beyond the training domains. It segments a target tree to 78.2 Intersection over Union (IoU) from a single click, 24.8 points above the strongest promptable baseline, and reaches every accuracy target with the fewest clicks, while using far fewer parameters and less inference time than prior promptable models. The source code is available at https://github.com/thanhhff/SelectAnyTree.
Authors:Raymond Yu, William Huey, Mustafa Mukadam, Anusha Nagabandi, Abhishek Gupta
Abstract:
Robots trained on real world data tend to be imprecise, slow, and brittle to perturbations. Improving these policies with reinforcement learning (RL) is an appealing alternative, but this process often requires expensive training in the real world. Performing policy improvement in simulation instead provides a far cheaper alternative, but unconstrained RL in simulation can exploit contact and dynamics mismatches, resulting in unsafe behaviors that do not transfer to hardware. Common forms of regularization can furthermore limit improvement by overconstraining to an imperfect behavior prior. In this work, we propose Support‑Constrained Off‑Domain REinforcement (SCORE), a real‑to‑sim‑to‑real framework that constrains RL in simulation to the support of a generative policy pretrained on real data. We instantiate this constraint through flow steering, restricting SCORE to actions the base policy can already produce, which ensures transferable behaviors while maximizing policy improvement. Improving a policy with SCORE requires minimal effort: it learns from sparse rewards, avoids distillation, and leaves the base policy untouched. Across eight real‑world dexterous multi‑fingered robotic manipulation tasks, SCORE improves average success rate from 37.8% to 89.9%, compared to 59.5% for the best baseline, and reaches success in 36.8% fewer steps than the base policy. Ultimately, through extensive experiments and ablations, we show that simulation can substantially improve real‑world manipulation policies when policy optimization is appropriately constrained, introducing a new paradigm for real‑to‑sim‑to‑real policy improvement. Videos and code are available at https://weirdlabuw.github.io/score/.
Authors:Vedant Patel
Abstract:
Large language model (LLM) agents operate over long, multi‑session interactions in which facts change: a user moves, a price updates, a plan is revised. Acting correctly requires using the current value of a fact and discarding values that have been superseded. We isolate this ability on real conversational data and show that it is a distinct, unsolved failure. On the knowledge‑update subset of LongMemEval, replacing an agent's full context with a bounded, self‑maintained memory drops accuracy from 92% to 77% even on a frontier model (gpt‑5.4), a gap that is statistically significant (paired McNemar p<0.005) and persists across model scale while full‑context accuracy saturates near 92%. The bottleneck is therefore memory maintenance, not comprehension, and is not closed by a stronger model. We then ask whether this is merely an undersized memory, and find it is not: as the conversation grows 24x, accuracy falls further (from 68% to 28%), and granting the agent proportionally more memory yields no detectable recovery (28% to 28%, n=25). The failure scales with the length of the conversation, not the compression ratio. We release Supersede, an open reinforcement‑learning environment (on the verifiers / prime‑rl stack) that turns this measurement into a training signal: agents are rewarded for answering from the current value and penalized for stale ones. Finally, we close the loop and show the gap is trainable: GRPO fine‑tuning a small open model (Qwen2.5‑3B) on this environment nearly doubles its held‑out supersession accuracy on real, unseen conversations (9.0% to 16.7%, a single run), along a monotonic checkpoint curve indicating the learned policy, not the harness, carries the gain. To our knowledge this is the first trainable environment whose reward targets temporal fact‑currency, and the first evidence the supersession gap can be trained down, not only measured.
Authors:Jingfeng Mao, Xuyang Chen, Qilin Zhang, Oussema Dhaouadi, Guangming Wang, Brian Sheil, Daniel Cremers, Yan Xia, Olaf Wysocki
Abstract:
Aerial 6DoF localization typically relies on precise GNSS signals or radiometrically rich 3D reconstructions, limiting scalability and on‑board deployment. We propose SemCityLoc, a semantic‑geometric alignment system that reframes aerial pose estimation as structured surface registration between foundation‑model‑derived visual priors and standardized LoD‑compliant 3D city models. Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models, increasing pose discriminability in repetitive and occluded urban environments. To enable accurate evaluation, we introduce SemCityLockeD, the first real‑world benchmark combining centimeter‑accurate UAV poses with standardized LoD1‑‑LoD3 semantic city models and challenging low‑altitude imagery. Experiments demonstrate substantial improvements over existing map‑based approaches, improving recall by up to 36% and reducing mean positional error from 9.89m to 2.62m in challenging urban canyons. Our results indicate that semantically structured geometry provides sufficient and scalable constraints for high‑precision aerial localization without radiometric scene reconstructions. The code and data are available at https://albertchen98.github.io/SemCityLoc.
Authors:Vassili Philippov, Pavel Katunin, Dmitry Andreev, Igor Ostanin, Anton Nikolaev
Abstract:
LLM coding agents make it tempting to automate empirical research by delegating experiments to them directly, but naive delegation does not scale to large projects: low‑rate instruction lapses compound into broken, irreproducible artefacts. To address this problem, we present Glite ARF, an open‑source Python framework for running many LLM coding agents in parallel on a research repository without sacrificing reproducibility or auditability. The framework defines a three‑role stack: a human researcher chooses which hypotheses to test, coding agents (Claude Code, Codex CLI) implement individual tasks under a fixed structure, and deterministic Python verifier scripts enforce task isolation, immutability of completed work, a corrections overlay, and a materialised project overview. We call this verifier‑driven research: the rules of the research process live in code that fails loudly when violated, not in prose that agents are merely asked to follow. Using Glite ARF, we developed our submission to the BEA 2026 vocabulary‑difficulty shared task, placing first in the closed track and second in the open track on all three target languages (Spanish, German, Mandarin) and reducing the official baseline RMSE by 29.9% (closed) and 35.9% (open). The campaign comprised 273 tracked tasks (146 experiment runs) across 129 feature sets, run by up to twelve parallel agents orchestrated from a single laptop ‑ with some model training on rented A100s ‑ at approximately \450 in LLM API spend (\498 total third‑party cost), and structured per‑fold provenance let us catch and strip four target‑leaking feature sets, correcting an implausible 0.609 RMSE to 0.802. Across three campaigns in three domains, the framework's structural machinery adds only about 1% of wall‑clock time. Framework and a public demo project accompany this paper.
Authors:Jingjun Sun, Chaowei Wang, Zhirui Liu, Jiaxu Tian, Ming Yang, Yaoxing Wang, Shan Gao
Abstract:
3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object‑relation‑object graphs, providing a compact relational abstraction for spatial understanding. In embodied intelligence settings, the same 3D scene may be observed by agents from viewpoints that differ by yaw rotations. However, current 3DSGG models often fail to produce relation predictions that follow the expected transformation behavior under such viewpoint shifts. This behavior reveals an empirical mismatch related to predicate‑level transformation heterogeneity: directional predicates such as left, front, right, and behind should transform with the observation frame, whereas most contact, support, and semantic predicates such as standing on and attached to should remain stable. To reduce this mismatch, we propose Transformation‑Aware Decoupling (TAD), a viewpoint‑robust 3DSGG framework that decouples relation reasoning according to predicate transformation behavior and is supported by viewpoint‑stable object representations. TAD decomposes relation reasoning into two parts: one learns cues that should stay stable across viewpoints, while the other learns directional cues that should change with the observation frame. The two parts are merged for standard multi‑label predicate prediction. Transformation‑specific descriptors and group‑aware auxiliary supervision encourage the two branches to capture complementary relation cues. Extensive experiments on 3DSSG show that TAD achieves state‑of‑the‑art robustness under yaw viewpoint changes without training‑time rotation augmentation, while maintaining competitive performance under the standard benchmark. The project page is available at https://tad‑predicate.github.io/.
Authors:Igor Itkin
Abstract:
Multi‑agent large language model (LLM) systems often rely on verifier and critic agents to suppress hallucinations, but verification is delayed. During this delay, false claims can propagate through the agent network. We model this process as delayed consensus on a graph with grounded corrector nodes. Spectral decomposition by the grounded Laplacian yields a closed‑form stability threshold for the verification dose: correction that is too strong or too delayed can turn consensus into oscillation. The most unstable regime occurs when the communication and verification delays coincide; for delay two, the threshold is the inverse golden ratio. The same framework gives a supermodular placement objective and a greedy (1‑1/e)‑approximation rule for assigning a limited corrector budget to influential nodes. Experiments across five open models confirm the predicted dose‑delay oscillations. By contrast, grounded factual answering makes truth an absorbing boundary and eliminates the effect, suggesting that the instability is specific to signed‑belief tasks while grounded verification remains stabilizing
Authors:Shravan Venkatraman, Ritesh Thawkar, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Salman Khan, Fahad Khan
Abstract:
Recently, self‑evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi‑role self‑play and self‑consistency reward schemes in existing self‑evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under‑conditioning, where the decoder relies on language priors rather than the image during generation, manifesting as insufficient attention to visual tokens. As a result, current self‑evolving LMMs struggle on vision‑‑language understanding tasks such as image captioning and visual question answering. To address this, we propose VISE (Visual Invariance Self‑Evolution), a purely unsupervised self‑evolving framework that directly regularizes the model's visual conditioning policy through two complementary invariance‑based rewards: a geometric invariance reward that enforces spatial consistency under known transformations, and a semantic invariance reward that penalizes evidence‑agnostic generation by requiring the model to recognize the absence of evidence when predicted regions are perturbed. VISE operates within a single model without specialist roles, external reward models, or annotations, and is trained on raw unlabeled images. Experiments on 18 benchmarks demonstrate the efficacy of our approach. Using Qwen3‑VL‑2B as the base model, VISE achieves gains of +16.85 CIDEr on COCO and +19.66 CIDEr on TextCaps, reduces object hallucination by 5.0 Chair‑I points, and generalizes across four model families and scales. Our code and models are available at https://mbzuai‑oryx.github.io/VISE
Authors:Yiming Chen, Yushi Lan, Andrea Vedaldi
Abstract:
We present PhysiFormer, a diffusion transformer for physically‑plausible 3D object motion. Unlike video world models that operate in view‑dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad‑hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation‑invariant multi‑object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed‑material settings, unseen real‑world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum‑based physical consistency. Our results position coordinate‑space diffusion as a promising step toward view‑invariant, geometry‑aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.
Authors:Danyal Rehman, Charlie B. Tan, Yoshua Bengio, Avishek Joey Bose, Alexander Tong
Abstract:
Efficient sampling of molecular systems at thermodynamic equilibrium is a hallmark challenge in statistical physics. This challenge has driven the development of Boltzmann Generators (BGs), which allow rapid generation of uncorrelated equilibrium samples by combining a generative model with exact likelihoods and an importance sampling correction. However, modern BGs predominantly rely on normalizing flows (NFs), which either suffer from limited expressivity due to strict invertibility constraints (discrete time) or computationally expensive likelihoods (continuous time). In this paper, we propose Autoregressive Boltzmann Generators (ArBG) ‑‑ a novel autoregressive modelling framework ‑‑ that overcomes these limitations by departing from the flow‑based BG paradigm. ArBG circumvents the topological constraints of flows and enables sequential inference‑time interventions, while offering enhanced scalability by leveraging architectures effective in Large Language Models. We empirically demonstrate that ArBG leads to significant improvements over flow‑based models across all benchmarks, but particularly in larger peptide systems such as the 10‑residue Chignolin. Furthermore, we introduce Robin, a 132 million parameter transferable model trained with the ArBG framework which improves over the previous state‑of‑the‑art, reducing the zero‑shot energy error, E‑W_2, on 8‑residue systems by over 60%. The code can be found at the following link: https://github.com/danyalrehman/autobg.
Authors:Haina Jiang, Liam Wang, Peng-Chen Chen, Min Seop Kwak, Seungryong Kim, Brian Bell, Jeong Joon Park
Abstract:
Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss‑‑Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill‑conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error‑conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching 10× on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero‑shot parameter changes and cross‑equation transfer, where its relative advantage is largest in the ill‑conditioned regimes where residual minimization is least reliable. Project website: https://neuralsolver.github.io/.
Authors:Yuemin Mao, Uksang Yoo, Jean Oh, Jonathan Francis, Jeffrey Ichnowski
Abstract:
Dexterous manipulation depends on contact events that are fast, local, and often visually occluded. Piezoelectric microphones offer a compact and high‑bandwidth way to sense these interactions, but the resulting vibro‑acoustic signals are difficult to simulate faithfully enough for end‑to‑end sim‑to‑real policy learning on dexterous robot hands. We propose VibeAct, a framework that bridges real vibrotactile sensing and simulation‑based reinforcement learning through a shared physical representation of contact and slip. In the real world, we embed piezoelectric microphones into a dexterous robot hand and collect vibro‑acoustic data through teleoperation, then replay the recordings in a calibrated digital clone to automatically label per‑finger contact and slip. A tactile estimator learns to predict contact and slip from real microphone waveforms, while manipulation policies are trained in simulation on the same representation computed directly from simulated contacts. This decoupling lets policies exploit rapid tactile feedback without simulating raw audio. Across five contact‑rich tasks spanning regrasping, in‑hand reorientation, and insertion, VibeAct consistently outperforms a proprioception‑and‑point‑cloud baseline in simulation, with the largest gains on tasks requiring sustained reactive control, where the continuous slip‑magnitude channel proves the most informative observation. The learned policies transfer to a physical dexterous hand‑arm platform, improving success rates on deployed tasks. Project videos and additional details are at https://vibeact.github.io/.
Authors:Tao Lin, Yuxin Du, Yiran Mao, Zewei Ye, Yilei Zhong, Bing Cheng, Yiming Wang, Jiting Liu, Yang Tian, Junchi Yan, Feiran Wu, Zenan Meng, Hu Wei, Yuqian Fu, Gen Li, Bo Zhao
Abstract:
Vision‑Language‑Action (VLA) models are commonly pretrained on robot demonstrations by jointly mapping visual observations and language instructions to actions. However, dense visual‑action supervision can dominate the comparatively sparse language‑action signal. As a result, policies may rely on visual shortcuts rather than learn how language conditions action execution, making them sensitive to visual variations. To address this limitation, we propose LA4VLA, a language‑action pretraining framework that enables policies to acquire language‑conditioned action priors without visual observations. These priors capture reusable manipulation skills shared across tasks and scenes, reducing reliance on scene‑specific visual cues. Specifically, LA4VLA decomposes expert demonstration trajectories into atomic action segments and pairs each segment with a corresponding low‑level action description. This yields LA‑33K, a dataset of 33K Language‑Action (LA) episodes derived entirely from existing demonstrations without additional robot data collection. We further develop LA4VLA‑1B, a lightweight 1B‑parameter VLA model, and investigate three paradigms for incorporating language‑action supervision into VLA learning: LA‑only pretraining, sequential LA‑to‑VLA pretraining, and mixed LA‑VLA pretraining. Across simulation and real‑world tasks, LA‑pretrained policies consistently outperform matched VLA‑pretrained counterparts, while combining LA and VLA supervision leads to further gains. In particular, mixed LA‑VLA pretraining improves the average success rate of LA4VLA‑1B over the no‑pretraining baseline by up to 17.8 and 45.0 percentage points in simulation and real‑world tasks, respectively. These results establish LA4VLA as an effective and complementary pretraining strategy for building stronger and more robust VLA policies.
Authors:Sourav Raxit, Abdullah Al Redwan Newaz, Jose Fuentes, Leonardo Bobadilla
Abstract:
This paper presents BOWConnect, a bidirectional parallel kinodynamic motion planner that addresses three fundamental limitations of existing sampling‑based methods: sample inefficiency in high‑dimensional state spaces, unreliable cost heuristics under dynamic constraints, and poor performance in narrow passage environments. Unlike classical planners that rely on random control sampling and geometric distance heuristics, BOWConnect integrates Bayesian Optimization over Windows (BOW) as a learning‑based steering function within a parallel tree‑based exploration framework, enabling each worker to learn local cost maps and constraints to guide sampling toward dynamically feasible and collision‑free controls. A bidirectional architecture simultaneously grows forward and backward trees from the start and goal regions in parallel threads, with a spatial hashing mechanism enabling fast connection queries and a boundary value problem solver generating kinodynamically consistent bridge trajectories. Extensive evaluations across ten benchmark environments demonstrate that BOWConnect achieves 100% success while delivering the fastest or near‑fastest planning time in complex scenarios, including narrow passages and non‑convex spaces where state‑of‑the‑art planners fail or degrade substantially. Real‑world deployment on a ground vehicle and a quadrotor confirms real‑time planning with no collisions. Videos of real‑world and simulated experiments, high‑resolution versions of the figures, and the open‑source code are available at https://bow‑connect.github.io/.
Authors:Preet Baxi, Jiannan Xu, Jane Yi Jiang, Stefanus Jasin
Abstract:
Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems. We study prompt injection in automated résumé screening, defined as subtle self‑promotional text that introduces no new qualifications but is designed to influence LLM evaluations. Using controlled experiments, we show that prompt injection reliably improves applicant rankings when résumé quality is homogeneous and few candidates inject. However, its effectiveness rapidly diminishes as more candidates inject, collapsing when manipulation becomes widespread. When candidate quality is heterogeneous, prompt injection is less effective on average, but can occasionally allow lower‑quality candidates to outrank higher‑quality ones, raising fairness concerns. Overall, LLM‑based screening is most vulnerable when manipulation is rare and candidate quality differences are small. Code and resources are publicly available at: https://github.com/preetb1199/Prompt_Injection_ACL26
Authors:Junwei Luo, Shuai Yuan, Zhenya Yang, Yansheng Li, Zhe Liu, Hengshuang Zhao
Abstract:
Earth Observation (EO) forecasting aims to predict future Earth surface dynamics from satellite observations under changing meteorological conditions. In this paper, we view this task as a partially observed, weather‑driven world modeling problem, in which weather acts as a conditioning signal, while forecasting remains uncertain due to sparse observations and unobserved land‑surface states. However, existing methods do not fully capture this setting: deterministic models collapse uncertainty into a single future prediction, while diffusion‑based methods typically treat weather variables as undifferentiated conditioning signals, and existing benchmarks focus mainly on reconstruction accuracy rather than whether forecasts respond correctly to changed weather forcing.We introduce EO‑WM, a video diffusion transformer for multispectral EO forecasting. EO‑WM incorporates a physically informed conditioning framework that represents meteorological forcing through a climatological baseline, weather anomalies, and cumulative physical stress signals. Specifically, it separates baseline and anomaly through distinct conditioning pathways, and accumulates anomalous forcing over time to capture sustained heat and drought stress. To evaluate weather‑response behavior beyond standard metrics, we introduce two diagnostic benchmarks: an Extreme Summer Benchmark for severity‑aware prediction of vegetation degradation under extreme weather, and a Seasonal Matched‑Pair Benchmark for testing response fidelity under changed weather forcing. Experiments show that EO‑WM reduces the error in predicted Normalized Difference Vegetation Index (NDVI) decline amplitude by a relative 5.63% and improves directional hit rate by a relative 7.80%, while remaining competitive on standard pixel‑level metrics. The benchmarks and model will be made open‑source at https://github.com/Luo‑Z13/EO‑WM.
Authors:Wen Ye, Peiyan Li, Tingyu Yuan, Yuan Xu, Xiangnan Wu, Chaoyang Zhao, Jing Liu, Nianfeng Liu, Yan Huang, Liang Wang
Abstract:
Recently, a few works have made early attempts to study test‑time scaling for embodied tasks. However, two major challenges remain unsolved: (1) reasoning can effectively improve the performance of the policy, but its scaling mechanism has seldom been studied; (2) historical information is essential, as embodied tasks are inherently long‑horizon and sequential, making sole reliance on current observations for action scaling inadequate due to the lack of historical context utilization. To address these challenges, we introduce E‑TTS, a modular and plug‑and‑play Embodied Test‑Time Scaling framework that unifies reasoning and action scaling for robotic manipulation via history‑aware iterative refinement with vision‑language verifiers. To support joint reasoning‑action scaling, E‑TTS performs reasoning‑action joint sampling and scoring in a pairwise manner. To better utilize historical information, E‑TTS uses a history buffer to store historical context, which is then used by reasoning and action verifiers to evaluate the sampled candidates. Unlike conventional open‑loop TTS methods, E‑TTS introduces feedback generation into the sampling process to form a closed‑loop iterative refinement mechanism, enhancing both inference efficiency and environmental adaptability. Each component functions as an independent and composable module, allowing flexible and adaptive configuration depending on task requirements. To evaluate the advantages of our framework, we conduct experiments across 4 different benchmarks, 6 environments, 3 embodiments, and 4 base vision‑language‑action models. The experimental results demonstrate that, without requiring additional expert data collection or retraining, E‑TTS consistently improves performance, achieving up to a 33.14% increase in simulation and 26.62% in real‑world scenarios.
Authors:Jiyong Kim, Shuang Song, Ronjgun Qin
Abstract:
Gaussian Splatting has been recently explored for satellite 3D reconstruction, demonstrating flexibility and efficiency in representing radiometrically diverse satellite scenes. However, the limited top viewpoint of satellite imagery results in insufficient supervision on building facades, leaving surface holes and degraded visual fidelity. Generative refinement, which leverages pretrained generative priors to iteratively refine and update the rendered images used as supervision targets, has recently been investigated to improve the visual fidelity of Gaussian‑rendered images. However, since these models refine each view independently, the resulting images can generate hallucinations and break photo‑consistency, leading to geometric degradation. To address these limitations, we propose SatSplatDiff, which aims to minimize geometric degradation prevalent in generative refinement. Building on photogrammetric DSM initialization and 2DGS‑based shadow casting established in our prior work SatSplat, we first introduce monocular depth supervision and multi‑scale geometric refinement to establish a geometrically accurate and well‑regularized surface representation. We then apply shadow‑guided generative refinement, where geometrically calculated shadow maps guide the Gaussians to maintain consistency with the underlying geometry, improving visual fidelity while reducing geometric degradation. Extensive evaluations on the IARPA2016 and DFC2019 datasets demonstrate state‑of‑the‑art performance, reducing geometric MAE by up to 18% and improving visual fidelity (FID‑CLIP) by 28‑45% over existing baselines. Our method delivers up to 5x resolution enhancement with minimal hallucination and sensor‑consistent appearance, demonstrating seamless cross‑tile consistency and strong scalability for large‑scale reconstruction. Source code is available at https://github.com/GDAOSU/SatSplatDiff
Authors:Hanli Xu, Fengxiang He, Sarat Moka
Abstract:
Global objectives, such as KL divergence and ELBO, are widely used in Bayesian inference for measuring distributional discrepancy. This paper studies their local‑mass behaviour that is not directly captured by such objectives. We introduce and use two mathematical tools: (1) Mass Index for recording the polynomial and logarithmic decay scales of local mass, and (2) regularised extended KL (RE‑KL), a set‑localised divergence that can be formulated in the presence of singular components. Mass Indices help characterise how Bayesian updating changes local mass: (1) power‑log likelihood factors shift it explicitly, and (2) parameter‑dependent supports, or their smooth softenings, may change the local scale through the amount of mass that remains near the parameter value. Using local RE‑KL, we prove absolute, relative, and directional inequalities for comparing local small‑ball masses under the two KL directions. Together, these results provide a local theoretical account of local mass behaviour. Experiments provide controlled illustrations of the local behaviour. Code is available at https://github.com/Forsythia0604/Local‑Mass‑Framework.
Authors:Ioannis Nasios
Abstract:
This paper presents solarsystem, a validated lightweight and dependency‑free Python package for planetary positions and solar‑lunar event calculations. The package provides heliocentric and geocentric positions for the major planets, selected dwarf planets, the Centaur Chiron, and the Moon, together with sunrise, sunset, moonrise, moonset, and lunar illumination calculations. Additional functionality includes coordinate transformations between commonly used astronomical reference systems. The implemented algorithms employ analytical models that avoid reliance on external ephemeris datasets, resulting in a portable and computationally efficient solution suitable for a broad range of astronomical applications. An optional precession correction model is included, enabling calculations either in a precession‑corrected reference frame or in a fixed epoch framework, depending on user requirements. The numerical performance of solarsystem was evaluated against the JPL DE440 planetary ephemerides using the Skyfield framework as a reference. Validation experiments spanning multiple bodies and extended temporal intervals demonstrate good agreement with the reference ephemerides, with mean planetary longitude and latitude deviations of approximately 0.44 and 0.16 arcminutes, respectively. Additional validation of solar and lunar event calculations yielded timing differences of only a few minutes relative to the reference solutions, while lunar illumination estimates differed by approximately 0.2%. The package can be installed directly through PyPI while the source code, documentation, validation notebooks and example workflows are publicly available through the project repository in https://github.com/IoannisNasios/solarsystem.
Authors:Andrei-Alexandru Oltean, Bogdan Macovei, Ioana Leuştean
Abstract:
We present a Lean formalization of a general hybrid modal logic with many‑sorted signatures and polyadic modal operators. The system borrows ideas from both algebraic specification and dynamic logics, and is designed to serve as a uniform axiomatic foundation for specifying and verifying programming languages and security protocols. We expose a DSL for users to define languages and protocols as many‑sorted signatures, specify the relevant domain‑specific axioms, and reason about program executions or protocol runs. We provide a machine‑checked proof of its soundness theorem and showcase the framework's versatility through several applications: an imperative programming language for code verification, the BAN logic for security protocols, and the modal system S5. We have designed our formalization to be intrinsically sorted, that is, well‑sorted formulas in the base language are well‑typed terms in Lean. Thanks to intrinsic sorting, all domain specific applications can be easily embedded in our framework via the DSL, at no additional syntactic overhead required for the user to prove. All code presented in this paper is openly accessible in the following repository: https://github.com/alexoltean61/msphml‑lean
Authors:Jinyu Liu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang
Abstract:
Unified multimodal models capable of both understanding and generation have achieved remarkable strides. However, despite their unified designs, existing evaluations typically assess understanding and generation capabilities in isolation, overlooking the synergy between comprehension and generation. To bridge this gap, we introduce Unison, a comprehensive benchmark comprising 2,169 high‑quality unified task samples, designed to evaluate joint understanding and generation in unified multimodal models. Unison offers three key strengths: 1) Comprehensive Dimensions: Unison encompasses internal consistency, understanding‑guided generation, generation‑guided understanding, and mutual enhancement to enable holistic evaluation. 2) Diagnostic Evaluation: it provides both unified and decoupled tracks for understanding and generation, allowing fine‑grained attribution of failure modes and quantitative analysis of the gains from unified modeling. 3) Human Alignment: we also introduce Unison‑Judge, an evaluation model well aligned with human judgments to ensure reliable assessment. Based on systematic evaluations of state‑of‑the‑art models on Unison, we uncover critical limitations in current unified multimodal systems and highlight promising directions for future research. Codes, Unison and Unison‑Judge are publicly available at https://github.com/FudanCVL/Unison.
Authors:Jiahe Chen, Qian Shao, Qiyuan Chen, Jiaying He, Jintai Chen, Jian Wu, Hongxia Xu
Abstract:
Open‑set semi‑supervised learning aims to leverage unlabeled data that may contain out‑of‑distribution outliers while maintaining performance on in‑distribution classes. Existing methods mainly follow two paradigms: filtering suspicious samples or incorporating unlabeled objectives with soft weighting. We argue that both face a common trade‑off: aggressive filtering can discard informative but hard ID samples, whereas utilization can introduce auxiliary gradients that conflict with supervised learning when pseudo labels are wrong. We therefore shift the focus from sample selection to gradient‑level control. We propose Geometric Gradient Rectification (GGR), a plug‑in framework that uses the supervised gradient as an anchor and projects conflicting auxiliary gradients onto an admissible region in gradient space. This makes the applied auxiliary update first‑order non‑opposing within the rectified coordinate block while preserving orthogonal components that may still carry useful representation signals. We further extend GGR with subspace‑aware rectification to stabilize the anchor under noisy mini‑batch gradients. Experiments on CIFAR and ImageNet benchmarks show that GGR improves representative OSSL baselines in most settings and yields gains in both closed‑set generalization and open‑set robustness. Code will be available at https://github.com/JiaheChen2002/GGR.
Authors:Ricardo da Rocha Carvalho, Eloísa Oliveira, Luiz Bernardo Martins Kummer, Emerson Cabrera Paraiso, Rayson Laroca
Abstract:
Introduction: Most Multiplayer Online Battle Arena (MOBA) analytics studies rely on structured data, which does not directly capture what each team could actually see during a match. Objective: This work introduces Dota2‑Vis, a video‑based dataset, and a baseline pipeline for visibility analysis in professional Dota 2 matches. Methodology: The dataset comprises all 144 matches from The International 2025, recorded from both team perspectives, totaling 288 Full HD videos, together with 2,477 manually annotated minimap images. We evaluate multiple variants of a modern object detector for player‑icon detection and use the best‑performing model to estimate opponent‑visible player presence over time. Results: YOLO11l (large) achieved the best overall performance, reliably identifying player icons even in dense and visually cluttered minimap scenes. The resulting visibility curves reveal player, hero, role, and team‑level patterns that complement conventional MOBA analytics, highlighting behavioral differences that are difficult to obtain from structured data alone. The dataset and code are publicly available at https://github.com/RicardoRCarvalho/dota2‑vis/.
Authors:Shuchao Duan, Alan Whone, Hossein Rahmani, Jun Liu, Majid Mirmehdi
Abstract:
Existing facial expression quality assessment (FEQA) methods typically produce only a severity score, without explicitly communicating the observable facial motion evidence that supports the prediction. This limits interpretability and makes it difficult to inspect the basis of model outputs in Parkinson's disease assessment. To address this gap, we propose TraMP‑LLaMA, a unified multimodal framework that jointly predicts severity scores and generates structured textual reports from facial motion cues. The framework integrates RGB appearance and landmark trajectory cues, and adopts a decoupled instruction‑tuning strategy to reduce task interference between severity prediction and language generation. To support this task, we further extend the PFED5 dataset with expert‑guided textual motion descriptions and construct PFED5‑plus. Experiments on PFED5‑plus show that TraMP‑LLaMA outperforms competitive video‑language baselines in report generation and achieves the best severity prediction performance among the compared methods under joint multi‑expression training, improving Spearman's rank correlation by at least 4.39 percent over all competing methods. The text annotations and code are available at https://github.com/shuchaoduan/TraMP‑LLaMA.
Authors:Kexu Cheng, Zicheng Liu, Mingju Gao, Chunhe Song, Hao Tang
Abstract:
Developing physically aware video generation models remains a significant challenge due to the difficulty in capturing diverse physical phenomena, such as thermal dynamics, mechanics, and optics. In this work, we introduce PhysRAG, a novel pipeline that enhances physical awareness in video generation through Retrieval‑Augmented Generation (RAG). To address the issue of limited high‑quality data, we design a two‑stage data filtering pipeline based on the WISA‑80K dataset, resulting in a curated set of 7K high‑quality videos for training. Furthermore, we construct a physical video database and develop a mechanism to inject physical knowledge into a video diffusion model using learnable queries. Our method achieves state‑of‑the‑art performance in both visual quality and physical rule compliance, surpassing existing models in benchmarks such as PhyGenBench and VBench. We conduct extensive ablation studies to validate the effectiveness of our key components, including the data filtering pipeline, RAG mechanism, and method for physical information extraction. To facilitate future research, our code, data, and models are prepared for release at https://github.com/sediment1024/PhysRAG.
Authors:Jinwook Jung, Taegyu Kim, Kumju Jo, Sungyong Baik
Abstract:
Multi‑task model merging aims to consolidate several task‑specific experts into a unified model, yet static merging consistently suffers from parameter interference. While dynamic merging models aim to bridge this gap, many works rely on the costly storage and loading of redundant expert components at inference. In this work, from the perspective of task expert, we view parameter interference as parameter perturbation introduced to each expert during merging process. We show that such parameter perturbations can be modeled as affine transformation, which can be approximated as additive offsets. Motivated by these, we propose Recover Task eXpert (ReTeX), a framework that predicts those offsets, in order to undo parameter interference and recover task‑expert performance from a single merged checkpoint. To recover the appropriate expert when task identity is unknown, we introduce a router‑free task identifier based on SVD subspace signatures computed offline before inference. At inference, the identifier selects the task whose subspace yields the smallest projection residual for a given input. As a result, ReTeX recovers over 95% of individual‑expert performance in both vision and NLP domains, while significantly improving generalization to unseen tasks. Crucially, we also show that the parameter offset prediction leads to emergent adaptive interpolation of expert knowledge for out‑of‑distribution (OOD) tasks. ReTeX adaptively interpolates seen expert knowledge to handle unseen tasks. Our code is available at https://github.com/BAIKLAB/ReTeX
Authors:Yulin Mao, Shiyu Li, Shuping Song, Yuling Zhang, Yajun Song
Abstract:
We present MedSWFlow, an open‑source, model‑agnostic LLM workflow for drafting medical social work case plans. The framework translates professional case‑planning tasks into six stages: assessment, problem analysis, goal setting, intervention planning, risk anticipation, and planned effect evaluation. Drawing on established social work and behavioral frameworks, MedSWFlow standardizes case inputs, builds structured case profiles, and generates reviewable assessment forms and service plans through staged prompting. The system is released as an open‑source research framework for reproducible case‑plan generation across LLM providers. Outputs are intended as practitioner‑reviewed drafts rather than final service decisions. Source code: https://github.com/santhiyacw‑droid/MedSWFlow/tree/main.
Authors:Snehal Raj, Brian Coyle, Léo Monbroussou, André J. Ferreira-Martins, Renato M. S. Farias, Elham Kashefi
Abstract:
Graphs provide a natural language for relational data in chemistry, biology and optimisation. Graph neural networks (GNNs) have driven much of the recent progress in learning from such data through message passing, a single primitive that generalises convolution and attention. Quantum counterparts have been proposed, but with limited connection to message passing and few guarantees on performance or scalability. More broadly, the trainability of variational quantum circuits is a recognised bottleneck for their wide applicability, and pre‑training has emerged as one way to address it. Yet for a quantum model to be useful, it must offer expressivity guarantees along with demonstrable scalability. Here we show how a quantum graph neural network can be built to perform message passing, to be permutation equivariant, and to sit at a chosen level of the Weisfeiler‑Leman hierarchy, the standard measure of how finely a model can tell graphs apart. We show that, as for classical GNNs, the training can be done first on small graph instances, allowing for a pre‑training that can mitigate usual training issues, and its output can be read out at a cost that stays low as the graph grows. We validate the framework in large‑scale simulations of up to 56 qubits across three datasets, on synthetic graphs that ordinary message passing cannot separate, on molecular property prediction, and on the travelling salesperson problem. Our framework opens a path for near‑term quantum algorithms with theoretical guarantees and practical scalability, bringing the principles of graph learning into quantum circuit design.
Authors:Daniel Barath
Abstract:
Rolling shutter (RS) cameras equip virtually all consumer devices, yet RS‑aware relative pose estimation has remained impractical: the state‑of‑the‑art solver requires a minimum of 20 point correspondences, making RANSAC‑based robust estimation prohibitively expensive due to the exponential dependence of the iteration count on the sample size. We make RS relative pose estimation practical by introducing affine correspondences (ACs) into the RS two‑view geometry. We derive novel \emphRS‑corrected affine constraints that account for the coupling between point perturbations and the row‑dependent essential matrix, providing two equations per correspondence beyond the standard epipolar constraint. Building on these constraints, we develop a linearized algebraic solver that estimates pose and RS motion from only 7 ACs. The solver exploits the physical smallness of RS parameters to linearize the constraints, eliminates the 12 RS unknowns via null‑space projection, and solves the remaining degree‑20 system via action matrices in 1.2\,ms. On the TUM RS benchmark, our method achieves the best pose and RS parameter accuracy among all tested methods and, uniquely among RS solvers, provides accurate translational velocity estimates ‑‑ which are poorly conditioned from point correspondences alone due to a \vecv‑\vect coupling. On the global‑shutter EuRoC MAV dataset, the solver achieves comparable accuracy to the standard 5‑point algorithm, demonstrating that it generalizes well to the GS setting. Code is at https://github.com/danini/rolling_shutter_made_practical.
Authors:Janik Bischoff, Anne Meyer, Uta Mohring, Fabian Dunke, Maximilian Barlang, Özge Nur Subas, Hadi Kutabi, Stefan Nickel, Kai Furmans
Abstract:
Order fulfillment in manual picker‑to‑goods warehouses involves interconnected decisions such as item assignment, order batching, and picker routing. While integrated models capture interactions between these decisions, practical warehouse systems often require decomposed approaches due to organizational boundaries, differing responsibilities, or limited data availability. Existing studies primarily evaluate algorithms for isolated subproblems or fixed subproblem combinations for specific warehouse settings, but lack a general mechanism to determine applicable algorithm configurations, compose them into valid solution pipelines, and assess their performance. With Context‑Aware Synthesis of Optimization Pipelines (CASOP), we propose a framework for constructing and evaluating context‑specific optimization pipelines and apply these to order fulfillment. The framework comprises: (1) a modular repository of algorithms for common order fulfillment problems; (2) semantic data and algorithm cards to describe warehouse context and algorithm requirements; (3) a taxonomy that structures order fulfillment problems into relevant subproblems; (4) a pipeline synthesizer that identifies applicable algorithms for a given warehouse context and composes all valid optimization pipelines; and (5) a pipeline evaluator that assesses all resulting pipelines. We demonstrate the framework on 7 benchmark instance sets covering four problem classes, resulting in 1,063,044 valid pipelines. The framework supports researchers and practitioners in designing, automatically synthesizing, and selecting valid, high‑performing algorithmic pipelines for warehouse operations. The software is open‑source and available at https://github.com/kit‑dsm/ware_ops_pipes and https://github.com/kit‑dsm/ware_ops_algos. Keywords: Warehouse optimization, Algorithm selection, Pipeline synthesis, Order fulfillment
Authors:Ke Chen, Ling Zhou, Guangqi Jiang, Gengshen Wu, Yi Liu, Shoukun Xu
Abstract:
General Salient Object Detection (SOD) aims to identify and segment visually interesting objects from uni‑modality or multi‑modality scenes, recently advanced by cutting‑edge State Space Models (SSMs). However, a critical limitation of current approaches is their neglect of the inherent spectral biases exhibited by different neural network paradigms. By digging to the dataset‑level spectral analysis of Convolutional Neural Networks (CNNs) and SSMs, their semantic representations are inherently complementary based on their complementary frequency preferences. Inspired by this, we harmonize heterogeneous representations from SSMs and CNNs to bridge their spectral biases for general salient object detection. To this end, inspired by the dynamic information propagation of Liquid Neural Networks (LNNs), we introduce a liquid fusion to dynamically integrates features from two backbones, including VMamba and ConvNeXt, referred to Liquid Fusion Network (LFNet). Concretely, by treating the continuous VMamba features and ConvNeXt features as evolving states and exogenous stimulus, respectively, LFNet employs a dynamic gating mechanism for content‑aware feature aggregation. Crucially, this state‑stimulus paradigm enables to scale to multi‑modal cues, resulting in flexibility in general SOD. Besides, a Saliency‑Guided Upsampling (SGU) operator to propagate the features to the shallow layer, which leverages a spectral‑spatial co‑design to suppress upsampling artifacts while preserving semantics. Extensive experiments across five diverse tasks (RGB, RGB‑D, RGB‑T, VSOD, and VDT) demonstrate that LFNet achieves state‑of‑the‑art performance, offering a superior trade‑off between detection accuracy and model efficiency. Code has been released at https://github.com/cke520/LFNet.
Authors:Xilai Li, Xiaosong Li, Haishu Tan, Tao Ye, Huafeng Li, Hongbin Wang
Abstract:
Multi‑modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross‑modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask‑guided MMIF method that integrates feature restoration and interaction. We first introduce "Pseudo Ground Truth" to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask‑guided cross‑modal cross‑attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the "Pseudo Ground Truth". Additionally, we propose a mask‑guided learning strategy and a task‑coupled degradation‑aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real‑world datasets demonstrate that our method surpasses state‑of‑the‑art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG‑Fuse.
Authors:Joy Bose
Abstract:
We present a computational corpus study of vocabulary relationships across eight tradition layers of Bengali and Sanskrit devotional literature spanning the 8th to 19th centuries, encompassing Buddhist Vajrayana, Shakta Tantra, Vaishnava, and Baul traditions. Using a corpus of 75 texts and TF‑IDF character n‑gram vectorization with cosine similarity analysis, we address the historically argued but previously unquantified claim that Buddhist Vajrayana vocabulary survived the collapse of the Pala monasteries and was absorbed into the Shakta Tantra tradition of Bengal. The central finding is a specificity result: the Gitagovinda (Vaishnava Sanskrit, 12th century) has zero cosine similarity to Shakta Kali texts, while Bridge Tara texts (Buddhist‑Shakta transitional, same century, same language) have cosine similarity 0.54 to Shakta Kali. This 8.5‑fold contrast between two Sanskrit traditions from the same century demonstrates that the Buddhist‑Shakta vocabulary overlap is not a generic property of Sanskrit devotional literature but is specific to the Buddhist‑Shakta transmission chain. Three Brihannilatantra Tara texts show Shakta‑to‑Buddhist vocabulary ratios of 2.0 to 4.0, constituting measurable evidence of lexical transition within that chain. Ramprasad Sen's 18th‑century Bengali Kali songs preserve Buddhist vocabulary residue including 56 occurrences of Tara alongside 103 occurrences of Kali. The Vaishnava Bengali tradition contributes a parallel chain to modern Baul vocabulary (similarity 0.29), slightly weaker than the Buddhist Sahajiya chain via Charyapada (0.31). These results provide the first quantitative multi‑tradition corroboration of historically argued Buddhist‑Shakta syncretism in Bengal.
Authors:Yuan Xu, Yixiang Chen, Kai Wang, Jiabing Yang, Peiyan Li, Qisen Ma, Yan Huang, Liang Wang
Abstract:
Vision‑Language‑Action (VLA) models have shown strong potential for generalizable robotic manipulation. During fine‑tuning, however, action supervision applies equally across all timesteps, without structured supervision on which manipulation stage the robot is in or what the next gripper‑event target should be. This causes failures to concentrate around challenging gripper‑event transitions. To address this, we propose StaKe, a plug‑in auxiliary supervision framework that automatically derives two complementary signals from demonstration gripper states without manual annotation: a stage classifier that identifies the current manipulation stage, and a keyframe predictor that estimates the target joint action at the next gripper transition. Both are modeled as lightweight auxiliary heads that enrich the learned representations during training, while leaving the base VLA policy architecture and inference loop unchanged. Experiments on bimanual simulation and single‑arm Franka real‑robot tasks show that StaKe consistently improves success rates (relative gains of 14% and 56%, respectively), with larger improvements on longer‑horizon tasks that involve more gripper‑event transitions. Ablation studies validate each design choice, and qualitative analysis confirms that the learned representations faithfully track manipulation stages. These results indicate that structured supervision is an effective and general strategy for enhancing VLA fine‑tuning in long‑horizon manipulation. Project website: https://hi‑yuanxu.github.io/StaKe‑Web/
Authors:Sicheng Zhang, Muzammal Naseer, Binzhu Xie, Naufal Suryanto, Shi Qiu, Jamal Bentahar, Naveed Akhtar, Mubarak Shah
Abstract:
CLIP and its variants are widely adopted visual backbones in multimodal systems, but their pretraining remains dominated by descriptive image‑text alignment. As downstream applications increasingly demand visually grounded commonsense inference and compositional reasoning, it remains unclear whether CLIP‑style encoders can support such reasoning without architectural changes. To address this, we present ReasonCLIP‑58M, a continual pretraining framework that integrates large‑scale reasoning supervision into CLIP‑style models through our two‑stage strategy, which progressively integrates reasoning signals while preserving descriptive alignment, followed by category‑structured reasoning supervision. To support this framework, we construct two complementary datasets and a benchmark: ReasonLite‑42M, with open‑form, visually verifiable reasoning captions; ReasonPro‑16M, with category‑specific reasoning supervision; and RCLIP‑Bench for diagnostic evaluation of visually grounded reasoning. We train a family of ReasonCLIP that improves visually grounded commonsense and compositional reasoning while also enhancing zero‑shot retrieval performance. As a drop‑in visual encoder for multimodal large language models such as LLaVA‑NeXT, ReasonCLIP delivers consistent gains without additional inference cost, demonstrating that structured reasoning supervision enhances the expressive capacity of CLIP‑style visual representations. All datasets, models, and training code are available at https://github.com/RISys‑Lab/ReasonCLIP.
Authors:Inderjeet Singh, Andrés Murillo, Motoyoshi Sekiya, Yuki Unno, Junichi Suga
Abstract:
Multimodal agentic retrieval‑augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct‑query attacks, and orchestrator‑level tool manipulation. Existing red‑teaming approaches are typically surface‑specific and often recycle known attack templates; on text‑poisoning benchmarks we measure 73‑84% exact duplication. We present MIRROR, a unified cross‑surface framework that performs memory‑guided Monte Carlo tree search while conditioning candidate generation on retrieved context under an explicit novelty constraint. A deterministic Novelty Gate rejects any candidate matching the retrieval set under normalized comparison, allowing retrieval to inform search priors without enabling prompt copying. Across four attack surfaces on a multimodal agentic RAG target, MIRROR attains 76% ASR on image poisoning compared with 52% for baselines, 97% ASR on orchestrator attacks at half the query cost, and the lowest cross‑surface variance (coefficient of variation 0.47). In contrast, specialized baselines collapse across surfaces: suffix optimization reaches 79% ASR on text poisoning but 1% on direct queries. We release ART‑SafeBench with 41,815 in‑package records and runtime adapters yielding 41,991+ total records across four surfaces.
Authors:Shuo Yang, Jinyang Wu, Zhengxi Lu, Yuhao Shen, Fan Zhang, Lang Feng, Shuai Zhang, Haoran Luo, Zheng Lian, Zhengqi Wen, Jianhua Tao
Abstract:
Outcome‑based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory‑level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On‑policy self‑distillation offers dense token‑level supervision, yet existing skill‑conditioned variants often rely on external skill memories or retrieved privileged context, which are costly to maintain and can be mismatched with the state distribution induced by the current policy in multi‑turn interaction. We propose OPID (On‑Policy Skill Distillation), a framework that extracts skill supervision directly from completed on‑policy trajectories. OPID represents trajectory hindsight as hierarchical skills: episode‑level skills capture global workflows or failure‑avoidance rules, while step‑level skills capture local decision knowledge at critical timesteps. A critical‑first routing mechanism uses step‑level skills when critical decisions are identified and falls back to episode‑level skills as default guidance otherwise. The selected skill is injected into the interaction history, allowing the old policy to re‑score the same sampled response under both original and skill‑augmented contexts. The resulting log‑probability shift yields a token‑level self‑distillation advantage, which is combined with the outcome advantage for policy optimization. OPID thus preserves RL as the primary training objective while introducing dense, distribution‑matched hindsight supervision. Experiments on ALFWorld, WebShop and Search‑based QA demonstrate that OPID generally improves agent performance, sample efficiency, and robustness over outcome‑only RL and existing skill‑distillation baselines. Our code is available at https://github.com/jinyangwu/OPID/tree/main.
Authors:Xuyue Huang, Zhe Chen, Wang Shen, Xiao-Ping Zhang
Abstract:
Diffusion Transformers (DiTs) have driven substantial progress in image and video generation but suffer from prohibitive computational costs. Feature caching accelerates inference by reusing intermediate representations. Existing methods rely on historical features for implementation simplicity, yet suffer from severe error accumulation at high acceleration ratios. To address this limitation, we investigate the nature of the requisite feature correction. We demonstrate that the optimal calibration update is characterized by a shared low‑rank subspace across diverse prompts. Guided by this structural insight, we propose LearniBridge, a learnable calibration mechanism for feature caching that bridges multiple timesteps through lightweight LoRA updates. This mechanism enables effective calibration requiring only 3‑5 training samples. Extensive experiments on image and video generation show that LearniBridge achieves up to 5.87×, 5.75×, and 4.10× acceleration on FLUX, HunyuanVideo, and WAN2.1, respectively. On WAN2.1, it improves VBench by 1.28% over the previous SOTA at 4.10× acceleration. Our code is available at https://github.com/Iiiiiiirene/LearniBridge.
Authors:Yiheng Cao, Gustavo Andrade-Miranda, Jiatian Zhang, Lingxiao Zhao, Xin Gao
Abstract:
Developing robust artificial intelligence models for 4D (3D + time) medical imaging is constrained by limited annotated data, inter‑device domain shifts, and privacy restrictions. To address this, we propose a 4D controllable generative framework for anatomically consistent data augmentation. A semi‑supervised variational autoencoder learns a compact latent representation of anatomical volumes while jointly predicting aligned segmentation masks in a unified framework. Anatomical structure is then disentangled from temporal dynamics through a cascaded latent diffusion model (LDM). A static LDM generates subject‑specific anatomy conditioned on clinical priors (diagnosis and volumes measures) and a subsequent motion LDM estimates residual latent motions, ensuring strict temporal coherence across the 4D sequence. The proposed approach was evaluated on cine cardiac MRI as a representative 4D imaging application. Experiments across multiple datasets demonstrate high controllability of static anatomy (Pearson r > 0.8) and strong temporal coherence (FVD = 288.08). In cross‑vendor generalization experiments, augmenting training sets with synthetic 4D sequences significantly improves downstream segmentation performance. Using nnU‑Net, the proposed augmentation strategy improves the average Dice score by 1.4% and reduces the Hausdorff Distance by 3.0mm compared to training on real data alone, for the left ventricle, Dice improves by 2.8% with a 5.4mm reduction in boundary error. Overall, this framework provides a scalable and controllable solution for 4D medical image synthesis, supporting the development of more robust models with limited annotations and cross‑vendor variability. Code available on https://github.com/cyiheng/4DCardiacMRISynthesis.
Authors:Honghang Chen, Xiujun Zhang, Xiaoli Sun, Mingqing Xiao
Abstract:
Implicit neural representation (INR) has emerged as a powerful prior for multi‑dimensional data (e.g., multispectral images and videos). However, most INR methods employing periodic activation functions (e.g., Sine) predominantly rely on function composition. This mechanism introduces optimization instability as network depth increases, thereby limiting their performance. Meanwhile, these methods fail to incorporate proper physical priors to effectively alleviate spectrum bias. To address these issues, inspired by the commonalities between deep periodic networks and generalized Fourier series, we propose a novel Calibrated Harmonic Overlaid Implicit Neural Representation (CHOIR). Specifically, we utilize Coordinated Harmonic Superposition (CHS) to replace the conventional function composition used in most INRs, thereby ensuring optimization stability when scaling network depth. Furthermore, we introduce a Perceptual Spectrum Calibration (PSC) to mitigate spectrum bias. This calibration embeds the ubiquitous power‑law spectrum prior of natural images and adjusts the globally fixed spectrum towards a physically plausible log‑uniform distribution. Extensive experiments on various multidimensional data recovery problems demonstrate that our method achieves superior performance over state‑of‑the‑art approaches. Code is available at https://github.com/chorl0229/CHOIR.
Authors:Majharulislam Babor, Giacomo Rossi, Annalisa Altavilla, Oliver Schlüter, Marina M. -C. Höhne
Abstract:
Edible insects offer an efficient source of alternative protein, requiring less land, water and emitting less greenhouse gas than conventional livestock. However, their successful integration into the food supply chain demands reliable species authentication to control allergen exposure, prevent adulteration, and meet regulatory standards. Near‑infrared spectroscopy provides a rapid analytical tool, but its performance drops when applied to production batches unseen during training due to batch‑to‑batch variation in spectral measurements. We introduce the Batch‑Invariant Spectral Network (BISN), an end‑to‑end framework that combines a learnable preprocessing module, initialised with Savitzky‑Golay filtering, with an entropy‑regularised adversarial objective to suppress batch‑specific spectral variation. In contrast to Domain‑Adversarial Neural Networks, which enforce domain adaptation only after feature extraction, BISN suppress batch‑effects before species‑specific features are learned. Using 2,700 spectra from three species (Acheta domesticus, Hermetia illucens, and Tenebrio molitor) collected across three independent production batches, BISN achieves a mean leave‑one‑batch‑out accuracy of 0.93 (standard deviation 0.04), outperforming the strongest baseline by four percent. Further insights gained by using explainable AI confirm that model decisions consistently rely on the lipid and protein absorption regions across all folds, connecting predictive performance to known insect biochemistry. BISN addresses both cross‑batch robustness and biochemical interpretability for automated insect species authentication under realistic industrial conditions. The source code and dataset are publicly available at https://github.com/majharB/bisn.
Authors:Zhihao Wen, Yixin Yang, Bojian Wu, Yang Zhou, Dani Lischinski, Daniel Cohen-Or, Hui Huang
Abstract:
While 3D Gaussian Splatting (3DGS) provides an efficient and explicit representation for novel view synthesis, enforcing stylistic coherence across viewpoints remains challenging. Existing 3D stylization methods typically apply 2D feature‑matching losses independently per rendered view, which leads to unstable style allocation, many‑to‑one feature reuse, and limited cross‑view consistency. We propose a capacity‑controlled framework for multi‑view stylization of 3DGS, grounded in optimal transport. Specifically, we reformulate local style matching as a semi‑balanced optimal transport problem. By introducing explicit column‑capacity constraints with tunable strength, our formulation mitigates many‑to‑one matching and enables controllable allocation of style features. This transport‑based objective provides a principled mechanism for balancing feature coverage and stylistic diversity while maintaining stable correspondences across viewpoints. To further enhance cross‑view coherence, we incorporate a novel cross‑view matching guidance to constrain correspondences between scene content and style patterns. In addition, we introduce several geometric regularizations to enhance the vanilla 3DGS, thereby enabling optimized Gaussian primitives to represent finer‑grained textures during stylization. Extensive experiments demonstrate that our approach significantly improves multi‑view stylistic consistency and produces stable, expressive 3D stylizations while preserving the core semantic structure of the scene.
Authors:Haofei Song, Siyuan Xu, Xintian Mao, Shaojie Guo, Qingli Li, Yan Wang
Abstract:
Arbitrary slice super‑resolution reconstructs isotropic volumes from anisotropic clinical acquisitions by synthesizing intermediate slices at arbitrary scales. However, treating this ill‑posed inverse problem as unconstrained residual‑based regression risks hallucinating anatomically implausible structures or altering the originally observed data. To address both concerns, this paper presents the Dual‑Prior Null‑space Learning (DP‑NSL) framework, which reformulates the task as a constrained recovery process guided by two complementary priors. A Measurement‑Consistent Projection (MCP) enforces a Deterministic Observation Prior: the reconstruction undergoes an exact orthogonal projection that reproduces every acquired slice with zero error, confining all learned details to the unobservable null space. Within this null space, a Mixture‑of‑Splines (MoS) module imposes a Geometric Continuity Prior by dynamically mixing B‑spline experts of different analytic orders, allowing each anatomical region to be modeled with a content‑aware level of continuity. To promote spatial coherence, a Local Spatial Consistency Decoder (LSCD) further injects local inductive bias. Experiments on three CT and one MRI benchmark show that DP‑NSL outperforms existing approaches while strictly preserving measurement consistency. Code is available at https://github.com/DeepMed‑Lab‑ECNU/Medical‑Image‑Reconstruction.
Authors:Kim Youwang, Jon Hasselgren, Peter Kocsis, Andrea Weidlich, Tae-Hyun Oh, Jacob Munkberg
Abstract:
Neural materials can represent complex specular reflections and scattering effects in a compact, universal basis. However, acquiring and authoring such materials remains challenging. We present NeuMatEx, a differentiable inverse rendering method for extracting spatially varying neural materials from images. The nonlinear structure of neural material latent spaces makes optimization with naive inverse rendering infeasible. To address this, we train a Large Material Reconstruction Model (LMRM) that directly predicts initialbase color, neural material latents, and aleatoric uncertainty guides from images. This material prior provides a good initialization and better constrains our subsequent optimization using inverse path tracing. The predicted uncertainty further helps by anchoring high‑confidence regions more tightly to the LMRM prediction, preventing lighting and complex specular effects from being baked into materials. Experiments on synthetic and real assets show that NeuMatEx extracts complex materials with better visual quality and material decomposition than PBR‑based methods.
Authors:Yuqi Jiang, Yumeng Liu, Zimu Li, Jinyuan Deng, Qian Jin, Yucheng Cui, Yu Li, Xunzhao Yin, Qi Sun, Cheng Zhuo
Abstract:
As semiconductor technology nodes scale, computational lithography is essential for ensuring yield and performance. However, lithography is a continuous physical process involving mask optimization, optical imaging, resist exposure, and development, which existing models fail to capture. To overcome this limitation, we present LithoDreamer, the first physics‑informed World Model (WM) framework for computational lithography, which formulates the ``Layout‑Mask‑Resist Image‑After Development Image (ADI)'' pipeline as a decision‑driven multi‑step evolution system. LithoDreamer captures feature changes between adjacent states to model stage‑specific physics‑informed latent spaces, in which it controls process intervention exploration and drives subsequent state transitions. To achieve interpretable intervention optimization without continuous supervision, we propose a contrastive variational optimization paradigm that contrasts the latent differences between intervention paths with variational evolution constraints, guiding the model to generate evolutions consistent with real lithography physics. Experiments show LithoDreamer achieves state‑of‑the‑art performance in forward evolution and inverse planning. Our lithography dataset is publicly available at GitHub (https://github.com/7jiangyq/lithodreamer.git).
Authors:Jingjun Gu, Chaojie Shen, Yifeng Cao, Wei Zhang, Yiliu Li, Aobo Fan
Abstract:
Skin lesion segmentation is a key task in computer‑aided dermatological diagnosis, where accuracy directly impacts downstream analysis and disease classification. However, dermoscopic images are challenging due to blurred boundaries, low contrast, large shape variations, and artifacts such as hair and shadows. Recently, diffusion models have shown strong performance in medical image segmentation thanks to their progressive denoising and distribution modeling capabilities. Nevertheless, existing diffusion‑based methods still suffer from limited cross‑level feature interaction and insufficient boundary detail recovery. To address these issues, we propose MLFFM‑SegDiff, a multi‑level feature fusion diffusion model for skin lesion segmentation. Built on a diffusion framework, the method introduces a dual‑path U‑Net encoder, a Multi‑Level Feature Fusion Module (MLFFM), and a boundary‑sensitive loss function. The dual‑path encoder enhances interaction between noisy mask features and dermoscopic image features. MLFFM improves skip connections via attention, scale alignment, and adaptive cross‑level fusion. These designs enable the decoder to jointly leverage shallow boundary cues and deep semantic representations, improving mask reconstruction quality. Experiments on ISIC2018, PH2, and HAM10000 demonstrate that MLFFM‑SegDiff outperforms representative methods including DermoSegDiff, U‑Net, and SwinUNETR across Accuracy, F1‑score, Jaccard index, Recall, and Dice. In particular, it achieves an average Jaccard index of 0.8546 and Dice coefficient of 0.9207. These results validate the effectiveness of the proposed multi‑level feature fusion strategy for improving lesion segmentation performance. The code will be released at https://github.com/Qacket/MLFFM‑SegDiff.git after publication.
Authors:Quan Zhou, Shaoqing Zhai, Qiang Hu, Jia Chen, Qiang Li, Zhiwei Wang
Abstract:
Transforming foundation segmentation models from human‑prompted tools into auto‑promptable annotators is critical for scalable medical data annotation. Current methods commonly depend on external feature matchers or auxiliary networks to automate geometric prompting, but introducing architectural overhead and limiting performance scalability. Although SAM3 natively supports concept segmentation via reusable text prompts, its direct use in medical imaging is hindered by a lack of fine‑grained clinical knowledge and the ambiguity of human‑written descriptions. In this work, we propose Mask to Concept (M2C), an efficient framework that adapts SAM3 for medical few‑shot annotation without external modules, parameter retraining, or manual text engineering. Using only a few labeled images, M2C enables SAM3 to automatically search for transferable visual concepts entirely within its frozen architecture: it initializes a learnable concept embedding, uses it to prompt segmentation, and updates the embedding by gradients of minimizing the concept segmentation error. We further introduce a Hybrid Uncertainty Estimation (HUE) module that calculates the prediction entropy and maps concept predictions back to the box prompts, measuring concept‑geometry prompting inconsistency. Highly uncertain samples are flagged actively for human correction, and the corrected masks are then fed back to M2C to continuously search for more precise concept embeddings, forming a self‑enhancing annotation loop with minimal expert effort. Experiments on medical segmentation benchmarks show that our method achieves SOTA few‑shot segmentation performance and outstanding annotation efficiency, offering a practical and efficient pathway toward scalable medical image labeling. Codes are at https://github.com/Huster‑Hq/M2C.
Authors:Bin Hu, Yanwen Ma, Jiehui Huang, Ziliang Zhang, Haoning Wu, Ruicheng Zhang, Yaokun Li, Zijun Wang, Yuechen Zhang, Chun-Mei Tseng, Hanhui Li, Shengju Qian, Jun Zhou, Kaipeng Zhang, Xiaodan Liang, Jiaya Jia, Xiu Li
Abstract:
Recent game world models can synthesize visually plausible, action‑conditioned rollouts. However, their interaction behaviors often remain limited to exploratory or wandering trajectories, and physical dynamics are typically learned as implicit correlations from data rather than as controllable variables. This limitation hinders their applicability to authored game environments, where physical rules are deliberately designed and require explicit manipulation. We introduce PhysEditWorld, a multimodal dataset with physical parameters, with a primary focus on gravity in this initial version. At its core, PhysEditWorld is built upon a replay paradigm implemented with a UE5 replay‑and‑rendering pipeline. Each scenario records a normalized action trace and replays the same initial state, character controller, action sequence, and camera policy under multiple gravity configurations, enabling controlled and attributable physical variation. PhysEditWorld contains 12 cinematic UE5 scenes, over 100 hours of gameplay interactions, and more than 60 million rendered rollout frames. Each sample provides synchronized multimodal signals, including RGB, depth, normals, audio, action traces, camera trajectory, engine states, semantic annotations, and explicit gravity labels. We further conduct initial utility studies on both generative video models and world understanding models, demonstrating that PhysEditWorld enables improved gravity‑faithful dynamics modeling, enhances consistency under physical edits, and provides a scalable foundation for controllable world modeling research.
Authors:Dongbin Na
Abstract:
In order to screen a prompt or a response, the recent guardrail methods generate a chain‑of‑thought (CoT) before they issue a verdict. This design follows a common belief that step‑by‑step reasoning improves a decision. However, CoT also makes the guard heavy and slow, because the model must generate many tokens before it decides. This may not match how guardrails are actually deployed. A guardrail sometimes should not be heavy and slow, and it often runs on‑device, for example on an embodied robot. In this paper, we pose a question whether a safety guardrail really needs to reason. To answer this question, we train a lightweight bidirectional encoder and a reasoning guard on the same corpus, and we then remove only the reasoning while we keep everything else fixed. With this controlled same‑base comparison, we show that the chain does not improve moderation accuracy. We name the resulting guard LeanGuard. A 395M label‑only encoder reaches an average F1 of 82.90 \pm 0.26 over public benchmarks. It matches a reasoning guard that is built on a much larger decoder, while it uses only a single forward pass over an input of at most 512 tokens. This is about a ~100x reduction in inference compute. We further show that this label‑only encoder stays robust under training‑label noise and retains far more recall at a strict false‑positive rate than the reasoning guard, so a heavier reasoning guard is not the more robust choice either. Our finding suggests that the current guardrail benchmarks may not be hard enough to reward reasoning, and that the necessity of CoT for moderation is still not proven. We release all source codes and models including LeanGuard at https://github.com/ndb796/LeanGuard.
Authors:Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Anbang Yao
Abstract:
In this paper, we present CAT‑Q, Cost‑efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state‑of‑the‑art ternary quantization methods that rely on data‑intensive and costly quantization‑aware training to mitigate severe performance degradation, CAT‑Q is a simple yet effective post‑training quantization scheme that is readily applicable to LLMs with diverse architectures and model sizes. It has two key components, learnable modulation (LM) and softened ternarization (ST), which are coupled from an optimization perspective. LM leverages a composition of learnable factors to modulate the distribution of pre‑trained high‑precision weights and the ternary threshold, making them less sensitive to ternarization. ST further introduces a differentiable transition function to guide the ternarization process toward stable convergence. We show that, for pre‑trained LLMs with 1.7B to 8B parameters, CAT‑Q can efficiently quantize them into ternary models using only 512 calibration samples, while achieving superior performance than the seminal BitNet 1.58‑bit v1 and v2 families (with 1.3B to 7B parameters) trained with 100B tokens, yielding about a 100,000X reduction in training tokens. Moreover, we show for the first time that CAT‑Q can quantize much larger pre‑trained LLMs having 14B to 235B parameters into leading ternary models within just 8 to 60 hours on 8 A100‑80GB GPUs. Code is available at https://github.com/IntelChina‑AI/BitTern.
Authors:Hongjae Lee, Sojung Kang, Jaeseong Yu, Seung-Won Jung
Abstract:
While traditional image restoration focuses on perceptual quality, Task‑Driven Image Restoration (TDIR) aims to maximize the performance of downstream high‑level vision tasks. Recent approaches leveraging generative priors have shown promise for TDIR; however, they typically suffer from computational inefficiency and potential semantic alteration by indiscriminately updating all latent tokens. In this paper, we posit that not all visual information is equally important for machine perception. Through an analysis of the latent token space, we observe that task‑relevant cues are unevenly distributed across the token sequence, exhibiting index‑wise specialization. This suggests that selectively refining a subset of tokens can be sufficient for task‑driven objectives. Leveraging this insight, we propose TaskTok, a novel framework that selectively restores only task‑relevant tokens via a learnable token switch and a lightweight token refinement module. Extensive experiments across image classification, semantic segmentation, and object detection demonstrate that TaskTok significantly enhances task performance with high computational efficiency. The source code is available at https://github.com/jimmy9704/TaskTok
Authors:Kuangshi Ai, Patrick Phuoc Do, Chaoli Wang
Abstract:
Large language model (LLM) agents enable natural language interaction for scientific visualization (SciVis). Still, prior systems have essentially prioritized autonomy over human analytical control, thereby limiting transparency and human oversight. We present HiLSVA, a human‑in‑the‑loop agentic system that supports mixed‑initiative SciVis workflows. HiLSVA integrates a plan‑first multi‑agent architecture with explicit human oversight, stepwise provenance tracking, and learn‑at‑test‑time adaptation from user feedback. The system supports fluid handoff between humans and agents through both natural language and direct manipulation of visualizations, while sandboxed execution ensures safe, reproducible workflows. In doing so, HiLSVA reframes agentic SciVis as a collaborative process that augments, rather than replaces, human analytical reasoning. We evaluate HiLSVA through representative case studies and a controlled user study with twelve participants of varying expertise across multiple autonomy settings. Results show that mixed‑initiative interaction improves task completion, user control, and workflow transparency across different levels of user expertise, while revealing a tradeoff between execution efficiency and human oversight. These findings highlight the importance of human‑centered design in agentic SciVis and guide the development of future collaborative visualization systems. We encourage readers to explore our demo video, case studies, and source code at https://hilsva.github.io/.
Authors:Hongjae Lee, Myungjun Son, Jaeseong Yu, Seung-Won Jung
Abstract:
Image restoration aims to reconstruct high‑quality images from degraded low‑quality inputs. As the computational demands of image restoration models continue to rise, there is growing interest in lightweight architectures optimized for fast and efficient inference. Logic gate networks (LGNs), which operate using fundamental logic operations such as NAND and XOR, have recently emerged as a promising direction for achieving highly efficient computation. However, their potential remains largely untapped in the domain of image restoration. In this work, we introduce LogicIR, the first LGN specifically designed for image restoration tasks. LogicIR incorporates a UNet‑inspired architecture composed entirely of logic gates. In addition, we propose a differentiable bit decoding layer and an index shuffling mechanism that improves information propagation across logic gates. Experimental results across multiple image restoration benchmarks demonstrate that LogicIR achieves strong performance with significantly reduced computational cost, establishing LogicIR as a viable and efficient alternative for image restoration. The source code is available at https://github.com/jimmy9704/LogicIR
Authors:Haoxin Sun, Zhongzhi Zhang
Abstract:
The forest matrix of a signed graph plays an important role in network science and social opinion dynamics, yet existing algorithms are mainly designed for unsigned graphs and are difficult to extend to signed graphs. In this paper, we study the problem of efficiently estimating the forest matrix of signed graphs with n nodes and introduce the signed forest matrix theorem, which establishes the relationship between generalized spanning converging forests and the forest matrix. Based on this result, we propose a novel algorithm GSCF, built on a variant of loop‑erased random walks, to generate generalized spanning converging forests in expected O(n) time. We further develop two sampling algorithms, FMDE and FMDE+, for estimating the diagonal of the forest matrix, both with time complexity O(ln), where l is the number of samples. Extensive experiments on various signed graphs show that our methods achieve high estimation accuracy, significantly improve computational efficiency, and scale to graphs with over twenty million nodes. Our source code is publicly available on https://github.com/HaoxinSun98/SignedForestDiagonal.
Authors:Vidullan Surendran, Neehar Peri, David Watkins
Abstract:
Handheld data collection systems, such as the Universal Manipulation Interface (UMI), enable scalable data collection across diverse environments but only capture observed actions rather than the desired actions executed by a robot controller. In contrast, teleoperation captures desired actions directly, but is prohibitively time‑consuming to collect. We revisit this trade‑off through the lens of action validity across task phases. We observe that handheld trajectories provide valid supervision in tolerant, free‑space phases, but lack dynamic feasibility in contact‑sensitive phases, where tracking observed trajectories at high stiffness produces large, unsafe contact forces. We study the interaction between these two supervision types for contact‑rich manipulation and find that training policies that combine handheld data with a small number of targeted teleoperated demonstrations provide an efficient hybrid strategy. Specifically, rather than teleoperating the entire task, we only collect partial teleoperated demonstrations for task segments where base handheld policies fail. However, naively mixing handheld and teleoperated phase‑specific data yields worse performance than training on handheld data alone. To address this mismatch between observed and desired supervision, we propose Bi‑modal Routing for Imitation Data via Gated Experts (BRIDGE), a mixture of diffusion policy experts that routes between specialist task phase heads conditioned on the current robot state. Notably, our approach enables task‑phase specific use of desired actions during contact sensitive segments and improves success rates over handheld‑only baselines by up to 36.7% across three contact‑rich manipulation tasks.
Authors:Geng Li, Yuxin Peng
Abstract:
Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive fine‑grained perception capabilities. However, existing benchmarks predominantly rely on explicit textual cues or low‑resolution inputs, failing to evaluate a model's ability to autonomously perceive implicit visual cues in high‑resolution. To bridge this gap, we introduce DiCoBench, a comprehensive, multi‑image high‑resolution benchmark designed for cross‑image fine‑grained perception. DiCoBench consists of 765 meticulously curated samples categorized into two progressive tracks: Differential Visual Cues and Commonality Visual Cues, covering 8 distinct perception tasks. By formulating the benchmark as a multiple‑choice question task and utilizing high‑resolution imagery (approaching 2K), we eliminate evaluation metric bias and pose a substantial challenge to current state‑of‑the‑art MLLMs. Our extensive evaluation of 18 diverse MLLMs reveals a striking performance gap compared to human accuracy (98.3%), with top‑performing models struggling significantly with micro‑scale detail capture. We believe DiCoBench will serve as a challenging testbed to drive future research in autonomous, high‑resolution multi‑image perception.
Authors:Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Huaqing Zheng, Xindian Ma, Peng Zhang
Abstract:
Low‑bit floating‑point formats and semi‑structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation compression remains challenging: activations contain input‑dependent outliers that dominate block scales in FP4 quantization, and directly applying N:M sparsity masks discards moderate values, coupling sparsification loss with quantization error. We introduce SharQ, a training‑free inference method that bridges activation sparsity and FP4 quantization through an online sparse‑‑dense decomposition. For each activation tensor, SharQ generates an input‑adaptive N:M mask to extract an outlier‑dominated sparse backbone, quantizes it to FP4, and defines a dense residual relative to the quantized sparse backbone rather than the unquantized sparse values. A sparse FP4 GEMM processes the backbone while a dense FP4 GEMM compensates for both mask‑induced activation loss and sparse‑path quantization error. The two paths share a single FP4 weight payload with path‑specific scale views, and a fused preparation kernel absorbs mask generation, residual construction, and layer normalization into one operator. SharQ requires no calibration data, retraining, or model‑specific tuning. Evaluated on Llama‑3.1‑8B, Qwen2.5‑7B, Qwen3‑30B‑A3B, and Qwen3‑VL‑8B, SharQ recovers 43‑‑63% of the NVFP4‑to‑FP16 accuracy gap across language and vision‑language tasks, and generalizes across NVFP4, HiF4, and MXFP4 formats. On an RTX 5090, SharQ delivers 2.2‑‑2.4× latency reduction over FP16 and 1.2‑‑1.4× throughput improvement over FP8 in language model serving, and up to 1.58× speedup on Wan2.2‑T2V‑A14B video generation when combined with SageAttention. Our code is available at https://github.com/actypedef/SharQ.
Authors:Feifan Luo, Ting Li, Zhao Li, Hongyang Chen
Abstract:
Non‑rigid 3D shape matching is a fundamental task in computer vision and graphics. In this paper, we propose a hybrid self‑supervised method based on a coarse‑to‑fine strategy, which ensures consistency between the coarse mapping and the refined correspondence produced by our refinement module. The architecture features a dual‑branch design, consisting of two symmetric functional map learning streams: one based on the Laplacian basis and the other utilizing the elastic basis. Extensive experiments show that our approach not only maintains computational efficiency, but also achieves state‑of‑the‑art performance across a variety of challenging scenarios, including non‑isometric deformations and topological noise. Finally, we rigorously demonstrate that contrastive energies promote feature discrimination. Furthermore, integrating these energies with existing methods yields consistent improvements, validating the overall efficacy of our approach. Our code is available at https://github.com/LuoFeifan77/Coarse‑to‑Fine‑Hybrid‑Self‑Supervised‑Matching.
Authors:Shengbin Guo, Shaokang He, Chaoyue Meng, Shengpeng Xiao, Xunzhi Xiang, Shaofeng Zhang, Qi Fan
Abstract:
While instruction‑based image editing, enabled by multi‑modal generative models, has advanced significantly, existing benchmarks lack a comprehensive evaluation of physics‑based reasoning, a critical capability for handling real‑world scenarios. To address this, we introduce PhyEditBench, a benchmark designed to assess the physical understanding of editing models. Guided by a hierarchical taxonomy, we establish 4 primary classes and 12 subclasses. It comprises 238 high‑quality, high‑resolution, real‑world instances meticulously extracted from videos to capture authentic physical dynamics, alongside 35 synthetic Anti‑Physics instances. Our empirical analysis of current SOTA editing methods exposes substantial limitations in their physics‑based reasoning. We further propose a training‑free baseline named PhyWorld that uses test‑time scaling and a latent reduction strategy. PhyWorld outperforms comparable models and suggests that the video generation process can effectively serve as a reasoning mechanism for image editing. The project page is available at https://github.com/Previsior/PhyEditBench.
Authors:Ao Hu, Liangjian Wen, Jiang Duan, Yong Dai, He Yan, Dongkai Wang, Jun Wang, Yukun Zhang, Ruoxi Jiang, Zenglin Xu
Abstract:
Long‑term time series forecasting (LTSF) plays a crucial role in fields such as energy management, finance, and traffic prediction. Transformer‑based models have adopted patch‑based strategies to capture long‑range dependencies, but accurately modeling shape similarities across patches and variables remains challenging due to scale differences. To address this, we introduce patch‑mean decoupling (PMD), which separates the trend and residual shape information by subtracting the mean of each patch, preserving the original structure and ensuring that the attention mechanism captures true shape similarities. Futhermore, to more effectively model long‑range dependencies and capture cross‑variable relationships, we propose Trend Restoration Attention (TRA) and Proximal Variable Attention (PVA). The former module reintegrates the decoupled trend from PMD while calculating attention output. And the latter focuses cross‑variable attention on the most relevant, recent time segments to avoid overfitting on outdated correlations. Combining these components, we propose PMDformer, a model designed to effectively capture shape similarity in long‑term forecasting scenarios. Extensive experiments indicate that PMDformer outperforms existing state‑of‑the‑art methods in stability and accuracy across multiple LTSF benchmarks. The code is available at https://github.com/aohu1105/PMDformer.
Authors:Zhixing Li, Yinan Yu
Abstract:
Current VLM evaluations often conflate language priors with genuine spatial reasoning. To address this, we introduce CRISP, a novel structural‑diagnostic evaluation paradigm that assesses visual spatial intelligence through consistency, the alignment between implicit perception and explicit reasoning. Unlike traditional black‑box QA, CRISP utilizes metric 3D Scene Graphs and an oracle intervention protocol to decouple latent reasoning capabilities from perceptual bottlenecks. This granular diagnosis uncovers a systematic perception‑reasoning disconnect. Crucially, we reveal that while proprietary models possess robust latent reasoning engines, they suffer from inaccurate metric estimation and a critical failure to leverage their implicit structural representations. Conversely, open‑source models remain fundamentally bottlenecked by their lack of multi‑hop compositional reasoning. By shifting the focus from merely ``guessing correctly'' via language priors to genuinely ``perceiving, verifying, and reasoning,'' CRISP offers a rigorous roadmap for multimodal alignment beyond end‑to‑end post‑training. The code and dataset are available at https://github.com/iiyamayuki/CRISP‑Bench.
Authors:Yuxuan Yang, Feiyang Li, Yile Wang
Abstract:
The Abstraction and Reasoning Corpus (ARC) contains tasks that require summarizing patterns from limited grid samples and predicting output grids. Recently, many large language model based approaches have attempted to transform it into a text‑based reasoning task. However, methods based on open‑source models have generally yielded unsatisfactory results, while those relying on closed‑source models are too costly. Current efforts mainly focus on data augmentation, constructing ARC‑like data for more comprehensive supervised fine‑tuning. In this work, we argue that solving ARC‑like problems requires not only positive sample supervision but also the ability to improve model reasoning by distinguishing negative samples. To this end, we draw on the idea of preference alignment and propose DiARC, a method that constructs preference pairs to enable the model to distinguish between them. Specifically, we propose three ways to construct negative samples, including output‑level visual transformations, DSL‑level rule inversion, and task‑specific rule editing. The resulting negative samples provide informative near‑miss alternatives while keeping the observed demonstrations unchanged. Experimental results across multiple ARC‑like benchmarks show that DiARC consistently improves performance over baseline models. The code is released at https://github.com/szu‑tera/DiARC.
Authors:Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu
Abstract:
Large language models (LLMs) are attractive for context‑aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language‑oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug‑and‑play temporal wavelet digit interface that maps each scalar observation into digit‑wise embeddings constructed from multi‑wavelet, multi‑scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine‑grained local fluctuations and macro global structures in a transformer‑compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context‑enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM‑based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state‑of‑the‑art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi‑resolution embeddings can better couple LLMs' contextual reasoning with precise forecasting. Our code is available at https://github.com/DC‑research/TempoWAVE and our model can be accessed at https://huggingface.co/Melady/TempoWAVE.
Authors:Xiao Wang, Xufeng Lou, Zikang Yan, Lan Chen, Sibao Chen, Yaowei Wang, Yonghong Tian, Jin Tang
Abstract:
RGB‑Event tracking improves localization robustness by fusing RGB appearance textures and dense temporal motion cues from event sensors. While this multi‑modal scheme broadens tracking applicability, real‑world scenes suffer diverse structured signal degradations that hinder traditional multi‑modal fusion. In harsh environments, either modality can lose reliability drastically, and targets frequently appear incomplete due to occlusion, edge truncation and foreground clutter.To tackle the above challenges, we present a hierarchical perturbation and retrieval framework tailored for RGB‑Event tracking with robustness against partial target missing and modal degradation, termed APRTrack. To mimic real‑world signal corruption, APRTrack constructs structured degradation via two adversarial perturbation branches at the modality and spatial levels, which separately simulate full‑modal failure and localized target region absence. A hierarchical routing mechanism is designed to disentangle the training pipelines of the two perturbation types, effectively eliminating feature collapse induced by superimposed degradation constraints. Furthermore, we devise Footprint‑guided Channel‑calibrated Hopfield Retrieval (FCHR) for reliable historical information compensation. This module evaluates retrieval confidence based on association footprints between queries and memory banks, and calibrates the retrieval metric space prior to Hopfield matching, realizing controllable historical feature compensation bounded to target regions. Extensive experiments on FE108, COESOT, VisEvent, and FELT datasets demonstrate the effectiveness of our proposed strategies for the RGB‑Event visual object tracking. The source code and pre‑trained models will be released on https://github.com/Event‑AHU/OpenEvTracking
Authors:Neelam Saini, Sourav Ghosh
Abstract:
Automatic singing quality assessment (SQA) requires evaluating lyrical correctness and musical fidelity while handling expressive variations. However, existing systems largely rely on either acoustic cues or lyric transcriptions exclusively, limiting holistic performance evaluation. Furthermore, their integration is non‑trivial due to challenges in robust singing transcription amid melisma, vibrato, and tempo elasticity. To this end, we propose MusicJudge, a modality‑guided framework for automated SQA that performs block‑aligned multimodal analysis by coupling lyric correctness with pitch‑rhythm fidelity. It detects semantically meaningful lyric blocks using multi‑signal matching that integrates semantic embeddings, lexical similarity, and phonetic alignment. To improve singing audio transcription, we introduce Modality‑Guided LoRA for ASR fine‑tuning. Experiments across datasets demonstrate strong agreement with human expert judgments and validate the generalizability of MusicJudge.
Authors:Baiqi Li, Ce Zhang, Yu Fang, Yue Yang, Shangzhe Li, Mingyu Ding, Gedas Bertasius
Abstract:
A robot working alongside people must reason about what they have done, in what order, and with what intent. Video carries the spatial layouts, object histories, and gestures that language leaves underspecified, yet today's manipulation benchmarks pair an instruction with a single current image, offering no way to evaluate reasoning over observed human behavior. We introduce WatchAct, a benchmark for robot manipulation grounded in observed human behavior. Each instance pairs a real‑world human‑action video and a language instruction with an aligned simulator scene and an executable LIBERO task, enabling scalable and reproducible evaluation. WatchAct comprises 3,000 long‑horizon instances across 14 tasks in four capability domains drawn from the cognitive demands of watching another agent: parsing events (Event Grounding), recovering procedural structure (Procedural Reasoning), inferring unstated intent (Implicit Intent Inference), and tracking how the scene was changed (Episodic Reasoning). We further propose a disentangled evaluation protocol that separately measures (i)~video‑to‑plan reasoning by vision‑language models, (ii)~policy execution under oracle plans, and (iii)~full task completion by integrated planner‑‑policy pipelines. In both simulation and on a Franka Research 3 robot, current systems remain far from solving WatchAct. The best pipeline, Gemini‑3.1‑Pro with π_0.5, reaches only 16.3% Success Rate (SR) in simulation and 14.0% on the real robot. Gemini‑3.1‑Pro attains just 36.8% Plan SR (vs. 97.1% for humans), while π_0.5 reaches only 21.5% Task SR under oracle plans and drops to 10.6% on out‑of‑domain scenarios. Dataset and code are available at https://baiqi‑li.github.io/watchact_page/.
Authors:Tianle Zhu, Haohua Que, Handong Yao, Hongyi Xu, Zhipeng Bao
Abstract:
High‑precision remote perception is often hindered by the severe bandwidth constraints of Vehicle‑to‑Everything (V2X) networks. We propose DinoLink, a token‑centric compression framework that replaces raw pixel streaming with discrete semantic communication for vehicle‑cloud collaborative inference. DinoLink employs a dual‑sparsity architecture: a saliency‑aware selector prunes redundant background tokens, while a Residual Vector Quantization (RVQ) module collapses features into compact codebook indices. By transmitting only lightweight indices and positional priors, DinoLink achieves a 139× bitrate reduction compared to uncompressed transmission while maintaining a competitive 32.8% mAP on the nuScenes dataset. Deployment simulations further demonstrate a 34.5× acceleration in narrow‑band environments, such as LoRa. Our results substantiate DinoLink as a robust, bandwidth‑efficient frontend for high‑fidelity remote perception in constrained V2X scenarios. The code is publicly available at https://github.com/UGA‑MOBILITY‑LAB/dino_link.
Authors:John Viljoen, Johanna Haffner, Masayoshi Tomizuka, Negar Mehr
Abstract:
Many robotics problems, including trajectory optimization, inverse kinematics, and contact‑rich motion planning, reduce to nonlinear programs (NLPs). Mature NLP solvers such as IPOPT can solve these problems, offering hard constraint satisfaction, optimality guarantees, and favorable scaling with problem dimension. These solvers underpin gradient‑based methods in robotics, yet remain CPU‑bound and solve only one problem at a time, preventing their integration into GPU‑batched learning pipelines. On the other hand, sampling‑based approaches such as reinforcement learning, model predictive path integral, and imitation learning have become the core of modern robotics research due to their ability to leverage GPU‑batched simulators. These simulators can generate orders of magnitude more dynamics rollouts per second than was previously possible. If a GPU‑batched NLP solver existed, it would unlock similar speedups in the number of constrained, locally optimal solutions generated per second. This regime of solving many problems concurrently versus solving a single problem at a time is a key requirement for integrating NLP solvers in modern GPU‑batched robotics frameworks. To this end, we introduce \textttjaxipm, the first GPU‑batched NLP solver, based on IPOPT, and implemented in JAX. We accomplish this by redesigning IPOPT's algorithm to eliminate control flow with heterogeneous iteration fusion, and by minimizing GPU idle time with iteration level batching. We evaluate \textttjaxipm on a variety of quadrotor nonlinear model predictive control benchmarks, including reference tracking in the presence of obstacles, multi‑quadrotor navigation without collision, and navigation in a cluttered environment. We demonstrate up to a 32.85× increase in throughput over IPOPT. Our complete open‑source codebase is available at https://github.com/johnviljoen/jaxipm.
Authors:Marla Eisman, Brian Lam, Samuel Sonnino, Francesco Borrelli
Abstract:
This paper presents a hierarchical control framework using model predictive control (MPC) and reinforcement learning (RL) for active roll control to manage lateral load transfer during autonomous racing of a wheeled quadruped. The framework integrates offline time‑optimal raceline generation, an online MPC planner that actively minimizes the lateral Load Transfer Ratio (LTR), and a low‑level, whole‑body RL policy deployed directly onto the robot's 16 actuators. The MPC is based on a vehicle dynamics bicycle model of the Unitree Go2‑W platform. The robot's leg actuators act as active suspension where knee joints generate anti‑roll torque to bank into turns. Physical track experiments demonstrate that active roll control reduces mean LTR by up to 44%, improves the fastest lap time by 8.7%, and boosts peak lateral acceleration capability by 21.3% to 1.98 m/s^2, maintaining robust high‑speed stability beyond the range of a non‑tilting baseline controller. Supplementary code and video can be found at https://github.com/meisman‑ucb/go2w‑roll‑control‑mpc
Authors:Tianyuan Zhou, Zhizheng Fu, Tianming Yang
Abstract:
Rating systems such as Elo serve as the gold standard for matchmaking in competitive chess. However, they inherently suffer from response lag due to their exclusive reliance on match outcomes, neglecting the granular quality of gameplay. Nevertheless, incorporating move‑by‑move information into rating adjustments presents a significant challenge given the substantial noise and the vastness of the game‑state space. To address this, we propose the Drift‑Diffusion‑Enhanced Elo Rating System (DD‑Elo), a novel skill assessment framework inspired by the drift diffusion model (DDM) from cognitive neuroscience. By modeling skill expression as a decision‑making process, our model integrates move‑level data to capture rapid skill fluctuations. We provide a rigorous mathematical derivation proving that DD‑Elo maintains a bounded deviation from the traditional Elo system, ensuring theoretical alignment. Extensive experiments demonstrate that DD‑Elo adapts to skill changes faster than Elo. Our findings suggest that DD‑Elo offers an explainable, highly responsive, and backward‑compatible solution for chess rating ecosystems. The implementation code is publicly available at https://github.com/Aquila‑zhou1/DD‑Elo .
Authors:Shuyi Chen, Zhengchang Hua, Nikos Tziritas, Georgios Theodoropoulos
Abstract:
Terrestrial AI training faces an unsustainable energy and water crisis, positioning Orbital Data Centers (ODCs) as a "zero operational carbon" alternative. However, the sub‑10μ\texts communication latency required for synchronized scientific workloads, such as distributed Large Language Model (LLM) training, forces ODCs into extreme physical density, triggering a critical "Proximity‑Thermal Paradox." As these high‑density systems scale into Monolithic Structures or Proximity Swarms, they suffer from intense thermal‑fluid crosstalk (heat traps in shared cooling loops) and thermal‑radiative crosstalk (mutual heating that blocks deep‑space cooling radiators). If left unmitigated, this persistent heat stagnation not only triggers severe thermal throttling that degrades training throughput, but also induces severe thermal fatigue, drastically shortening hardware lifespans and generating premature space e‑waste. To make orbital AI truly sustainable, this position paper challenges traditional uniform load‑sharing. We propose the Thermal‑Aware Heterogeneity Thesis, which treats spatial cooling variances as a primary resource management dimension. Building on this, we introduce Thermal‑Load Balancing (TLB), a software framework that dynamically migrates these intensive workloads to the coolest available units based on instantaneous fluid temperatures or absorbed radiation. Our analysis demonstrates that TLB resolves thermal bottlenecks to restore Model Flops Utilization (MFU), while simultaneously reducing physical thermal stress. Extending the operational lifespan of orbital hardware is crucial to amortize the massive embodied carbon of rocket launches, outlining a necessary pathway to scale orbital AI without accelerating e‑waste.
Authors:Gaurang Katyal
Abstract:
Bot detection on social platforms has historically relied on a mix of account‑metadata features and features extracted from the text of posts and profile fields. The arrival of capable language models complicates the latter. A bot operator can run every post through GPT‑4 or Claude and produce text whose surface statistics are difficult to distinguish from those of human writing, which weakens the predictive value of content‑derived features. This paper asks how much of the detection problem can be solved by features that an attacker cannot easily manipulate at low cost: the age of the account, follower and friend counts and their ratios, profile completeness, and the structural properties of the handle. On a publicly redistributed corpus of 2,432 Twitter accounts with manually verified labels (43.0% bots), a random forest using only these account‑history features achieves ROC‑AUC of 0.977 in five‑fold cross‑validation, against 0.830 for a content‑only baseline and 0.981 for the fusion model. The behavioral‑versus‑content gap is large and statistically significant by DeLong's test (z = 9.36, p < 0.001). We then evaluate two adversarial settings. In the first, we rewrite the text of bot tweets to match human surface statistics for URLs, hashtags, mentions, and casing; the content classifier's ROC‑AUC degrades from 0.842 to 0.785 while the behavioral classifier is essentially unchanged. In the second, more aggressive setting we directly perturb the content feature values toward the human distribution; the content classifier falls below chance (AUC 0.466) while behavioral performance is invariant. We replicate the score distribution qualitatively on a 100‑account sample of TwiBot‑20. We conclude that operational bot detection should not treat content features as the primary signal; account‑history features carry most of the load already and are not eroded by adversarial text rewriting.
Authors:Tianyi Wu, Xiaoxi Sun, Yanhua Jiao, Yulin Li, Yixin Chen, YunHao Cao, YiQi Hu, Zhuotao Tian
Abstract:
Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms. However, their computational complexity scales on the order of L cubed with the sequence length L. This poses significant challenges for long‑sequence and real‑time applications, primarily due to the lack of compatibility with key‑value caching and the non‑autoregressive nature of denoising steps. Existing acceleration methods rely on static caching or parallel decoding strategies, which fail to account for the dynamic behavior of token properties across layers and decoding steps. We propose Dynamic‑dLLM, a training‑free framework that enhances dLLM inference efficiency through two components: Dynamic Cache Updating (DCU), which adaptively allocates cache‑update budgets based on layer‑wise token dynamics, and Adaptive Parallel Decoding (APD), which dynamically calibrates decoding thresholds to balance generation quality and efficiency. Extensive experiments on models like LLaDA‑8B‑Instruct, LLaDA‑1.5, and Dream‑v0‑7B‑Instruct across benchmarks such as MMLU, GSM8K, and HumanEval demonstrate that Dynamic‑dLLM significantly improves inference speed. It attains an average speedup exceeding 3 times while maintaining performance. Dynamic‑dLLM outperforms state‑of‑the‑art acceleration methods and provides a plug‑and‑play solution for efficient dLLM deployment without compromising performance. The code is available at https://github.com/TianyiWu233/DYNAMIC‑DLLM.
Authors:Derek Thomas
Abstract:
Large language models (LLMs) exhibit strong capabilities in short‑context reasoning but degrade in performance over long conversational horizons due to context window limitations and inefficient token usage. We introduce ContextForge, a system for context recycling that maintains task‑relevant information across turns by combining structured query generation, external memory retrieval, and controlled synthesis. The system enables efficient reuse of prior computation without relying on full context replay, reducing token overhead while preserving answer quality. We evaluate ContextForge using a 15‑turn conversational benchmark that tests multi‑turn reasoning, back‑references, and domain shifts across structured healthcare queries. Compared to a baseline agent using identical underlying models, ContextForge demonstrates improved consistency and reduced token consumption, while maintaining comparable response accuracy. These results suggest that context recycling provides a practical approach for extending LLM capabilities in long‑horizon tasks without requiring larger context windows or model retraining. Code and evaluation artifacts are available at https://github.com/Betanu701/ContextForge.
Authors:Renwei Meng, Bowen Zhang, Jian Wang, Xican Wang, Haoyi Wu, Xuanyan Qiu, Shengan Yang
Abstract:
Reliable evaluation of large language models should separate supported answering from unsupported guessing without conflating either with data contamination, prompt idiosyncrasy, or generic refusal behavior. We present a contamination‑aware, multi‑zone benchmark for measuring the transition from answerable knowledge to abstention‑expected unknowns under frozen build‑time labels. The benchmark contains 1,200 items across five domains, explicit abstention expectations, contamination‑risk metadata, and dual parsing with an official strict parser plus a normalized robustness parser. We evaluate FLAN‑T5, Qwen2.5‑Instruct, and Llama‑3‑Instruct models under locked answer‑or‑abstain prompts, answer‑only controls, and prompt‑template variants. The benchmark is not solved by generic non‑answer behavior: FLAN baselines remain weak on productive abstention, while stronger instruction‑tuned models expose a selective but incomplete transition from answering to abstaining. Qwen2.5‑3B‑Instruct achieves the best overall reliability, but answer‑expected zones remain difficult, calibration remains poor, and benign‑item refusal persists. Prompt and parser robustness analyses preserve the main ranking and qualitative conclusions. The benchmark therefore provides a reproducible protocol for auditing answerability, abstention, refusal, and contamination as distinct but interacting dimensions of LLM reliability.The dataset is publicly available at https://github.com/renweimeng/Know2Guess‑A‑Contamination‑Aware‑Multi‑Zone‑Benchmark.
Authors:Hao Sun, Hao Yan, Mengting Chen, Quanjian Song, Yu Li, Juan Cao, Jinsong Lan, Xiaoyong Zhu, Bo Zheng, Sheng Tang
Abstract:
While Video Virtual Try‑on (VVT) has achieved remarkable progress in synthesizing realistic garment overlays on dynamic subjects, existing paradigms remains fundamentally constrained by a passive dependency on source camera trajectories, failing to accommodate the requisite interactive freedom for omnidirectional viewpoint exploration. To address this limitation, we define a pioneering research frontier: Camera‑controllable Video Virtual Try‑on (CaM‑VVT). Unlike conventional VVT, CaM‑VVT not only necessitates viewpoint‑agnostic texture hallucination but also strict structural synchronization between non‑rigid human dynamics and background contexts under arbitrary, unconstrained camera movements. To tackle these challenges, we present TryOnCrafter, the first unified DiT‑based framework specifically architected for the CaM‑VVT task. Departing from implicit pixel‑space manipulation, we introduce a Renderable 4D Try‑on Proxy that explicitly decouples the human subject from the environment. This is achieved by distilling high‑fidelity 2D try‑on priors into a clothed 3DGS‑based avatar, which is subsequently animated via SMPL‑X sequences and metric‑aligned into a reconstructed background point cloud. This proxy establishes a robust structural foundation with superior texture density and motion integrity. Our Proxy‑Anchored Video DiT leverages this robust structural foundation as a primary geometric anchor, ensuring that the synthesized photorealistic videos are strictly constrained by prescribed trajectories and physically plausible deformations. Benefiting from the inherent editability of the 4D proxy, TryOnCrafter facilitates diverse downstream applications, including human relocalization, ``bullet time'' effects, and 360‑degree orbital viewing.
Authors:JoungBin Lee, Jaewoo Jung, Jongmin Lee, Tongmin Kim, Hyunsung Kim, Takuya Narihira, Kazumi Fukuda, Jahyeok Koo, Jisang Han, Yuki Mitsufuji, Seungryong Kim
Abstract:
Synthesizing a novel‑view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off‑the‑shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos. In contrast, camera‑conditioning‑only methods can achieve high visual quality but often struggle to preserve geometric and motion consistency. In this work, we introduce MVTrack4Gen (Multi‑View point Tracking for Novel‑View Generation), a motion‑aware training framework that leverages multi‑view point tracking as an additional geometric and motion supervision signal for camera‑conditioning‑only novel‑view video diffusion models. Our key finding is that specific attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding locations across views and over time, and the misalignment of these correspondences causes motion inconsistency. Based on this observation, we route these features into an auxiliary multi‑view tracking head and jointly train the diffusion model with a point‑tracking objective. By explicitly strengthening these motion‑aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintain cross‑view geometric consistency. Across diverse benchmarks, our method achieves state‑of‑the‑art geometric consistency and competitive camera accuracy.
Authors:Juliana Li, Diya Sreedhar
Abstract:
Midway through an ordinary pretraining run, a small language model learns the pronoun‑gender rule: cued with a girl's name ("Sue cried because"), it resolves the next pronoun to she, generalizing to held‑out probes (0.94 by step 925). By step 3,500 the same model scores near zero on the same probes, although the rule's evidence is still in the training data. We call this within‑run reversal natural ungrokking: the corpus decides, with no trace in the loss curve, which learned rules a model keeps. Which rules survive is predictable from one corpus statistic: how often the training stream shows the rule winning. Across un‑intervened runs (two corpora, three budgets, three seeds), support frequency decides a rule's fate; the data‑to‑parameter ratio only modulates how deeply a doomed rule falls. The same emerge‑then‑collapse dynamics appear in public Pythia checkpoints, collapse depth ordered by model scale as predicted. The forgetting is a displacement: a competing surface pattern out‑competes the rule, and the log‑probability margin between them crosses zero within 100 training steps of the behavioral collapse. Control over this fate is asymmetric: the same edit that destroys a rule on demand cannot restore it. Flipping support to counter‑evidence in place kills the rule with monotone dose‑response in two unrelated rules; but injecting support back, even to 450 times the level that naturally sustains it, buys no recovery. Every confirmatory threshold and prediction was pre‑registered before the data it governed was read.
Authors:Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao
Abstract:
Tool use enables large language models (LLMs) to perform complex tasks, and recent agentic reinforcement learning (RL) methods show promise for enhancing model capabilities. However, RL alone often leads to instability or limited gains in tool‑use tasks. In our experiments, some models exhibit catastrophic collapse, where performance abruptly drops and tool‑invocation structures fail. The analysis reveals that these failures stem from unexpected probability spikes in specific control tokens, disrupting structured execution, yet the underlying tool‑use capability remains intact, merely obscured by specific formats. To address this, we systematically investigate a diverse set of supervisory signals, including off‑policy supervision, hint‑based guidance, erroneous example supervision, and others, applied under both synchronous and interleaved training schemes. We find that interleaving supervised fine‑tuning (SFT) with RL substantially improves stability, but exhibits degraded performance under format and content out‑of‑distribution (OOD) evaluation. We also analyze the impact of learning rates and generalization across settings. These results highlight the importance of understanding RL failures and demonstrate how diverse supervisory signals can guide exploratory learning, enabling robust training of LLMs for complex, multi‑step tool‑use tasks. Our Code is available at https://github.com/hypasd‑art/Tool‑RL‑Box.
Authors:Yang Chen, Xiaowei Xu, Shuai Wang, Xinwen Zhang, Qiushi Guo, Tiezheng Ge, Limin Wang
Abstract:
Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low‑level pixel details, hindering the capture of high‑level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end‑to‑end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low‑frequency semantic manifold, while a specialized decoder handles high‑frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256×256 show that MIMFlow‑L reaches 71.3% linear probing accuracy and an FID of 2.50. Despite using only 128 tokens (50% fewer than standard models), it yields a 32.8% performance gain over similar‑scale NF baselines. Our code is available at https://github.com/MCG‑NJU/MIMFlow.
Authors:Maximilian Leitenstern, Marcel Weinmann, Patrick Haft, Tobias Lasser, Dominik Kulmer, Markus Lienkamp
Abstract:
Robust and accurate odometry estimation is essential in modern robotics. In environments characterized by highly dynamic motion and sensor noise, odometry estimation becomes increasingly challenging. Autonomous racing combines both factors in an unstructured setting, where minimizing odometry latency is essential for stable closed‑loop control. This paper introduces FAR‑LIO, a highly optimized CUDA‑accelerated LiDAR‑inertial odometry framework developed for Fast, Accurate, and Robust performance. Our system leverages a novel CUDA‑based voxel hashmap to enable parallelized nearest‑neighbor search and efficient map updates. We employ a sparsity‑aware Generalized Iterative Closest Point algorithm with adaptive thresholding on top of the CUDA‑based voxel hashmap with adaptive density to achieve low‑latency without compromising accuracy. An Extended Kalman Filter serves as a robust backend. It utilizes an upsampling and delay compensation strategy to fuse the LiDAR odometry with high‑frequency IMU data, thereby ensuring a robust and smooth odometry output. We evaluate FAR‑LIO across four different sensor setups, using both public datasets and data from two autonomous racecars driving at speeds of up to 250 km/h. FAR‑LIO achieves an average 6.9% reduction in the positional error and 38.4% lower runtime compared to state‑of‑the‑art baselines on target hardware using a single parameter set. This demonstrates its computational efficiency and broad applicability. To build upon our work, our code is available open‑source on https://github.com/TUMFTM/FAR‑LIO.
Authors:Long Cao, Zhongquan Wang, Jie Li, Yuhan Chen, Kefei Qian, Xiangfei Huang, Guofa Li
Abstract:
Image priors can synthesize target conditions for 3D Gaussian street scenes, but independently edited views do not define a coherent 3D target. Direct fitting can propagate view‑specific noise, while existing pipelines do not jointly handle imperfect sparse anchors and standard‑rasterizer deployment. To address this gap, teacher‑relative appearance residual distillation is introduced for appearance baking. A structured space for frequency decomposition, confidence estimation, and primitive‑level lifting is formed by residuals between teacher anchors and original renders. The direct optimization signal is supplied by renderer‑space matching, while primitive assignment is regularized by support‑aware Gaussian‑space aggregation. Supported detail is admitted and unsupported noise is suppressed through confidence‑gated coarse‑to‑fine optimization, after which all residuals are baked into fixed‑geometry spherical‑harmonic coefficients. The teacher and auxiliary training modules are discarded at inference. Evaluation across Waymo street assets, Tanks and Temples scenes, and multiple target conditions shows a favorable overall balance of target alignment, content preservation, artifact suppression, and cross‑view consistency over editing‑based baselines. Ablations confirm the effectiveness of the main components. Code will be released at https://github.com/Cagares/Baking‑for‑3D‑Gaussian.
Authors:Liang-Yuan Wu, Zih-Ching Chen, Tongshuang Wu, Chao-Han Huck Yang, Hua Shen
Abstract:
As multimodal conversational systems increasingly engage in spoken interaction, their ability to navigate paralinguistic social cues has become a critical bottleneck for natural human‑AI communication. However, existing evaluations of machine emotional intelligence assess reasoning exclusively through isolated text or passive acoustic perception, overlooking the complex cross‑modal reasoning required for active, multi‑turn dialogue. We introduce \textscSpeechEQ, a comprehensive framework designed to evaluate the sociolinguistic reasoning of Speech‑Language Models (SLMs). The framework includes a validated dataset of 2,265 dialogues across 15 Emotional Quotient (EQ) subscales grounded in EQ‑i 2.0 theory, along with a multi‑turn evaluation protocol measured by our proposed Spoken EQ (SEQ) score inspired by human EQ assessments. Experiments show limitations in how both existing Speech Emotion Recognition and end‑to‑end Speech‑Language Models understand and apply paralinguistic cues through speech. While end‑to‑end architectures outperform cascaded systems, \textscSpeechEQ reveals that current multimodal models remain bottlenecked by a text‑reliant ``modality shortcut,'' an alignment‑induced ``safety trap,'' and ``contextual amnesia,'' highlighting the barriers to truly emotionally aware AI. Our benchmark can be accessed at https://huggingface.co/datasets/SpeechEQ/SpeechEQ and demo page at https://binomial14.github.io/speecheq‑demo/
Authors:Alexandre Bouayad
Abstract:
Large language models (LLMs) attain remarkable surface fluency on code, yet they neither formally guarantee the syntactic validity of their output nor leverage the hierarchical structure defining the target language. While existing constrained‑decoding frameworks address the former, they operate under rigid assumptions that preclude critical lexical mechanisms ‑‑ including context‑sensitive lexing, maximal‑munch tokenization, and keyword extraction ‑‑ and only approximate vocabulary masking, sacrificing completeness. For the latter, code LLMs typically inject grammatical structure via predetermined policies rather than learning which structural information to expose. In this work, we introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations. First, we present a formal engine and constrained decoder that is sound and complete with respect to the full Tree‑sitter specification. By augmenting generalized LR (GLR) parsing with a speculative‑lexing construction that maintains concurrent lexer‑state hypotheses synchronized with a GLR graph‑structured stack, our decoder admits every subword token extending to a valid program prefix and rejects all others. Second, we present a latent‑variable fine‑tuning method training the language model to interleave non‑terminal grammar symbols directly into generation. Utilizing the reweighted wake‑sleep (RWS) algorithm to optimize the importance‑weighted evidence lower bound (IW‑ELBO) of the surface text, the model learns to selectively retain formal derivations as an adaptive structural scratchpad. For Python, fine‑tuning StarCoder2‑3B with our RWS objective reduces per‑token cross‑entropy by 14.3% relative to a text‑only SFT baseline, demonstrating that discretionary latent syntax recovers critical structural information that flat autoregressive training discards.
Authors:Giulio Turrisi, Ozan Pali, Luca Oneto, Claudio Semini
Abstract:
Legged robots deployed in planetary exploration and other remote environments must maintain reliable locomotion despite actuator failures and challenging terrain conditions. Although reinforcement learning has achieved strong results in legged locomotion, monolithic policies can struggle to efficiently represent the diverse control strategies required to compensate for different fault conditions. In this work, we propose a fault‑aware modular control architecture that explicitly leverages fault‑diagnosis information to activate specialized control experts associated with distinct actuator failure modes. Experimental results show that explicit fault‑conditioned modular policies consistently outperform monolithic policies of comparable size, achieving higher locomotion performance across failure scenarios. Moreover, the proposed modular architecture retains competitive performance even under significantly reduced network capacity, highlighting its suitability for compute‑constrained robotic platforms, such as those typically employed in space applications. The code associated with this work is available at: https://github.com/iit‑DLSLab/fault‑locomotion‑isaaclab.
Authors:Nathan Painchaud, Tristan Habémont, Morgane des Ligneris, Allan Serva, Pierre Croisille, Laurent Bertoletti, Thomas Lampert, Johannes F. Lutzeyer, Odyssée Merveille
Abstract:
Risk stratification for pulmonary embolism (PE) is critical for clinical decision‑making. Stratification guidelines are based on patient medical records, parameters measured from computed tomography pulmonary angiography (CTPA), and blood tests. However, blood tests are often missing in routine practice. This work studies whether state‑of‑the‑art models can accurately classify risk stratification from only medical records and biomarkers extracted from CTPA images. We benchmark different approaches to combine medical records and cardiac biomarkers with rich pulmonary vascular information; we add vascular biomarkers to tabular models and apply graph neural networks (GNNs) on the vascular tree's intrinsic graph representation. We use a private dataset (n=353) with uniquely complete data for PE risk stratification. Our results show that, among global features, medical records and cardiac biomarkers are the most significant predictors, while vascular biomarkers do not further improve stratification. Even more surprising, even GNNs on vascular graphs fail to outperform strong tabular baseline on global features. We consider hypotheses, on both models and data, that could explain this suboptimal performance. Our investigation suggests that, counter‑intuitively, vascular graphs might hold no discriminative information for PE risk stratification. Code is available from https://github.com/creatis‑myriad/GENESIS.
Authors:Pengwei Wang, José Morano, Virginia Mares, Hrvoje Bogunović
Abstract:
Color fundus photography (CFP) is the most common ophthalmic imaging modality for large‑scale screening. However, it is highly susceptible to degradations, making robust fundus image quality assessment (FIQA) crucial. The criteria for what constitutes high‑quality at the image level vary across clinical tasks, making FIQA dependent on expert knowledge. This motivated the development of automated methods and datasets. While existing datasets aim to standardize image‑level quality, their criteria often differ. Furthermore, image‑level labels preclude the quantitative evaluation of localized degradations, which is essential for trustworthy FIQA. We argue that pixel‑level FIQA based on anatomical visibility represents a more task‑agnostic, explainable approach. In this work, we introduce FunPiQ, the first FIQA benchmark to provide pixel‑level quality annotations. In addition, we propose EFIQA‑CP, an explainable‑by‑design (EBD) method that uses quality pseudo‑labels based on anatomical visibility to train a CNN via Non‑Negative Positive‑Unlabeled learning. Extensive evaluations of classification methods with post‑hoc explanations, anomaly detection methods, and EBD methods demonstrate the superior performance of the last and, particularly, of EFIQA‑CP.
Authors:Jiacheng Sui, Tianyu Hao, Bingjie Gao, Li Niu, Guangtao Zhai
Abstract:
Diffusion models have shown promise in drag‑style editing. Previous works mainly focus on point‑based drag, which is inherently ambiguous. This paper focuses on region‑based drag and introduces a novel In‑Context Region‑based Drag (ICRDrag) method. Under the in‑context learning framework, ICRDrag consumes a source image, a source region mask, and a target region mask, producing the target dragged image. Built upon the basic in‑context learning model, we introduce two novel attention regularization: 1) image‑mask attention consistency to ensure that a target region attends to similar source regions for image and mask modalities; 2) source‑target attention correspondence to ensure the mutual correspondence between source and target regions. To facilitate region‑based drag, we also construct Paired Region Dataset (PRD), a large‑scale dataset with paired masks and images. Extensive experiments show that ICRDrag significantly outperforms existing methods in both quantitative metrics and user studies, achieving superior editing accuracy and visual fidelity. The dataset, code, and model are available at https://github.com/bcmi/ICRDrag‑Region‑Drag‑Editing.
Authors:Yuchen Xie, Xinyu Zhou, Kuangji Zuo, Yanshuo Lu, Fengrui Huang, Boyu Ma, Jianfei Yang
Abstract:
Embodied Visual Tracking (EVT) requires an agent to continuously follow a specified target while actively moving through dynamic environments. However, prevailing EVT paradigms predominantly rely on language‑based target indication. While language is expressive and convenient, cluttered scenes often contain multiple objects that satisfy the same semantic description, leading to ambiguous target grounding. We therefore propose a paradigm shift, reframing target indication in EVT from text‑only specification to unified spatial‑semantic prompting. Based on this paradigm, we introduce Unified Spatial‑Semantic Prompts for Embodied Visual Tracking with Latent Dynamics Learning, USS, an end‑to‑end embodied tracking framework that supports text, point, bounding box, and mask prompts within a unified architecture. USS encodes heterogeneous prompts with modality‑specific encoders, fuses prompt tokens with visual features through hybrid attention, and decodes compact prompt‑conditioned representations into egocentric waypoints. To further improve temporal robustness, USS incorporates a latent world model that predicts future representations through self‑supervised alignment. Real‑robot experiments demonstrate that explicit spatial target cues yield higher success rates than text‑only prompts, particularly in scenarios involving similar distractors and longer‑horizon tracking where maintaining instance‑level target identity is critical. In the simulation benchmark, USS also achieves state‑of‑the‑art performance among non‑MLLM‑based methods and competitive results against recent MLLM‑based approaches with faster inference speed. Our findings reveal that spatial‑semantic prompting provides a more precise and flexible target indication interface for embodied visual tracking. Project site: https://arescheah.github.io/uss‑project‑page/.
Authors:Ke Zhao, Zixiang Di, Hong Qian, Xiang Shu, Yaolin Wen, Qitao Shi, Bingdong Li, Xingyu Lu, Xiangfeng Wang, Jun Zhou, Ke Tang, Yang Yu
Abstract:
Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization‑oriented large language models (LLMs). Existing approaches typically rely on large‑scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead. To address these challenges, we propose MiniOpt, a reinforcement learning framework that learns to solve optimization problems through an "reasoning‑to‑model‑and‑solve" paradigm. MiniOpt decomposes optimization reasoning into structured optimization modeling and executable solver generation. Building upon this paradigm, we introduce OptReward, a reward function with hierarchical score structure that jointly evaluates formulation and solution, enabling effective policy learning without expert demonstrations. We further develop an optimization‑oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models. Extensive experiments show that MiniOpt‑3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains. For models with fewer than 10B parameters, MiniOpt series achieves the highest average solving accuracy (SA). For models with more than 10B parameters, MiniOpt still shows competitive performance. These results suggest that optimization‑oriented reward design and reinforcement learning provide an effective pathway for developing compact optimization‑specialized language models with strong optimization generalization capabilities. The code is available at https://github.com/Hsiang‑1/MiniOpt.
Authors:Yang Tian, Zhengpeng Shi, Yu Zhou, Bo Zhao
Abstract:
Large language models are increasingly deployed as agents that solve tasks by interacting with external tool environments. Although recent tool‑use benchmarks increasingly cover complex task settings, they still largely assume clean, stable, and trustworthy tool environments, leaving tool‑environment unreliability insufficiently examined. We introduce ToolBench‑X, a benchmark for evaluating agents under recoverable reliability hazards. ToolBench‑X contains executable multi‑step tasks across diverse domains and sequential, parallel, and mixed workflows, each paired with deterministic tools and a canonical final answer for automatic evaluation. Starting from clean tool environments, ToolBench‑X injects five structured hazard types: Specification Drift, Invocation Error, Execution Failure, Output Drift, and Cross‑source Conflict. Crucially, each injected instance remains solvable through at least one valid recovery path, such as retrying, fallback, verification, or cross‑checking. Experiments reveal a substantial reliability gap: agents that perform well with reliable tools often fail under recoverable hazards. Further analysis shows that failures are driven less by tool‑use volume or inference budget than by limited hazard diagnosis and ineffective recovery. Targeted recovery hints recover many failed tasks, while test‑time scaling yields more limited gains. These results suggest that tool‑use evaluation should move beyond function‑call accuracy toward task completion under unreliable tool environments. The code and data is available at https://github.com/Foreverskyou/ToolBench‑X.
Authors:Khawar Islam, Arif Mahmood, Xin Jin, Naveed Akhtar
Abstract:
Data augmentation is known to improve generalization of deep visual models. Recent methods favor mixup strategies that generate interpolated samples to improve model performance. However, these techniques not only incur significant computational overhead, they also lead to semantic disruption of augmentation data due to cross‑sample mixing. We first propose Self‑Saliency (S^2) Mixup, which constructs challenging yet label‑consistent samples by extracting multi‑scale salient patches and reinserting them into non‑salient regions of the same image. This promotes scale‑invariant feature learning while avoiding cross‑sample interference. To further enhance model robustness, we introduce FracMix, a mixing scheme that injects self‑similarity patterns into salient regions using adaptive ratios. Collectively, our unified framework, S^2‑FracMix, enables simultaneous learning from fractal and non‑fractal structures within a single image, yielding a targeted and structurally coherent augmentation strategy. We theoretically analyze the advantage of our technique, and empirically establish its superiority over the existing methods by achieving state‑of‑the‑art performance in extensive evaluation with seven benchmarks across classification (coarse and fine‑grained), robustness, calibration, object detection, and transfer learning tasks. Project page is available at \hrefhttps://fracmix‑data‑augmentation.github.io/fracmix‑data‑augmentation.github.io
Authors:Muhammed Furkan Dasdelen, Fatih Ozlugedik, Anastasia Litinetskaya, Nassir Navab, Carsten Marr, Ario Sadafi
Abstract:
Data scarcity is a major bottleneck in medical Multiple Instance Learning (MIL), especially for rare diseases or expensive modalities. We introduce a statistically grounded patient augmentation approach that generates realistic patients directly in embedding space. Using Gaussian Mixture Models as a probabilistic clustering approach on pooled instance embeddings from all patients, our method learns disease‑specific "recipes"‑statistical distributions of instances across unsupervised clusters. New patients are then generated by sampling embeddings from clusters based on learned recipes. Unlike existing methods that require examples from all categories, our method can generate patients offline by re‑mixing pooled embeddings. Generated patients are further selected based on uncertainty quantification to improve MIL performance. We evaluate our method across three clinically relevant scarcity scenarios: (i) cross‑dataset transfer, where an entirely missing "healthy" class is generated using statistics from an external cohort; (ii) low‑data regimes, where class sizes are extremely limited; and (iii) small‑cohort non‑image tasks, including single‑cell RNA‑seq and flow cytometry. Across all experiments, our method improves performance over baseline, often outperforming other bag‑mixing strategies. Notably, in the missing‑class scenario, a performance comparable to full‑dataset training is achieved, demonstrating its potential for rare disease diagnostic and privacy‑preserving patient augmentation. The code is available at https://github.com/marrlab/RECIPE
Authors:Xincheng Tang, Youhan Xie, Zhengjie Shu, Wanyu Li, Lai Jiang, Wenkang Hu, Yitong Li, Ruigang Yang
Abstract:
Climbing hollow stairs remains a challenging problem for quadruped robots due to the high risk of leg trapping, severe depth sparsity, and high‑frequency depth‑sensing noise. In this paper, we propose StairMaster, a novel three‑stage reinforcement learning framework for stable locomotion on such extreme discontinuous terrains. Our architecture integrates a Cross‑Attention mechanism to extract structural features from noisy depth data, alongside a Spatial‑aware Recurrent Unit (SRU) that maintains robust spatio‑temporal memory to mitigate perception blind spots. To bridge the sim‑to‑real gap in depth perception, we propose a high‑fidelity sim‑to‑real depth sensor modeling pipeline that faithfully replicates real‑world sensor artifacts. Additionally, we employ a 3D waypoint‑guided active perception reward for proactive sensing, alongside hollow gap kinematic and stair edge penalties to ensure precise foothold placement. We successfully deployed StairMaster on a Unitree Go2 robot, demonstrating its ability to conquer hollow stairs with an unprecedented incline of up to 55^\circ through zero‑shot transfer. To the best of our knowledge, this is the first RL‑based policy to achieve such steep hollow stair climbing in real‑world environments. Project Website: https://sivan666666.github.io/StairMaster/.
Authors:Jiayu Li, Yixiao Fang, Tianyu Hu, Wei Cheng, Ping Huang, Zheheng Fan, Gang Yu, Xingjun Ma
Abstract:
Real‑world photography requires capture‑time guidance for both camera framing and subject pose. Yet existing aesthetic cropping benchmarks mainly evaluate post‑hoc crop prediction and overlook subject‑side recommendations, leaving the capture‑time guidance capabilities of multimodal large language models (MLLMs) underexplored. To address this gap, we introduce CaptureGuide‑Bench, a benchmark with two complementary tasks: photographer‑side composition decision and refinement, and subject‑side scene‑conditioned pose recommendation. Our evaluation reveals limitations: general‑purpose MLLMs can make composition decisions but lack precise refinement localization, while specialized aesthetic cropping models localize crops effectively but are limited to refinement; neither provides actionable pose guidance. To support model development, we further construct CaptureGuide‑Dataset, comprising 130K samples with textual rationales and structured visual annotations, and develop ShutterMuse, a unified MLLM trained with supervised and reinforcement fine‑tuning. Experiments on CaptureGuide‑Bench show that ShutterMuse achieves the best overall photographer‑side performance among evaluated baselines and competitive subject‑side pose recommendation with substantially lower inference cost, demonstrating the potential of MLLMs as interactive assistants for photography during image capture.
Authors:Wenjie Zhu, Yabin Zhang, Liang Xu, Xin Jin, Wenjun Zeng, Lei Zhang
Abstract:
While test‑time adaptation (TTA) empowers vision‑language models to adapt without costly retraining, it remains highly vulnerable to out‑of‑distribution (OOD) outliers prevalent in real‑world applications. This discrepancy motivates Noisy TTA (NTTA), an online task to filter noisy OOD samples on the fly while maximizing in‑distribution (ID) classification accuracy. Existing zero‑shot NTTA approaches typically rely on test‑time discriminative training, leading to overconfident misclassifications and significantly degraded inference efficiency. To address these limitations, we propose a novel framework named Dual Distribution Estimation (DDE), shifting the zero‑shot NTTA paradigm from instance‑level learning to training‑free Gaussian distribution modeling. DDE incorporates two novel modules: Positive Feature Distribution Estimation (PFDE) and Negative Label Distribution Estimation (NLDE). PFDE explicitly models class‑wise inclusion and exclusion Gaussian distributions to formulate a calibrated contrastive score, robustly enhancing ID accuracy. In parallel, NLDE improves OOD identification by explicitly modeling the negative label distribution to mine highly discriminative labels, effectively mitigating spurious correlations. Extensive experiments show that on the large‑scale ImageNet benchmark, DDE achieves an improvement of 3.70% in harmonic mean accuracy and reduces the FPR95 for OOD detection by 6.20%, while ensuring highly scalable and efficient online inference. Furthermore, DDE is zero‑shot and training‑free, demonstrating remarkable robustness in data‑scarce scenarios. Codes are available at https://github.com/ZhuWenjie98/DDE.
Authors:Wenxuan Jiang, Zining Fan, Zijian Zhang, Xuecheng Wu, Hongming Tan, Haoyang Dai, Xiaoyu Li, Xuezhi Cao, Ninghao Liu
Abstract:
Reinforcement Learning (RL) has enabled LLMs to excel in objective reasoning tasks such as mathematics and code generation. However, applying RL to open‑ended tasks, such as creative writing, remains challenging because LLM‑as‑a‑judge reward models often exhibit stylistic biases and positional inconsistencies, leading to unstable supervision. To address this, we propose OPERA (Objective Perplexity‑based Reflective Alignment), which replaces unreliable external judges with intrinsic rewards derived from perplexity dynamics. Specifically, we derive an intrinsic reward signal from perplexity dynamics, quantifying uncertainty reduction at critical reflective states. During the cold‑start phase, we introduce a data synthesis method that leverages carefully designed guiding words to generate diverse reasoning traces, along with perplexity‑prioritized rollouts that utilize internal log‑probabilities to identify logically consistent reasoning branches. This pipeline yields a large‑scale dataset comprising 20,000 high‑quality reasoning trajectories. Empirical evaluations consistently demonstrate the scalability and efficacy of our approach in alignment for open‑ended tasks. Implementing OPERA on Qwen3‑8B establishes a new state‑of‑the‑art among open‑source models, achieving parity with or surpassing proprietary models like Gemini2.5 and MiniMax‑M2.5 in some open‑ended tasks. The code is available at https://github.com/pangpang‑xuan/OPERA.
Authors:Yifei Qu, Ru Li, Junjie Chen, Jinyuan Wu
Abstract:
Real‑world single image dehazing is highly ill‑posed due to spatially and spectrally varying scattering, while practical deployment demands lightweight and low‑latency models. Existing approaches either rely on fragile physical inversion under simplified assumptions or adopt heavy blind architectures unsuitable for edge deployment. To overcome these limitations, we propose PGL‑Net (Physics‑Inspired Global‑Local Decoupling Network), a lightweight framework that incorporates physical inductive biases via operator‑level emulation, avoiding explicit parameter estimation. It decouples dehazing into global distribution rectification and local structural refinement. A Physics‑Inspired Affine Fusion (PAF) module performs globally conditioned alignment across hierarchical skip connections to compensate for haze‑induced bias, while a compact Degradation‑Aware Modulation (DAM) block adaptively restores spatially and spectrally variant details through dynamic feature modulation. Extensive experiments on multiple real‑world benchmarks demonstrate that PGL‑Net achieves state‑of‑the‑art restoration quality with significantly reduced complexity. Compared with the recent SOTA SGDN, the Tiny variant (PGL‑Net‑T) improves PSNR by up to 2.6dB and consistently enhances downstream object detection accuracy, while achieving over a 10x reduction in inference latency. Code is publicly available at: https://github.com/sc‑30‑bit/PGL‑Net.
Authors:Mithil Ramteke
Abstract:
We extend the cone‑ray exact‑NMF pipeline of Ramteke (arXiv:2606.22451) from the uniform‑support regime r_+ = r to the gap regime r_+ > r, and classify recoverable nonnegative factorisations by the rank of the W‑factor into a three‑regime taxonomy. Regime A (rank(W) = r_+, full column rank): a two‑sided SVD‑gauge cone‑ray pipeline W = U_r+(G) Q, H = P V_r+(K)^T with G, K on Stiefel manifolds and square consistency Q P = diag(S_r, 0). On 10x10 dense random gap matrices it gives 100/100 recovery at r_+ = 5 and 6. We explain this via two geometric facts: slack enclosure (the data cone has codimension r_+ ‑ r in the outer cone) and NRF‑variety thickness (valid gauges form a positive‑measure set, so the blind SVD lands on one with probability one). Regime B (rank(W) = r, W a column subset of M): a rank‑deficient‑W branch enumerating r_+‑subsets of M's columns with per‑column LP tests. On the block‑diagonal family diag(C, J_k), where additivity of nonnegative rank collapses the valid gauges to a single point and the blind SVD pipeline fails, the column‑subset branch restores recovery in milliseconds. Regime C (r < rank(W) < r_+, W not a column subset): exposed by the regular octagon's slack matrix. An exact size‑6 NRF exists and is reachable by the symmetric formulation at an oracle gauge derived from a known factorisation (residual 1.5e‑10), but the blind problem is open: 50 Haar random restarts and Riemannian gradient descent on the Stiefel/Grassmann gauge all stall, because the alt‑LP residual is piecewise constant on cells of gauge‑space, so local descent cannot cross cell walls. A combined toolkit (Regime B then A) covers regimes A and B with no regression on dense draws; Regime C remains open, with the regular octagon as the cleanest unsolved test case.
Authors:Baiyang Song, Yuli Lin, Qiong Wu, Tao Chen, Jun Peng, Xiao Chen, Yiyi Zhou, Rongrong Ji
Abstract:
Currently, streaming video understanding is still a daunting task for existing \emphmultimodal large language models (MLLMs). Its difficulties not only lie in handling the ever‑increasing video frames, but also in the unpredictability of future video content and input instructions. In this paper, we study this task from the perspective of constructing a dynamic but fixed‑budget memory bank, and propose a novel and training‑free approach termed \emphCausalMem. CausalMem is dedicated to constructing a dynamic visual memory update mechanism, thereby maximizing the amount of information in streaming video within a limited memory space, much like the human brain. In practice, CausalMem estimates the redundancy of visual tokens and updates the memory bank via an online semantic basis, which models the principal semantics of the observed video stream. To validate CausalMem, we apply it to two representative MLLMs, namely LLaVA‑OneVision and Qwen2.5‑VL respectively, and conduct extensive experiments on both streaming and offline video understanding benchmarks. The experimental results not only show the great advantages than existing methods under both streaming and offline settings, \emphe.g., +3.2% and +3.0% average accuracy gains respectively, but also witness the superior semantic preservation for streaming videos, \emphe.g., using 12k token budgets to memorize hour‑long streaming videos, which achieves more than 20× visual token compression ratio and only occupies about 82 MB storage. Our code is given in \hrefhttps://github.com/hktk07/CausalMemCausalMem.
Authors:Congbo Ma, Hu Wang, Yichun Zhang, Farah E. Shamout
Abstract:
As Large Language Models (LLMs) are increasingly deployed in healthcare settings, accurate error detection and correction in generated or existing text becomes critical, as even minor mistakes can pose risks to patient safety. Existing methods for error detection and correction, including automated checks and heuristic‑based approaches, do not generalize well across unseen datasets. In this paper, we propose MedGuards as a medical safety guardrail, which is a new framework that treats medical error detection and correction as a multi‑agent in‑context learning task. Specialized agents separately detect, localize, and correct errors, while a confidence‑guided arbitration mechanism resolves disagreements using reasoning traces and confidence scores. This design enhances interpretability, robustness, and adaptability, without requiring additional training of the base LLMs. Additionally, we introduce the Keyword‑Prioritized Correction Score (KPCS), a new evaluation metric that considers whether critical keywords within the reference text are generated correctly, providing a more comprehensive assessment than conventional metrics. Experiments across four multilingual medical datasets consisting of clinical notes demonstrate significant improvements by the proposed framework across several metrics and models. Our aim is to enable safer deployment of LLMs in real‑world healthcare applications. For reproducibility, we make our code publicly available at https://github.com/congboma/MedGuards.
Authors:Tianchen Guo, Chen Liu, Ling Chen, Xin Yu
Abstract:
Multimodal Large Language Models (MLLMs) have shown remarkable progress in single‑image perception, yet their ability to reason about complex cross‑view human‑centric scenes remains largely unverified. Current multi‑view benchmarks evaluate models using a fixed "bag of frames" and thus conflate a model's robustness to visual distraction with its genuine ability to fuse fragmented cross‑view evidence. To address this issue, we introduce SSMNBench, a diagnostic benchmark comprising 3,300 curated QA pairs for cross‑view human and human‑object understanding. SSMNBench uniquely categorizes tasks into Single‑View Sufficiency (SVS) and Multi‑View Necessity (MVN). By systematically perturbing view availability across 17 state‑of‑the‑art MLLMs, critical limitations are revealed: models suffer from severe "distraction degradation" when presented with redundant views (SVS), and fail to integrate fragmented geometric evidence across cameras (MVN). Our evaluations demonstrate that modern MLLMs rely on multiple single‑image semantic averaging and view preference rather than genuine cross‑view synthesis. By exposing these fundamental vulnerabilities, SSMNBench provides a rigorous diagnostic framework to drive the advancement of future cross‑view‑aware multimodal architectures. The code is available at: \hrefhttps://github.com/gtc‑gh/SSMNBench\textSSMNBench
Authors:Felipe Moreno, Sharifa Alghowinem, Hae Won Park, Cynthia Breazeal
Abstract:
Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention. As a multidisciplinary topic, these objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. Even though current automated depression diagnosis approaches improved over the last decade, a critical gap exists as they often lack affect‑specificity and interpretability, limiting their practical application and potential impact on mental health care. In particular, interpretability from temporal activities from videos when deep models are used is not fully explored. In this study, we present a novel framework for analyzing Deep Neural Networks' decisions when trained on facial videos, specifically focusing on automatic depression severity diagnosis. By fine‑tuning Deep Convolutional Neural Networks (DCNN) pre‑trained on Action Recognition datasets on depression severity facial videos from AVEC depression dataset, our framework is able to interpret the model's saliency maps by examining face regions and temporal expression semantics. Our approach generates both visual and quantitative explanations for the model's decisions, providing greater insight into its reasoning. In addition to this interpretability, our video‑based modeling has improved upon previous single‑face benchmarks for visual depression diagnosis, resulting in enhanced predictive performance. Overall, our work demonstrates the successful development of a framework capable of generating hypotheses from a facial model's decisions while simultaneously improving depression's predictive capabilities.
Authors:Fangzheng Li, Aimin Zhang, Chen Lv
Abstract:
Tool Calling and Structured Output are two core capabilities of modern Agent systems, yet their interaction under joint deployment conditions remains insufficiently understood. This paper reports a reproducible phenomenon observed in a production Agent system: when Tool Calling and JSON Schema constraints are simultaneously enabled, multiple open‑weight models cease invoking tools despite maintaining high schema compliance. We refer to this behavior as Tool Suppression. Through controlled experiments across multiple model families and deployment settings, we consistently reproduce Tool Suppression under joint constraints, while tool execution and schema compliance remain functional when evaluated independently. Further analysis reveals that JSON Schema constraints are compiled into grammar‑based token masks, causing tool‑call tokens to become unreachable during decoding. This provides an implementation‑level explanation for the observed behavior. To interpret the phenomenon, we formulate the Constraint Priority Inversion (CPI) hypothesis, which suggests that schema satisfaction may dominate action‑selection behavior under multiple simultaneous constraints. We present CPI as a behavioral hypothesis consistent with the observed evidence rather than a verified internal mechanism. To mitigate the problem, we propose Transparent Two‑Pass Execution, an inference‑time strategy that decouples tool execution from schema‑constrained response generation. Experimental results show that this approach restores tool invocation while preserving structured output guarantees without requiring model retraining. These findings suggest that evaluating tool use and structured output separately may overlook important reliability issues in production Agent systems. Code, data, and docs will be released at https://github.com/Fzsama/Constrain‑Tax‑26‑06.git.
Authors:Seulgi Jeong, Yunseong Cho, Sanghun Park
Abstract:
Hairstyle transfer has practical applications such as virtual try‑on, yet remains challenging when the source and reference exhibit large head‑pose discrepancies. We propose H‑Adapter, which improves pose robustness by training with a region‑specific loss that disentangles hair and non‑hair objectives and thereby induces spatially disentangled cross‑attention, from which a source‑aligned hair edit mask is derived to guide diffusion‑based inpainting. Experiments on pose‑agnostic and pose‑different subsets demonstrate strong quantitative results, including the best FID, \mathrmFID_\mathrmCLIP, and CLIP‑I under pose differences, while maintaining competitive non‑hair preservation and improving qualitative fidelity to fine‑grained reference hairstyle details. Beyond source‑conditioned transfer, H‑Adapter supports practical extensions including text‑to‑image generation, auxiliary prompt‑based hair color control, and compatibility with an identity‑preserving IP‑Adapter variant. We also introduce a VLM‑as‑a‑judge protocol and observe consistent gains in hairstyle faithfulness, non‑hair preservation, and artifact quality.
Authors:Hanyang Wang, Weijieying Ren, Yuxiang Zhang, Ding Cao, Zhizhao Zeng, Ke Zeng, Tianxiang Zhao
Abstract:
Stepwise group‑based RL is an attractive way to train long‑horizon LLM agents without a learned critic: it reuses multiple sampled rollouts to estimate local advantages. Its weakness is less visible but more fundamental: every group‑relative estimator assumes that the steps it compares are equivalent for credit assignment. We show that current agentic variants violate this assumption through a state‑action credit mismatch. The observation‑hash partition is overly fine on the state side, creating singleton groups with zero step‑level signal, while a single within‑group mean is too coarse on the action side, mixing state‑value estimation with action‑specific credit. We introduce BiPACE (Bisimulation‑Guided Policy Optimization with Action Counterfactual Estimation), a drop‑in advantage estimator that fixes both sides without adding a critic, auxiliary loss, or extra rollouts. BiGPO clusters steps by cosine distance in the actor's own hidden‑state geometry, an empirical policy‑induced proxy for bisimulation that substantially lowers the singleton rate left by observation hashing. PACE then recenters returns within each behavioral cluster using action‑conditioned peer baselines; its Q‑style instance estimates a local Q(s,a)‑V(s) nonparametrically. On ALFWorld/Qwen2.5‑7B, BiPACE_Q raises overall validation success from GiGPO's 90.8 to 97.1\pm0.9 over three seeds, and crosses the 95% threshold on every seed, which GiGPO never does within the same budget. On Qwen2.5‑1.5B it reaches 93.5\pm1.2 versus GiGPO's 86.7, and on WebShop and TextCraft it improves over GRPO and GiGPO at both model scales. The measured BiPACE‑specific overhead is 11.3% of a single training‑step wall time. Yet it changes the estimator's comparison unit from surface identity to approximate behavioral equivalence plus action‑side counterfactuals. The code is available at https://github.com/TianxiangZhao/BiPACE.
Authors:Sitong Cheng, Weizhen Bian, Songjun Cao, Jin Li, Bei Liu, Chunyang Jiang, Yike Zhang, Weihao Wu, Yiming Li, Chi-Min Chan, Long Ma, Wei Xue
Abstract:
Speech‑to‑speech translation (S2ST) should preserve not only lexical meaning, but also expressive attributes: emotion, scenario style (e.g., news reporting vs. dramatic dialogue), and nonverbal vocalizations (NVs). Moreover, collecting cross‑lingual target speech that is both translation‑faithful and expressively aligned with the source is difficult at scale, making reference‑based evaluation impractical. We introduce STEB (Speech‑to‑Speech Translation Expressiveness Benchmark), a 32.6‑hour Chinese‑‑English benchmark that evaluates both standard dimensions (translation fidelity, speaker similarity, duration alignment) and expressiveness dimensions (emotion, scenario style, NV preservation). For expressiveness evaluation, STEB uses a caption‑then‑summarize framework that converts speech into structured expressive attributes and compares source and hypothesis attributes with an LLM judge. Human validation shows statistically significant correlations with listener judgments across all expressive dimensions. We evaluate six S2ST systems covering cascaded systems, end‑to‑end models, and speech large language models. Many systems, especially cascaded ones, achieve strong translation fidelity, but they still struggle with emotion preservation (best: 3.82/5) and NV preservation (best: 2.31/5). These results reveal a gap between semantic transfer and expressive transfer, identifying expressiveness preservation as an open challenge for S2ST. Audio samples are available at https://cmots.github.io/steb.github.io/.
Authors:Bang Giang Le, Viet Cuong Ta
Abstract:
Cooperative multi‑agent reinforcement learning assumes each agent shares the same reward function and can be trained effectively using the Trust Region framework of single‑agent. Instead of relying on other agents' actions, the independent actors setting considers each agent to act based only on its local information, thus having more flexible applications. However, in the sequential update framework, it is required to re‑estimate the joint advantage function after each individual agent's policy step. Despite the practical success of importance sampling, the updated advantage function suffers from exponentially high variance problems, which likely result in unstable convergence. In this work, we first analyze the high variance advantage both empirically and theoretically. To overcome this limitation, we introduce a clipping objective to control the upper bounds of the advantage fluctuation in sequential updates. With the proposed objective, we provide a monotonic bound with sub‑linear convergence to ε‑Nash Equilibria. We further derive two new practical algorithms using our clipping objective. The experiment results on three popular multi‑agent reinforcement learning benchmarks show that our proposed method outperforms the tested baselines in most environments. By carefully analyzing different training settings, our proposed method is highlighted with both stable convergence properties and the desired low advantage variance estimation. For reproducibility purposes, our source code is publicly available at https://github.com/giangbang/Low‑Variance‑Trust‑Region‑MARL.
Authors:Sining Ang, Yuan Chen, Liu Haiyan, Xuanyao Mao, Jason Bao, Xuliang, Bingchuan Sun, Yan Wang
Abstract:
Large language models (LLMs) can improve autonomous driving planning but are costly to query online, and existing fast‑slow planners often rely on hand‑designed triggering rules that either over‑call the slow system or call it at the wrong times. We formulate slow‑system invocation as a resource‑aware sequential decision problem and propose the Adaptive Slow‑System Control Gate (ASSCG), which makes frame‑level Query/Cache/Drop decisions to refresh, reuse, or suppress slow guidance. ASSCG uses an RWKV backbone for efficient long‑horizon gating and is trained with supervised fine‑tuning followed by GRPO‑style compute‑aware reinforcement fine‑tuning. We apply ASSCG to two different fast‑slow architectures: (i) AsyncDriver on nuPlan Hard20 closed‑loop evaluation, where ASSCG improves score to 67.28 (+2.28) while reducing average end‑to‑end inference latency by 60%; and (ii) a RecogDrive‑based dual system that we build by replacing its original VLM‑2B module with a lightweight ViT‑based fast planner and adding an LLM slow planner, evaluated on NAVSIM, where ASSCG achieves 91.4 PDMS (+0.6) and increases average speed by 25%. The project page, including video visualizations and additional results, is available at https://williamxuanyu.github.io/asscg/.
Authors:Hualong Zhang, Siyang Feng, Zihan Huan, Yi Qian, Zhenbing Liu, Rushi Lan, Xipeng Pan
Abstract:
Histopathological tissue segmentation is essential for computer‑aided diagnosis, yet weakly supervised methods often suffer from noisy pseudo‑labels generated by Class Activation Mapping (CAM). Existing CAM approaches tend to focus on staining‑driven appearance cues rather than true causal tissue morphology, resulting in spurious localization and poor structural consistency. To address this issue, we propose C^2RM‑Seg, a two‑stage framework that integrates causal pseudo‑label refinement with structure‑aware semantic enhancement. For classification, we introduce a Causal Counterfactual Reasoning Module (C^2RM) that decomposes features into latent factors and performs counterfactual intervention via a learned causal structure matrix, suppressing confounding context and producing morphology‑aligned CAMs. For segmentation, we design a Dual‑Path Structural‑Semantic Architecture that combines fine‑grained structural features from ResNeSt with global semantic priors from a frozen DINOV3 foundation model. A cross‑path gating mechanism adaptively regulates semantic injection using local structural cues to preserve boundary fidelity. To further mitigate residual pseudo‑label noise, we propose an Uncertainty‑Gated Margin (UGM) loss, which dynamically balances margin enforcement and confidence learning based on prediction uncertainty. Extensive experiments on two public histopathological tissue datasets show that C^2RM‑Seg achieves state‑of‑the‑art performance.
Authors:Shaocheng Jin, Tao Zhou, Rui Wang, Ziheng Chen, Xiaoqing Luo, Xiaojun Wu, Josef Kittler
Abstract:
Brain‑Computer Interface (BCI) based on electroencephalography (EEG) enables direct interaction between the brain and external environments and has significant applications in assistive technologies, medical rehabilitation, and entertainment. Recently, EEG decoding methods based on Symmetric Positive Definite (SPD) learning have demonstrated superior performance. However, these methods typically employ basic network architectures and do not explicitly capture local relationships between EEG signals. This limitation is problematic for EEG signals due to their inherently low Signal‑to‑Noise Ratio (SNR). Moreover, most existing Riemannian manifold‑based methods are restricted to specific metrics. The most widely used is the Affine‑Invariant Metric (AIM). However, it has a quadratic dependency on the SPD matrices and cannot handle ill‑conditioned SPD matrices, which hinders the effectiveness of networks. In contrast, the Bures‑Wasserstein Metric (BWM) exhibits linear dependence on SPD matrices and demonstrates superior performance for ill conditioning. To overcome these challenges, we propose a Riemannian self‑attention network based on the BWM. Additionally, the recently introduced power‑deformed generalized Bures‑Wasserstein metric reveals a nonlinear relationship between SPD matrices and matrix power deformation. This metric provides a more nuanced representation of the geometric structure of the SPD manifold. Consequently, we extend our model to a learnable version. For simplicity, we refer to it as GBWAtt. Experimental results on three EEG benchmarking datasets validate the robustness and effectiveness of our proposed method. The code is available at https://github.com/jissc/GBWAtt.
Authors:Chang Wu, Junfeng Fang, Houcheng Jiang, Kai Tang, Pengyu Cheng, Xiaoxi Jiang, Guanjun Jiang, Xiang Wang
Abstract:
Safety alignment of large language models (LLMs) typically depends on high‑quality supervision data, such as safe demonstrations or preference pairs. However, in real‑world deployment, emerging safety requirements are often specified as natural‑language policies, while corresponding supervision data may be costly, delayed, or unavailable. This creates a mismatch between rapidly evolving safety policies and conventional data‑driven alignment methods. To address this, we propose PolicyAlign, a simple yet effective framework for directly aligning LLMs with safety policies. Given a safety policy, PolicyAlign first synthesizes policy‑violating instructions and then performs on‑policy self‑distillation to internalize policy‑guided behavior. To improve training stability and data efficiency, we further introduce Policy‑Sensitive Filtering, which selects instructions where the policy induces the largest behavioral shift. Experiments across multiple models show that PolicyAlign consistently improves safety while maintaining low over‑refusal and preserving general capabilities. PolicyAlign also generalizes to medical, legal, and financial safety scenarios, highlighting its potential as a scalable and maintainable approach to policy‑based LLM safety alignment. The code is released at https://github.com/Qwen‑Applications/PolicyAlign.
Authors:Francesc Wilhelmi, Sergio Barrachina-Muñoz, Boris Bellalta
Abstract:
The upcoming IEEE 802.11bn amendment marks a paradigm shift in Wi‑Fi, which will pose ambitious performance targets under the paradigm of Ultra‑High Reliability (UHR). To understand the implications of such a new technology and to support early research and protocol design for Wi‑Fi~8, we present \textttKom8ndor. This discrete‑event network simulator extends the open‑source Komondor platform (a simulator validated against ns‑3 and other analytical tools) with 802.11bn features. Among the newly added functionalities, we highlight Multi‑Access Point Coordination (MAPC) ‑‑ including Coordinated Time‑Division Multiple Access (Co‑TDMA), Coordinated Spatial Reuse (Co‑SR), and Coordinated Beamforming (Co‑BF) ‑‑ , Non‑Primary Channel Access (NPCA), and Dynamic Subband Operation (DSO). Beyond Wi‑Fi~8 implementations, \textttKom8ndor introduces novel functionalities (e.g., a machine learning wrapper for building AI‑based protocols) and a modular design to boost the prototyping and research of future Wi‑Fi technologies. \textttKom8ndor is open‑source (GNU GPLv3) and available at https://github.com/wn‑upf/Komondor.
Authors:Deyvik Bhan
Abstract:
On Apple Silicon the fp32 GEMMs dominating LLM prefill are dispatched by Accelerate to a matrix coprocessor (AMX) on the M1‑M3. We ask where a hand‑written kernel's throughput over Accelerate comes from on the M1 AMX, and reach a structural conclusion: not a faster inner loop. By microbenchmark the inner loop is load‑issue bound ‑‑ once any operand load interleaves with the FMA32 stream, single‑thread throughput falls to a 610‑to‑680 GFLOPS band, under half the load‑free rate. The gain comes from two deployment‑level levers Accelerate underuses: fine multi‑thread panels filling the M1's second on‑chip AMX block (winning the K >= N shapes), and pre‑packing the constant weight at load (winning the N > K shapes). A bit‑exact direct‑AMX kernel using both is the fastest bit‑exact fp32 GEMM path we find on the M1: it exceeds all three Accelerate fp32 paths (cblas_sgemm, BNNSMatMul, and the BNNS Graph compiler) at all twelve LLM prefill GEMMs at S = 128 (GPT‑2 to Llama‑7B scale), leading the fastest, BNNS Graph, by 1.17 ‑‑ and by 1.09 at the three shapes where it too holds fp32 ‑‑ with geometric means of 1.58 over BNNSMatMul and about 2.0x over cblas_sgemm. Every output is bit‑identical to Accelerate, whereas BNNS Graph is bit‑exact at only three of twelve shapes, the rest at reduced precision (error up to 1.4e‑3). Dropped into llama.cpp for its cblas_sgemm prefill matmuls, it raises measured full‑forward throughput from 291 to 420 tokens/s (1.44x, bit‑identical) at 128‑token prefill ‑‑ end‑to‑end, not a GEMM‑only ratio. The contribution is this shape‑resolved M1‑AMX characterization (microbenchmark, two‑block aggregate, per‑core occupancy probe), leaving fine‑panel scheduling and pre‑packing as the only two levers above an inner loop at the hardware limit; mis‑tuning the single column‑panel width costs nearly 2x.
Authors:Fenghe Guo, Runjie Shen, Chenyang Sun, Junrui Zhang, Quanxi Zhan, Yongchun Wang, Junjie Zhang
Abstract:
Hydropower tunnel inspection is critical for infrastructure integrity yet remains inefficient and hazardous using manual methods. We propose FLISP (Fast LiDAR‑IMU Synchronized Path Planner), a mapless planning framework for cooperative UGV‑UAV inspection. Unlike traditional map‑based paradigms, FLISP features three core contributions: (1) a unified architecture where a single UGV‑mounted LiDAR‑IMU suite drives synchronized path generation for both platforms; (2) platform‑specific solvers utilizing an enhanced Firefly Algorithm for UGV obstacle avoidance and a dynamic iterative optimizer for UAV flight; and (3) a hierarchical refinement strategy ensuring kinematic feasibility without state estimation drift. Benchmarks in a 1.2 km operational tunnel demonstrate that FLISP circumvents structural bottlenecks of map‑based methods, eliminating map rasterization overhead (Fast‑LIO2 + A) and sampling instability (LIO‑SAM + RRT). FLISP achieves a 100% success rate with 7 ms latency, representing a 7‑fold speedup over grid‑based and a three‑order‑of‑magnitude improvement over sampling‑based baselines. Validated in operational hydropower tunnels, this approach offers a scalable solution for robotic inspection in feature‑degraded linear infrastructure. A demonstration video is available at https://youtu.be/Y_ezs1PfLJ4, and the code at https://github.com/ArchibaldGuo/FLISP.git.
Authors:Salman Shaik, Truong Thanh Hung Nguyen, Hung Cao
Abstract:
Accurate classification of diffuse gliomas is often hindered by domain shifts across centers and a lack of large, annotated datasets. We propose the Anatomically‑conditioned Latent Diffusion Model (ALDM), a novel framework for data‑efficient, few‑shot 3D volumetric MRI synthesis. ALDM utilizes a two‑stage approach: a 3D variational autoencoder learns anatomical priors from a data‑rich source domain, while a conditional latent diffusion model, guided by tumor masks via a ControlNet, generates structurally coherent volumes for a data‑scarce target domain. Evaluated in an extreme few‑shot setting with only 16 target images, ALDM outperformed GAN and hybrid baselines, achieving a superior Frechet Inception Distance (FID) of 85.40 and a downstream classification AUC of 0.987. Qualitative results confirm that the model preserves sharp pathology boundaries and cross‑modal consistency, with visual fidelity improving progressively during training. By capturing essential diagnostic features, ALDM provides a robust tool for clinical data augmentation in low‑resource settings. Our implementation is available at https://github.com/Analytics‑Everywhere‑Lab/anatomically‑conditioned‑LDM.
Authors:Hogyun Kim, Jiwon Choi, Juwon Kim, Geonmo Yang, Seokhwan Jeong, Hyungtae Lim, Younggun Cho
Abstract:
By maintaining global consistency across robot teams, multi‑robot LiDAR map merging enables faster exploration and efficient area coverage. However, map merging requires exchanging massive sensor data between the server and robots, making communication the bottleneck, especially in communication‑constrained environments. Therefore, we present Commerge, a communication‑efficient map merging framework that achieves bandwidth reduction through graph‑theoretic selective data exchange. By doing so, our Commerge reduces inter‑robot communication by up to 5,000x while maintaining alignment accuracy. Our key insight is that only a small subset of carefully selected scans is sufficient for robust map merging. We formulate this as a three‑stage cascaded optimization problem on an exchange graph, where vertices represent robot keyframes and edges denote candidate inter‑robot loops. Through three cascade stages, we select a sequentially overlapped, balanced‑transmission‑cost, and geometrically‑perceptually optimal scan subset that preserves alignment quality while reducing communication. Unlike existing approaches that either transmit whole scans, which require GB‑scale data exchange, or employ naive downsampling, our approach exchanges only MB‑scale data while achieving comparable alignment accuracy. Extensive evaluation on five public datasets and four in‑house datasets covering cave, planetary‑analog, indoor, and outdoor campus environments shows up to 99.98% reduction in data exchange (e.g., from 7,000MB to 1.3MB on the HeLiPR dataset), while maintaining alignment performance across embedded to desktop platforms. The supplementary materials are available at https://sparolab.github.io/research/commerge.
Authors:Joy Bose
Abstract:
We present a computational stylometric analysis of the Tipitaka across all three Pitakas in English translation, extending earlier work on the Sutta Pitaka alone. The corpus spans 134,831 segments from Bhikkhu Sujato's Sutta Pitaka (114,591 segments, CC0), Bhikkhu Brahmali's Vinaya Pitaka (7,923 segments, CC0 2026), I.B. Horner's 1938 Vinaya translation (2,826 segments), three English translations of the Abhidhammattha Sangaha compendium (2,077 segments), and cross‑tradition Vinaya texts from the Dharmaguptaka and Mulasarvastivada schools. We compute Zipf rank‑frequency distributions with OLS‑fitted exponents, Moving Average TTR (MATTR‑500), numeral‑word density, and vocabulary overlap (Jaccard and Szymkiewicz‑Simpson coefficients). Main findings: (1) all corpora show Zipf‑consistent distributions (R2 > 0.989); the Vinaya is closest to ideal Zipf slope ‑1 and the Sangaha corpus deviates most, with 'consciousness' displacing grammatical particles at rank 8; (2) MATTR‑500 shows the Sutta and Vinaya Theravada are nearly identical in lexical diversity (0.399 and 0.400), while the Sangaha corpus is genuinely more diverse (0.560), confirmed by size‑controlled subsampling; (3) the Sangaha corpus has the highest numeral‑word density (3.26%), consistent with its systematic enumeration of mental and material categories; (4) the Mulasarvastivada Vinaya shares 20.0% vocabulary (Jaccard) and 49.1% (overlap coefficient) with the Theravada Vinaya, reflecting shared legal heritage across two millennia; (5) two English translations of the same Vinaya source text share only 24.2% of their vocabulary across 88 years, with 'musing' versus 'absorption' for jhana and 'defeat' versus 'expulsion' for parajika as the most diagnostic shifts. All results are point estimates; no significance testing is conducted. Code and data are released as open‑source extensions to the Darshana Graph corpus (arXiv:2606.18222).
Authors:Lianbo Liu, Shiao Zhu, Kai Washizaki, Reo Yoneyama, Haesung Jeon, Mengjie Zhao, Yusuke Fujita, Hao Shi, Nao Yoshida, Yuan Gao, Roman Koshkin, Yukiya Hono, Yui Sudo
Abstract:
While large language model (LLM)‑based text‑to‑speech (TTS) systems have achieved high‑quality speech synthesis, most existing systems focus on English and Chinese. Japanese, however, remains under‑explored, and its unique linguistic challenges, such as widespread context‑dependent kanji polyphony, have yet to be adequately tackled. Here we introduce Sarashina2.2‑TTS (https://github.com/sbintuitions/sarashina2.2‑tts), a Japanese‑centric LLM‑TTS system that tackles these challenges through a dual approach: data strategy and evaluation methodology. First, we scale training to approximately 361k hours of speech, incorporating a balanced mix of Japanese and English data. Furthermore, we design a targeted data augmentation pipeline covering all 2,136 Joyo (regular‑use) kanji designated by Japan's Agency for Cultural Affairs to efficiently address kanji polyphony disambiguation. Second, we introduce the Joyo Kanji Yomi Benchmark (https://github.com/sbintuitions/JoyoKanji‑Yomi‑Benchmark), covering all 2,136 Joyo kanji and their 4,378 readings. Alongside this benchmark, we propose Kana‑CER, a metric that compares synthesized speech against reference readings in the kana space, eliminating orthographic variations to directly measure pronunciation correctness. Experiments demonstrate that our targeted data augmentation significantly improves reading accuracy. Overall, Sarashina2.2‑TTS achieves state‑of‑the‑art kanji‑level reading accuracy and matches top baselines on general sentence‑level pronunciation, while delivering the highest speaker similarity in zero‑shot Japanese speech synthesis. Furthermore, cross‑lingual evaluation reveals that Sarashina2.2‑TTS is the only system that maintains stable Japanese pronunciation regardless of the prompt language, confirming that our balanced training approach improves cross‑lingual robustness.
Authors:Hongye Xu, Bartosz Krawczyk
Abstract:
Exemplar‑free class‑incremental learning (EFCIL) requires stable decision boundaries within a shifting feature space. While maintaining class‑conditional Gaussian statistics provides a principled classification strategy, these parametric summaries remain sensitive to anisotropic representation drift. Existing methods often transport these statistics across tasks using a decoupled, post‑hoc paradigm: optimizing a backbone without explicit geometric constraints can distort the legacy manifold, limiting the precision of retroactive alignment. In this paper, we formulate feature transport as an endogenous training constraint rather than a separate post‑task step, presenting the Geometry‑Anchored Transport Framework. First, we derive an Analytic Geometric Anchor via Mahalanobis‑aligned regression to mitigate macroscopic anisotropic drift. Second, we introduce a Topology‑Aware Evolution objective that regularizes localized manifold degradation while calibrating a residual network against the analytic prior. By coupling manifold evolution with transport constraints during the primary training phase, our framework mitigates evaluation errors without requiring decoupled fine‑tuning. Experiments across CIFAR‑100, TinyImageNet, and ImageNet‑100 demonstrate that the proposed framework consistently improves upon existing post‑hoc alternatives under strict exemplar‑free constraints.
Authors:Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, Ji-Rong Wen
Abstract:
Modern large language models are predominantly trained with autoregressive factorization and causal attention. We present \emphiLLaDA, an 8B masked diffusion language model trained from scratch with fully bidirectional attention. iLLaDA keeps the masked diffusion objective throughout pre‑training and supervised fine‑tuning (SFT), scaling pre‑training to 12T tokens and fine‑tuning on a 25B‑token instruction corpus for 12 epochs. We further use variable‑length generation for efficiency and introduce confidence‑based scoring for multiple‑choice evaluation. Compared with LLaDA, iLLaDA improves broadly across general, mathematical, and code benchmarks; for example, iLLaDA‑Base improves by 21.6 points on BBH and 14.9 points on ARC‑Challenge, while iLLaDA‑Instruct improves by 14.5 points on MATH and 16.5 points on HumanEval. Despite its non‑autoregressive training, iLLaDA also remains competitive with Qwen2.5 7B on several benchmarks. These results show that fully bidirectional diffusion training from scratch is a competitive path toward strong language models. Model weights and codes: https://github.com/ML‑GSAI/LLaDA.
Authors:Qinzhe Yang, Chenyang Liu, Jia Xu, Zhenwei Shi, Zhengxia Zou
Abstract:
State Space Models (SSMs), designed for long‑range modeling, offer linear computational complexity and strong capabilities in capturing long‑range dependencies. In the field of remote sensing, SSMs have gained popularity due to their effectiveness in addressing unique challenges such as dense visual predictions, multi‑modal remote sensing data, and temporal remote sensing data, which have also yielded significant advancements in customized architectures. This paper presents a comprehensive review of SSM‑based approaches in remote sensing, covering most of the relevant studies since SSMs were first introduced to the field. We offer a multi‑dimensional analysis examining SSM applications in remote sensing tasks and discussing advancements in architecture design. This paper not only synthesizes the rapid progress in SSM‑based research but also identifies key challenges and future opportunities. By providing a detailed perspective, this paper aims to serve as a foundational resource for remote sensing researchers, offering actionable insights to foster further advancements in this evolving domain. We will keep tracing related works at https://github.com/QinzheYang/Awesome‑RS‑State‑Space‑Model.
Authors:Qinzhe Yang, Keyan Chen, Jia Xu, Zhenwei Shi, Zhengxia Zou
Abstract:
The computational complexity of Transformers scales quadratically with the number of tokens, which significantly constrains the efficiency of vision models, particularly recent ViT‑based foundation models in dense prediction tasks. Instance segmentation, a typical dense visual prediction task in the remote sensing field, faces similar challenges. In this paper, inspired by the recent advances of knowledge distillation in large language models, we introduce RS4D ‑ a new remote sensing instance segmentation method with linear computational complexity, which addresses the inefficiency of long sequence modeling through distilled state space modeling (SSM). We propose an adaptive noise and masking knowledge distillation training method for pre‑training lightweight SSM backbones, which effectively compresses knowledge from the vast self‑attention space into a compact, dense linear state space. We also design a remote sensing image instance segmentation architecture based on this lightweight visual encoder, where we explore variants of three different backbones and two segmentation heads. Extensive experiments are conducted on multiple benchmark datasets, including SSDD, WHU, and NWPU. Compared to ViT‑based approaches, our proposed SSM backbone achieves an 8x reduction in parameters and a 9x reduction in FLOPs while maintaining comparable or superior accuracy to both ViT‑ and CNN‑based instance segmentation methods. The implementation codes have been publicly available at https://github.com/QinzheYang/RS4D.
Authors:Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv, Tao Wang
Abstract:
Fine‑grained visual reasoning requires multimodal large language models (MLLMs) to identify task‑relevant visual evidence and ground their reasoning in local image regions. Existing agentic methods typically rely on reinforcement learning with verifiable rewards or supervised fine‑tuning on large‑scale annotated reasoning traces, leading to costly exploration, hand‑designed verification rules, or heavy dependence on textual supervision. A natural way to avoid such external answer labels is to learn from trajectories sampled by the student itself, which points to On‑Policy Distillation (OPD). To understand what OPD can and cannot provide for visual reasoning, we revisit it as negative‑free stop‑gradient alignment. This perspective shows that, although OPD provides effective token‑level correction, its ceiling is constrained by the absence of trajectory‑level discrimination. Motivated by these observations, we propose V‑Zero, an answer‑label‑free framework for visual reasoning with contrastive evidence gating. V‑Zero uses no annotated textual answer labels; instead, during training it pairs a question‑relevant regional crop with a negative visual view to evaluate student‑sampled trajectories and gate dense token‑level distillation. Experiments on multiple visual reasoning benchmarks show that V‑Zero consistently improves fine‑grained visual reasoning while preserving strong generalization. Notably, V‑Zero is more than 5× faster than previous supervised fine‑tuning methods and more than 10× faster than reinforcement learning baselines. Code and dataset will be released at https://github.com/eVI‑group‑SCU/V‑Zero
Authors:Qinzhe Yang, Dongyu Wang, Haohan Niu, Jia Xu, Zhenwei Shi, Zhengxia Zou
Abstract:
Remote sensing object detection has advanced rapidly with the development of large‑scale benchmarks and modern detection architectures. However, existing datasets and detectors remain fragmented. Most benchmarks focus on limited categories, fixed spatial resolutions, or a single sensor, while detectors still struggle to work across different sensors and categorical systems. In this paper, we introduce LEVIRDet‑159, the largest and most comprehensive remote sensing object detection dataset to date, with 159 categories, 2.56 million bounding boxes, and 700k fine‑grained annotations under a multi‑level taxonomy. In each key scale dimension, LEVIRDet‑159 exceeds the corresponding largest existing remote sensing object detection dataset, containing approximately (7x) more images, (6x) more object instances, and (4x) more categories. Based on this dataset, we design LEVIRDetNet, a scale‑hierarchy‑aware detection foundation model for universal remote sensing object detection. LEVIRDetNet couples online visual Ground Sampling Distance (GSD) prediction, GSD‑conditioned query modulation and allocation, and a hierarchy‑aware detection head for mixed‑granularity remote sensing supervision. Under stringent evaluation settings, LEVIRDetNet demonstrates strong cross‑domain generalization. Even without target‑domain training or fine‑tuning, it achieves state‑of‑the‑art detection performance on 9 external benchmarks, improving the strongest fully supervised competing methods by 5.02 mAP on average under each benchmark's primary metric. We hope this study will facilitate the development of strongly generalizable remote sensing object detection across diverse category systems, spatial resolutions, and sensor platforms. The dataset and trained models will be released at https://qinzheyang.github.io/LEVIRDet/, accompanying the final paper.
Authors:Atin Pothiraj, Jaemin Cho, Yue Zhang, Elias Stengel-Eskin, Mohit Bansal
Abstract:
Video generation models are increasingly capable of producing realistic videos, but they still struggle to generate videos that follow basic physical laws. Compounding this is a lack of reliable granular evaluation methods for localizing and specifying physical law violations in videos. We address this by introducing Physics Question Scene Graph (PQSG), a hierarchical question‑based evaluation pipeline. PQSG evaluates generated videos by checking their faithfulness to a prompt across objects, actions, and adherence to physical laws using a graph‑based hierarchy of questions generated by a vision‑language model (VLM), guided by high‑quality in‑context examples. By representing questions as a graph, PQSG introduces logical dependencies within questions, ensuring that each query is contextually valid. Moreover, PQSG provides granular assessments of which qualities of the video violate physical plausibility constraints. We validate PQSG by creating FinePhyEval, a dataset with physics‑based prompts and corresponding generated videos from diverse state‑of‑the‑art video generation models (Sora 2, Veo 3, and Wan 2.1), with each video annotated across multiple categories by humans. Using FinePhyEval, we measure the correlation between PQSG's fine‑grained scores and human judgments, showing higher overall correlations than prior work. We also find that PQSG ranks closed‑source models higher than Wan 2.1 on physical realism. Lastly, we show that the annotations we provide in FinePhyEval can also be used for subtask evaluation: we benchmark two strong VLMs on generating and answering questions, finding that while models can create human‑like questions, they still fall short of human performance in answering them.
Authors:Hoai-Danh Vo, Trung-Nghia Le
Abstract:
Generative models have significantly advanced image generation, resulting in synthesized images that are increasingly indistinguishable from authentic ones. However, the creation of fake images with malicious intent is a growing concern. Low‑configured smart devices have become highly popular, making it easier for deceptive images to reach users. Consequently, the demand for effective detection methods is increasingly urgent. In this paper, we introduce a simple yet efficient method that captures pixel fluctuations between neighboring pixels by calculating the gradient, which highlights variations in grayscale intensity. This approach functions as a high‑pass filter, emphasizing key features for accurate image distinction while minimizing color influence. Our experiments on multiple datasets demonstrate that our method achieves accuracy levels comparable to state‑of‑the‑art techniques while requiring minimal computational resources. Therefore, it is suitable for deployment on low‑end devices such as smartphones. The code is available at https://github.com/vohoaidanh/adof.
Authors:Ke Xu, Jiaqi Wan, Wenhao Hu, Han Pu, Xiaoyun Wang
Abstract:
Post‑Training Sparsity (PTS) has emerged as a crucial paradigm for compressing Large Language Models to facilitate efficient deployment on resource‑constrained devices. However, existing PTS methodologies are typically confined to Single‑Sparsity optimization, necessitating a separate, time‑consuming optimization session for each specific sparsity level. This rigid paradigm significantly hinders flexible deployment across diverse hardware scenarios, as adapting to a new sparsity requirement mandates a complete re‑optimization process. To address these limitations, we propose Elastic Post‑Training Sparsity (EPTS), a unified Multi‑Sparsity framework that produces a single elastic model capable of maintaining robust performance across diverse sparsity configurations through a one‑shot optimization process. Specifically, we design a Multi‑Sparsity Hierarchy LoRA (MS‑HiLoRA) mechanism that facilitates knowledge inheritance from low‑ to high‑sparsity groups, effectively mitigating the competition for parameter reconstruction. Furthermore, we introduce a Multi‑Sparsity Feature Mixer (MSFM), which significantly enhances the model's adaptability to pruning perturbations by dynamically fusing feature representations of varying sparsity granularities. Extensive experiments on LLaMA and OPT families demonstrate that EPTS achieves competitive performance compared to state‑of‑the‑art methods like SparseGPT and Wanda, while offering significant efficiency gains by enabling multi‑scenario deployment from a single optimization. our source code is available at https://github.com/xuke225/EPTS.
Authors:Ke Xu, Xinle Wang, Yanning Hou, Xueliang Ma, Juan Xie, Jianfeng Qiu
Abstract:
Zero‑shot 3D anomaly detection is essential for industrial quality inspection, where labeled anomaly samples are scarce. Meanwhile, existing methods lack an effective mechanism to fuse complementary 2D color images with 3D geometric structures, limiting their ability to detect both surface and structural defects in a unified framework. To address these issues, we propose CoGeoAD, a unified CLIP‑based framework that fuses color and geometric features by constructing pixel‑aligned paired multi‑view images. The framework introduces a Data‑Driven Multi‑View Attention (MVA) mechanism to adaptively aggregate 3D features and a Multi‑Stage Color‑Geometric Fusion (MS‑CGF) module to hierarchically integrate multi‑level features from both modalities. Extensive experiments on the MVTec3D‑AD and Eyecandies benchmarks demonstrate that CoGeoAD achieves state‑of‑the‑art performance, effectively capturing both structural and textural anomalies in complex industrial scenarios. our source code is available at https://github.com/kingdomShu/CoGeoAD.
Authors:Qing Lian, Kent Yu, Lei Zhang
Abstract:
Most vision‑language‑action (VLA) models are reactive: they predict the next action from the current instruction and observation, implicitly assuming that the current observation fully specifies the action‑relevant state. In embodied control, however, embodiment‑specific factors such as camera‑to‑robot geometry, robot calibration, or systematic actuation bias are often hard to identify from a single observation. As a result, reactive policies cannot reliably disambiguate these factors in general, overfitting to training environments and generalizing poorly at deployment. We propose Reflective VLA, which conditions each decision on a context of observation‑action‑consequence triplets. Each triplet records not only what the robot observed and executed, but also how the scene changed afterward, exposing the deployment‑specific mapping from actions to observed effects. Architecturally, Reflective VLA routes all observation modalities through the VLM under shared attention, so the action expert reasons directly over past triplets and the current observation. A block‑causal mask enables parallel multi‑frame training without leakage and supports KV‑cached real‑time inference. On standard LIBERO and SimplerEnv‑Bridge, Reflective VLA preserves strong in‑distribution performance. Under distribution shift on LIBERO‑Plus and the harder LIBERO‑Plus‑Hard, it improves average success rate by 5.4 and 4.2 percentage points over a matched reactive baseline. Ablations with a matched history‑only baseline further show that action consequences ‑‑ rather than additional context length alone ‑‑ are the key to cross‑environment generalization. Project page: https://lianqing11.github.io/reflective‑vla‑page/
Authors:Jungseob Lee, Chanjun Park, Heuiseok Lim
Abstract:
Multi‑agent document assessment for retrieval‑augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training‑free interventions on 7B‑9B instruction‑tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per‑document isolation. Astoundingly, assessment‑free isolation matches full multi‑agent assessment, demonstrating that resolving multi‑document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce Reasoning‑Score Coupling, a label‑free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model‑adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero‑shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.
Authors:Yusheng Zheng, Tianyuan Wu, Quanzhi Fu, Tong Yu, Wenan Mao, Tao Ma, Dan Williams, Wei Wang, Andi Quinn
Abstract:
AI agents increasingly run in production through harnesses, the software around the LLM, including an engine that enforces safety and effectiveness policies, e.g., 'run tests before committing.' Enforcing these policies requires bridging a semantic gap: policy intent is expressed in underspecified natural language, while enforcement must act on concrete system actions, e.g., which test to run. Many policies also define event ordering or data flow actions. Yet existing approaches fall short. Tool‑call guardrails miss system actions that bypass the tool layer, while OS sandboxes control resource access instead of actions, returning opaque errors that confuse the agent. Our key insight is that policy context lives within the agent closest to the task, while enforcement must happen at the OS to cover all execution paths. We introduce ActPlane, a policy engine that lets agents declare policies and enforces them in the OS kernel with semantic feedback and isolation. ActPlane uses a simple information‑flow control (IFC) DSL to support cross‑event policies. We implement ActPlane with eBPF and evaluate it on policies from the empirical study, coding‑task benchmarks, and safety benchmarks. ActPlane improves policy compliance, including on indirect execution paths that tool‑call interception cannot observe, with 1.9%‑8.4% overhead. ActPlane is at https://github.com/eunomia‑bpf/ActPlane
Authors:Yongjin Yang, Jiarui Liu, Yinghui He, Lechen Zhang, Bernhard Schölkopf, Zhijing Jin
Abstract:
Reinforcement learning with verifiable rewards (RLVR) has been extended from single‑domain training to multi‑domain reasoning suites spanning mathematics, programming, and science. However, the training curriculum (how often each domain is sampled) is typically fixed or hand‑tuned, even though reasoning skills transfer unevenly across domains. Existing learnability‑based curricula adapt to where the policy is currently improving, but are blind to whether a gradient step on the selected domain benefits the remaining domains. In this paper, we propose Transfer‑Aware Curriculum (TAC), a bandit‑style online curriculum that prioritizes domains whose updates broadly benefit the rest of the training suite. TAC repurposes signals already produced by RL training: per‑domain advantages capture local learnability, and projected gradients, taken from the GRPO step being computed, estimate cross‑domain transferability via gradient‑geometry alignment, at negligible cost (<1% wall‑clock overhead). Across a six‑domain reasoning suite, TAC achieves the best macro‑averaged accuracy on both Qwen3‑1.7B and Llama3.2‑3B, outperforming proportional random sampling, a hand‑designed schedule, and a learnability‑only bandit, and improving over the last of these by up to 2.8 points (10% relative). Ablations show performance degrades sharply when the transferability term is removed, and TAC remains robust on imbalanced training mixtures where learnability‑only curricula over‑commit to dominant domains. Our findings establish cross‑domain transferability as a key signal for curriculum design in multi‑domain RLVR.
Authors:Habibullah Akbar
Abstract:
Modern large language models based on softmax scaled‑dot‑product attention are constrained by their training sequence length: as the key‑value sequence grows, softmax probability mass can dilute across a wider distribution, inducing activation shift and long‑context performance collapse. Moreover, long‑context language modeling faces a structural tension: a sliding‑window attention core maintains a bounded local representation and low perplexity but is blind to long‑range dependencies, while full‑context attention preserves global recall but suffers from out‑of‑distribution perplexity explosion. To resolve these limitations, we introduce ATMA, a hybrid convolutional‑attention architecture that integrates a novel three‑channel attention mechanism. ATMA factorizes the attention mixing step into: (1) a count‑blind, unit‑vector direction channel, (2) a bounded magnitude channel driven by the participation ratio of effective matches over an extreme‑value‑corrected null sink, and (3) a long‑term recurrent compression memory optimized via a gated‑delta fast‑weights rule. Neither the Polar Attention core nor the recurrent memory is sufficient alone; their combination enables monotonic perplexity reduction and high‑fidelity long‑range retrieval simultaneously. We evaluate ATMA using a 120‑run factorial ablation sweep, demonstrating that the combined Polar + memory model maintains induction needle‑in‑a‑haystack retrieval accuracy above 90% out to 64K tokens (32 times the training length of 2K) while its document perplexity improves monotonically, outperforming softmax‑based memory baselines which collapse at extreme context lengths. Code: https://github.com/kreasof‑ai/atma
Authors:Kevin Ren, Manish Raghavan, Nikhil Garg
Abstract:
Deployed approaches for AI text detection often rely on training‑time access to labeled datasets of both human‑written and AI‑generated text. This approach is vulnerable to three types of distribution shifts that occur continually post‑deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and temporal drift in human writing. Simultaneously, existing approaches do not leverage a key signal of LLM usage: inference‑time homogeneity. We propose a test‑time adaptation (TTA) approach, using semi‑supervised learning, that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time. Empirically, we find that state‑of‑the‑art supervised detectors systematically fail when they encounter distribution shifts in AI‑generated and human writing, both adversarial and natural, while test‑time adaptation with semi‑supervised learning is largely robust; e.g., the commercial model Pangram detects just 24.1% of our adversarial AI‑generated text, compared to 90.5% for our test‑time approach. We establish that test‑time adaptation is a promising framework for AI text detection in the wild. We publicly release our code (which includes code for model training, evaluation, and plots) at https://github.com/kkr36/llm_detection.
Authors:Yong Yi Bay, Kathleen A. Yearick
Abstract:
Every Cartesian three‑dimensional PDE solver hides a structural secret that production CFD codes have used for half a century and that graduate‑level textbooks rarely state plainly. The derivative matrices, the compact Padé line solves, the Galerkin mass inversions, the alternating‑direction‑implicit substeps, and even the fast Poisson and Helmholtz diagonalization transforms all factor along the coordinate axes and collapse into repeated one‑dimensional banded kernels executed along the grid lines. The three‑dimensional operator exists only on paper; it is never assembled, factored, or stored. This paper is the manual for that collapse. We derive the Kronecker‑product algebra that makes it exact, carry it cleanly through central differences, compact schemes, tensor‑product Galerkin, B‑spline and isogeometric methods, collocation, ADI time stepping, and direct Poisson and Helmholtz solves, and bring into the open the three production tricks that turn the reduction into hardware‑conscious floating‑point throughput on real machines: the multi‑right‑hand‑side reshape that exposes a sweep as one batched line kernel (a dense BLAS‑3 GEMM when the line factor is dense or element‑local, a banded or stencil kernel when it is not), the sum factorization that rescues high‑order Galerkin from the O(p^2d) quadrature trap, and the pencil decomposition that keeps every direction contiguous across an MPI cluster. For fixed stencil width or fixed polynomial degree, the compute cost stays O(N) in the total number of unknowns N = N_x N_y N_z; the operator storage drops to O(N_x + N_y + N_z) up to bandwidth constants; direct separable Poisson and Helmholtz solvers add the expected transform cost; the line kernels are embarrassingly parallel. These facts are familiar to practitioners but rarely assembled in one place; this paper collects them and shows how to use them.
Authors:Abhishek Phadke, Karthik Kumar Vasudeva, Abhishek Joshi
Abstract:
The initial development phase of UAV swarms largely depends on simulation for experimental design and validation, yet existing open‑source tools are often unmaintained, have steep learning curves, or are built around a single fixed scenario. The need for a comprehensive, modular simulation platform is a recognized research gap. This paper presents SwarmFly, a MATLAB‑based simulation and test platform for multi‑ UAV swarms that addresses these gaps. SwarmFly combines a real‑time operational map, four swarm coordination modes (leader‑follower, decentralized, heterogeneous relay, and heterogeneous speed), simulated IMU telemetry, and IP‑based geolocation with a plugin architecture that lets researchers add behaviors, fault models, and analysis tools without touching the core code. Eight bundled plugins extend the base simulator into a full test harness. The SwarmFly platform exposes multi‑agent aerial swarms to a wide range of internal and external disruptions, enabling observation and quantification of underlying swarm control and behavioral mechanisms. This study verifies and characterizes each subsystem through eight experiments that measure formation accuracy, wind tolerance, fault recovery, energy endurance, and airspace compliance. The platform runs entirely in MATLAB. Its modular design supports straightforward extension toward hardware‑in‑the‑loop testing, larger swarms, and higher‑fidelity dynamics. An open‑source release is available at [https://github.com/abhishekphadke/SwarmFly.git]
Authors:Rıfat Volkan Şenyuva
Abstract:
We consider wideband channel estimation for extremely large‑scale multiple‑input multiple‑output (XL‑MIMO) arrays under hybrid analog‑digital compression, in which a uniform linear array (ULA) is observed through far fewer radio‑frequency (RF) chains than antennas. At a carrier frequency of 28 GHz with bandwidths reaching several hundred MHz, the standard narrowband polar‑domain channel model fails: the near‑field Fresnel curvature becomes subcarrier‑dependent, and the compressed observation destroys the per‑subcarrier spatial covariance structure that narrowband methods exploit. We propose the Wideband Cross‑subcarrier Kullback‑‑Leibler (WB‑CL‑KL) estimator, which jointly estimates angle and range directly from the compressed sample covariance, without full‑array reconstruction, by fitting a structured Fresnel covariance model across orthogonal frequency‑division multiplexing (OFDM) subcarriers via a cross‑subcarrier Kullback‑‑Leibler (KL) divergence criterion. We also derive the wideband compressed‑domain Cramér‑‑Rao bound (CRB) ‑‑ the performance lower bound for this hybrid architecture ‑‑ from the Slepian‑‑Bangs formula, and decompose its gain over the narrowband bound into a data‑diversity component of +27.093 dB and a geometric‑diversity component of +0.701 dB, totalling +27.793 dB at B = 400 MHz (Propositions 1 and 2). In the single‑path line‑of‑sight regime, WB‑CL‑KL attains a range root‑mean‑square error of 19.8 mm against a 19.9 mm bound at signal‑to‑noise ratio (SNR) = 10 dB, a ratio of 0.996. Under the 3GPP Urban Micro (UMi) path‑loss and shadow‑fading SNR distribution, it achieves a bound ratio of 0.959 at the median deployment SNR of 9.6 dB, indicating near‑CRB operation at the representative deployment point, where the compressed‑domain bound is evaluated at the scene‑median geometry.
Authors:Tony Salomone, Deep Gandhi, Ali Asaria
Abstract:
Sparse Mixture‑of‑Experts (MoE) models route each token to a few of many experts, inviting the hypothesis that experts form functional modules tied to capabilities or languages. We test this causally on Command A+, a frontier open‑weights MoE (218B total / 25B active; 128 experts, 8 active, +1 shared). We build a routing‑mass atlas, pre‑register six family‑to‑axis hypotheses before any intervention, and ablate each family at inference time against a size‑matched random‑expert null, measuring whether it selectively breaks its own axis (worst off‑target effect at most one third of on‑target). Crucially, we test the same families under four metrics and a held‑out, independent‑corpus run with bootstrap confidence intervals. Our finding is cautionary: robust functional modularity is rare and measurement‑dependent. Of six pre‑registered families, only one, the Arabic‑language family, is a clean selective module that survives an independent corpus and a conservative statistical bar (1/6; a more permissive pre‑registered point rule admits 3/6, but that count is threshold‑sensitive). Every other family has a real causal effect yet fails selectivity, and its apparent modularity flips with the measurement: with the corpus, the metric, and the statistical bar. A positive control on Qwen3‑30B‑A3B recovers its published disjoint structure, confirming the method detects modularity when present. The verdict reproduces on the un‑quantized BF16 model, ruling out a 4‑bit quantization artifact. We conclude that ablation‑based modularity verdicts are not safe unless the corpus, metric, and statistical bar are controlled. We release the atlas and ablation data.
Authors:Giulia Perciballi, Ahmad Fall, Federica Granese, Edi Prifti, Jean-Daniel Zucker
Abstract:
Tabular foundation models (TFMs) achieve strong performance on microbiome abundance data, yet their robustness under realistic distribution shift remains poorly characterized. We introduce a benchmark that evaluates the robustness of TFMs to biologically inspired perturbations across six gut microbiome datasets spanning four disease contexts. In this in‑context learning setting, models receive unperturbed support sets as context and are evaluated on perturbed query samples. To isolate robustness beyond "shortcut" features, we preserve the most discriminative taxa and apply three controlled perturbation strategies: (i) removal of high‑abundance (uninformative) taxa, (ii) sparsification via increased zero‑inflation, and (iii) zero‑imputation via spurious non‑zero injections. Our results show that protecting discriminative features is insufficient to guarantee stability under support‑query shift: across datasets, all perturbations degrade model performance, with zero‑imputation consistently the most harmful, indicating that corrupting global feature structure can break generalization even when key taxa are retained. Sparsification disproportionately affects TFMs relative to a classical random forest baseline, suggesting greater sensitivity to zero‑inflation‑type shifts. The code is publicly available at: https://github.com/UMMISCO/metagenomics‑fm/.
Authors:John Pavlopoulos, Spyros Barbakos, Lavinia Ferretti, Dionysis Voulgarakis, Asimina Paparrigopoulou, Maria Konstantinidou, Giuseppe De Gregorio, Isabelle Marthot-Santaniello, Paraskevi Platanou, Holger Essler
Abstract:
Learning representations that remain robust across centuries of variation in handwriting is a key challenge in diachronic representation learning. Taking one of the longest continuously used writing systems, ancient Greek, as a case study, we introduce three datasets for diachronic representation learning: Hell‑Char, a curated training set spanning the 3rd‑1st centuries BCE, and two evaluation sets, PaLit‑Char (2nd‑5th c. CE) and Med‑Char (9th‑14th c. CE). To address the challenges of symbolic variation, scarce data, and systematic degradation, we propose: a similarity‑weighted supervised contrastive loss that biases embeddings using dynamically estimated inter‑class similarities, and a lacuna‑driven augmentation scheme that simulates realistic manuscript corruptions. Trained with these strategies, both a lightweight CNN and a pretrained ResNet achieve strong recognition performance and produce embeddings that more coherently separate character classes than PCA or generic pretrained models. These embeddings enable clustering, identification of stylistic subgroups, and construction of prototype images that visualize diachronic evolution and transitional letterforms. Our results demonstrate that respecting intrinsic inter‑letter relationships and augmenting with domain‑informed corruptions yield robust, interpretable representations, offering a transferable paradigm for representation learning under scarce, temporally evolving, and noisy conditions. Code and data available at: https://github.com/ipavlopoulos/diachronic‑greek‑letterforms.
Authors:Shuai Zhang, Yancheng Chen, Chuan Zhou, Yang Liu, Xixun Lin, Xiangyu Zhao, Jun Zhu, Zhi-Ming Ma
Abstract:
Modeling and sampling from the underlying distribution of asynchronous event sequences are crucial in various real‑world applications, including social networks, medical diagnosis, and financial transactions. Existing autoregressive methods suffer from error accumulation during multi‑step generation, while non‑autoregressive diffusion methods are typically limited to fixed‑length output sequences. In this paper, we propose Latent Block‑Diffusion Temporal Point Processes (LBDTPP), a novel semi‑autoregressive TPP framework that introduces a latent block diffusion mechanism for high‑quality and variable‑length event sequence generation. The core idea is to define an autoregressive probability distribution over event blocks in latent space and perform Gaussian diffusion within each block. By sequentially generating blocks while simultaneously sampling events in each block, LBDTPP preserves the length flexibility of autoregressive TPPs and inherits the parallel high‑quality generation capability of diffusion models. Theoretically, we derive Wasserstein error bounds showing that, under suitable local approximation and prefix‑stability assumptions, block‑wise generation can reduce error accumulation compared with event‑wise autoregressive generation. Extensive experiments on six real‑world benchmark datasets demonstrate that LBDTPP outperforms state‑of‑the‑art TPP baselines in both unconditional and conditional generation tasks. Further empirical analyses verify the benefits of latent‑space diffusion and block‑wise generation, and reveal the trade‑off between generation quality and block size. Our code is available at https://github.com/Zh‑Shuai/LBDTPP.
Authors:Anirban Das, Joanne Boisson, Irtaza Khalid, Sumita Garai, Steven Schockaert
Abstract:
Reasoning about relational structures remains a significant challenge for neural models, particularly when they must systematically apply learned knowledge to problem instances that are harder than those seen in training. Progress is hampered by the difficulty of evaluating such generalization, since a priori, it is rarely clear what makes an instance hard. We study how this issue can be addressed by using large language models (LLMs) to automate benchmark generation, learning to produce increasingly challenging instances in an end‑to‑end manner. Concretely, given a world parametrized by Datalog rules, and an Edge Transformer as the reasoning evaluator, we use LLM‑driven evolutionary search (based on FunSearch) and autonomous agentic search to discover sampling functions that yield hard problem instances. We also show that the Edge Transformer can be improved using this data such that it generalizes well to further data perturbations. Finally, we show that the same machinery can be applied to novel worlds proposed by LLMs, opening the door to autonomous research on neural relational reasoning.
Authors:Randhir Kumar
Abstract:
Knowledge graph embedding (KGE) models predict single‑hop links well but have no mechanism for zero‑shot compositional queries: multi‑hop questions whose relation chains never appeared during training. Holographic Reduced Representations (HRR), which bind and unbind symbols via circular convolution, are a theoretically attractive candidate, since binding is approximately invertible and associative. We test whether this promise holds. We study two holographic memory variants, real‑valued HRR and phase‑only Fourier HRR (FHRR), each with a modern Hopfield cleanup, on FB15k‑237 over five seeds. Four findings follow. First, both are competitive single‑hop retrievers (filtered MRR 0.358 +/‑ 0.002 for HRR, 0.350 +/‑ 0.021 for FHRR). Second, neither composes zero‑shot: accuracy stays at chance across all cleanup temperatures. Third, the main contribution, we localise the failure mechanistically. A hop‑1 probe shows the memory recovers the correct intermediate entity with high fidelity (MRR 0.896 +/‑ 0.002 for HRR), yet composition still fails even with a verified‑correct intermediate. A second probe shows why: posing the ground‑truth second‑hop fact as a standalone atomic query, bypassing composition entirely, already recovers it at only 0.26 to 0.48x average atomic accuracy, uniformly across relation fan‑out. The bottleneck is not the bind‑unbind algebra or the cleanup; it is that facts compositional chains pass through are intrinsically harder for the superposed memory to retrieve, a capacity and interference effect present already at a single hop. Fourth, we prove (Lemma 4.1) that FHRR's softmax cleanup is not phase‑equivariant, compounding the primary failure on the minority of chains where hop‑1 itself errs. Fixing zero‑shot composition requires improving retrieval capacity under superposition, not just redesigning the cleanup.
Authors:Hongbo Wang
Abstract:
We ask a representation‑learning question about physical world models: when does a conservation law remain certifiable after a model learns a latent representation? A certified horizon bounds ‑‑ in advance, from measurable model defects ‑‑ how many steps a rollout provably stays on a physical invariant's level set. The key design choice is what is certified: not a learned latent Hamiltonian or a learned scalar witness (a model can conserve either while drifting in true energy), but the decoded physical invariant obtained by decoding the latent state and evaluating the known invariant. Around this object we derive shell‑horizon certificates whose budget decomposes into representation, readout, and latent‑dynamics defects, with a monotone alignment bridge through which a soft learned witness yields a certified horizon for the decoded invariant, and test them across state, learned‑lift, and pixel observations on conservative systems. Conservation certificates can survive learned representation, but not all geometric priors survive equally. Hard canonical symplectic structure yields the longest horizons in known phase coordinates yet does not cross a learned chart, whereas a controlled‑Lipschitz‑aligned soft invariant survives in the nonlinear learned‑representation settings we test ‑‑ two lift systems, with the gain growing with nonlinearity, and pixels. Pixel certification is recovered on a readout‑stable sub‑tube, and the Kepler problem exposes a geometric boundary. The central object is therefore not a latent Hamiltonian, but a decoded physical invariant whose robustness to representation learning can be measured, certified, and falsified.
Authors:Anzhe Xie, Weihang Su, Jiaxin Mao, Yiqun Liu, Shaoping Ma, Qingyao Ai
Abstract:
Large language models have shown strong fluency in scientific writing, yet the evaluation of related work generation (RWG) remains limited. Existing RWG evaluations largely inherit summarization‑oriented metrics, using lexical or semantic similarity to reference sections as proxies for quality. However, related work writing is fundamentally a citation‑level scholarly positioning task: it requires selecting, organizing, and framing prior work to clarify how a target paper relates to, differs from, and contributes beyond existing research.As a result, models may generate coherent and semantically‑relevant text while exhibiting academically critical failures, such as inappropriate citation selection or misplaced references, that conventional metrics do not capture.To this end, we introduce RWGBench, a benchmark that evaluates RWG from the perspective of citation decision‑making rather than text similarity. RWGBench is constructed from a large‑scale collection of 40,108 computer science papers and a retrieval corpus of 1.09 million documents, with a carefully curated test set comprising 100 papers and their corresponding published related work sections.We propose a multi‑dimensional evaluation framework that assesses citation selection, contextual appropriateness, organization, and discourse structure.Experiments reveal systematic limitations in current systems that are obscured by standard evaluations, while Oracle studies further disentangle retrieval‑level and generation‑level bottlenecks. Human evaluation further shows that our citation‑centric metrics align substantially better with expert judgment than surface‑level text metrics. RWGBench offers a citation‑centric testbed for developing and evaluating related work generation systems that are better aligned with scholarly writing practices.
Authors:Ahmad Pouramini, Hesham Faili
Abstract:
Prompt‑based learning has emerged as a dominant paradigm in natural language processing. This study explores the impact of diverse pre‑training objectives on the performance of encoder‑decoder pre‑trained language models across generation and question answering tasks, with a focus on commonsense knowledge retrieval and completion. We highlight the benefits of incorporating multiple objectives during both pre‑training and fine‑tuning stages. We introduce the Match Task to Objective (MTO) framework and methods for determining the appropriate objective for a given task. This framework offers automated methods to prepare task‑related data for adaptation through unsupervised training, based on the identified objective. In the fine‑tuning stage, we design novel templates that align with the objectives of the pre‑training and adaptation stages. When aligned with task requirements, these strategies can achieve a performance gain of over 120% compared to conventional methods in few‑shot settings. They significantly outperform related works in few‑shot settings and exceed the baseline even in full‑dataset scenarios. Furthermore, we extend this approach to include prompt‑tuning methodologies, providing guidance for more effective soft prompt engineering and optimization. Our strategies significantly enhance prompt‑tuning performance as well. These insights hold substantial value, precisely guiding the selection and optimization of models customized for specific tasks. Code is available at https://github.com/puraminy/MTO/
Authors:Leshu Li, Jie Peng, Yang Zhao
Abstract:
3D Gaussian Splatting (3DGS) has garnered significant attention in Simultaneous Localization and Mapping (SLAM) due to its advances in capturing fine‑grained geometry features and synthesizing novel views. For SLAM in large‑scale scenes, such as autonomous driving, 3DGS‑SLAM faces a critical limitation: memory consumption increases continuously over time as Gaussian points accumulate, leading to poor memory efficiency and limiting its applicability. In this work, we propose a rendering‑area‑aware pruning strategy that selectively removes Gaussians based on their contribution to the effective rendering area, rather than solely relying on Gaussian‑level heuristics such as opacity or gradient magnitude. This perspective directly targets the sources of memory redundancy, effectively reducing the peak memory footprint of 3DGS‑SLAM during runtime. Evaluations on the EuRoC and KITTI datasets demonstrate that our method consistently outperforms existing pruning approaches in large‑scale outdoor scenes, achieving over 60% memory reduction and more than 2 times FPS improvement while preserving localization and mapping accuracy. These results highlight rendering‑area‑aware pruning as a promising direction for scaling 3DGS‑SLAM to real‑world autonomous driving scenarios. Our code is publicly available at https://github.com/UMN‑ZhaoLab/Pocket‑SLAM.git.
Authors:Oren E. Livne
Abstract:
Graph‑Laplacian systems Lϕ=b underlie spectral clustering, semi‑supervised learning, finite‑element analysis, and network‑flow solvers. We present LAMG+, a lean, parameter‑free, empirically linear‑time algebraic multigrid solver: a Julia re‑derivation of Lean Algebraic Multigrid (LAMG) with two targeted refinements. We establish three facts. (1) Benchmarking against approximate‑Cholesky (AC) and four other solvers (BoomerAMG, PETSc GAMG, pyAMG, CMG): LAMG+ and AC are complementary peers ‑‑ AC is faster on social/citation graphs; LAMG+ is faster on finite‑element/structural matrices (fastest robust solver, most memory‑frugal, 2.2× faster than the robust AC variant on large graphs). Only LAMG+ and AC converge across all 13 test classes; the others fail or slow by an order of magnitude off their home turf. (2) Linear scaling: LAMG+ is empirically O(m) with m nonzeros over the full 1,711‑graph SuiteSparse set (100% converged, median 4 cycles, log‑log slope 1.01), verified up to 2.4× 10^8 nonzeros. (3) Robustness: prior benchmarking reported LAMG non‑convergent on certain families; running the unmodified LAMG 2.2.1 under identical conditions establishes full convergence, indicating an evaluation artifact. A Local Fourier Analysis proves a strict interpolation‑order deficit on grid‑aligned anisotropy. Two lean local refinements ‑‑ a strength‑of‑connection aggregation veto and selective caliber‑2 interpolation ‑‑ resolve LAMG's anisotropy failure (convergence factor \approx 0.99 \to 0.11) with negligible overhead.
Authors:Dimitri Gominski, Maurice Mugabowindekwe, Qiue Xu, Xiaowei Tong, Martin Brandt, Hieu Le, Rasmus Fensholt, Dimitris Samaras, Loic Landrieu
Abstract:
Counting individual trees is a fundamental task for environmental monitoring, yet remains largely unexplored with satellite imagery. At these resolutions, isolated trees may still be identifiable, but crown boundaries become ambiguous in dense forests, making the notion of an individual tree inherently ill‑defined. Moreover, large‑scale manual annotations of individual trees are prohibitively expensive. While scalable supervision can be derived from airborne LiDAR, the resulting annotations are noisy and difficult to exploit effectively. We address these challenges by formulating tree counting as a spatial density matching problem supervised through Unbalanced Optimal Transport. This formulation naturally accommodates both precise localization of isolate trees and robust density estimation in dense forests. We further introduce a self‑correction mechanism that leverages transport residuals to progressively refine noisy supervision during training. We evaluate our approach on TinyTrees, a new benchmark spanning three continents and three satellite sensors, comprising over 216 million tree annotations (including 639k manually verified instances) across 25\,890 km^2. Our method consistently outperforms detection‑based, regression‑based, and transport‑based distribution‑matching baselines, demonstrating the effectiveness of unbalanced transport and reliability‑aware supervision for large‑scale tree counting from satellite imagery. Code, data and models are available at https://github.com/dgominski/treematch.
Authors:Daniel Lengerer, Mathias Pechinger, Klaus Bogenberger, Carsten Markgraf
Abstract:
High‑resolution aerial imagery has recently emerged as a complementary modality for automated driving perception and has shown potential to improve birds‑eye‑view (BEV) scene understanding when fused with onboard sensors. Prior work demonstrated performance gains for online high‑definition (HD) map construction through aerial‑onboard fusion; however, conventional end‑to‑end fusion does not fully exploit the structural information contained in aerial representations. In this work, we introduce AerialFusionMapNet, a fusion‑based mapping framework with a structured two‑stage training strategy that explicitly enhances the contribution of aerial features within a unified pipeline. The proposed training scheme enables more effective integration of structural aerial priors. On the nuScenes geographic split, AerialFusionMapNet achieves up to 54.7 mAP, improving over prior aerial‑onboard fusion baselines from 48.8 mAP by +5.9 absolute and +12.1% relative. The results suggest that structured training design, rather than increased architectural complexity, plays a more decisive role in unlocking the full potential of aerial imagery for online HD map construction. Code and trained models are available at https://github.com/DriverlessMobility/AerialFusionMapNet.
Authors:Wei Zhou, Xuanhe Zhou, Shaokun Han, Hongming Xu, Guoliang Li, Zhiyu Li, Feiyu Xiong, Fan Wu
Abstract:
Memory for large language model (LLM) agents has rapidly evolved from simple retrieval‑augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end‑to‑end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system‑level concerns, including operational costs, architectural trade‑offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end‑to‑end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine‑grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long‑horizon stability. Finally, we reveal cost‑performance trade‑offs under realistic workloads, showing localized maintenance is more cost‑efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent‑native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.
Authors:Xiaowei Gao, Pengxiang Li, Yitai Cheng, Ruihan Xu, James Haworth, Stephen Law, Yun Ye
Abstract:
Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade‑off between temporal reasoning and spatial precision. Models that rely on single‑frame or low‑resolution inputs often miss small, distant, or partially occluded hazards, while language‑centric driving models frequently provide limited grounded evidence for their explanations. To address this gap, we propose UniDrive, a unified visual‑language and grounding framework for interpretable risk understanding in autonomous driving. UniDrive combines a temporal reasoning branch that models scene dynamics from multi‑frame visual input with a high‑resolution perception branch that preserves fine‑grained spatial details from the latest frame. The two branches are integrated through a gated cross‑attention fusion module, enabling dynamic context to be aligned with precise spatial evidence. Based on the fused representation, UniDrive jointly generates natural‑language risk descriptions and grounded bounding‑box outputs for risk objects. Experiments on the DRAMA‑Reasoning benchmark show that UniDrive outperforms representative image‑based and video‑based baselines in both captioning and risk‑object grounding. In particular, UniDrive achieves the best overall performance on the validation split and demonstrates clear advantages in small‑object localization, zero‑shot generalization to NuScenes and BDD100K, and human‑rated interpretability and trustworthiness. These results suggest that explicitly combining temporal semantics and high‑resolution perception provides a stronger foundation for interpretable and safety‑oriented autonomous driving systems. The code is available at https://github.com/pixeli99/unidrive‑dev.
Authors:Faris Alasmary, Taif Nono, Orjuwan Zaafarani, Kholood Al Tabash, Ahmad Ghannam, Anas Salamah, Shouq Sadah, Lahouari Ghouti
Abstract:
Handling repeated characters in text can be tricky, since they can represent either the correct spelling of a word or informal character elongation often seen in social media posts. We present CANDLE, a lightweight system for character‑level Arabic noise deduplication that addresses this challenge without relying on handcrafted rules, dictionaries, or morphological analyzers. At the heart of CANDLE is a novel application of Connectionist Temporal Classification (CTC) to this task, a formulation not previously explored for character deduplication, which frames normalization as a sequence alignment problem over a character‑based encoder. Evaluated on three benchmarks spanning clean newspaper, manually curated ambiguous cases, and real‑world social media text, the CTC model achieves a Sentence Error Rate (SER) as low as 5.37% and consistently outperforms a classification‑based baseline by a large margin. To reduce inference overhead, we distill the 6‑layer CTC model into a 2‑layer student, achieving a 3× depth reduction with minimal performance degradation. Beyond deduplication accuracy, normalization yields a practical downstream benefit: a relative reduction in tokenizer fertility of up to 12.8% across a diverse set of Arabic LLM tokenizers, directly lowering inference costs and improving context window utilization. We release all code and models publicly to support reproducibility and advance future research\footnotehttps://github.com/abjadai/candle.
Authors:Jiaxiang Liu, Tianxiang Hu, Juwei Guan, Yujie Wu, Yusong Wang, Yao Mu, Zuozhu Liu, Mingkun Xu
Abstract:
Recent advances in vision‑language models (VLMs) such as CLIP have demonstrated strong generalization across natural‑image domains. However, adapting these models to biomedical imaging is non‑trivial: full‑model fine‑tuning is computationally expensive, while medical data are often scarce and exhibit subtle, fine‑grained inter‑class differences, making parameter‑efficient adaptation particularly critical. Visual Reprogramming (VR) offers a parameter‑efficient alternative by injecting learnable perturbations into the input space, but existing VR approaches for VLMs mainly focus on positive class prompts and overlook confusing negatives, leading to miscalibrated predictions in fine‑grained medical scenarios. We present BioMedVR, the first VR‑based framework for biomedical imaging, enabling few‑shot adaptation of pretrained VLMs through compact learnable VR modules. To mitigate class confusion, we introduce a Confusion Minimization Mechanism that leverages LLM‑generated confusion‑aware attributes together with a Confusion‑Suppression Loss to explicitly reduce false‑positive alignment. Moreover, the designed Mixture‑of‑Prompt Experts combines a positive expert for main‑class discrimination and a negative expert for confusion suppression, balanced via adaptive gating. Extensive experiments on 18 datasets, including 11 biomedical datasets and 7 natural image benchmarks, demonstrate that BioMedVR achieves superior accuracy and generalization, effectively bridging VR and VLMs in biomedical domains.
Authors:Jonas Klotz, Cassio F. Dantas, Pallavi Jain, Diego Marcos, Begüm Demir
Abstract:
Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human‑grounded evaluation framework that quantifies alignment between SAE latents and human‑annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention‑style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully‑Binary Matching Pursuit (FBMP), a coalition‑based matching procedure that supports many‑to‑one mappings between SAE latents and annotated concepts, and consistently outperforms one‑to‑one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image‑level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade‑off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae‑concept‑eval.
Authors:Kunyu Ni, Lei Cao, Jie He, Xiaotong Zhang, Jianfeng Jin, Junyu Dong, Yanwei Yu
Abstract:
Data preparation pipelines improve data quality in machine learning by transforming raw tables into learning‑ready data through sequential cleaning and feature transformation operators. However, automatically constructing such pipelines is computationally difficult because operator sequences are combinatorial and end‑to‑end evaluation is expensive. Existing state‑of‑the‑art (SOTA) Multi‑DQN methods still face three key limitations: decoupled value estimators weaken long‑horizon credit assignment, dataset context is only weakly injected into the policy, and exploration is inefficient in a sparse search space with many invalid states. To address these issues, we propose FlowPipe, a unified framework that formulates pipeline synthesis as conditional probabilistic flow generation over a directed acyclic graph. FlowPipe uses Conditional Generative Flow Networks (C‑GFlowNets) with a Trajectory Balance objective to connect terminal validation rewards with early pipeline decisions. It further introduces Deep Semantic Modulation through Feature‑wise Linear Modulation (FiLM), allowing LLM‑derived logical priors to condition the policy's internal activations according to dataset semantics. In addition, FlowPipe incorporates failure awareness into the flow objective to avoid invalid states and concentrate search on high‑potential regions. Experiments on two benchmark suites with 74 real‑world datasets show that FlowPipe outperforms SOTA baselines, improving accuracy by 11.96% on average and achieving 12.5x faster training convergence. Source code is available at https://github.com/KunyuNi/FlowPipe.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Time‑series, geospatial, and ontology systems each maintain a hierarchy ‑‑ day <= month <= year, zip <= city, is‑a / part‑of ‑‑ and each indexes it in a separate silo. We observe these are all subsumption posets, with one recurring workload: order testing (is x under y?) and hierarchical roll‑up (aggregate a measure over everything under y). We present OEH, a single declarable index that, by a cheap structural probe, encodes a hierarchy as a nested‑set order‑embedding (trees) or a chain decomposition (low‑width DAGs), and answers both subsumption and index‑resident monoid roll‑up from one structure. On five real hierarchies ‑‑ Gene Ontology, NCBI Taxonomy (1.3M), GeoNames (330k), a 2.6M‑node calendar, and git commit DAGs ‑‑ OEH on trees matches a 2‑hop index on query latency using about half the space and building 6‑‑7x faster, and adds roll‑up that 2‑hop cannot. Its roll‑up matches TimescaleDB's continuous aggregates exactly and in the same latency regime, while also answering subsumption. On high‑width DAGs the chain index is declined and 2‑hop dominates. Order‑embedding is classical; our contribution is the unification and the structure‑selected index over subsumption and index‑resident roll‑up.
Authors:Petr Nyoma
Abstract:
We present Harmonic, a hierarchical state space model (SSM) for language modeling. The architecture stacks three recurrent levels at progressively slower timescales; each level receives the prediction error of the level below as input, rather than its raw hidden state. On enwiki8 with equal token budgets, Harmonic outperforms a comparable Transformer (28M params) by +1.4% at 1K tokens, +6.7% at 8K tokens, and +11.4% at 32K tokens (bpt, lower is better). It also outperforms Mamba at every tested length by 0.7‑‑1.8%. At 64K tokens, both Mamba and Transformer run out of memory on an 80GB H100; Harmonic trains successfully, reaching 6.169 bpt. Results replicate on WikiText‑103 (H‑TF gap +1.7% to +7.2% across 1K‑‑32K). At 1B parameter scale, replacing all attention layers in TinyLlama 1.1B with HarmonicBlock eliminates the RoPE positional encoding limit: the resulting Hallamonic model maintains stable loss across sequence lengths 1K‑‑8K on two independent clean benchmarks (Lambada and fineweb‑edu held‑out), while TinyLlama degrades catastrophically past its 2K‑token RoPE limit (gap: +9.4 bpt at seq=8K on Lambada). Compute is O(L) per forward pass vs. O(L^2) for attention. Logs: https://github.com/Omibranch/harmonic‑logs.
Authors:Wenxin Wang, Bo Zhang, Feng Chen, Zixuan Wang, Wen Li, Changsheng Li, Yinjie Lei
Abstract:
Recent advancements have explored agentic zero‑shot 3D understanding by reformulating it as video keyframe understanding with Multimodal Large Language Models (MLLMs). However, existing methods face an intrinsic bottleneck due to the finite observation perspectives inherent in videos and the implicit perception of 3D scenes. In this paper, we propose a collaborative multi‑agent framework that assigns a Planning Agent to handle high‑level viewpoint planning and supplement novel perspectives, and a Perception Agent to explicitly summarize the 3D scene into a structured holistic cognitive map. Specifically, Planning Agent first analyzes this cognitive map to determine query‑relevant viewpoints and supplements missing critical perspectives to ensure comprehensive observation. Subsequently, Perception Agent documents object‑level attributes from these views by assigning consistent instance identifiers across viewpoints, thereby integrating fragmented observations into the holistic cognitive map. In parallel, it provides feedback to filter out mismatched candidate objects and guide subsequent viewpoint planning. Through this closed‑loop iterative process, two agents collaboratively figure out candidates until Perception Agent determines that sufficient information has been captured to complete the task. Extensive experiments demonstrate that our method achieves state‑of‑the‑art performance on 6 benchmarks, with improvements of 11.1% Acc@0.5 on ScanRefer, 14.6 BLEU‑1 on 3D‑assisted dialog, and 2.1 EM on SQA3D.
Authors:Xinyu Mao, Yuhui Zeng, Xiaokun Liu, Wenyu Qin, Meng Wang, Xin Tao, Pengfei Wan, Xiaohan Xing, Max Meng
Abstract:
Cinematographic captioning aims to describe how a video is filmed using professional film‑language concepts such as camera movement, shot size, depth of field, composition, and shooting angle. This capability is important for fine‑grained video understanding and controllable movie‑quality video generation, yet remains underexplored in existing multimodal large language models. Unlike question‑answering‑based evaluation of cinematic understanding, cinematographic captioning requires a unified open‑form description over multiple cinematographic dimensions. This task is challenging for two main reasons: the model must infer professional cinematographic concepts from subtle visual evidence, and it must generate captions that are both comprehensive and accurate. Accordingly, we propose CineCap, a framework that combines structured reasoning with spatio‑temporal anchors and reinforcement learning with comprehensiveness, accuracy, and gated coverage rewards. The former grounds professional cinematographic descriptions in explicit visual evidence and organizes them into compact atomic reasoning for supervised fine‑tuning, while the latter improves the balance between descriptive completeness and factual correctness. In addition, we construct CineCap Bench, a benchmark of 472 manually annotated video‑caption pairs for systematic evaluation. Extensive experiments show that CineCap consistently outperforms strong proprietary and open‑source baselines, establishing a new state of the art for cinematographic captioning. The code, model checkpoint, and benchmark are publicly available in https://github.com/Hectormxy/CineCap.git.
Authors:Yuanhe Zhao, Tianyu Zhang, Huafei Xing, Derek F. Wong, Jianbin Li, Tao Fang
Abstract:
Retrieval‑Augmented Generation enhances large language models by incorporating external knowledge, but deploying it in sensitive scenarios risks privacy leakage via malicious prompts. To address this, we propose a multi‑agent framework that sanitizes retrieved content through semantic rewriting. By employing three specialized agents for privacy extraction, semantic analysis, and reconstruction, our approach collaboratively removes sensitive identifiers while preserving the semantic core. We evaluate the framework on the ChatDoctor and Wiki‑PII datasets across six large language models. Experimental results demonstrate a significant reduction in privacy leakage under targeted attacks. For instance, we reduced targeted information exposure in LLaMA‑3‑8B from 144 instances in the baseline to just 1. Furthermore, we maintain strong contextual fidelity with a BLEU‑1 score of 0.122, outperforming the existing SAGE method's 0.117. Finally, the framework operates as an asynchronous preprocessing module, introducing no additional latency to online inference, as all rewriting is executed as a one‑time offline preprocessing step. To promote reproducibility, the source code of this work is publicly available at https://github.com/foursoils/Privacy‑Preserving‑RAG.
Authors:Yuxin Zuo, Zikai Xiao, Li Sheng, Fei Huang, Jianhong Tu, Yuxuan Liu, Tianyi Tang, Xiaomeng Hu, Yang Su, Qingfeng Lan, Yantao Liu, Qin Zhu, Yinger Zhang, Bowen Yu, Haiquan Zhao, Haiyang Xu, Jianxin Yang, Jiayang Cheng, Junyang Wang, Lianghao Deng, Mingfeng Xue, Tianyi Bai, Yang Fan, Yubo Ma, Yucheng Li, Zeyu Cui, Zhihai Wang, Zhihui Xie, Zhuorui Ye, An Yang, Dayiheng Liu, Jingren Zhou, Ning Ding
Abstract:
A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen‑AgentWorld‑35B‑A3B and Qwen‑AgentWorld‑397B‑A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain‑of‑thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real‑world environments, we develop Qwen‑AgentWorld through a three‑stage training pipeline: CPT injects general‑purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next‑state‑prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric‑and‑rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real‑world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen‑AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen‑AgentWorld supports scalable and controllable simulation of thousands of real‑world environments for agentic RL, yielding gains that surpass real‑environment training alone. Second, as a unified agent foundation model, world‑model training acts as a highly effective warm‑up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen‑AgentWorld
Authors:Khanak Khandelwal
Abstract:
Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real. We present AdversaBench, an end‑to‑end red‑teaming pipeline that mutates seed prompts with five structured operators, queries a target model, and confirms failures through a three‑judge panel with a meta‑judge tiebreaker. We report experiments on 45 seeds across three categories: reasoning, instruction‑following, and tool use. Every seed produced a confirmed failure. Four findings stand out. First, operator effectiveness varies sharply by category: inject_distractor scores 0.00 mean reward on instruction‑following seeds but 0.80‑0.83 on reasoning and tool‑use. Second, binary failure rate hides difficulty: instruction‑following seeds required 2.4 attacker iterations on average versus 1.1 for other categories, a gap visible in survival curves. Third, pairwise judge agreement of 80‑87% coexists with near‑zero Cohen's kappa due to label skew; category‑level disagreement rates are more informative. Fourth, adversarial prompts generated against Llama 3.1 8B transfer zero‑shot to Llama 3.3 70B, suggesting the mutations exploit general behavioral patterns rather than model‑specific weaknesses. Code, dataset, and analysis scripts are available at https://github.com/khanak0509/AdversaBench .
Authors:Zhenyang Li, Lutao Jiang, Yizhou Zhao, Ying-Cong Chen, Xin Wang, Weikai Chen, Yifan Peng
Abstract:
Reconstructing realistic, physically plausible garments from a single image remains a fundamental challenge. Template‑free methods capture surface geometry but lack explicit sewing structure for simulation; while programmatic systems are simulation‑ready but constrained by predefined templates. This reveals a fundamental representation gap between geometric reconstruction and structured garment construction. We present PatternGSL, a structured garment representation in the form of a template‑free and learnable specification language that encodes complete sewing patterns, including panel boundaries, parameterized seams, and explicit stitch topology, in a compact and standardized form. PatternGSL preserves the physical rigor of pattern‑based models while removing template dependence, elevating sewing structure as a first‑class target for generative modeling. We further propose a vision‑language framework that predicts PatternGSL specifications directly from a single image and decodes them into garments using lightweight deterministic validity handling, without optimization‑based refinement or manual cleanup. In addition, we introduce PatternGSLData, the first large‑scale image‑to‑GSL paired dataset comprising 300K samples with complete sewing pattern annotations, enabling supervised VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing‑structure recovery, reliable cloth simulation, and pattern‑level editing through the same deterministic decoding pipeline. Code and data‑processing scripts will be released at https://lagrangeli.github.io/PatternGSL/.
Authors:Jiayi Lei, Yuandong Pu, Xingyu Han, Rongpeng Zhu, Jing Xu, Jinyao Wang, Zijian Zhou, Bin Fu, Yuewen Cao, Yihao Liu, Hongsheng Li
Abstract:
Text‑to‑image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural language prompts. Yet it remains unclear whether their success reflects genuine causal understanding or sophisticated pattern matching over visual‑textual correlations. Inspired by Russell's inductivist turkey, we introduce Counterfactual‑World (CF‑World), a counterfactual benchmark designed to investigate whether text‑to‑image models can generate images under rules that systematically contradict real‑world priors. CF‑World organizes each scenario into three progressive levels: factual generation under ordinary world knowledge, explicit counterfactual generation with direct visual instructions, and implicit counterfactual generation requiring causal deduction from altered rules. We evaluate both open‑source and closed‑source T2I models using a Vision Language Model (VLM)‑based evaluator (CF‑Eval). Furthermore, we introduce two metrics: Prior Resistance Rate (PRR), which measures a models' ability to overcome entrenched real‑world priors, and Reasoning Retention Rate (RRR), which assesses whether models can maintain reasoning‑dependent counterfactual generation without explicit visual cues. Experiments show that all models exhibit sharp degradation from factual to counterfactual settings. Further analyses suggest that these failures arise because current T2I models encode world knowledge and visual appearances as tightly coupled patterns. Consequently, their heavy reliance on frequent visual co‑occurrences within the training data forces them to default to familiar commonsense priors when tasked with rendering counterfactual worlds.
Authors:Ling Li, Bowen Liu, Zinuo Zhan, Jianhui Zhong, Ziyu Zhu, Bingcai Wei, Kenglun Chang, Zhidong Deng
Abstract:
Pointing‑based visual grounding requires models to precisely locate target objects by deciphering complex spatial relationships between the visual scene and pointing gestures. Traditional methods typically encode input images into static feature representations and perform reasoning primarily within the linguistic domain, often overlooking the rich perceptual cues and explicit spatial geometry inherent in images. In this study, we aim to mitigate the cognitive vulnerability of models in interpreting gestural spatial relations by proposing PointVG‑R, a reasoning‑guided Multi‑modal Large Language Model (MLLM). PointVG‑R introduces geometric‑aware reasoning for pointing‑based grounding, enabling the model to think with images through the strategic integration of Reinforcement Learning (RL) and cold‑start data. Specifically, we design a novel geometric reasoning pipeline that simulates the iterative cognitive process humans employ when interpreting pointing gestures. Furthermore, we construct EgoPoint‑CoT, a high‑quality visual Chain‑of‑Thought (CoT) dataset featuring detailed reasoning trajectories to guide the model via Supervised Fine‑Tuning (SFT) and RL. To address the varying quality of learning signals encountered during training, we further propose an Adaptive Importance Weighting strategy based on Group Variance, which dynamically adjusts reward signals to optimize the learning process. Experimental results demonstrate that PointVG‑R achieves SOTA performance, outperforming the baseline by 15.86 points in mIoU. Extensive ablation studies further validate the efficacy of our proposed modules. Code: https://github.com/lingli1724/PointVG‑R.
Authors:Yakiv Shavidze
Abstract:
Random access into compressed data is normally confined to a single layer. Entropy‑layer methods (Recoil) seek within rANS by storing intermediate decoder states; dictionary/match‑layer methods seek within LZ‑style references. We are not aware of a format that supports a single position‑invariant seek through both an entropy layer and a match layer addressed by one coordinate. We show that ACEAPEX's absolute‑offset design provides exactly this: because the match layer resolves every back‑reference to an absolute position at encode time, and the entropy layer is applied per block, an arbitrary block can be decoded through both layers using one coordinate, bit‑perfect, in isolation. We prove this with a three‑phase verification that closes the empty‑buffer trap. The seek of one 16KB block through ANS‑entropy and match completes in 0.334ms. We verify the full entropy+match pipeline end‑to‑end on four data profiles and characterize the hardware ceiling the format reaches: the absolute‑offset structure unrolls to as many as 25,344 independent parsers on one H100, which sequential LZ77 cannot do. We state explicitly what is not claimed: this is a round‑trip correctness proof, not a disk‑archive format; throughput figures are match‑phase; and the unified‑seek result is demonstrated for two layers, with three‑layer generalization left as a hypothesis. Code and the verification harness are in the project repository.
Authors:Yuru Wang, Lejun Cheng, Yuxin Zuo, Sihang Zeng, Bingxiang He, Che Jiang, Junlin Yang, Yuchong Wang, Kaikai Zhao, Weifeng Huang, Kai Tian, Zhenzhao Yuan, Jincheng Zhong, Weizhi Wang, Ning Ding, Bowen Zhou, Kaiyan Zhang
Abstract:
We introduce NatureBench, a cross‑discipline benchmark of 90 tasks distilled from peer‑reviewed Nature‑family publications, designed to evaluate whether AI coding agents can move beyond reproduction toward discovery on real scientific problems. NatureBench is built on NatureGym, an automated pipeline that constructs a standardized, per‑task containerized environment from a source paper, addressing the environment‑fragmentation problem that has limited the credibility of prior agent‑on‑research benchmarks. Evaluating ten frontier agent configurations under a strict web‑search‑disabled protocol, we find that the strongest model surpasses SOTA on only 17.8% of tasks under the g>0.1 criterion. Analysis of method pathways reveals that agents succeed primarily through methodological translation, converting scientific tasks into familiar supervised prediction problems, rather than through genuine scientific invention. Failures are dominated by wrong method choice and insufficient compute budget, not by task misunderstanding. We release the benchmark, the NatureGym pipeline, and a public leaderboard with maintainer‑side reproduction. Code: https://github.com/FrontisAI/NatureBench
Authors:Nouran Khallaf, Serge Sharoff
Abstract:
This paper describes UOL@IDEM's closed‑track submission to the BEA 2026 shared task on L1‑aware vocabulary difficulty prediction. We model the task as regression and train separate systems for Spanish, German, and Mandarin Chinese\footnoteBelow we use \emphChinese for brevity.. Our system combines multilingual contextual representations with engineered features capturing frequency, surface form, retrieval evidence, semantic alignment, cognate similarity, and masked‑language‑model predictability. Development results show consistent gains over the official closed‑track baselines, with sentence‑embedding encoders such as BGE‑M3, multilingual E5, and LaBSE performing best. Official submissions achieve RMSE scores of 1.132, 1.037, and 0.891 for Spanish, German, and Chinese, respectively. Feature analysis identifies frequency as the most stable predictor, while contextual predictability, form similarity, retrieval, and semantic features provide complementary L1‑sensitive signals. Error analysis shows strong ranking performance but weaker calibration for the easiest items, which are often overpredicted. See https://github.com/Nouran‑Khallaf/UoL‑IDEM‑BEA2026‑Vocabulary‑Difficulty‑Prediction
Authors:Ling Li, Zhizhen Cai, Xinkun Wu, Ziyu Zhu, Jiaqing Lyu, Bowen Liu, Zhidong Deng
Abstract:
Grounding deictic gestures in natural images is fundamental to AR and human‑robot collaboration, providing a basis for seamless spatial interaction. While Transformer‑based visual models have achieved significant progress in general object detection, their global attention mechanisms often neglect micro‑geometric relationships, degrading orientation accuracy. In pointing tasks, this deficiency manifests as an inability to accurately capture the pointing ray implied by finger poses, which results in pointing drift and localization ambiguity when dealing with distant or densely packed objects. To address this, we propose VistaRef, a framework designed to explicitly enhance spatial orientation awareness. First, we develop the Local Hand Entity Modeling (LHEM) module, which incorporates hand‑pose embeddings to strengthen the model's capability to capture subtle finger deviations. Second, drawing inspiration from multi‑view geometry, we construct the Geometric Ray Modeling (GRM) module to transform implicit orientation information into explicit spatial geometric features, guiding feature aggregation and deep fusion via attention mechanisms. Furthermore, we introduce a novel Orientation‑Consistent Alignment Loss (OCAL) to synergistically supervise hand presence and pointing consistency, ensuring that all architectural improvements collectively serve the core objective of spatial localization. Experimental results demonstrate that VistaRef significantly outperforms the baseline, achieving a 14‑point absolute gain in grounding accuracy. Qualitative analysis further confirms that VistaRef effectively models the geometric correlation from hand to target, bridging the spatial perception gap inherent in traditional Transformers for complex scenarios. Code: https://github.com/lingli1724/VistaRef.
Authors:Inam Ullah, Imran Razzak, Shoaib Jameel
Abstract:
Learning causal models from fragmented biomedical data is challenging because clinical, molecular, and imaging variables are often incomplete or not jointly observed. We propose RetiSEM, a domain‑constrained structural equation modelling (SEM) framework for causal graph recovery and mediation analysis under limited multimodal resources. This proposed work organises variables into biologically informed blocks, applies forbidden‑edge constraints, and decomposes pathway‑level effects into TE, NDE, and NIE components. We evaluate RetiSEM across ten synthetic benchmark scenarios that vary in dimensionality, nonlinearity, causal depth, and pathway structure, together with a fragmented real‑world setting that combines NHANES clinical variables with externally derived retinal representations. This approach achieves lower structural error and higher causal accuracy than unconstrained baselines across the synthetic benchmarks. In the real‑data analysis, retinal variables behave mainly as downstream biomarker‑like indicators, with smaller but detectable indirect effects. These findings support our strategy as an interpretable framework for testing structured causal hypotheses in limited‑resource biomedical AI. The code and resources for this work are publicly available at: https://github.com/Inamullah‑Colab/ReitSEM.
Authors:Xingsong Ye, Yongkun Du, Jiaxin Zhang, Haojie Zhang, Chong Sun, Chen Li, Jing Lyu, Zhineng Chen
Abstract:
WordArt (artistic text) features highly customized fonts, textures, and layouts, making WordArt‑oriented scene TExt Recognition (WATER) substantially more challenging than general Scene Text Recognition (STR). Existing STR datasets and methods, typically built around regular scene text and fixed‑template inputs, struggle to scale to WATER. Thus, we aim to advance this task from both data and model perspectives. On the data side, we construct a 2M synthetic dataset, WATER‑S, with the scale improved by hundreds of times compared to existing artistic text data. WATER‑S consists of two complementary subsets. One rendered by an upgraded rendering pipeline (SynthWordArt), which provides highly accurate and controllable synthetic WordArt data. The other is generated by combining Qwen3‑VL for prompt mining and Z‑Image for image synthesis, which improves the coverage of realistic and diverse data. On the model side, we propose WATERec. It adopts an visual encoder supporting arbitrary‑shaped inputs and an autoregressive decoder to model complex layouts, structurally breaking the bottleneck of fixed‑template STR on WordArt. Experiments show that this architecture outperforms prior STR methods, achieving state‑of‑the‑art performance on irregular texts such as WordArt. Together with WATER‑R, carefully reorganized from existing real STR data, our strong baseline with the new synthetic data and model design reaches 90.40% accuracy on WordArt‑Bench, surpassing both general‑purpose and OCR‑specialized vision‑language models by a large margin. Code and data are available at https://github.com/YesianRohn/WATER.
Authors:Peize Li, Fanhu Zeng, Tongda Xu, Xingguo Xu, Xinjie Zhang, Xingtong Ge, Haotian Zhang, Yan Wang
Abstract:
In‑camera JPEG previews are ubiquitous in raw image formats and provide an sRGB reference at negligible storage cost. Although existing metadata‑based reconstruction frameworks can exploit this side information when recovering raw images, their context models often become computationally expensive especially at high resolution, eg, 4K raw image, given that attention mechanisms scale quadratically with feature maps, hindering its practical application. To address these limitations, we propose MambaRaw, a JPEG‑conditioned metadata‑based raw image reconstruction framework that uses State Space Models (SSMs) to estimate entropy parameters efficiently. Our key contribution comprises a Spatial‑Energy Coupled Context Modeling mechanism with two lightweight modules: (1) TileMambaBlock, which performs Mamba‑style selective scanning only on information‑dense tiles to improve the efficiency; and (2) Energy‑Aware Refinement (EAR), an identity‑initialized residual module that enhance feature representation to match the long‑tail energy distribution of raw signals. Extensive experiments on three camera datasets (Sony, Olympus, Samsung) show consistent improvements over strong metadata‑based baselines and set a new state of the art for JPEG‑guided raw reconstruction with great efficiency. Notably, at low metadata bitrates, MambaRaw increases PSNR by 1.2‑‑1.4 dB and reduces end‑to‑end coding latency by about 9%. Code is released at https://github.com/Peizeli1/MambaRaw.
Authors:Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, Grace Li Zhang
Abstract:
Long‑context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key‑value (KV) caches, limiting sustainable deployment on resource‑constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA‑based LLMs. These methods ignore the different functionalities of attention heads, leading to the eviction of critical tokens and thus degrading the performance of LLMs. To address this issue, we propose CompressKV, a resource‑efficient KV‑cache compression framework for GQA‑based LLMs. Instead of aggregating attention scores from all heads, CompressKV identifies Semantic Retrieval Heads (SRHs) that capture both the initial and final tokens of a prompt and semantically important mid‑context evidence, and uses them to select tokens whose KV pairs should be retained. Furthermore, CompressKV allocates cache budgets across layers according to offline estimates of layer‑wise eviction error. Experiments on LongBench and Needle‑in‑a‑Haystack show that CompressKV consistently outperforms existing KV‑cache eviction methods across memory budgets. Notably, it preserves over 97% of full‑cache performance using only 3% of the KV cache on LongBench question‑answering tasks and achieves 90% accuracy with just 0.7% KV storage on Needle‑in‑a‑Haystack. These results demonstrate an improved resource‑‑performance trade‑off for long‑context LLM inference. Our code is publicly available at: https://github.com/TUDa‑HWAI/CompressKV
Authors:Tianyu Zhu, Yingping Liang, Hesong Li, Ying Fu
Abstract:
Text‑driven Referring Video Object Segmentation (RVOS) aims to locate and segment target objects in videos given natural language. However, existing models are typically trained on 2D image or video datasets with naive segmentation losses, which overlooks the geometric consistency across frames and leads to weak spatial understanding. In this paper, we propose Geometry‑enhanced Language‑guided Video segmentation (GeoLaV), a two‑stage framework that distills 3D geometric knowledge from images to enhance text‑driven video segmentation. In the first stage, we perform monocular geometry pretraining with monocular novel‑view synthesis, enabling the model to acquire geometry‑consistent visual representations via spatial alignment on large‑scale single‑image datasets. In the second stage, we introduce geometry‑aware distillation and fine‑tune the model on video segmentation datasets, transferring 3D structural knowledge from a general 3D prior model. This process reinforces 3D awareness and improves both spatiotemporal coherence and language grounding in segmentation. Extensive experiments show that our method using only image segmentation data already provides notable zero‑shot generalization in RVOS. When combined with geometry‑aware distillation for fine‑tuning on videos, our method achieves state‑of‑the‑art performance across multiple RVOS benchmarks. The code is available at https://github.com/Tony1882880/GeoLaV.
Authors:Junpeng Jing, Ronglai Zuo, Zhelun Shen, Shangchen Zhou, Rolandos Alexandros Potamias, Stefanos Zafeiriou, Krystian Mikolajczyk, Jiankang Deng
Abstract:
Recent advances in stereo matching have achieved remarkable accuracy, but often rely on large models, heavy computation, or additional foundation‑model priors, making them difficult to deploy on resource‑constrained platforms. In contrast, efficient stereo models offer faster inference but are commonly considered less capable of strong zero‑shot generalization. In this paper, we challenge this assumption by introducing Lite Any Stereo V2 (LAS2), an ultra‑fast model series designed for efficient zero‑shot stereo matching. LAS2 is developed from both architecture and training perspectives. Architecturally, we revisit efficient stereo design under practical deployment settings and propose a 2D‑only cost aggregation framework, optimized for real inference latency rather than theoretical MACs alone. For training, we develop a three‑stage strategy that combines synthetic supervision, self‑distillation, and real‑world knowledge distillation. To improve the reliability of real‑world pseudo supervision, we further introduce pseudo‑label filtering and an error‑clamping operation, enabling smoother synthetic‑to‑real transfer. We instantiate LAS2 as a family of models, including feed‑forward variants for different efficiency budgets and an iterative variant for higher accuracy. Extensive experiments show that LAS2 achieves state‑of‑the‑art accuracy among efficient stereo methods while maintaining significantly lower latency. Specifically, LAS2‑H achieves stronger overall zero‑shot performance than the iterative method Fast‑FoundationStereo, with 1.8x and 2.7x faster inference on H200 and Orin, respectively. The project page, demos, and code are available at https://tomtomtommi.github.io/LiteAnyStereoV2/.
Authors:Soham Patil, Avirup Das, Sourabh Bhosale, Spandan Roy
Abstract:
Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer‑based multimodal framework that fuses RGB‑D vision with the robot's proprioception to infer binary contact states as a pseudo‑tactile signal for hand‑object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in‑hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real‑world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no‑contact‑no‑worries/
Authors:Mohamad Alansari, Yonathan Michael, Hasan AlMarzouqi, Muzammal Naseer, Naoufel Werghi, Sajid Javed
Abstract:
We revisit the memory update mechanism in SAM2‑based visual object tracking and identify confidence‑only mask selection as the dominant cause of drift under occlusion, rapid motion, and distractors. We introduce SENTRY, a training‑free, plug‑and‑play, refine‑before‑write module that validates each memory update for short‑horizon temporal consistency before committing it. SENTRY aggregates diverse segmentation hypotheses per frame, backtracks them into short tracklets, and uses neighbor‑aware cycle‑consistent matching against recent trajectories to favor temporally and geometrically consistent masks. It leaves the base architecture untouched, replacing confidence‑driven writes with consistency‑validated ones. For fair evaluation, we re‑evaluate major open‑source SAM2‑based trackers across all available scales and datasets, filling gaps in prior reports. Integrated into five strong baselines, SENTRY delivers consistent gains across nine benchmarks, achieving new zero‑shot SOTA on LaSOT, LaSOT_ext, GOT‑10k, VOT20, VOT22, and DiDi. Despite these checks, the SAM2‑L version runs at 32.8 FPS on an A100, and across compatible hosts adds only about 0.4‑‑0.6 GB VRAM. Our results provide the first unified all‑scale evaluation of SAM2‑based trackers and show that enforcing temporal validity at write time stabilizes memory‑augmented tracking without retraining. Project page: https://hamadya.github.io/SENTRY/page/
Authors:Michael Krieger, Markus Gierlinger, Farooq Shaikh, Mario Kahlhofer
Abstract:
Kubernetes has become the industry standard for orchestrating containers in microservice‑based software architectures. While several hardening guidelines and scanning tools for securing Kubernetes clusters and deployments have emerged in recent years, their differing guidance and outputs often lead to inconsistent configuration and prioritization decisions. This work presents a systematic comparison of eight commonly used Kubernetes hardening guidelines. Through this comparison and the inclusion of best practices, we established a benchmark of 79 Kubernetes configuration recommendations and conducted the a structured empirical evaluation of ten popular static configuration scanning tools and their scoring outputs. Our findings reveal substantial disparities in the coverage of configuration issues across hardening guidelines and scanners, as well as inconsistencies in how configuration issues are scored and ranked by different scanners. These results highlight the need for more standardized, transparent, and consistent approaches to risk and severity assessment of Kubernetes configuration issues.
Authors:Shiding Zhu, Yudi Qi, Yajie Wang, Jiaze Li, Chao Song, Yaorui Shi, Yibo Miao, Hanqi Gao, Kai Zhang
Abstract:
Experience‑driven self‑evolution is critical for large language model (LLM) agents to improve through open‑world interaction. However, existing experience learning methods mostly rely on single‑agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self‑Confirmation Trap: wrong‑but‑self‑consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute‑Distill‑Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third‑party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor‑centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self‑reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long‑horizon benchmarks: tau2‑bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self‑evolution. Our code is available at https://github.com/shidingz/EDV.
Authors:Yijia Lei, Jinzhao Li, Yichi Zhang, Jiacheng Hua, Yin Li, Miao Liu
Abstract:
We introduce EgoSAT, the first comprehensive benchmark for egocentric video reasoning in streaming settings, designed to evaluate the capabilities of modern vision‑language models (VLMs). The benchmark targets streaming interaction understanding, where video frames arrive sequentially and models must continuously interpret evolving visual context. EgoSAT unifies several previously distinct tasks within a single streaming framework. In this formulation, queries about completed events correspond to retrospective reasoning, queries about ongoing activities require online understanding, and queries about future actions involve prospective anticipation. This unified setting requires models to reason about the past, present, and future while operating under the constraint that only previously observed frames are available. EgoSAT contains 1,997 unique videos spanning 165 hours of egocentric footage and around 4,800 high‑quality question‑answer pairs, carefully designed to probe reasoning across varying temporal contexts. Using this benchmark, we evaluate a diverse set of both open‑weight and closed‑weight VLMs, providing a systematic assessment of their ability for streaming interaction understanding. By distinguishing answerability and conducting diagnostics on confidence of models, we find existing models not only struggle with prospective and retrospective modeling, but also exhibit severe mis‑calibration: confidence often fails to track inherent answerability, leading to dangerous "confidently wrong" behaviors. Project page: https://leiyj23.github.io/EgoSAT/
Authors:Hojun Choi, Seulbin Hwang, Dae Jung Kim, Kisung Kim, Hyunjung Shim, Jinhan Lee
Abstract:
Bird's‑eye view (BEV) perception fuses multi‑camera images into a unified top‑down representation for autonomous driving. Despite recent progress, state‑of‑the‑art methods remain confined to closed‑set scenarios, making them vulnerable to unpredictable real‑world environments. In this work, we introduce open‑vocabulary BEV segmentation (OVBS), which leverages vision‑language models (VLMs) to recognize categories beyond the training set while maintaining precise BEV perception and real‑time efficiency. A key challenge in OVBS lies in the 3D geometric inconsistency inherent in the ill‑posed lifting of 2D VLM semantics into BEV. To address this, we propose OVBEVSeg, a geometry‑aware OVBS framework that enhances efficient Gaussian splatting (GS)‑based unprojection by leveraging robust 3D geometric constraints across three progressive stages: (1) 2D‑to‑BEV pseudo‑labeling via reliable 3D projection for OV generalization; (2) joint 2D‑BEV per‑scene optimization with BEV structural constraints for 3D geometric consistency; and (3) 3D geometric distillation for online efficiency. On the nuScenes dataset, OVBEVSeg achieves state‑of‑the‑art performance, outperforming closed‑set methods by 15.3 mIoU on unseen categories. Remarkably, even with no novel‑class ground‑truth labels, it remains competitive with self‑ and semi‑supervised baselines trained with up to 40% of ground‑truth annotations. Furthermore, it achieves 2.5x faster inference with only 0.22x the memory consumption of projection‑based methods. Project page: https://hchoi256.github.io/projects/ovbevseg/.
Authors:Yang Zhou, Wenxue Li, Peng Zhang, Yifei Chen, Fei Wang, Daiguo Zhou
Abstract:
Face Video Restoration (FVR) aims to recover high‑fidelity facial videos from degraded input while preserving identity and semantic consistency across frames. Existing methods often struggle to simultaneously address three key challenges: identity shift, viewpoint‑entangled guidance, and perceptual realism. To tackle these issues, we propose TIGER, a structured tri‑prior fusion framework that Tames Identity, Geometry, and gEnerative pRiors for high‑quality FVR. Specifically, an Identity Prior is first established by injecting subject‑discriminative embeddings into the latent space, effectively anchoring the subject's identity against severe degradations. Then, to provide temporally consistent structural guidance for dynamic videos, TIGER constructs a Geometry Prior by lifting 2D reference cues into a disentangled 3D parameter space, creating a geometric anchor through cross‑source parameter fusion. Moreover, to achieve maximum efficiency without compromising realism, we harness the video generation model's Generative Prior through a one‑step rectified flow. We further design a progressive three‑stage training optimization strategy that refines structural fidelity, textural reconstruction, and distribution‑level realism to ensure robust optimization. We also construct a large‑scale FVR dataset to facilitate robust training and standardized evaluation. Extensive experiments demonstrate that TIGER achieves state‑of‑the‑art performance in both identity fidelity and temporal stability, delivering a high‑quality, efficient and identity‑consistent FVR. Project page: https://yzhoulv.github.io/Tiger/.
Authors:Jiahao Lyu, Pei Fu, Zhenhang Li, Shaojie Zhang, Jiahui Yang, Yu Zhou, Can Ma, Zhenbo Luo, Jian Luan
Abstract:
In‑Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserving the overall visual appearance. Recent unified multimodal models provide a promising solution by combining visual‑text understanding and image generation within a single framework. However, directly adapting such models to IIMT remains challenging. In particular, they often suffer from understanding‑generation conflicts, where the translation inferred during understanding is inconsistent with the text supervision used in generation, and spatial position misalignment, where the rendered text does not accurately match the target text regions. To address these issues, we present UniTranslator, a unified multimodal framework for IIMT that tightly couples translation understanding and text editing. Specifically, we introduce an Understand‑Generation Alignment Module (UGAM) to bridge the representation gap between understanding and generation, encouraging semantic consistency between translated content prediction and text rendering. We further propose a Spatial Mask Decoder (SMD) with pixel‑level supervision over text regions to improve spatial grounding, geometric alignment, and layout controllability during generation. Extensive experiments on multiple benchmarks demonstrate that UniTranslator achieves state‑of‑the‑art performance across diverse language directions and complex real‑world layouts. Moreover, our results reveal a strong mutual reinforcement effect between translation understanding and image generation, highlighting the advantage of unified translation multimodal learning. Code is available at https://github.com/SeerRay‑Lab/Unitranslator.
Authors:Yinji Ge, Guixu Zheng, Wulong Guo, Qian Feng, Xu Wu, Kai Zhou, Xinyuan Liu, Fei Xing
Abstract:
Vision Foundation Models (VFMs) have significantly advanced dense feature matching, yet severe in‑plane rotation remains a critical challenge. Existing solutions face a fundamental dilemma: data‑driven methods require inefficient parameter scaling to implicitly learn rotations, whereas strictly equivariant networks lack the semantic capacity of modern VFMs. Consequently, current frameworks typically freeze VFMs and shift the entire burden of rotation generalization to the downstream decoder. To break this architectural bottleneck, we propose REDI‑Match, an efficient framework driven by a novel Rotation‑Equivariant Distillation (REDI) paradigm. Instead of relying on rotation data augmentation to establish rotational correspondences, REDI distills the non‑equivariant semantic representations of a VFM into a lightweight, strictly rotation‑equivariant encoder, leveraging an equivariant geometric architecture to constrain robust high‑dimensional semantics. To fully exploit these features, we equip the decoder with an entropy‑driven spatial alignment module. By evaluating discrete rotation hypotheses, this mechanism explicitly locks onto the canonical coordinate system, eliminating global ambiguity before continuous refinement. Extensive experiments demonstrate that REDI‑Match establishes a new state‑of‑the‑art (SOTA) across multiple benchmarks. Notably, it achieves a 13.89% absolute pose accuracy improvement on the highly challenging SatAst dataset while operating 1.9x faster than the current SOTA (RoMa v2), enabling real‑time inference (~41 FPS) on a single RTX 4090 GPU. Code: https://github.com/YinjiGe/REDI‑Match.
Authors:Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close, Beren Millidge
Abstract:
We present ZONOS2 8B, our latest TTS model, which achieves state‑of‑the‑art naturalness, prosody, and voice cloning fidelity. We improve upon Zonos‑v0.1 across scale, data, and training recipe. We scale the model from 1.6B to 8B total parameters (900M active) with a novel mixture‑of‑experts (MoE) backbone, improving inference latency and throughput. We expand our training corpus from 200K to over 6M hours using a new data processing pipeline, and we simplify our post‑training and conditioning recipes to improve naturalness and voice cloning fidelity. We evaluate ZONOS2 8B on quality, speaker similarity, WER, and ZTTS1‑Eval, our novel TTS benchmark, where it performs competitively with state‑of‑the‑art systems while maintaining good streaming latency. We release our model weights and example inference code under an Apache 2.0 license on GitHub and Hugging Face.
Authors:Sachin Sharma, Michele Flammini, Federico Simonetta
Abstract:
Fine‑tuning transformer‑based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre‑trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine‑tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th‑century Italian manuscript (I‑CT 91 "Cortonese") and replicate the same experimental grid on the public READ‑16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain‑specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross‑dataset validation on READ‑16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset‑specific re‑validation. Finally, we use Grad‑CAM gradient attributions and decoder cross‑attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR‑analysis
Authors:Hongli Xiao, Youjian Zhang, Yucai Bai, Chaoyue Wang, Yaohui Jin, Xiaoguang Ren, Wenjing Yang, Long Lan
Abstract:
Recovering realistic 3D vehicle models from autonomous driving scenes is crucial for synthesizing training data and building simulation environment. However, most existing vehicle generation methods fail to fully exploit multimodal sensors i.e. multi‑view images and LiDAR point clouds) and rely on neural rendering based reconstruction, leading to low‑quality mesh. Recently, native 3D generative models have made significant progress, yet they are not built for arbitrary multi‑view inputs and often struggle with in‑the‑wild driving images. In this work, we present MM‑TRELLIS, a multi‑modal version of TRELLIS for in‑the‑wild 3D vehicle generation that integrates LiDAR and image sensors from autonomous driving datasets into native 3D generative models. Specifically, multi‑view images are cycled as conditioning inputs, while LiDAR point clouds provide test‑time guidance to ensure geometric accuracy and cross‑view consistency. During denoising, we first align the guidance point cloud with the model priors, then enforce consistency between the generated geometry and the guidance point cloud. Finally, we introduce a voxel filtering strategy based on the opacity of 3D Gaussian Splatting to suppress floaters and produce clean meshes. Comprehensive experiments on Waymo dataset demonstrate our method outperforms existing methods in high‑fidelity 3D vehicle generation. Code is available at https://github.com/HongliXiao/MM‑TRELLIS.
Authors:Yajing Wang, Chao Bi, Junshu Sun, Shufan Shen, Zhaobo Qi, Shuhui Wang, Qingming Huang
Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated impressive vision‑language understanding, yet still struggle with fine‑grained perception in high‑resolution images. While existing training‑free methods typically rely on attention‑based localization or coarse‑to‑fine search, they are often misled by distractors and fail to locate multiple targets. Our investigation attributes these failures to Contextual Dominance, where salient distractors overwhelm target attention and cause inaccurate localization, and Semantic Bias, where global semantics cause the model to fixate on the most salient concept, resulting in incomplete localization in multi‑object scenarios. Built on these insights, we propose ActiveScope, a training‑free framework that enhances MLLMs by actively seeking and correcting perception. ActiveScope features two modules. The Semantic Anchor Localization (SAL) utilizes fine‑grained semantic anchors to independently localize key targets, thereby mitigating semantic bias. The Interference‑Suppressed Refinement (ISR) refines localization by suppressing attention on salient distractions to overcome contextual dominance. Extensive experiments on high‑resolution image understanding benchmarks demonstrate that ActiveScope outperforms existing training‑free methods (e.g., 96.34 percent accuracy on V^ Bench), validating the superiority of the active search and self‑correction paradigm. Our code is available at https://github.com/jasmine‑ww/ActiveScope.
Authors:Kim Youwang, Zhengyu Yang, Liuhao Ge, Yu Rong, Timur Bagautdinov, Su Zhaoen, Nir Sopher, Jovan Popović, Teng Deng, Tae-Hyun Oh, Chen Cao
Abstract:
We introduce FiCA, a Feed‑forward, instant Gaussian Codec Avatar generation pipeline that creates lifelike avatars from a single portrait image. Generating a photorealistic and drivable avatar from just a single image is significantly challenging due to the limited visual information available to accurately infer the 3D appearance and geometry of human heads. To address this, we develop a novel system that combines human‑centric vision foundation models with a diffusion model. This system is designed to fully exploit partial visual observations to generate lifelike human avatars. Our proposed diffusion model learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction. Additionally, we introduce a feed‑forward mesh refinement network that enhances the fidelity and identity preservation of the generated avatars, eliminating the need for person‑specific test‑time optimization. By leveraging a universal prior model that decodes a generated mesh into a set of 3D Gaussians, we generate a photorealistic 3D Gaussian avatar, capable of being driven with novel expressions in real‑time. Our experiments demonstrate that the avatars generated by our feed‑forward approach faithfully represent diverse identities and surpass the visual quality of avatars produced by recent competing methods.
Authors:Xirui Li, Zhe Liu, Xiaoqing Ye, Wenhua Han, Yifeng Pan, Junyu Han, Hengshuang Zhao
Abstract:
Multimodal driving planning faces a long‑standing tension between two paradigms: scoring‑based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor‑based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground‑truth trajectory. In this work, we propose FlowR2A, which resolves this tension by reframing simulation‑based rewards from discriminative targets into generative conditions. By learning the reward‑conditioned action distribution from dense trajectory‑reward pairs with a flow‑matching decoder, FlowR2A unifies the dense supervision of scoring‑based methods with the proposal generation of anchor‑based methods in a single generative model, forcing the model to internalize the correlation between an action and its outcomes in safety, progress, comfort, and rule compliance. To balance hard safety constraints against soft progress objectives, we introduce fine‑grained per‑timestep reward conditioning and reward noise augmentation. The generative formulation naturally supports controllable test‑time sampling via reward guidance and anchored sampling, producing high‑quality proposals. FlowR2A achieves state‑of‑the‑art results on the NAVSIM v1 and v2 benchmarks, with multimodal proposals of substantially higher quality than prior methods.
Authors:Chirui Chang, Xiaoyang Lyu, Yi-Hua Huang, Haoru Tan, Shizhen Zhao, Yikang Ding, Jianmin Bao, Xin Tao, Pengfei Wan, Xiaojuan Qi
Abstract:
Object‑level geometric edits, including translating, rotating, scaling, duplicating, or removing an object, are routine operations in digital content creation (DCC) workflows, yet they remain unreliable in generative video editing. The key challenge lies in specifying the target object's 3D state change unambiguously across viewpoint and time, while consistently updating geometry‑dependent secondary effects such as shadows and reflections. We introduce GIVE, a geometry‑instructed video editing framework that represents edits through a unified object‑state formulation. Two video‑aligned geometry streams describe the target object before and after editing: a depth‑box encoding coarse 3D placement and extent, and an orientation‑box providing an appearance‑agnostic orientation cue. Together, these streams provide a compact pre/post geometric specification for object‑state transitions. To provide paired supervision for learning these edits, we build a scalable graphics‑engine pipeline that executes object‑level edit programs and renders controlled before/after pairs, isolating the intended geometric edit while keeping secondary effects consistent with the transformation. Experimental results demonstrate that GIVE produces faithful geometric edits with temporal coherence and consistent secondary effects across operators in a unified framework, and shows promising transfer to in‑the‑wild videos. Project page: https://geometry‑instructed‑video‑editing.github.io/give/
Authors:Fuyou Mao, Yifei Chen, Beining Wu, Lixin Lin, Jinnan Dai, Zhiling Li, Yilei Chen, Yaqi Wang, Hao Zhang, Yan Tang, Huiyu Zhou, Feiwei Qin
Abstract:
Accurate pulmonary vessel segmentation remains challenging due to the sparse, tortuous, and multi‑scale nature of vascular structures, where small branches are easily lost and topology integrity is difficult to preserve under voxel‑wise supervision. Existing deep segmentation models primarily optimize binary masks, lacking explicit geometric constraints, thus struggling to recover continuous tubular morphology and fine vascular connectivity. In this study, we introduce MorVess, a morphology‑aware segmentation framework that integrates differentiable geometric priors with large‑scale foundation model adaptation to achieve fine‑grained vascular parsing. MorVess jointly predicts vessel masks, distance maps, and thickness maps, providing explicit supervision for vascular boundaries, centerline consistency, and smooth diameter transitions. A lightweight 2.5D adapter bridges 3D spatial context and 2D SAM representations, while a global‑local fusion block aggregates multi‑level semantics and geometric cues for high‑fidelity topology reconstruction. Across two challenging pulmonary CT benchmarks, MorVess delivers superior Dice, clDice, and HD95 scores, substantially improving small‑vessel recovery and global connectivity. These results demonstrate that embedding geometric intelligence into pretrained vision models offers a principled and scalable pathway toward precise vessel analysis and clinically reliable structural quantification. Our source code is available at https://github.com/MaoFuyou/MorVess.
Authors:Hengji Zhou, Yufeng Liu, Ye Liu, Yong Xu, Lianghao Xia, Liqiang Nie
Abstract:
Modern AIGC pipelines deliver high‑fidelity images and videos but presuppose a well‑formed creation instruction, while end users rarely articulate visual details, leaving generators misaligned with user demand. We study personalized content generation, which turns a user's interaction history into an executable instruction for downstream synthesis, and identify two obstacles: behavior must be encoded in a form legible to language reasoning, and the model must acquire instruction‑writing skill absent from both pretraining and behavior data. We propose NaviGen, which represents each item with a dual identifier coupling a collaborative code and a textual code as a behavioral substrate and a semantic bridge in one token stream. On this representation, a two‑stage SFT+RL pipeline first distills preference reasoning and instruction writing from evolutionarily searched supervision, then aligns generation with user intent through hierarchical and self‑consistent rewards. Experiments across product, game, and short‑video domains show that NaviGen improves personalized image and video generation, strengthens next‑item prediction, and yields more specific, relevant, and visually generatable instructions. Our code is released at: https://github.com/iLearn‑Lab/NaviGen.
Authors:Miso Kim, Georu Lee, Yunji Kim, Hoki Kim, Jinseong Park, Woojin Lee
Abstract:
Unlearning has emerged as a key technique to mitigate harmful content generation in diffusion models. However, existing methods often remove not only the target concept, but also benign co‑occurring concepts. As illustrated in Fig.1, unlearning nudity can unintentionally suppress the concept of person, preventing a model from generating images with person. We define these undesirably suppressed co‑occurring concepts that must be preserved CARE (Co‑occurring Associated REtained concepts). Then, we introduce the CARE score, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose ReCARE (Robust erasure for CARE), a framework that explicitly safeguards CARE while erasing only the target concept. ReCARE automatically constructs the CARE‑set, a curated vocabulary of benign co‑occurring tokens extracted from target images, and leverages this vocabulary during training for stable unlearning. Extensive experiments across various target concepts (Nudity, Van Gogh style, and Tench object) demonstrate that ReCARE achieves overall state‑of‑the‑art performance in balancing robust concept erasure, overall utility, and CARE preservation.
Authors:Anton Morgunov, Victor S. Batista
Abstract:
Retrosynthetic planning seeks to connect a target molecule to commercially available starting materials through a multistep route. Classical planners construct such routes by iteratively applying single‑step reaction models within a search procedure; constrained variants often require specialized algorithms or architectural changes. Direct route generation reframes retrosynthesis as sequence generation, but existing direct‑generation methods still train separate models for different planning specifications. We introduce Ariadne, a decoder‑only route generator that represents the target, optional constraints, and route in one prompt‑completion sequence. On the RetroCast/PaRoutes mkt‑cnv‑160 benchmark family, one 24‑layer checkpoint follows route‑depth and required‑starting‑material prompts: adding the corresponding prompt fields raises Solv‑0 by 13.7 points for depth constraints and 31.2 points for required‑leaf constraints. Ariadne also improves over DESP, a bidirectional search planner, on required‑leaf Top‑10 and Solv‑0 in 24 GPU‑minutes versus 6.8 GPU‑hours. On standard reconstruction, Ariadne is comparable to DMS Explorer XL at about half the reported inference time. Across additional target‑only benchmarks, Ariadne's clearest gains are on route‑holdout reconstruction, whereas AiZynthFinder MCTS remains stronger on several Solv‑0 comparisons. These results extend sequence generation from specialist retrosynthesis models to prompt‑conditioned structural route generation. We release the codebase and training scripts to support further work, but do not introduce Tier‑1‑‑3 route checkers; those remain the main bottleneck before models of this kind can become useful to experimental chemists.
Authors:Inam Ullah, Imran Razzak, Shoaib Jameel
Abstract:
Automated diabetic retinopathy (DR) grading from colour fundus photographs can achieve strong predictive performance, but clinical interpretation requires more than an image‑level label. It requires understanding how lesion evidence is distributed around retinal vessels and how this evidence relates to quantitative vascular biomarkers. We present a dual‑edge spatial‑Jacobian image graph for interpretable DR grading. Each fundus image is represented as a graph node with four aligned evidence streams: AutoMorph vessel information (X_1), DR‑XAI‑style lesion evidence maps (X_2), a 128‑dimensional lesion‑based contrastive image embedding (X_3), and AutoMorph morphometric biomarkers (X_4). The spatial edge branch (X_12) encodes vessel‑lesion geometry, while the Jacobian branch (X_34) models embedding‑biomarker sensitivity. Lightweight two‑token attention fuses both edge families into a final image graph. On 2,910 matched non‑augmented APTOS images, the full graph achieves 0.8076 accuracy, 0.8312 quadratic weighted kappa, 0.5915 macro‑F1, and 0.9330 adjacent‑grade accuracy; referable DR reaches 0.9055 accuracy and 0.9711 AUROC. The framework is positioned as an explainable representation‑learning tool for lesion‑biomarker hypothesis generation, rather than as a deployment‑ready clinical classifier. The code is available at https://github.com/Inamullah‑Colab/dual‑edge‑dr‑graph‑xai.
Authors:Jin Huang, Yutong Xie, Wanli Song, Xingjian Zhang, Walter Yuan, Matthew O. Jackson, Qiaozhu Mei
Abstract:
Foundation models have been increasingly applied to behavioral science domains such as psychology, sociology, and economics. While these models show promise in individual tasks such as survey response prediction and human‑subject experiment simulation, there remains no systematic understanding of how well they perform across diverse behavioral science tasks, contexts, and populations. We introduce BehaviorBench, a comprehensive benchmark that evaluates foundation models along four core capabilities: (1) behavior prediction and simulation, (2) strategic decision‑making, (3) subject‑trait inference, and (4) behavioral knowledge application. Crucially, BehaviorBench evaluates model outputs at both the individual and distributional levels, capturing not only per‑subject accuracy but also population‑level alignment, an essential requirement for behavioral validity. Leveraging the tasks in BehaviorBench, we further develop Be.FM‑1.5, extending the Be.FM family of behavioral foundation models fine‑tuned on behavioral data. Our results reveal a considerable gap: proprietary general‑purpose models excel at individual‑level prediction and knowledge‑intensive tasks, whereas behavioral foundation models, fine‑tuned on behavioral data, achieve substantially stronger distributional alignment. Notably, Be.FM‑1.5 leads on distributional metrics and remains competitive on individual‑level metrics, suggesting that proper behavioral adaptation can close the gap. Our results highlight the importance of distributional evaluation, establish BehaviorBench as a foundation for developing and assessing behaviorally aligned AI systems, and demonstrate Be.FM‑1.5's potential for a broad range of behavioral science studies. Our BehaviorBench and Be.FM‑1.5 models can be accessed via https://umich‑foreseer.github.io/behaviorbench/.
Authors:Muyuan Zhang, Jiancheng Zhang, Haijin Zeng, Yin-ping Zhao
Abstract:
While Deep Unfolding Networks (DUNs) dominate video Snapshot Compressive Imaging (SCI), they remain constrained by a uniform design philosophy. Existing methods repeatedly stack high‑complexity priors with identical structures, ignoring the fact that optimization trajectories converge toward static states. This results in representation stagnation, where high‑cost computations are wasted on minimal feature updates. To address this inefficiency, we present Differential Unfolding (DU), a heterogeneous framework that replaces uniform repetition with dynamic evolution. Central to DU is the Differential Evolutionary Framework (DEF), which partitions the unfolding process into two complementary roles: structural anchoring and differential evolution. In this scheme, high‑parameter general stages are sparsely deployed to generate high‑fidelity feature foundations. Complementing these, lightweight differential stages employ a Differential Representation Prior (DRP) to propagate and refine these foundational features through a differential mechanism. By integrating Differential Representation Attention (DRA) for evolving attention maps and a Differential Modulated FFN (DM‑FFN) for feature rectification, DRP effectively models cross‑stage variations with minimal overhead. By focusing computational resources on dynamic evolution rather than static redundancy, DU achieves a superior trade‑off between accuracy and efficiency. Extensive experiments verify that our method establishes new state‑of‑the‑art results while significantly slashing computational overhead. https://github.com/Muyuan‑Zhang/DU
Authors:Min Hyeok Bang, Jun Hyeong Kim, Seung-Wook Kim, Se-Ho Lee
Abstract:
In this paper, we present a novel geometry‑aware style transfer framework for 3D Gaussian splatting (3DGS) that simultaneously transfers appearance attributes and geometric structures. Unlike prior works that primarily focus on color‑based stylization and often overlook structural adaptation, our method explicitly incorporates geometry adaptation through a decoupled optimization scheme that alternately updates color and geometry parameters. This strategy alleviates potential interference between color and geometry updates, leading to stable and consistent scene‑level geometry transformation. The decoupled optimization is enabled by the proposed geometry‑aware contrastive feature matching (GCFM). GCFM integrates RGB, depth, and edge cues into a contrastive objective and is employed in both optimization phases to effectively transfer structural characteristics from style images to Gaussian primitives. Extensive experiments show that our approach achieves superior performance in both qualitative fidelity and quantitative metrics, significantly outperforming existing 3DGS‑based stylization methods. Our code is available at \hrefhttps://github.com/oweixx/gasthttps://github.com/oweixx/gast.
Authors:Wonjun Kang, Kevin Galim, Seunghyuk Oh, Minjun Kang, Sanghyun Park, Donghoon Kim, Minjae Lee, Minseo Kim, Rishabh Tiwari, Yuchen Zeng, Hyung Il Koo, Kangwook Lee
Abstract:
On‑policy distillation (OPD) trains a student on its own rollouts guided by teacher feedback and is becoming increasingly important for large language model (LLM) post‑training. Like reinforcement learning (RL), however, OPD faces an on‑policy systems bottleneck, as rollouts can dominate training time for reasoning workloads. Asynchronous training pipelines can alleviate this bottleneck by decoupling rollout generation from learner updates, but doing so introduces stale‑policy data. While prior work has studied stale data in asynchronous RL, its effects in OPD remain underexplored. We present the first systematic study of staleness in asynchronous OPD, focusing on a practical setting where teacher feedback is implemented through local KL losses and full‑vocabulary teacher logits are too expensive to store or transfer, necessitating finite teacher‑score caches. We first show that KL direction changes the stale‑data problem: teacher‑weighted forward KL is more robust to stale rollouts, whereas student‑weighted reverse KL is vulnerable. Second, for this vulnerable reverse‑KL case, we study whether methods designed to stabilize asynchronous RL can mitigate OPD staleness. In our experiments, they do not improve over a simpler OPD‑specific surrogate: recomputing the reverse‑KL signal under the current student at learner time. Third, we analyze how finite teacher‑score caches create a bias‑variance tradeoff for sparse and sampled reverse‑KL OPD estimators. This motivates multi‑sample Monte Carlo (MC), which preserves MC correctability while reducing one‑sample variance. Finally, we present and open‑source AsyncOPD, a fully asynchronous OPD training pipeline built from these estimator choices. Experiments show that AsyncOPD improves training throughput by 1.6× to 3.8× over strict synchronous training while reaching comparable accuracy.
Authors:Tongyan Hua, Dongli Wu, Jinjing Zhu, Yinrui Ren, Zhongcheng Hong, Ying-Cong Chen, Hui Xiong, Wufan Zhao
Abstract:
Generating explicit 3D city assets from a single satellite image is important for digital twins, urban simulation, and geospatial intelligence. Unlike satellite‑to‑street‑view synthesis, the task requires a reusable textured mesh with plausible geometry and controllable appearance rather than a 3D proxy optimized only for rendering a small set of images or videos. The ICCV Sat2City framework made a first step by conditioning cascaded sparse‑voxel latent diffusion on satellite‑derived height maps, but its appearance was random, its training data were synthetic, and its task‑specific VAE did not scale well to noisy real‑world reconstructions. We present Sat2City v2, a journal extension that adapts a pretrained native structured‑latent 3D foundation model to weakly aligned satellite images and textured meshes. We build a real‑world dataset with 16,241 satellite‑mesh pairs across 24 regions in 9 cities. Instead of learning a 3D representation from noisy city meshes, Sat2City v2 encodes each mesh into a pretrained native 3D latent space, fine‑tunes a satellite‑conditioned geometry flow, and uses the decoded shape to anchor satellite‑conditioned texturing. This retains Sat2City's geometry‑to‑appearance cascade while enabling appearance‑controllable generation from the satellite input. Experiments on metric‑scale DSM reconstruction and generative city‑asset benchmarks for geometry and appearance show that Sat2City v2 achieves the best overall performance among evaluated baselines. Overall, Sat2City v2 advances satellite‑to‑city generation from rendering‑oriented 3D proxies to explicit textured mesh assets, supported by, to the best of our knowledge, the first documented satellite‑mesh paired dataset collected from matched geographic crops for this asset‑level task. Project page: https://ai4city‑hkust.github.io/Sat2City‑v2/
Authors:Chenhao Dang, Jing Ma, Mingjie Liao
Abstract:
The composition of training data, governed by the diversity of sources and their mixing strategy, is a cornerstone of Large Language Model (LLM) pre‑training. Online Data Mixing (ODM), the technique of adaptively adjusting data mixtures during training, has emerged as a promising direction to improve efficiency. However, existing methods are constrained by their reliance on a singular optimization perspective, which fundamentally overlooks the need for complex LLM pre‑training to consider the dynamic data composition from multiple dimensions. To overcome this limitation, we introduce the Holistic Data Scheduler (HDS), a novel online data mixing framework. HDS formulates the data scheduling challenge as a reinforcement learning problem in a continuous control space and leverages the Soft Actor‑Critic (SAC) algorithm for its stability and sample efficiency in exploring the high‑dimensional policy space. At the core of HDS lies a novel multi‑objective, holistic reward function that integrates three critical perspectives: a data‑driven reward for quality, a loss‑driven reward capturing inter‑domain influence, and a model‑driven reward based on weight norms. To validate our design and determine its optimal configuration, we conducted systematic experiments on LLMs of various sizes. On The Pile benchmark, HDS reaches the final validation perplexity of the next best method with 44% fewer training iterations. Furthermore, it achieves a 7.2% improvement on the MMLU 0‑shot task along with consistent gains on other benchmarks, showcasing its ability to enhance both training efficiency and final model capability.
Authors:Lucky Verma, Pratik Yadav
Abstract:
Discrete diffusion language model (DLM) fine‑tuning inherits inexpensive diagnostics from denoising‑time confidence monitors, but their PEFT‑training meaning is untested. We test top‑1 argmax concentration as a collapse warning. Across 816 LoRA/PEFT configurations from three DLM families, the warning fires for every configuration while logs record 0/816 actual collapses at the 200 step horizon, giving zero precision. The cause is pre‑equilibrium saturation: top‑1 concentration is already high before optimization and quickly becomes insensitive to final training stability. We then evaluate max LoRA gradient norm, a parameter‑side signal that samples gradient routing rather than token concentration. On a pooled held‑out LLaDA‑family split, a train‑optimized threshold identifies top‑decile final‑loss configurations with precision 0.68 and F1=0.79, above the all‑positive top‑1 baseline even at the lower split‑bootstrap confidence bound. Autoregressive controls and cross‑family threshold failures bound the result to short‑horizon DLM‑LoRA inspection rather than a universal collapse detector. Workflow: drop top‑1 as a PEFT alarm, log max‑gradient early in training, and calibrate thresholds per DLM family before routing runs for inspection.
Authors:Feihong Nan, Zhengyi Zhong, Pan Wang, Weidong Bao, Xiongtao Zhang, Quan Wen, Ji Wang
Abstract:
Federated unlearning (FU) is critical for complying with legal mandates like the right to be forgotten in decentralized systems, yet current methods face a persistent dilemma between non‑target knowledge loss and high request latency. To resolve these issues, we propose FedUP, a one‑shot federated unlearning framework utilizing lightweight pluggable filters that act as a "knowledge funnel" to screen out target data while preserving original model performance. By freezing original model parameters and training filters at the server side using differentially private (DP)‑protected class centroid samples, FedUP bypasses the need for multi‑round client‑server communication and complex retraining, reducing unlearning latency from minutes to mere seconds. Additionally, the framework's pluggable architecture ensures inherent reversibility, enabling the seamless restoration of forgotten knowledge by simply removing the filters. Extensive experiments on diverse image and text tasks demonstrate that FedUP effectively reduces non‑target knowledge loss and achieves superior unlearning precision and efficiency across various scenarios. Code is available at: https://github.com/suows/FedUP‑code.
Authors:Chenhao Dang, Dantong Zhu, Jun Yang, Conghui He, Weijia Li
Abstract:
Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text‑‑image framing errors. Existing benchmarks and methods remain poorly matched to this setting: they usually isolate short captions, single images, binary labels, or one manipulation source, while agentic verification remains costly under realistic evidence search. We present ReMMD, a realistic multilingual multi‑image agentic verification framework for multimodal misinformation detection. ReMMD includes ReMMDBench, a real‑world multimodal misinformation detection benchmark with 500 samples, 2,756 images, five monolingual languages, two cross‑lingual settings, three text‑length tiers, multi‑image posts, five‑way veracity labels, eight distortion labels, evidence provenance, and rationales. It also includes ReMMD‑Agent, a persistent‑memory verifier that decomposes posts into atomic points, builds a reusable evidence set, and predicts structured L1/L2/L3 outputs. Across proprietary systems, open LVLMs, MMD‑Agent, and T2‑Agent, ReMMD‑Agent obtains the best five‑way veracity performance, with 41.80% accuracy and 39.12% macro‑F1 using GPT‑5.2, while reducing cost by 17.5% relative to MMD‑Agent and 79.9% relative to T2‑Agent. The project is available at https://dang‑ai.github.io/ReMMD.
Authors:Hengji Zhou, Sijie Liu, Jianrun Chen, Xingchen Zou, Lianghao Xia, Liqiang Nie
Abstract:
Short dramas, with their rapid shot rhythms, dialogue‑driven focus shifts, and demanding cinematographic grounding, pose challenges that prompt‑level or text‑only video generation pipelines struggle to meet. We study plot‑to‑short‑drama generation, where a global plot and local context are transformed into visually grounded multi‑shot videos. We propose DramaDirector, a geometry‑grounded framework that lets the planner borrow cinematographic geometry from a gallery of real short‑drama shots indexed by depth and pose. DramaDirector decouples each shot into static visual and dynamic narrative conditions, trains the planner with schema‑constrained SFT and GRPO under a learned text‑visual alignment reward, and retrieves depth‑pose references to guide first‑frame generation and image‑to‑video synthesis. We also introduce DramaBoard, a benchmark built from 35 live‑action dramas, 2.8K episodes, and 81K shots, with structured storyboards and multi‑dimensional evaluation protocols. Experiments show that DramaDirector improves over representative multi‑agent and video generation baselines on faithfulness, consistency, and controllability. Our code is released at: https://github.com/iLearn‑Lab/DramaDirector
Authors:Tian Qiu, Jifeng Shen, Xin Zuo
Abstract:
Effective cross‑modal feature alignment and interaction are central challenges in multispectral object detection. Although global cross‑attention provides strong long‑range modeling ability, its quadratic complexity with respect to feature size limits deployment on resource‑constrained platforms. We therefore propose Progressive Pixel‑Neighborhood Deformable Cross‑Attention for multispectral feature fusion, termed PNAFusion. The proposed framework is motivated by two observations: weak misalignment between visible and thermal images is usually concentrated around local neighborhoods, and semantic correspondence across modalities often follows non‑linear spatial mappings that fixed receptive fields cannot model well. To address these issues, PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and alignment on the most relevant neighborhoods. Specifically, a Pixel‑Neighborhood Cross‑Attention (PNCA) module is introduced to avoid redundant global feature matching and suppress background noise. Meanwhile, an Adaptive Deformable Alignment (ADA) module captures non‑linear spatial correspondences through learned pixel‑wise offsets. These components are further integrated through an iterative feedback mechanism to progressively refine cross‑modal feature alignment. Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 mAP@0.5, respectively, under the YOLOv5 detector, and further reaches 86.8 mAP@0.5 on FLIR and 90.8 mAP@0.5 on M3FD when transferred to Co‑DETR. Efficiency analysis indicates that PNAFusion reduces allocated GPU memory by 33.0% compared with ICAFusion and reduces theoretical FLOPs from 194.8 G to 156.4 G, although the deformable sampling and iterative refinement introduce additional latency. Our code will be available at https://github.com/DanielQiuTian/PNAFusion.
Authors:Wenhao Gao, Yifan Wang, Yijia Ma, Carl Yang, Wen Li, Chenyu You
Abstract:
Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging. EEG provides a weak, spatially diffuse, and highly variable measurement of distributed cortical activity, whereas speech is organized as a coherent acoustic trajectory with strong harmonic and temporal structure. The resulting mismatch makes waveform regression unstable and causes stochastic multi‑step generation to be sensitive to artifact‑dependent conditioning and subject variability. We introduce NeuroSonic, a conditional flow‑matching framework for EEG‑to‑speech reconstruction. Instead of predicting waveforms directly or refining them through stochastic denoising, NeuroSonic learns a deterministic probability‑flow velocity field that transports a noise‑corrupted acoustic state toward clean speech under EEG conditioning. EEG and audio are embedded into a shared token space and processed by a time‑conditioned gated Transformer that parameterizes the transport ordinary differential equation. This formulation models trajectory evolution explicitly while avoiding iterative stochastic sampling. We evaluate NeuroSonic on the CineBrain and EAV benchmarks under cross‑subject evaluation. Across both datasets, the proposed method improves distributional realism, spectral fidelity, and perceptual quality over representative GAN‑, diffusion‑, and mean‑flow baselines, with up to a 26.3% gain in overall perceptual quality. The performance gap is most evident in artifact‑heavy segments, where conditioning variability is strongest. These findings indicate that deterministic conditional transport provides a stable and effective formulation for EEG‑driven speech reconstruction. Code is available at https://github.com/Y‑Research‑SBU/NeuroSonic/ .
Authors:Morayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt
Abstract:
"Talk short. Drop grammar. Save token." This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything depends on which channel (the user's prompt or the model's response) is being compressed. We present Cavewoman, a two‑channel evaluation protocol that scores every generation on task accuracy, realized per‑item cost, and reference‑text agreement against the model's unconstrained reference. We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items. Output compression cuts realized cost on most API models (1.4‑2.4x per model, up to 3x in the best case) and on all four open‑weight models under public‑tier pricing. Input compression has the opposite effect, a strict lose‑lose: it raises net cost rather than lowering it (~1.15x on the five‑benchmark mean, up to 1.8x on the worst dataset and 2.7x under stronger compression), because models compensate with longer responses even as accuracy collapses. Under the same setting, surface text diverges from the unconstrained reference: on the non‑reasoning models, roughly half of all generations are correct yet their surface text no longer entails the model's own unconstrained baseline generation. The divergence survives length‑controlled re‑scoring, multiple‑comparisons correction, and replication under complementary semantic measures. Code and data are available at https://github.com/danielle34/cavewoman.
Authors:Gabriel Bravo-Palacios, Jianghan Zhang, Zachary Pestrikov, Brian Plancher, Thomas Lew
Abstract:
Robotics increasingly relies on GPUs for parallel simulation, large‑scale learning, and neural‑network inference. For model predictive control (MPC) to scale with this paradigm, solvers must run efficiently on this hardware while remaining fast, differentiable, and compatible with expressive MPC formulations used in robotics. We present TurboMPC, a differentiable MPC solver that runs entirely on the GPU and supports state and control inequality constraints, implicit integrators, cross‑time‑coupled costs, and slack variables. TurboMPC combines sequential quadratic programming (SQP), an alternating direction method of multipliers (ADMM) inner solver, implicit differentiation, and a co‑designed JAX‑CUDA implementation for efficiency and ease of use. In simulation, we validate TurboMPC on constrained planning, humanoid imitation learning, and reinforcement learning with neural‑network cost function tasks, achieving up to 15× and 58× speedups over state‑of‑the‑art CPU and GPU differentiable solvers, respectively. We deploy TurboMPC on a full‑scale car for minimum‑time racing and find that batched, GPU‑accelerated tuning of MPC parameters via Bayesian optimization yields significantly faster driving than a hand‑tuned baseline. TurboMPC also scales to planning horizons of over 8000 knot points while maintaining control of the vehicle. We open‑source TurboMPC at: https://github.com/ToyotaResearchInstitute/turbompc
Authors:Yujia Chen, Bowen Weng
Abstract:
This note describes an integration of the sim‑to‑real performance estimate with betting (from Chen et al.) and the safe anytime‑valid inference (from Ramdas et al.). Using the scaled simulators. The method produces efficient, reliable certificates for the mean estimate, an approach that is especially valuable in robot performance testing. This note gives a primary, self‑contained account of the construction; preliminaries of the respective methods are kept at a minimum, and one shall refer to the original works for full detail. Some synthetic examples demonstrating the proposed algorithm can be found at https://github.com/ISUSAIL/Bet4Sim2Real‑EProcess.
Authors:Fengfeng Liang, Yuechen Zhang, Jiaya Jia
Abstract:
Existing low‑bit KV‑cache quantizers often treat each cached key as a flat vector. Under RoPE, however, a key's contribution to a future attention logit decomposes into a position‑dependent sum over two‑dimensional frequency blocks. This makes key‑cache quantization a block‑wise bit‑allocation problem: high‑energy RoPE blocks are more sensitive to quantization error and should receive more bits. We introduce Block‑GTQ, a RoPE‑aware bit allocator for key‑cache quantization built on TurboQuant‑MSE(TQ‑MSE). For each layer and KV head, Block‑GTQ computes a label‑free energy score for each RoPE block and greedily allocates integer bit widths by marginal gain. Under matched K/V bit budgets, Block‑GTQ better preserves RoPE query‑key logits on a ten‑model diagnostic panel, cutting per‑layer MAE by 32‑80% at 2 and 3 b/dim K‑only quantization and winning all 367/367 layer comparisons against uniform TQ‑MSE. These fidelity gains translate to stronger downstream long‑context retrieval, understanding, and reasoning. At K2V2 on Llama‑3.1‑8B‑Instruct, Block‑GTQ raises the six‑task NIAH average from 70.6 to 97.4, and the LongBench‑EN average from 36.87 to 53.31. On AIME 2024/2025 with DeepSeek‑R1‑Distill‑Qwen‑7B, without an fp16 recent‑key buffer, Block‑GTQ at K3V2 scores 51.7/37.5, close to fp16's 54.2/37.9, whereas uniform TQ‑MSE collapses to 0.0/0.0. We further implement a packed‑cache serving path. On a single H800 GPU with Qwen2.5‑3B‑Instruct, packed K3V3 achieves 3.24x KV‑cache compression with fp16‑comparable quality, runs 1.34x faster than fp16 FlashAttention2 at 128K context, reduces peak memory from 56.31 GB to 19.85 GB, and remains feasible at 256K and 512K where fp16 OOMs. Code is available at https://github.com/JIA‑Lab‑research/blockgtq.
Authors:Zixin Ding, Shaghayegh Emami, Giovanna Salvi, Cecilia Tosciri, Abhijith Gandrakota, Jennifer Ngadiuba, Nhan Tran, Christian Herwig, David W. Miller, Yuxin Chen
Abstract:
High‑throughput scientific facilities such as the Large Hadron Collider depend on real‑time event filtering (triggering) under tight constraints on bandwidth, latency, and storage. In practice, trigger menus are largely static and hand‑tuned and can become suboptimal as detector conditions, pileup, and background composition drift over time. We cast online threshold tuning as a sequential decision‑making problem: a reinforcement learning agent ingests streaming summaries of recent rates and signal‑sensitive features and updates trigger thresholds to maximize signal efficiency while tracking a target background rate within a tolerance band. We adapt Group‑Filtered Policy Optimization (GFPO) to streaming control and introduce two variants (GFPO‑F, GFPO‑FR) that enforce background rate feasibility during training. On a benchmark that emulates realistic collider operation, we study two representative triggers: a total transverse energy (H_T) trigger sensitive to pileup variation, and an anomaly‑detection (AD) trigger based on reconstruction loss for rare or non‑standard signatures. On Monte Carlo streams, our agent increases the fraction of in‑tolerance time intervals by 48% (H_T) and 28% (AD), with a cumulative gain of up to 2% in signal efficiency on those in‑tolerance intervals. Transferring from simulation to \emphreal collision data (CMS Run 283408), the same agent, without fine‑tuning, achieves a 56% (H_T) and 28% (AD) in‑tolerance improvement over baselines, with further signal‑efficiency gain on both triggers. To our knowledge, this is the \emphfirst demonstration of RL‑based trigger control on real Large Hadron Collider collision data. Code is available at https://github.com/Zixind/GFPO_LHC (see repo for details).
Authors:Amirhossein Kardoost, Lion Gleiter, Tingying Peng, Carsten Marr
Abstract:
Self‑supervised learning in fluorescence microscopy often relies on 2D projections, despite the inherently three‑dimensional nature of cells. We present a systematic comparison of 2D and 3D masked autoencoders (MAE‑2D vs. MAE‑3D) on volumetric microscopy data. Under matched architectures and training protocols, MAE‑3D consistently outperforms 2D max‑projection and slice‑based variants on downstream single‑cell tasks. We further align visual representations with a pretrained protein language model (ESM2) and show that cross‑modal supervision yields larger gains for volumetric models. Channel cross‑attention and frequency‑domain regularization are critical for leveraging 3D spatial context. On a protein‑‑protein interaction task, MAE‑3D achieves a ROC‑‑AUC of 0.865, outperforming prior methods by up to +0.025. For protein localization, our best 3D model attains state‑of‑the‑art AUC_\textmicro (0.952) and F1_\textmicro (0.742), improving over previous approaches by +0.003 and +0.010 absolute, respectively. Overall, these results demonstrate the advantages of native 3D modeling and multimodal alignment for representation learning in single‑cell microscopy.
Authors:Mohammed Nagdi, Evangelos-Marios Nikolados, Alexey Yermakov, Mars Gao, Nathan Kutz, Filippo Menolascina
Abstract:
Learning Koopman operators with autoencoders enables linear prediction in a latent space, but long‑horizon rollouts often drift off the learned manifold, leading to phase and amplitude errors on systems with switching, continuous spectra, or strong transients. We introduce two complementary components that make Koopman predictors more robust. First, we add an attention‑free latent memory (AFT) block that aggregates a short window of past latents to produce a corrected latent before each Koopman update. Unlike multi‑head attention, AFT operates in linear time and adds only \approx30k parameters (3d^2 + T^2, fewer than matched multi‑head attention), yet captures the local temporal context needed to suppress error divergence. Second, we propose dynamic re‑encoding: lightweight, online change‑point triggers (EWMA, CUSUM, and sequential two‑sample tests) that detect latent drift and project predictions back onto the autoencoder manifold. Across three benchmark systems ‑‑ Duffing oscillator, Repressilator, IRMA ‑‑ our model consistently reduces error accumulation compared to a Koopman autoencoder and matched‑capacity multi‑head attention. We also compare against GRU and Transformer autoencoders, evaluated both from initial conditions and with a 50‑step context, and find that Koopman+AFT (with optional re‑encoding) attains markedly lower long‑horizon error while maintaining lower inference latency. We report improvements over horizons up to 1000 steps, together with ablations over trigger policies. The result is a fast, compact predictor that stays on the learned manifold over long horizons.
Authors:Qian Wang, Zhenyu Li, Abdelrahman Eldesokey, Peter Wonka
Abstract:
Subject‑driven image generation faces an "Identity‑Diversity Paradox", where strong identity preservation often leads to rigid and low‑diversity outputs. We propose a post‑training framework called DivRL that jointly optimizes identity consistency and structural diversity simultaneously by leveraging disentangled visual features from a robust similarity model. Specifically, we introduce a Negative Self‑Similarity Measure (nSSM) to quantify structural diversity, and Visual Semantic Matching (VSM) to evaluate identity consistency. We propose an "Explore‑and‑Suppress" strategy that treats VSM as a gated constraint: the model freely explores structurally diverse configurations, and only samples that violate the identity threshold are penalized via a quadratic hinge loss. This converts identity preservation from a competing objective into a feasibility constraint, allowing nSSM and VSM to improve jointly. Experiments demonstrate that our method effectively pushes the model to generate both consistent and diverse images and improves structural diversity while maintaining comparable identity consistency through a gated optimization formulation.
Authors:Sudarshan Srinivasan, Deepak Gangadharan, Dip Goswami
Abstract:
Memory bandwidth contention in multi‑core systems severely impacts application performance and quality‑of‑service (QoS) guarantees. Regulating the shared memory bandwidth mitigates the memory performance uncertainty thereby making it a manageable resource and improving trustworthiness of multi‑core systems. In this work we propose a memory bandwidth regulation mechanism LMS‑AR, i.e., LMS Prediction‑based Adaptive Regulator within a Linux kernel module to distribute the memory bandwidth as a resource among the CPU cores. We describe a design in which both monitoring and regulation is enforced from outside by a master core ‑ which is not a dedicated controller for regulation. This allows for plugging in computationally heavy prediction and regulation algorithms without interfering with the regulated core. An adaptive filtering technique was employed for prediction of per‑core bandwidth requirement. We conducted several experiments with SPEC CPU 2017 benchmarks distributed across multiple cores. Our proposed approach demonstrated significant improvement over Memguard with respect to slowdown ratios caused due to memory contention. Our solution is hosted publicly at \hrefhttps://github.com/ss22ongithub/LMSAdaptiveRegulatorhttps://github.com/ss22ongithub/LMSAdaptiveRegulator.
Authors:Xiangbo Gao, Xiukun Huang, Boyu Lu, Junge Zhang, Mengjie Mao, Jiachen Li, Wei Xiong, Zhengzhong Tu
Abstract:
Driving VLA models incorporating Chain‑of‑Thought (CoT) reasoning are attractive because they leverage pretrained VLM representations and expose intermediate decisions in natural language, yet current rationales often lack the step‑by‑step decision semantics needed to keep the rationale causally connected to the planned motion. We introduce Neuro‑Symbolic Drive, a neuro‑symbolic driving framework that supervises a driving VLA with rule‑grounded reasoning traces extracted directly from classical rule‑based planners. Our key observation is that rule‑based planners are symbolic AI systems that already function as executable reasoning engines: they reason about active safety constraints, search over candidate maneuvers, and select a final trajectory. We instrument these planners in simulation to capture both the executed trajectory and the internal decision trace at each rule‑evaluation step. Each trace is serialized into structured rule‑grounded reasoning and paired with the trajectory to fine‑tune Qwen3.5‑4B as a driving VLA. Because these traces are derived directly from the planner states that determine the action, they ensure reasoning is structurally coupled to motion generation by construction, rather than by post‑hoc alignment. On our simulator‑generated benchmark, detailed rule‑grounded reasoning reduces ADE@3s from 0.47 to 0.26 and miss rate from 8.30% to 6.40% under three‑camera perception, and from 0.54 to 0.26 and 10.13% to 5.99% under eight‑camera perception. Neuro‑Symbolic Drive thus converts neuro‑symbolic planning logic into structured supervision. Code base: https://github.com/XiangboGaoBarry/Neural‑Symbolic‑Drive.
Authors:Yifei Zhao, Qian Lou, Mengxin Zheng
Abstract:
Vision‑language models (VLMs) are increasingly used as perception‑reasoning backbones for embodied intelligence in safety‑critical physical systems, where perception or reasoning errors can lead to unsafe decisions or actions. Although many red‑teaming methods have been developed to probe VLM vulnerabilities, their evaluation remains fragmented across datasets, metrics, and threat models, making direct comparison difficult and obscuring whether observed differences arise from stronger attacks, more vulnerable models, or incompatible evaluation settings. Existing chatbot‑centric red‑teaming benchmarks mainly standardize jailbreak and content‑safety evaluation, but they do not systematically capture physically grounded functional failures or cover red‑teaming methods that target physical‑world VLMs. This raises the key challenge of comparing diverse attack methods under a unified protocol while targeting the same scenario‑specific failures. We introduce REALM, to our knowledge the first unified red‑teaming benchmark for physical‑world VLMs. REALM integrates 12 red‑teaming methods, 3 model‑agnostic defenses, and 13 VLMs under a practical black‑box threat model with shared datasets and metrics. To align adversarial objectives across attack families, REALM introduces an agentic target‑generation pipeline that constructs shared, scenario‑specific, and physically grounded attack objectives for each scene, enabling fair comparison of diverse red‑teaming methods under aligned adversarial goals. Our evaluation shows that text and typographic injection attacks induce the most failures, multimodal co‑optimization yields the strongest visual‑perturbation transfer, single‑pass attacks approach iterative methods at much lower cost, and model scale alone does not confer adversarial robustness. Code is available at https://github.com/UCF‑ML‑Research/REALM.
Authors:Qian Ma, Qiong Wu, Zhengyi Zhou, Yao Ma
Abstract:
Knowledge‑Based Visual Question Answering (KB‑VQA) requires grounding visual queries to external knowledge beyond directly observable content in images. While recent multi modal large language models (MLLMs) show strong perceptual abilities, they struggle on KB‑VQA tasks requiring groundings from both fine‑grained entity and evidence levels. Most existing multi‑modal retrieval augmented generation (MM‑RAG) methods tightly couple entity discrimination and section‑level evidence ranking into a single re‑ranking stage, leading to high cost and limited generalization. In this work, we revisit existing MM‑RAG solutions from a workflow perspective and argue both entity‑level and fact‑level groundings are key bottlenecks. We observe that although MLLMs often fail under open‑ended entity naming, they can better identify the correct entity when selecting from a small set of candidate names. Based on this insight, we propose a simple and training‑free identify‑before‑answer IBA framework that decouples entity identification from section‑level re‑ranking. Our approach prompts an MLLM to select high‑confidence entities using only candidate names, followed by an off‑the‑shelf textual re‑ranker for evidence selection. Experiments on Encyclopedic‑VQA and InfoSeek show that our method consistently outperforms fine‑tuned multi‑modal re‑ranking baselines while reducing training and inference complexity. Additional analyses reveal that the improvements arise not only from better entity identification, but also from selecting more informative evidence once correct entity is fixed. Our implementation is made public to ease reproducibility.
Authors:William Guey, Pierrick Bougault, Wei Zhang, Vitor D. de Moura, José O. Gomes
Abstract:
Who sets the rules for artificial intelligence, and on what terms, has become a defining question of global governance. For several years that contest ran through principles and ethics codes; it now runs through institutions. China's proposed World Artificial Intelligence Cooperation Organization (WAICO) is the most consequential recent entrant and the least examined. We place WAICO within the emerging regime complex for AI and argue that its importance lies not in any single commitment but in the position it is designed to hold. Coding a cross‑section of fifteen international AI governance instruments and institutions on how they admit members, how they are organized, and what they prioritize, we find that WAICO's proposed design joins three features that no constituted multilateral body currently combines: membership open to any sovereign state, no values or regime‑type test for entry, and an agenda built around development and the global capability divide. The incumbent Western‑led bodies gate membership by shared values and concentrate on rights and safety; the universal United Nations bodies are open but anchored in human rights; a development‑first agenda is otherwise carried by the regional strategies of the Global South. Among constituted institutions, the only occupant of WAICO's intended position is China's own 2023 precursor initiative. We read this as the formation of a second, still‑proposed pole in global AI governance, organized around sovereignty and development rather than rights and safety, and argue that WAICO would be the first standing organization built to anchor it. We report the full coding, state testable expectations against which the claim can be judged as the organization takes shape, and release the dataset for replication.
Authors:Bin Qiu, Yanming Shao, Guanyu Cai, Yao Mu
Abstract:
Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high‑DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose Adversarial Posture Regularization (APR). It avoids expensive, song‑aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human‑like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer‑grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human‑likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: https://github.com/APRProject/APRPianist.
Authors:T. Lucas Makinen, Deaglan J. Bartlett, Niall Jeffrey, Benjamin D. Wandelt
Abstract:
When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish. Degeneracies render both label prediction and inverse problems difficult, since both machine learning algorithms and probabilistic samplers rely on the distinguishability of data and its gradients with respect to parameters. However, identifying degeneracies in physical models or real‑world datasets can be elucidating about the choice of model or the underlying process that produces the data. We present the degeneracy distillery, a method that (1) detects and (2) resolves degenerate parameter combinations (a) automatically and (b) symbolically, from parameter‑data (or parameter‑simulation) pairs alone, through estimation and flattening of the Fisher information matrix. By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model, requiring no realised data observation. We demonstrate our approach on a range of synthetic and real‑world problems, discovering symbolic coordinate transformations that identify the combinations of parameters of a model which yield independent effects on the data. The resulting coordinates flatten the Fisher information in expectation globally, in contrast to posterior‑based methods that flatten only at a single point, and substantially reduce the simulation budget required for downstream neural posterior estimation. In test cases we require up to 10× fewer simulations for posterior estimation at matched validation calibration whilst simultaneously gaining physical insight on the system.
Authors:Anindya Mondal, Sauradip Nag, Anjan Dutta
Abstract:
ABACUS is a unified vision‑language model that handles object counting, crowd counting, referring‑expression counting, and count‑faithful image generation without any benchmark‑specific training required. Our model is built on existing 3B‑parameter unified foundation model and is adapted for object localization tasks using three key innovations: density‑aware adaptive zooming with objectness maps for spatial grounding; a boundary‑aware count policy via GRPO to eliminate crop‑boundary errors; and a cycle‑consistent GRPO strategy where the understanding branch self‑critiques generated outputs, closing the understanding‑generation gap without any external annotations. ABACUS achieves state‑of‑the‑art results across seven benchmarks, outperforming both task‑specific specialists and larger generalist models.
Authors:Yiming Wang, Arthur N. Montanari, Adilson E. Motter
Abstract:
The 2025 Iberian blackout has renewed concerns about the resilience of power grids with high shares of renewable generation. This commentary argues that renewable generation can not only advance decarbonization but also strengthen grid stability through synthetic inertia, advanced inverter‑based control, and coordinated transmission planning. Rapid advances in energy storage and power electronics make this transition increasingly viable.
Authors:Wietse Stienstra
Abstract:
Headline accuracies on the Tuebingen cause‑effect pairs are routinely compared across papers even though each is measured under its authors' own protocol ‑‑ different pair subsets, weightings, model‑selection, and decision rates. We argue this is the wrong comparison and run the right one: a same‑hands re‑evaluation in which every method is run by us on the identical 102 pairs, with one strict rule ‑‑ no tuning and a decision forced on every pair. As a clean reference point we introduce a deliberately minimal baseline: sorted‑conditional compression, which feeds quantized, sorted, first‑differenced data to an off‑the‑shelf compressor (bz2) and has zero fitted parameters. Under the common ruler the ranking differs sharply from the literature. Our baseline reaches 74.7% weighted accuracy (p = 3.7e‑7); on the same 100 pairs that SLOPE is evaluated on it scores 76.0%, a 1.2‑point gap below the authors' own forced‑decision SLOPE (77.2%) that is well inside noise (McNemar p = 0.39). A faithful re‑run of RECI lands at 70.7% ‑‑ inside the original authors' reported error bar, not the 77.5% often quoted (which we trace to a mis‑copied cell). SLOPE's published 82.4% is a decided‑subset figure: scoring the authors' own stored output only on the pairs its significance test chose to answer reproduces 81.7%. Under the common ruler the methods cluster in the low‑to‑mid 70s and the zero‑parameter compressor ties the strongest of them. We document the mechanisms that inflate published figures (test‑set model selection, significance‑gated abstention) and contribute two further results: compression score magnitude is a model‑free confounding flag (p = 2.8e‑68), and a pre‑registered falsification test fails in an instructive way that bounds the method's theoretical interpretation. Code, pre‑registrations, and per‑pair outputs are released.
Authors:Yihan Wang, Cheng Liu, Jiazheng Zhang, Lei Zhang, Long Cheng, Xiaowei Li, Huawei Li
Abstract:
Verilog debugging remains one of the most time‑consuming stages in digital circuit design. Recent advances in Large Language Models (LLMs) have enabled automated debugging; however, most existing approaches rely solely on test outputs and compiler feedback in an end‑to‑end manner, limiting their effectiveness on complex bugs. A key challenge is that the root cause of an error may be far removed from its observable outputs, making it difficult for LLMs to trace long dependency chains in code. This challenge is further exacerbated in large codebases, where long context lengths hinder efficient reasoning. To address these limitations, we propose VeriPilot, an LLM‑powered debugging framework that leverages golden reference models to enable fine‑grained bug localization and repair. VeriPilot goes beyond output‑level comparison by aligning internal variable semantics between the Verilog design and its corresponding golden model through LLM‑based analysis. It then performs step‑by‑step signal tracing using Control‑Data‑Flow Graphs (CDFGs) derived from static analysis, identifying a minimal set of suspicious code regions along with their correct counterparts from the golden model. These structured insights are subsequently provided to the LLM to guide reasoning and automated code repair. Experimental results on the Comprehensive Verilog Design Problems (CVDP) benchmark from NVIDIA demonstrate that VeriPilot improves the repair success rate of GPT‑4o from 54.3% to 85.71%, significantly enhancing both bug localization accuracy and repair effectiveness for complex Verilog designs. The source code and benchmark are publicly available at Github https://github.com/YihanWn/VeriPilot.git.
Authors:Vinamra Singh
Abstract:
CABAC, the entropy coder of H.264/AVC and the basis for HEVC and VVC, decomposes multi‑symbol values into bins via a binarization scheme before a binary arithmetic coder. H.264 uses Truncated Unary plus k‑th order Exp‑Golomb (UEG); alternatives include canonical Huffman and the entropy‑conserving binarization (ECB), which provably preserves entropy mapping m‑ary data to m‑1 binary strings but has not been evaluated inside a production binary arithmetic coder. We integrate ECB into a from‑scratch CABAC implementation alongside UEG, single‑context Huffman, and a Huffman variant with per‑bin‑position contexts (HuffmanPos), all sharing one M‑coder backend. We benchmark all four on synthetic sources, DCT residuals from a procedural image, and the full 24‑image Kodak suite (2,480 round‑trip trials, bit‑exact verified). On the procedural image, a sparsity‑driven crossover at Q=8 lets ECB overtake single‑context Huffman, reaching 27 percentage points below at Q=32. On Kodak the crossover shifts below the tested range and ECB beats single‑context Huffman at every Q, the gap growing from 0.031 to 0.113 bits per symbol. HuffmanPos, sharing Huffman's codewords but allocating one context per bin position, beats ECB on 12 of 15 source cells and loses by at most 0.56 percentage points on the other three, despite the same per‑symbol bin count as single‑context Huffman. This isolates the dominant mechanism: at low source entropy the rate gap is driven primarily by context allocation over the bin stream, not the binarization's per‑symbol bin count. ECB's rate efficiency costs 7 to 10x in decoder latency on large alphabets, traced to an O(Nm) decoder; we sketch an interleaved single‑pass variant that would close this gap. Code, benchmarks, and raw data are open source.
Authors:Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov
Abstract:
We present an automated large‑scale search pipeline for heterogeneous 4‑Expert Mixture‑of‑Experts (MoE4) architectures within the LEMUR neural network dataset ecosystem. Building on a hand‑crafted heterogeneous MoE reference model, we replace manual design with a deterministic code‑assembly generator that systematically combines base architecture families drawn from the LEMUR database into MoE4 ensembles, each governed by a convolutional gating network with temperature scaling, mixup augmentation, and cosine‑annealed learning rate scheduling. Over a 28‑day campaign on an NVIDIA RTX 4090, the pipeline generated 4,463 candidate models across 197 batches, of which 1,021 were evaluated successfully. A critical finding emerged from the campaign: due to alphabetical enumeration via itertools.combinations, the entire explored search space (4.8% of the theoretical 23,751 possible 4‑family combinations) is anchored to a single family, AirNet. We characterise this coverage bias precisely, identify the root cause in the generator, and propose a stratified random sampling fix. Within the AirNet anchored scope, ShuffleNet and MobileNetV3 consistently co‑produce the highest‑accuracy ensembles (mean accuracy up to 0.632), while FractalNet and MNASNet are identified as low‑yield families warranting exclusion in future campaigns. The pipeline, analysis artefacts, and corrected generator are released as part of the open‑source NNGPT project at https://github.com/ABrain‑One/nn‑gpt
Authors:Ammar Daskin
Abstract:
We establish a direct quantum‑classical duality based on the degree‑2 Sum‑of‑Squares (SoS) semidefinite programming cone: the matrix of two‑qubit Pauli‑Z correlation functions obtained from \emphany quantum state ρ is automatically a feasible point of the classical Goemans‑Williamson (GW) relaxation. This observation provides a universal ``safety net'' for quantum optimization algorithms: applying GW random hyperplane rounding to the quantum‑driven moment matrix yields a certified expected cut value \mathbbE[\mathrmCut] \ge α_\mathrmGW\langle\mathcalH\rangle_ρ, valid for every state produced by variational algorithms such as QAOA or the Variational Quantum Power Method (VQPM), regardless of convergence quality. We further show that the same moment matrix reveals the tensor‑product structure of the underlying unitary circuit, enabling a polynomial‑time, correlation‑based circuit cutting procedure with rigorous error bounds. The framework is validated numerically on Max‑Cut instances for variational quantum algorithms and on random states for circuit cutting, demonstrating that the cheap two‑point correlation data are sufficient to locate near‑optimal bipartitions and that the theoretical error bounds hold in practice.
Authors:Ronglai Qian, Liang An, Xiaoyan Wang, Qing Fan, Ziwei Huang, Yang Ye
Abstract:
Modulation recognition systems rely on heterogeneous signal representations. 2D signal‑image modalities such as time‑frequency and cyclostationary maps capture structural patterns, while 1D statistical descriptors such as higher‑order power spectra encode complementary cues. Under distribution shift, these modalities degrade unevenly, making robust fusion a central challenge for practical deployment. Progress is further limited by the lack of a unified evaluation protocol that systematically separates different shift types. This paper addresses both challenges through a joint benchmark‑and‑model study in underwater acoustic modulation recognition. UAMR‑ShiftBench is the first benchmark to jointly cover in‑distribution, low‑SNR, unseen‑environment, unseen‑communication‑parameter, and measured sea‑trial evaluation under a single matched protocol, with two independent real‑world subsets collected during two sea‑trial campaigns conducted in March and November in the South China Sea. SCP‑TriCA fuses STFT, cyclostationary, and P2/P4 (second‑ and fourth‑order power spectra) modalities hierarchically: the two 2D modalities are first aligned through bidirectional cross‑attention, and the 1D statistical modality is then incorporated through a sample‑adaptive selective gate. On UAMR‑ShiftBench, SCP‑TriCA achieves 95.33% in‑distribution accuracy and 74.59% simulated OOD average, outperforming the strongest baseline by 5.12 percentage points, and reaches 91.14% and 94.86% on the two sea‑trial subsets, exceeding the best baseline by 15.71 and 23.00 percentage points respectively. Ablation results confirm that the gains stem from modality complementarity and the hierarchical fusion design. Code and models are available at https://github.com/ronglaiqian/UAMR‑ShiftBench.
Authors:Yucheng Wu, Jundong Xu, Mingzhen Ju, Yue Yu, Chenpeng Wang, Haoxuan Li, Liangming Pan
Abstract:
Logical reasoning is essential for reliable AI, yet existing benchmarks are largely first‑order‑logic‑centric, focusing on object‑level deduction over fixed predicates. This misses many realistic scenarios where models must reason over rules, predicates, functions, constraints, and decision procedures themselves. We introduce HOLMES (Higher‑Order Logic Meets real‑world Explainable Symbolic reasoning), the first real‑world benchmark for higher‑order symbolic reasoning in LLMs, containing 1379 instances. Built on higher‑order logic, HOLMES pairs natural‑language problems with HOL formalizations, ground‑truth answers, verifiable reasoning traces, and fine‑grained controllable reasoning factors across law and finance. Experiments show that current LLMs still struggle on HOLMES, with an average accuracy of only 50.64% and the best model reaching 59.54%. Our analyses further reveal that high final‑answer accuracy can mask shortcut reasoning in conflict‑resolution settings, while performance drops sharply under scope‑conditioned and compositional reasoning. These findings identify higher‑order symbolic reasoning as a key bottleneck for building reliable and verifiable LLMs. The project code and dataset are publicly available at https://github.com/wuyucheng2002/HOLMES.
Authors:Mostapha Benhenda
Abstract:
Finance Agent v2 (by Vals AI) has emerged as the reference benchmark for evaluating both Anthropic Claude and OpenAI ChatGPT frontier language models on financial tasks. However, it narrowly deals with periodic reporting from publicly traded companies (SEC 10‑K and 10‑Q filings), and its agentic harness relies on naive, unenriched chunk retrieval. Neither the task design nor the retrieval approach addresses the distinct challenges of IPO due diligence. SEC S‑1 filings combine historical financial statements, governance structures, pro forma and common‑control accounting treatments, capital‑formation narratives, and underwriting‑sensitive risk disclosures within substantially longer documents than typical periodic filings. That is why we introduce IPO Finance Agent, which extends the Finance Agent v2 framework along two directions: task domain and retrieval architecture. During our experiments, the original Finance Agent v2 harness basically failed to deliver any output related to the SpaceX S‑1 filing, due to document length. We therefore had to improve the agentic harness with contextual retrieval, a more realistic and industry‑standard approach for long documents. We also built a dataset of 1,000 IPO‑diligence questions, and publicly release 70 questions on the SpaceX (SPCX) S‑1 filing to support reproducibility, while the remainder are held private to guard against benchmark contamination. In addition, we introduce an evaluator‑optimizer pipeline to automatically generate evaluation rubrics for the benchmark: candidate facts are extracted from model answers, consolidated into draft criteria, then automatically audited for omissions, hallucinations, mistiered items, and redundancy, with LLM feedback driving iterative repair, targeted enrichment, and deduplication. Human experts only review final rubrics before deployment. Results show that the best‑performing evaluated model, Zhipu GLM‑5.2, reaches 79.8% accuracy, and the most cost‑efficient model on the resulting Pareto frontier, Xiaomi MiMo‑2.5 Pro, reaches slightly lower accuracy (77.2%) at 0.05 USD per query, while exceeding the current Finance Agent v2 leaderboard ceiling, Google Gemini 3.5 Flash at 57.9% for 2.51 USD per query, and undercutting even FABv2's cheapest entry (MiniMax M3: 48.3% at 0.32 USD) on cost‑efficiency. Code and data are released on GitHub https://github.com/benstaf/ipoagent
Authors:Jianuo Cao, Yuxin Chen, Yuzhen Song, Masayoshi Tomizuka, Chenran Li, Thomas Tian
Abstract:
Text‑conditioned motion generation is a promising interface for programming humanoid robots, yet current generators are often trained on human motion datasets retargeted to robot morphologies. Although such data provides rich semantic and kinematic priors, it fails to capture the nuances of whole‑body tracking controllers, including balance, contact dynamics, actuation limits, and controller‑specific failure modes. As a result, generated motions can be semantically plausible but difficult or impossible for the robot to execute. We introduce TEXEDO, a test‑time scaling framework for humanoid motion generation that improves motion quality without requiring a stronger underlying generator. Given a text prompt, TEXEDO samples multiple candidate motions from a pretrained text‑conditioned generator and selects the best motion that is both executable and task‑aligned. The reward model combines a dynamic feasibility verifier, distilled from whole‑body tracking rollouts to predict physical executability, with a semantic alignment verifier that measures text‑motion alignment in a learned co‑embedding space. Our pipeline treats dynamic feasibility as a hard constraint and semantic alignment as the selection objective within the feasible set. Through large‑scale simulation studies and real‑world deployment on a Unitree G1 humanoid robot, we show that TEXEDO consistently improves both tracking fidelity and text alignment. These results demonstrate that grounded verification is an effective path toward deployable language‑guided humanoid motion generation. Project website: https://jianuocao.github.io/TEXEDO/
Authors:Chanyoung Park, Minsung Yoon, Andrew Jeong, Sung-eui Yoon
Abstract:
Multi‑arm manipulation demands precise spatiotemporal coordination, yet many centralized approaches scale poorly as team size increases. To address this, we propose CLS‑DP, a decentralized multi‑agent framework that enables implicit coordination under partial observability without shared global views, explicit state information, or inter‑agent communication. Under the centralized training and decentralized execution (CTDE) paradigm, CLS‑DP distills privileged multi‑agent dynamics into a latent space. At deployment, each agent infers a collaborative latent from its local RGB observation and a shared task instruction; it then conditions the diffusion denoising process on this latent. This design enables implicit coordination with a per‑agent cost independent of team size. Across six RoboFactory benchmark tasks spanning two to four agents, CLS‑DP achieves a 38% mean success rate, outperforming the best centralized baseline (20%) and a decentralized ablation without the collaborative latent (9%). It also maintains superior parameter efficiency across all agent configurations. Attribution maps show that an agent conditioned on the collaborative latent places high attribution on the joints and grippers of both itself and its teammates throughout execution. This suggests that the learned latent efficiently encodes collaborative dynamics from local observation, which facilitates implicit coordination in realistic settings characterized by partial observability.
Authors:Yubo Zhou, Jianghao Wu, Ping Ye, Shaoting Zhang, Guotai Wang
Abstract:
Concept segmentation models like Segment Anything Model 3 (SAM3) show strong generalization on natural images, yet their performance degrades in medical imaging due to the domain gap caused by different imaging principles and styles. Test‑Time Adaptation (TTA) is essential for improving the testing performance by updating the model on the fly without annotations. However, existing vision‑language TTA methods are mainly driven by image‑level uncertainty minimization, which does not necessarily reflect region‑level semantic correctness in medical segmentation. Moreover, they often lack mechanisms to maintain stability in continual one‑pass adaptation, leading to limited performance when reliable dense supervision is missing for segmentation. To address these issues, we propose Concept Alignment Contrast and LongShort Prompt Memory for Test‑Time Adaptation (CM‑TTA) of SAM3 for medical images. First, for a test sample with multiple augmentations, we introduce a novel Concept Alignment Contrast (CAC) metric, which leverages textual‑visual semantic consistency to robustly evaluate prediction quality to select the best augmented view as the supervision. Second, to balance rapid and stable adaptation, we design a Long‑Short Prompt Memory (LSPM) module. The short memory dynamically fuses recent prompts based on CAC scores for agile local adaptation, while the long memory maintains a stable global prompt to generate enhanced pseudo‑labels. Finally, a Densely Supervised Prompt Update (DSPU) strategy is proposed to optimize the prompt embeddings with enhanced pseudo labels as dense supervision. Extensive experiments on prostate and skin lesion segmentation demonstrate that our CM‑TTA framework significantly outperforms existing methods for TTA of SAM3. The code is available at https://github.com/SherlockZYB/CM‑TTA.
Authors:Junrong Huang, Zhiyuan Zhang, Rui Tang, Hongbo Fu, Jnig Liao
Abstract:
Realistic integration of user‑specified textures into scene images is a fundamental task in computer graphics and image editing. While existing material transfer and reference‑guided inpainting methods can edit surface appearances, they often fail to address the specific requirements of texture tiling. This task necessitates precisely repeating a reference pattern according to user‑defined parameters such as frequency, orientation, and scale. Furthermore, current generative approaches often struggle to maintain the structural fidelity of the reference texture, limited by either destructive pixel‑level resampling or the lack of fine‑grained spatial information in semantic image encoders, and they frequently fail to preserve the coherent lighting and geometry of the original scene. In this paper, we propose a novel framework for controllable and high‑fidelity texture tiling based on Diffusion Transformers. Our approach introduces two key technical innovations to decouple spatial manipulation from content generation. First, we propose a Coordinate‑Transformed Rotary Embedding mechanism. By applying 2D affine transformations directly to the relative positional embeddings between the target latent and the image condition, we achieve precise control over tiling patterns without explicit pixel warping, thereby utilizing the full information of the reference condition without degradation. Second, a Disjoint Attention Mask is employed to shield reference features from semantic leakage. This preserves structural integrity while seamlessly blending the synthesized texture with the scene's original lighting and geometry. Extensive experiments demonstrate that our method outperforms state‑of‑the‑art baselines in both control accuracy and texture fidelity.
Authors:Pengfei Zhou, Zhiwei Tang, Yixing Ma, Jiasheng Tang, Yizeng Han, Zhenglin Wan, Fanqing Meng, Wei Wang, Bohan Zhuang, Wangbo Zhao, Yang You
Abstract:
Real‑world users typically have access to multiple Large Language Models (LLMs) from different providers, and these LLMs often excel at distinct domains, yet none dominate all. Consequently, routing each task to the most suitable model becomes critical for both performance and cost. Existing routers treat this as a static, one‑off classification problem. However, we identify the performance bottleneck for these routers as information deficit: simply augmenting a vanilla LLM router with performance statistics at the task‑dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same dimension‑level priors. Motivated by this finding, we propose Agent‑as‑a‑Router, a framework that formalizes routing as a C‑A‑F loop (Context‑>Action‑>Feedback‑>Context). It closes the information gap by accumulating execution‑grounded experience during deployment. We instantiate this framework as ACRouter, composed of an Orchestrator, a Verifier, a Memory module, and introduce CodeRouterBench, an evaluation environment comprising ~10K task instances with verified scores from 8 frontier LLMs, enabling regret‑based router comparison on streaming tasks. Experiments show that ACRouter achieves the lowest cumulative regret on in‑distribution tasks and generalizes to out‑of‑distribution agentic‑programming tasks, demonstrating that our routing framework actively closes the information gap. Codes and benchmarks are released at https://github.com/LanceZPF/agent‑as‑a‑router.
Authors:SingGuard Team
Abstract:
Vision‑language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from multimodal question answering, assistant responses, and cross‑modal composition, while moderation policies may vary across products, regions, and deployment stages. Most existing guardrails either rely on fixed taxonomies or target only a narrow set of interaction settings, which limits their adaptability when safety rules change at deployment time. We present SingGuard, a policy‑adaptive multimodal guardrail model family for safety assessment in multimodal conversations. SingGuard treats the active policy as a runtime input: given natural‑language rules, it checks the target content against the active policy rule by rule and predicts both the safety label and the triggered rule. To balance efficiency and interpretability, SingGuard supports fast, hybrid, and slow inference regimes along a fast‑to‑slow reasoning spectrum, ranging from direct safety judgments to policy‑grounded deliberation. We further optimize this behavior with fast‑‑slow decoupled reinforcement learning. We also introduce SingGuard‑Bench, a multimodal guardrail benchmark with 56,340 examples spanning 80+ fine‑grained risk types across multimodal QA, adversarial attack, and dynamic‑rule evaluation settings, including cross‑modal joint‑risk cases where each modality is harmless in isolation but their composition implies unsafe intent. Across six benchmark families (35 datasets), SingGuard achieves state‑of‑the‑art average F1 in every family. Dynamic‑rule evaluation further shows improved policy‑following accuracy from 0.6465 to 0.7415 under runtime policy shifts. Our code is available at https://github.com/inclusionAI/Sing‑Guard.
Authors:João Guilherme Alves Santos, Giovana Kerche Bonás, Thiago Laitz, Thales Sales Almeida, Helio Pedrini
Abstract:
Although Large Language Models (LLMs) excel in many tasks, their assessment in Portuguese has received less attention, particularly for open‑ended, discursive tasks that demand deeper reasoning and generation capabilities. While the original BLUEX benchmark addressed the scarcity of Portuguese evaluation datasets through multiple‑choice questions from Brazilian university entrance exams, it did not cover the more challenging second‑phase examinations, which require free‑form written responses. In this work, we introduce BLUEX v2, a benchmark derived from the second‑phase entrance exams of Brazil's two leading universities: UNICAMP (Comvest) and USP (Fuvest), spanning exam years 2022‑‑2025. Our dataset comprises 395 questions unfolding into 919 graded subquestions, with 55.7% of questions containing associated images (represented as context‑aware captions during inference to enable evaluation across both vision‑capable and text‑only models). Each question is annotated with subject area, official reference answers, LLM‑generated rubric criteria, and six cognitive capability tags. We evaluate 21 state‑of‑the‑art LLMs using an LLM‑as‑a‑judge protocol. Results reveal a 4.92‑point performance spread across models (4.18‑9.10 on a 0‑10 scale), with Mathematical Reasoning and Image Understanding emerging as the hardest capability dimensions. The evaluation code, model outputs, and dataset are publicly available at https://github.com/TropicAI‑Research/BLUEXv2 and on Hugging Face at https://huggingface.co/datasets/Tropic‑AI/BLUEX‑v2.
Authors:Francesco Di Salvo, Sebastian Doerrich, Christian Ledig
Abstract:
Foundation models provide highly descriptive representations for medical images, yet their reliability degrades under distribution shifts arising from changes in patients, devices, or acquisition conditions. Reliable out‑of‑distribution (OOD) detection is therefore essential for safe deployment. Recent post‑hoc detectors efficiently exploit frozen embeddings (e.g., kNN), whereas reconstruction‑based OOD detection in latent feature space has seen limited adoption due to inconsistent performance. In this work, we show that the limitation of reconstruction‑based methods in latent space does not stem from poor reconstruction quality, but from how reconstruction errors are scored. Standard L2 residual norms collapse the anisotropic residual structure, thereby suppressing informative deviations. To address this limitation, we introduce MaRS (Mahalanobis Residual Scoring), a label‑free OOD detector that learns an in‑distribution manifold using a lightweight autoencoder and measures deviation via a Mahalanobis distance on reconstruction residuals, yielding variance‑aware OOD scores. Across three imaging modalities, multiple types of distribution shift, and different model families and scales, MaRS outperforms established confidence‑, distance‑, and reconstruction‑based baselines, while remaining fully post‑hoc and lightweight. The code is available at https://github.com/francescodisalvo05/mars.
Authors:Andreas Kosmas Kakolyris, F. Nisa Bostanci, Ataberk Olgun, Ismail Emir Yuksel, Harsh Songara, Konstantinos Marios Sgouras, Umut Baser, Konstantinos Kanellopoulos, A. Giray Yaglikci, Onur Mutlu
Abstract:
Modern DRAM chips are vulnerable to read disturbance phenomena such as RowHammer and RowPress, which induce bitflips after accessing nearby rows a certain number of times (the read disturbance threshold). ColumnDisturb is a new, fundamentally different DRAM read disturbance phenomenon. Specifically, ColumnDisturb (i) disturbs DRAM columns instead of rows, and (ii) increases the number of affected DRAM cells from those in only a few neighboring rows to all cells across three consecutive DRAM subarrays. We propose ColumnKeeper, the first set of ColumnDisturb mitigations, in two variants: ColumnKeeper‑D (CK‑D), a deterministic mechanism, and ColumnKeeper‑P (CK‑P), a probabilistic one. CK‑D exploits DRAM's open‑bitline architecture to provide deterministic security guarantees at low performance and energy overheads: it uses two counters per subarray to track activations affecting the odd and even columns, and refreshes one row in a subarray when either counter reaches a predetermined threshold. CK‑P instead refreshes one row in three consecutive subarrays upon a row activation in the middle subarray, with a predetermined probability, providing configurable security guarantees at low area overhead. Both mechanisms prevent ColumnDisturb bitflips at low performance, energy, and area overheads. At the current experimentally‑demonstrated ColumnDisturb threshold (1M), CK‑D and CK‑P incur very low average single‑core performance overheads of 0.15% and 0.36%, respectively. For near‑future thresholds (128K), these rise to a still low average of 1.70% and 2.73%. Mitigating ColumnDisturb at low thresholds (e.g., 16K) remains possible by adopting smaller subarray sizes or enabling subarray‑level parallelism. CK‑D and CK‑P require low area overheads of 0.1 mm^2 and 0.03 mm^2, respectively. ColumnKeeper is freely available at https://github.com/CMU‑SAFARI/ColumnKeeper .
Authors:Srinivas Venkatanarayanan, Clement Pakkam Isaac
Abstract:
Vision‑language models (VLMs) are increasingly used to read maps for logistics, delivery, and accessible navigation, where the output is an actionable decision (a route, a pin, a parking choice) that must respect the road network. Yet most map benchmarks grade free text or multiple‑choice answers that cannot be verified against the underlying graph. We present MapReason‑OSM, a benchmark and evaluation harness for graph‑verifiable mobility decisions on self‑rendered OpenStreetMap panels. We render fixed‑style maps for ten U.S. downtowns at two aligned zoom scales, overlay a consistent marker grammar, and pair each panel with a hidden street graph and exact oracles, yielding 6,000 instances (12,000 panels across the two zooms) over 12 routing, facility‑location, and visual disambiguation tasks. Models return structured decisions that we snap back to the graph and score for validity, legality, optimality, and constraint satisfaction, plus cross‑zoom consistency. Across seven VLMs, models read maps and route simply but fail at graph cost reasoning (single‑facility pin placement is near chance even for frontier reasoning models), and are frequently scale‑inconsistent. We release the benchmark, harness, and deterministic generator. Code and data: https://github.com/Vi‑Sri/mapreason‑osm
Authors:Nicolò Savioli
Abstract:
Modern Referring Image Segmentation (RIS) systems generate multiple candidate masks per expression but rely on a simple heuristic‑‑typically the argmax detection score‑‑to select the final output. We identify query selection as a failure‑case bottleneck: although heuristic selection succeeds on 82‑93% of samples, the residual 7‑18% of failures dominate the error budget, leaving a best‑query selection gap of 3‑11% mIoU. We introduce Venice‑H1, a lightweight, backbone‑decoupled post‑hoc re‑ranking module that encodes each candidate through multi‑scale grid signatures‑‑compact spatial descriptors pooled onto 4x4, 8x8, and 16x16 grids‑‑and feeds them to a Transformer‑based re‑ranker with a Failure Gate (ROCAUC 0.78‑0.82) that intervenes only when the default choice is likely suboptimal. Instantiated on DeRIS‑L and DeRIS‑B, Venice‑H1 achieves delta_fail of +1.40 and +0.89 mIoU with strictly positive 95% CIs on all 16/16 (split, backbone) pairs and harmful‑switch rates below 0.53%. Zero‑shot transfer to medical referring segmentation (MS‑CXR, M3D‑RefSeg‑2D) yields +1.16 and +0.51 mIoU without RIS‑backbone fine‑tuning. The module adds approximately 11.3M parameters and under 1 ms latency.
Authors:Xianghui Wang, Feng Chen, Wenbo Zhang, Hua Yan, Zixuan Wang, Changsheng Li, Yinjie Lei
Abstract:
Vision‑Language‑Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real‑world deployment is often bottlenecked by execution efficiency. While existing efforts predominantly focus on compute‑centric efficiency to reduce per‑step inference latency, the intrinsic policy efficiency of these models remains largely unexplored. Policy efficiency is fundamentally affected by two factors, namely the effective executable length of predicted action chunks and the total physical steps required to complete a task. These two factors jointly determine the total number of forward inference calls during execution. We observe that current VLA policies struggle with planning unreliability and action redundancy, suffering from severe prediction degradation at the tail of action chunks and tending to generate unnecessarily redundant physical steps. To address this, we propose PolicyTrim, a reinforcement learning‑based post‑training framework that extends the reliable action chunk length and reduces redundant physical steps. For reliable chunk extension, we employ a dynamic exploration strategy that explicitly rewards the successful completion of longer executable lengths, progressively pushing the trustworthy prediction horizon to its empirical limit. For step efficiency, we design a redundancy‑aware reward that directly favors successful task completions with fewer steps while penalizing unreproducible shortcuts, effectively eliminating redundant physical actions. Extensive experiments across three benchmarks and three VLA models demonstrate that PolicyTrim improves action chunk utilization by 3× and reduces physical execution steps by 51.4%. Ultimately, our framework delivers up to a 5.83× end‑to‑end deployment speedup without compromising task success rates.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
A recent Nature Medicine study reports that general‑purpose frontier LLMs outperform specialized retrieval‑augmented clinical tools on medical benchmarks, and that retrieval can hurt strong models. We ask the natural follow‑up: does structured knowledge‑graph (KG) grounding change this, and when does grounding help at all? We contribute two results. First, a reproduction: the study's headline HealthBench score (~88) is the Consensus variant, not full HealthBench, where frontier models and ideal completions both score ~46‑47 under a physician‑calibrated grader (agreement 82.5%); we reproduce GPT‑5.2 Consensus =90.9 and flag a score‑deflating grader bug. Second, a knowledge‑boundary result. Using a graph+vector engine (samyama‑graph) over the public biomedical KG PrimeKG, neither naive triple retrieval nor an agentic natural‑language‑to‑Cypher loop (82% successful queries) improves MedQA across a weak‑to‑strong model ladder (all |Delta| <= 3.4). On a synthetic counterfactual KG, and on a hybrid benchmark mixing known and novel facts, the identical pipeline lifts out‑of‑training accuracy from chance to ~100% (+68 to +79) while adding nothing on known facts (a no‑LLM arm answers both). Across three regimes (no‑knowledge, graph‑aided, hybrid), grounding helps only insofar as the decisive fact lies outside the model's training ‑‑ public‑KG facts are redundant, private and novel data are where it pays ‑‑ matching the study's institutional‑data caveat.
Authors:Li Kong, Qi Qi, Yinyu Ye, Zijie Zhou
Abstract:
The explosive demand for interactive Large Language Model serving has highlighted the management of the Key‑Value cache's dynamic memory footprint as a critical area for performance optimization in inference engines. Modern inference systems overwhelmingly rely on time‑centric scheduling heuristics, such as Shortest Job First. However, their theoretical optimality is rooted in traditional schedule modeling, failing to capture the highly dynamic, 2D spatio‑temporal geometric growth specific to LLM inference mechanisms. To resolve this, we propose the geometry‑aware online scheduling by introducing the Smallest Volume First (SVF) algorithm and its highly efficient variant, 1‑bit SVF. Theoretically, we provide a rigorous mathematical foundation for our approach. Via a novel volume‑certificate proof, we sharpen SVF's worst‑case competitive ratio from the prior best of 48 towards 3 in the high‑concurrency regime of LLM serving. Building upon this core breakthrough, we complete a comprehensive theoretical taxonomy analyzing our algorithms across different traffic scenarios and information availability. Practically, we seamlessly integrate our approach as a plug‑and‑play layer in vLLM. Extensive evaluations on Llama‑3.1 models demonstrate comprehensive performance gains: SVF delivers strong reductions in both average and tail latency, while 1‑bit SVF, with merely a single bit information, achieves competitive throughput and latency. This work establishes a theoretically sound and empirically proven approach for resolving memory‑constrained scheduling in modern LLM deployments. To facilitate future research, our code is available at https://github.com/Aurora‑Kl/Geometry‑Aware‑Online‑Scheduling.git.
Authors:Samat Zharassov
Abstract:
Standard autoregressive Transformer decoders can often exhibit substantial forgetting under sequential fine‑tuning on shifting curriculum distributions. This technical report evaluates SamatNext v0.2‑B, an experimental 356M‑parameter hybrid sequence decoder that alternates Differential‑Attention‑style layers with DeltaNet‑inspired simplified linear‑state mixer layers using RMS normalization and output scale calibration. We study the model under a controlled staged Python code curriculum and compare it with a parameter‑matched Transformer baseline. In this setting, SamatNext v0.2‑B achieves a 100.0% pass rate on the controlled Stage 5 holdout while retaining 98.8% of adjacent Stage 3 semantic behavior and reaching 12.0% on the Stage 2E early syntax holdout. The strongest Transformer baseline reaches 97.6% on Stage 5 but retains only 6.0% of Stage 3 behavior. Both architectures remain weak on long‑horizon early‑stage retention, so the result should be interpreted as evidence of an altered retention/plasticity tradeoff in this controlled setting, not as a general solution to catastrophic forgetting. Code, model specifications, evaluation scripts, and result tables are provided for independent verification.
Authors:Yangtao Chen, Zixuan Chen, Peiyang Wang, Yong-Lu Li, Jing Huo, Jieqi Shi, Yang Gao
Abstract:
Scaling dexterous manipulation requires generalization across objects, scenes, and tasks, yet existing data sources face a trade‑off between scale and scene/embodiment alignment: teleoperation data is well aligned with robot deployment but expensive to collect; simulation is scalable but limited by the sim‑to‑real gap; and real egocentric videos scale effectively but remain misaligned with robot deployment. We propose Wh0, a framework that uses generative video world models as scalable and controllable sources of egocentric human‑hand manipulation data to unlock the manipulation capabilities of pretrained dexterous VLA models. Conditioned on language, objects, and scenes, Wh0 uses a generative world model to produce WM‑H, a 50k‑episode dataset of egocentric human‑object interaction videos. Wh0 then converts the generated videos into robot‑trainable supervision through hand motion reconstruction and visual editing. Co‑trained with a limited amount of real robot data, WM‑H adapts pretrained VLA models to dexterous manipulation deployment. Across 18 real‑world dexterous manipulation tasks, compared with a model post‑trained only on robot data, Wh0 improves zero‑shot success on unseen tasks from 8.3% to 38.9%. Ablation studies further show that scalable generation and scene/embodiment alignment are key drivers of performance gains. Videos and open‑source code can be found on our project website: https://chenyt31.github.io/wh0.github.io/.
Authors:Syed Hamzah Rizvi, Yash Vardhan Tomar
Abstract:
Simulation‑to‑reality transfer, often called sim‑to‑real transfer, is a central challenge in robot learning. Yet, the tradeoff between measuring a system more accurately and training over a broader range of simulated dynamics is still poorly understood. In this work, we focused on the allocation of real‑robot measurement time between system identification and domain randomization. We studied this tradeoff in a controlled sim‑to‑sim pendulum setting, where a hidden‑parameter model stands in for the physical robot, and the experiment sweeps identification rollouts against the width of the randomization distribution. Across the reality gaps and noise levels we tested, the measurement budget did most of the work. A small number of identification rollouts closed most of the transfer gap, and once any real data was available, policies performed best when trained at the estimated parameters rather than over a widened randomization band. Broad randomization that contained the true system still did not substitute for measurement. These results hold in a benign regime where the dynamics are identifiable and only two parameters are unknown, so structural model mismatch remains the setting where randomization breadth may become more valuable. Overall, our results suggest that sim‑to‑real pipelines should first measure the parameters they can and reserve randomization for the uncertainty that remains.
Authors:Irene Iele, Elena Mulero Ayllón, Paolo Soda, Matteo Tortora
Abstract:
Vegetation monitoring under climate stress requires answering not only how it will evolve given the expected weather, but how it would respond to alternative meteorological conditions. Forecasting models return the expected vegetation state for the observed weather and cannot answer these scenario‑conditioned questions, because future weather is fixed to the recorded trajectory. We present VegSim, a geospatial world model for scenario‑conditioned vegetation simulation. VegSim infers a latent vegetation state from sparse satellite‑derived NDVI histories, past meteorological covariates, and static spatial context, propagates it forward under future weather forcing through recurrent latent dynamics, and decodes predictive NDVI quantiles at each lead time. Because future forcing enters as a controllable input, the same trained model supports probabilistic forecasting under observed weather and conditional simulation under user‑defined meteorological forcing, without supervision on scenario responses. We evaluate VegSim on GreenEarthNet across in‑distribution data and spatial, temporal, and joint spatial‑temporal shift, where it achieves strong point and probabilistic accuracy against time series and Earth observation forecasting baselines while using a compact architecture. We then simulate vegetation responses across Europe under four meteorological scenarios, and in a France summer 2022 case study, obtaining spatially coherent patterns consistent with known sensitivity to temperature and precipitation. The code is available at https://github.com/arco‑group/vegsim.
Authors:Abdul-Kazeem Shamba, Kerstin Bach, Gavin Taylor
Abstract:
Supervised learning demands large quantities of labeled data, a bottleneck that is expensive and reliant on domain‑specific expertise. Self‑supervised learning, particularly contrastive learning, has emerged as a compelling alternative, enabling rich representation learning directly from unlabeled data. Yet its success hinges critically on the design of positive and negative sample pairs. Existing approaches for time series rely on hand‑crafted augmentations and masking heuristics that embed strong domain assumptions, often limiting generalization across diverse temporal patterns and potentially introducing spurious correlations. In this work, we challenge this paradigm by demonstrating that explicitly encoding temporal shift invariance through a simple, deterministic view construction is sufficient to learn strong representations for time series classification. By exploiting temporal structure, our method, Shift Invariant Feature Training (ShiFT), achieves state‑of‑the‑art performance on six diverse real‑world time series benchmark datasets, as well as the UCR and UEA archives, while reducing training time. Beyond empirical performance, we present a systematic analysis of contrastive learning dynamics in time series settings, examining the effects of batch size and the number of negatives on downstream performance. Our findings provide practical insights for designing efficient contrastive learning frameworks for time series representation learning. The source code is publicly available at https://github.com/sfi‑norwai/ShiFT.
Authors:Xinlong Chen, Jiafu Tang, Yue Ding, Yizhuo Jia, Bozhou Li, Bohan Zeng, Yang Shi, Shihao Li, Yiyan Ji, Qiang Liu, Weihong Lin, Yuanxing Zhang, Pengfei Wan, Liang Wang, Tieniu Tan
Abstract:
Accurate and comprehensive video captions with consistent subject references are critical for downstream understanding and generation tasks. However, few existing benchmarks can objectively and comprehensively evaluate these properties across diverse durations and scenarios, thereby hindering the advancement of video captioning models. To bridge this gap, we propose CapRiCorn‑1K, a comprehensive benchmark designed to evaluate both video captioning quality and subject referential consistency across long temporal horizons and diverse video domains. To accommodate varied evaluation needs, our benchmark supports both audiovisual and visual‑only settings. Extensive experiments on CapRiCorn‑1K reveal that current models generally struggle to generate accurate and comprehensive captions while maintaining consistent subject references. Moreover, as video duration increases, both the overall caption quality and subject referential consistency decline. Notably, our evaluation metrics exhibit strong correlations with the performance of downstream understanding and generation tasks conditioned on the generated captions, further validating their effectiveness. The project is available at https://github.com/xlchen0205/CapRiCorn‑1K .
Authors:Jayadratha Gayen, Bradly Alicea
Abstract:
Understanding how a nervous system wires itself from birth to adulthood is a fundamental challenge in developmental neuroscience. We present DevoTG, a temporal graph framework that applies Temporal Graph Neural Networks (TGNs) to two complementary representations of C. elegans neural development: a Continuous‑Time Dynamic Graph (CTDG) of cell division events derived from cell lineage data, and a Discrete‑Time Dynamic Graph (DTDG) of the developing synaptic connectome spanning eight reconstructed electron‑microscopy datasets. On the lineage prediction task, our TGN achieves a mean test AUC of 0.839 +/‑ 0.007 (5 seeds; validation AUC 0.937 +/‑ 0.001), outperforming a static GNN with the identical architecture by 26 AUC points (0.577 +/‑ 0.080), demonstrating that temporal memory is the decisive factor. Applied to the connectome DTDG, DevoTG identifies three connection stability classes (stable, developmental, and variable) across 225 neurons and 858 to 2,496 connections over development (L1 birth to adult), providing a temporal‑graph‑theoretic complement to the individual‑variability classification of Witvliet et al. Analysis of hub command interneurons AVA, AVB, and AVE reveals their persistent centrality and how their integration roles are progressively reinforced across larval stages. Accompanying interactive visualizations (3D animated networks, centrality heatmaps, and a spatiotemporal lineage graph) make developmental dynamics accessible for biological hypothesis generation. DevoTG is open‑source and designed for extension to other developing nervous systems. Code is publicly available at https://github.com/DevoLearn/DevoGraph/tree/main/DevoTG.
Authors:Wei Xue, Junlan Feng, Shilei Zhang, Yue Wang, Ruosong Yang, Bei Liu, Liumeng Xue, Sitong Cheng, Jiahao Pan, Weizhen Bian, Boyi Kang, Bin Long
Abstract:
Recent advances in text‑to‑speech (TTS) have greatly improved speech naturalness, speaker similarity, and controllability. However, most existing controllable TTS systems still rely on explicit user‑provided style prompts, making it difficult to automatically determine how a sentence should be spoken in long and complex conversational scenarios. This proposal introduces the ISCSLP 2026 CoT‑TTS Challenge, which aims to evaluate whether a system can infer the intended speaking manner from contextual information and generate speech consistent with both the reasoning output and the surrounding scene. The challenge contains two tracks: text‑context‑aware CoT‑TTS and audio‑context‑aware CoT‑TTS. We construct a large‑scale bilingual training set from speech‑rich media and provide carefully filtered evaluation data for leaderboard comparison. Each system is required to output both a chain‑of‑thought reasoning analysis and the generated speech waveform. The official evaluation combines objective metrics, multimodal LLM‑based evaluation, and human subjective assessment. To facilitate reproducibility, we provide inference code together with a fine‑tuning recipe for a 0.6B Qwen3‑based model trained via a three‑stage strategy. This challenge is expected to support research on context understanding, chain‑of‑thought reasoning, and expressive speech generation for applications such as film dubbing, audiobook production, virtual characters, and spoken dialogue agents. Further information about the associated challenge is available at:https://iscslp2026‑cot‑tts.github.io/challenge‑website/
Authors:Abdirashid Omar, Jonghyuk Park
Abstract:
Remote sensing change detection (CD) from bi‑temporal imagery is critical for applications such as urban monitoring, disaster assessment, and environmental management, yet robust localization remains challenging under sparse changes, noisy labels, and appearance variations. In this paper, we propose Context Sampling Attention (CoSA), a lightweight decoder‑side refinement module that explicitly leverages bi‑temporal feature correlation as a control signal for adaptive change‑aware feature enhancement. This differs from conventional attention mechanisms that rely on implicit feature weighting without explicit temporal control. In the implemented FC‑Siam setting, CoSA computes normalized same‑location cross‑correlation between paired decoder features, converts low correlation into a change gate, and injects the resulting gated residual at native 1/8 and 1/16 feature scales through learnable residual scaling. This design enables effective discrimination between stable and ambiguous regions without relying on computationally expensive global attention. Extensive experiments on four benchmark datasets (LEVIR‑CD, S2Looking, DSIFN, and CLCD) demonstrate consistent improvements over strong baselines, achieving 1.5‑2.6% gains in changed‑class F1 while introducing negligible parameter overhead. Ablation studies confirm that multiscale placement and learnable residual gating are both important for peak performance. These results indicate that CoSA establishes a practical and effective refinement paradigm for enhancing temporal discriminability in Siamese change detection frameworks.
Authors:Qing Xu, Xiangjian He, Wenting Duan, Jiebo Luo, Zhen Chen
Abstract:
Cell segmentation is critical for computational pathology and biomedical discovery. While recent Vision Foundation Models (VFMs) have demonstrated remarkable universal feature representations, unlocking their full potential for cellular imaging is currently bottlenecked by resource‑intensive adaptation paradigms. Existing methods typically rely on fine‑tuning heavy visual encoders, leading to extensive computational overhead and a dependency on large‑scale annotations. To address this, we propose the EffiCell‑Seg framework for highly efficient cell segmentation without re‑training the visual encoder. Our core insight is that pretrained VFMs intrinsically encode complementary structural priors: global saliency for localizing potential cells, and local morphological patterns for delineating cellular structures. To harness these priors, we devise a Cell Structure Prompt Encoder (CSP‑Encoder) that synthesizes semantic‑aware saliency and principal morphological features from frozen VFM representations into explicit structural prior maps. Moreover, we propose a Synergistic Mask Decoder (SM‑Decoder) that enforces contextual consistency by jointly predicting geometric distance fields and semantic maps via mutual cross‑guidance. Extensive experiments demonstrate that EffiCell‑Seg outperforms state‑of‑the‑art methods across diverse cell imaging modalities while requiring only ~5M trainable parameters, over 130x fewer than fully fine‑tuned VFM counterparts. The code is available at https://github.com/xq141839/EffiCell‑Seg.
Authors:Weiqin Yang, Yue Pan, Chongming Gao, Sheng Zhou, Xiang Wang, Can Wang, Jiawei Chen
Abstract:
We identify a critical pitfall in scaling transformer‑based sequential recommenders: while increasing model size improves recommendation accuracy, it simultaneously amplifies popularity bias. This bias drives systems to over‑recommend popular items at the expense of niche ones, which not only undermines fairness but also degrades the broader ecosystem by reinforcing the Matthew effect and filter bubbles. Consequently, this bias amplification emerges as a fundamental obstacle to sustainable model scaling. Through comprehensive theoretical and empirical analyses, we uncover the root cause of this amplification. Our findings reveal that as model depth increases, the two core components of the transformer architecture, i.e., attention aggregation and feed‑forward projections, synergistically induce severe spectral collapse in model predictions, which directly translates to the amplification of popularity bias. To address this challenge, we propose SPRINT (Scalable Popularity Regularization IN Transformers), which mitigates spectral collapse during scaling by constraining (i) the maximum column‑sums of the attention score matrices and (ii) the spectral norms of the feed‑forward parameters. Extensive experiments demonstrate that SPRINT significantly improves both accuracy and long‑tail fairness. Crucially, it yields more favorable scaling behaviors when expanding model sizes from 0.05M to 0.34B parameters. The code is available at https://github.com/Tiny‑Snow/GenRec.
Authors:Yu-Syuan Xu, Hao-Lun Sun, Hao-Wei Chen, Hsien-Kai Kuo, Chun-Yi Lee
Abstract:
Arbitrary‑scale image super‑resolution (ASISR) aims to reconstruct high‑resolution images from low‑resolution inputs over a continuous range of upscaling factors. While traditional pixel‑regression approaches often produce overly smooth results that lack realistic details, recent diffusion methods can produce sharper and more realistic textures. However, these diffusion techniques frequently introduce the risk of structural hallucinations. To address these issues, we propose Fidelity‑ and Perception‑Aware Local Implicit Attention (FPLIA), a framework that effectively integrates fidelity‑oriented features into a diffusion pipeline to produce realistic and faithful reconstructions for ASISR. We introduce a Fidelity and Perception Attention Module (FPAM), which applies both self‑attention and cross‑attention to fidelity‑oriented and perceptual features to enhance representational capacity. To further exploit their complements, we design a Fidelity and Perception Select Module (FPSM) that adaptively selects the most representative features for RGB values prediction. We conduct extensive experiments to validate the effectiveness of these components. Both qualitative and quantitative results show that FPLIA delivers superior perceptual realism while maintaining reconstruction accuracy on standard ASISR benchmarks. The source code is accessible at the following repository: https://github.com/XUSean0118/FPLIA.
Authors:Harsh Patel
Abstract:
It is tempting to assume any task solvable by a short program can be taught to a model as its chain‑of‑thought: write the steps out, fine‑tune, and the model follows. This paper shows the assumption fails for an identifiable class of procedures. The testbed is nine reasoning tasks, each from a deterministic generator; public and hidden splits share generators, so held‑out data proxies test accuracy. I reverse‑engineer the generators into Python solvers, render them as chain‑of‑thought, and distill into a rank‑<= 32 LoRA over a 30B (3.5B‑active) Nemotron model. Forward‑computable tasks install readily: lookup/arithmetic and an 8‑bit boolean task transfer (>= 0.99 and 0.68). Cryptarithm does not: distilling its backtracking search holds at 0.01‑0.07 across eleven chain‑of‑thought designs, RL from verifiable rewards, and self‑training, even though a search solver answers 71% of instances. This is not a capability gap. The model does the arithmetic on 97‑100% of lines and ranks the correct cipher in its top eight on 71%; it cannot carry the search forward as a left‑to‑right derivation. Fine‑tuning learns the shape of a verifiable elimination step while its verdicts become unconditional templates, correct only 16‑57% of the time ("verdict‑as‑token"). The ceiling holds across backbones from 3B to 671B and across fine‑tuning and prompting; a controlled intervention isolates the cause: revealing the cipher key, which turns the derivation forward, lifts the same instances from 0.03 to 0.57. When a procedure's only solution is search over information‑free structure, no faithful forward chain‑of‑thought exists to imitate. The task becomes learnable only by removing the search, precomputing its combinatorial core into a catalog and reducing the trace to recall plus verification; the 1st‑place solution reaches Private LB 0.92 this way. What distills is memorization and verification, not search.
Authors:Rome Thorstenson
Abstract:
The Categorical Jacobian (CJ) of Zhang et al. (2024) reads protein contacts from a language model by perturbing every residue with every alternative amino acid, about 19L forward passes. We show the signal it reconstructs is already concentrated in a small subset of attention heads: averaging the top‑K contact‑relevant heads, selected on as few as 10 labeled proteins, recovers contacts in one forward pass and beats CJ on leakage‑clean data for every bidirectional model where CJ is defined, and matches or beats it in‑distribution (the exceptions being the smallest 8M model and a statistical tie on ESM Cambrian). Ablations localize the gain to labeled head selection, not averaging: at a matched label budget the unweighted mean ties a supervised L1 logistic regression on the same heads, so the parameter‑free mean is selection's minimal form, not the source of the advantage. Our primary test is leakage‑clean: on a CAMEO split where neither selection nor evaluation touches data the models have plausibly memorized, the head readout beats CJ on ESM‑2‑650M by +9 pp (N=29, p<0.001), with the within‑model margin reproducing across architectures on a wider pretraining‑aware set. Both methods fall 30‑36 percentage points from their in‑distribution Zhang numbers to the leakage‑clean numbers, consistent with substantial pretraining overlap inflating prior numbers (a CAMEO‑vs‑Zhang difficulty shift contributes too, so we read it as an upper bound on the leakage component). We additionally introduce representation‑CJ, a hidden‑state generalization of the Jacobian for architectures without a masked‑LM head; show that the optimal K tracks how diffusely a model spreads its contact heads; and find that both methods lose the contact signal on both causal LMs we test (ProGen2), suggesting attention‑encoded pair structure may depend on bidirectional pretraining.
Authors:Yulun Zhuang, Yue Qin, Justin Lu, Zelin Shen, Yichen Wang, Sicheng He, Yanran Ding
Abstract:
Co‑design of legged robots with elastic elements is challenging due to the non‑differentiability of contact dynamics and mechanism engagement. This paper presents SurGE, a framework that computes surrogate gradients of the design objective through a differentiable pipeline consisting of a kinodynamic single‑rigid‑body (Kino‑SRB) model and a design‑aware control policy, and injects them into CMA‑ES via mean shift with cosine‑annealed step decay. On a 4‑DOF design space of a hopping robot with unidirectional parallel spring, SurGE achieves 6 times lower cross‑seed standard deviation and 18% tighter population concentration compared to vanilla CMA‑ES, while matching or improving the best objective. Hardware experiments on a 2D design subspace show that, starting from a hand‑tuned initial design, SurGE reduces the design objective by 37.65% on hardware, with the improvement trend identified in simulation transferring consistently to the physical system. SurGE provides the potential to accelerate non‑differentiable co‑design problems in legged robots via surrogate model gradients.
Authors:Yanghui Song, Nanqing Liu, Haonan Yin, Yingjie Gao, Chengfu Yang, Qi Ming
Abstract:
Open‑vocabulary semantic segmentation (OVSS) in remote sensing images aims to segment categories beyond a fixed label space. Recent SAM 3‑based methods provide a promising training‑free foundation, yet three key issues remain: (1) a single class‑name prompt lacks sufficient semantic coverage for complex remote sensing categories; (2) expanding each category into multiple prompts introduces redundant online text encoding; and (3) directly aggregating multiple prompt responses propagates noisy activations into the final prediction. To address these issues, we propose ProC‑SAM3, which calibrates SAM 3's prompt interface for remote sensing OVSS from three complementary aspects. First, we construct an offline prompt pool where a Category Matcher groups MLLM‑generated candidates into per‑category sets, and Expansion Constraints further refine each set using category‑specific prior knowledge. Second, the resulting text embeddings are cached and reused across all test images, eliminating repeated text encoding. Third, we introduce Presence‑Guided Residual Fusion to gate unreliable decoder outputs by prompt presence and confidence, followed by peak‑preserving class aggregation that retains fine‑grained activations for small and sparse objects. Experiments on eight benchmarks show that ProC‑SAM3 achieves an average mIoU of 56.1%, outperforming the previous best training‑free method by 3.9 percentage points. Code will be available at https://github.com/YanghuiSong/ProC‑SAM3.
Authors:Wangxuan Fan, Xiaoyu Nie, Zhongxiang Dai
Abstract:
The increasing deployment of large language model (LLM) agents in collaborative workflows demands robust multi‑user, multi‑principal interaction mechanisms capable of enforcing access permissions, resolving authoritative conflicts, and preventing unauthorized data disclosure. However, a fundamental mismatch exists between the single‑user training paradigm of contemporary LLMs and the hard constraints required for multi‑principal governance, rendering probabilistic, prompt‑based safeguards vulnerable under multi‑turn adversarial interactions.Our key insight is that governance constraints ‑‑ who is authorized, what is restricted, and whose instructions take precedence ‑‑ are deterministic runtime variables that should be enforced by execution hooks rather than entrusted to the LLM. We present Harness‑MU, the first model‑agnostic, zero‑tuning infrastructure framework for multi‑user LLM agents. By decoupling language generation from safety orchestration, Harness‑MU guarantees unbreakable permission boundaries while maximizing compliant demand satisfaction. Across four frontier open‑weight and proprietary models on the Muses‑Bench benchmark, Harness‑MU achieves the goal of privacy preservation across all access‑control attacks, outperforming the standard baseline by 0.28‑‑0.39 in utility score and improving instruction‑following accuracy by up to 48.9 percentage points. Harness‑MU advances the philosophy of Harness Engineering, establishing that systematic infrastructure is essential for solving LLM multi‑principal governance challenges. The code and data are available at https://github.com/YuanJrShiuan/Harness‑MulUser.
Authors:Chao Han, Haozhe Hu, Fei Ma, Wei Zhang, Xiaoyu Shen
Abstract:
Low‑rank decomposition serves as a promising compression paradigm for large language models, however, rank allocation remains challenging: manual rules lack generalizability, and learning‑based approaches incur heavy computational overhead. To address these issues, we formulate global low‑rank allocation as a sorting‑and‑truncation pipeline, and score each singular component via dual criteria: Local singular energy ratio that quantifies the intrinsic importance within the decomposed parameter matrix and Global functional importance (measured by input‑output cosine similarity) that evaluates the functional significance of decomposed modules. We verify the strong correlation between high input‑output cosine similarity and low effective rank through geometric interpretation and experimental validation. Furthermore, we propose rank‑preserving fine‑tuning, which performs direct LoRA tuning on decomposed weights and avoids extra information loss caused by re‑truncation in conventional merging pipelines. Empirical results confirm that our method delivers sustained performance enhancements when combined with models featuring distinct decomposition schemes, model sizes and architectural designs, e.g. in one‑shot compression without further fine‑tuning, our method reduces perplexity by up to 50% compared with uniform and heuristic allocation baselines. Code will be available at https://github.com/EIT‑NLP/LLM‑Pruning.
Authors:Yuxuan Chen, Haipeng Xie, Yichi Zhang, Shuo Dai, Zhaohong Bie
Abstract:
Accurate distributed energy resources (DERs) forecasting is critical for downstream optimal operations. However, such forecast‑based operation can be highly vulnerable to cyberattacks. While existing research mainly focuses on adversarial attacks, we pivot to a more controllable and persistent threat: backdoor attacks. In time series forecasting, a backdoored model generates an attacker‑specified target pattern whenever a trigger is embedded in historical inputs. This paradigm naturally fits the entire DER forecast‑optimization‑operation chain. In this paper, we investigate whether and how backdoor attacks can compromise distribution network operations and propose GridTroj, a unified backdoor framework tailored for this scenario. Unlike standard time series backdoor approaches that train a poisoned model to match a predefined target only in terms of forecasting error, GridTroj explicitly incorporates the attacker's intention and optimizes the attack toward operational disruption. Specifically, GridTroj coordinates two key modules. The Intention Planner designs operation‑damaging targets and poisoning strategies, while the Backdoor Realizer constructs the corresponding network architecture and training strategy to learn the trigger‑target association. Experiments on three downstream optimization tasks demonstrate that GridTroj can effectively compromise grid operations and outperforms existing baselines. Our code is available at https://github.com/YuxuanCEE/GridTroj.
Authors:Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, Carolyn Rose
Abstract:
End‑to‑end code agent training is resource‑intensive and plateaus on the strategy‑level reasoning needed to resolve code issues, since jointly optimizing code‑level execution and strategy‑level reasoning leaves the latter underdeveloped. Instead, we freeze the agent and add a critic model to supply that signal. Prior code critics are post‑hoc, scoring completed trajectories rather than steering the agent; we instead train a small critic that provides intra‑trajectory feedback via Supervised Fine‑Tuning. On SWE‑bench Verified, a critic trained on CWM‑32B trajectories transfers to two unseen agents (gains of +3.0 to +3.8 points), and adding target‑agent trajectories to the corpus increases the gain to +3.8 on CWM‑32B and +4.4 to +5.2 on two Qwen agents, at 30‑92x lower critic cost than a strong teacher. On Qwen3‑Next‑80B‑A3B, the critic‑guided system is both more accurate (25.2% vs. 20.8%) and cheaper (\0.04 vs. \0.11) than the agent alone, because the critic also shortens trajectories. Our results show that a small, well‑trained critic is a practical complement to scaling agent training. Code: https://github.com/shubhamrgandhi/critic‑training. Data and models: https://huggingface.co/collections/shubhamrgandhi/critic‑training‑for‑code‑agents
Authors:Ashwin Vinod, Ying Ding, Elias Stengel-Eskin
Abstract:
LLM agents in knowledge intensive question answering take retrieval and reasoning actions with incomplete knowledge about whether their current answer is uncertain, unsupported, or already complete. This produces two failure modes: committing to confident but unsupported answers, which hurts accuracy, and over‑retrieving when the evidence in hand already suffices, resulting in wasted compute. To give agents a more complete picture of the state space they are operating in, we introduce calibrated verifier telemetry (CalVerT), which augments the agent's state with additional telemetry: a calibrated self‑confidence score and a grounding verifier score. We show that CalVerT can improve agents in both training‑free and training‑based settings. On four QA benchmarks, we find that CalVerT raises F1 by triggering retrieval in cases where agents over‑rely on parametric knowledge, while cutting redundant retrieval in cases where agents have sufficient context to answer. We show that CalVerT can augment existing QA frameworks without training. Moreover, CalVerT also improves trained systems: by simply augmenting an agent's state with telemetry, we observe improvements after reinforcement learning, as compared to an agent with identical training but no CalVerT telemetry.
Authors:Prithvi Raj Singh, Satyendra Singh
Abstract:
We present MARLNet (Motion‑Aware Reinforcement Learning Network), a PPO‑based bounding‑box refinement agent that incorporates a constant‑velocity motion prior into the observation state and an action smoothness penalty into the reward function. The agent operates on 268‑dimensional observations encoding the current proposal, a kinematic prediction, the previous action, and a 256‑dimensional EfficientNet‑B0 crop feature, and learns a five‑dimensional policy controlling coordinate adjustments and a binary termination trigger. Evaluated on Pascal VOC 2012 and VisDrone 2019, MARLNet trains stably across all regularization strengths tested and achieves consistent gains in detection success rate at \textIoU \geq 0.5: up to +0.011 on VOC (λ_\textphys=0.10), where the motion prior prevents the overshooting that causes plain PPO to regress on this metric, and +0.007 on VisDrone (λ_\textphys=0.70), where unconstrained PPO achieves a larger gain (+0.025) owing to the weaker base detector. Through reward design ablations and training dynamics analysis, we identify a reward interference in which combining a constant‑velocity deviation penalty with an absolute IoU term causes trigger collapse, and show that replacing it with the action smoothness penalty resolves this failure. We further characterize a representational ceiling facing crop‑feature refinement agents that share a backbone with their base detector, confirmed through a global‑plus‑local observation ablation. Project page: https://prithviraj97.github.io/marl‑net
Authors:Chu-Hsuan Lin, Alberto Mario Ceballos-Arroyo, Jisoo Kim, Shrikanth M. Yadav, Huaizu Jiang, Lei Qin, Geoffrey S. Young
Abstract:
In this work, we present SemanticVessel, a dataset for fine‑grained brain vessel segmentation in computed tomography angiography scans. Based on the detailed contrast provided by dynamic 4D‑CTA scans, we generate segmentation traces for arteries and veins. We then use intensity‑guided region growing to obtain segmentations of the majority of vascular territories in the human brain, which are refined and annotated with 20 unique arterial classes by an expert radiologist. Unlike existing datasets, where minor arteries are discarded as background content, we merge these minor arteries into a generic arterial class. Due to the multiple‑phase acquisition of dynamic 4D‑CTA, labels for a single phase can be re‑used for other phases in the same series, greatly increasing the size of our dataset with no additional annotation cost. The results show that models trained with the additional generic artery class produce better fine‑grained segmentations across the board. We will make our code, annotation GUI, and model weights available to the scientific community. Code, weights, and data will be made available on https://github.com/alceballosa/robust‑vessel‑segmentation
Authors:Sachin Sundar, Sandeep Kumar, Mothish M
Abstract:
Detecting online polarization remains a critical challenge, particularly in multilingual and multicultural contexts where intergroup hostility is prevalent. The problem is particularly challenging due to the data scarcity for these tasks in the low‑resource languages. Identifying such phenomena has become an active area of research and is addressed in SemEval‑2026 Task 9: Multilingual, Multicultural Online Polarization Detection. To address this problem we propose an architecture that leverages LaBSE embeddings ‑ an unconventional choice typically reserved for retrieval tasks, to obtain strong cross‑lingual learning which enhances scores in low‑resource language by a score up to 0.2 macro F1. Furthermore, we provide a comprehensive ablation study evaluating the performance of diverse encoder models in the Qwen model family within a retrieval‑based prompting framework. Our code will be soon available at https://github.com/carrycurious/PolarMind.
Authors:Tairan Xu, Leyang Xue, Zhan Lu, Jinfu Deng, Hongyang Xiao, Yinsicheng Jiang, Congjie He, Matej Sandor, Le Xu, Luo Mai
Abstract:
Batch inference has become a central mode of AI computation, yet existing inference engines still rely on execution models designed for interactive serving. When scaled to millions of sequences, batch workloads reveal two fundamental requirements: the ability to handle extreme inter‑ and intra‑sequence load variation that emerges only at runtime, and the ability to sustain high utilization across large fleets of GPUs. Existing systems fail to meet these requirements, losing substantial fractions of achievable throughput. We introduce a new architectural foundation for batch inference: the sequence coroutine compute model, which represents each sequence as a fine‑grained, event‑driven coroutine. This model exposes expressive primitives that allow the runtime to reorganize work dynamically, enabling larger expert‑level batches, mitigating stragglers, reallocating work across devices, and maintaining utilization even on cost‑effective or memory‑constrained GPUs. Building on this abstraction, we implement BatchGen, a production‑ready system that uses the coroutine model at cluster scale. On a 128‑GPU cluster, BatchGen reduces batch completion time by up to 2.3×, and on memory‑constrained accelerators it outperforms the strongest offloading baseline by up to 9.6×. We will open‑source BatchGen at https://github.com/batchgen‑project/batchgen
Authors:Saifelden M. Ismail, Aser O. Ibrahim, Omar A. Mahmoud
Abstract:
Phishing is a multi‑modal threat. We present a hybrid pipeline that scores each modality with its own engine and fuses the results. Three engines are built, deployed, and independently benchmarked: a four‑stage URL stack (Domain Guard, lexical model, threat intelligence, and an asymmetric L2 fusion sidecar); a generalization‑hardened DistilBERT NLP classifier whose held‑out real‑phishing recall rises from 0.8% to 87.3%; and a threat‑intelligence synchronizer with end‑to‑end OpenTelemetry instrumentation confirming 1:1 message conservation. A decision‑level fusion stage, characterized on a 10,677‑email whole‑system benchmark, reaches F1 = 0.914 with a calibrated probabilistic‑OR over URL, header, and phishing‑probability channels while cutting held‑out real‑spam false positives to 3.6%. Because that benchmark uses proxy URL and header channels and an operating point still needing recalibration, we present it as a preliminary integrated result. The binding constraint for deployable detection is generalization rather than same‑distribution accuracy.
Authors:Tianyou Wang, Anson Lei, Joe Watson, Ingmar Posner
Abstract:
Imitation learning has emerged as a powerful paradigm for learning visuomotor policies, but its generalisation and stability are limited by the scale and quality of demonstration data needed. A promising direction is to leverage more abundant but heterogeneous data sources, which differ in action space and often lack action labels altogether. Existing co‑training approaches that combine heterogeneous data sources rely on heuristic and hand‑engineered alignment techniques. In contrast, we argue that action representations should be grounded in prediction: actions that produce the same effect on the environment should share the same representation, regardless of their sources. To this end, we instantiate this principle by using a grounded latent‑action world model (GLAM), a pair of generative models with a shared latent action space across data sources that is grounded by predicting future observations consistently across sources. This latent action space is used to train downstream behavioural cloning (BC) policies which map observations to latent actions and decode them back to robot actions, providing a paradigm for learning from heterogeneous data. Empirically, we demonstrate that GLAM successfully learns an aligned latent action space that facilitates action transfer across data sources with and without action labels. Across five manipulation tasks in simulation and in the real world, GLAM‑aligned policies significantly outperform BC baselines and prior latent‑action methods, achieving an average of +48% improvement in task success rate with the same data‑scarce setting. Videos and code are available at https://viccccciv.github.io/glam/.
Authors:Jiehui Huang, Yuechen Zhang, Bin Xia, Jiahao Wang, Xu He, Zhenchao Tang, Meng Chu, Xin Tao, Pengfei Wan, Jiaya Jia
Abstract:
Generating a coherent multi‑shot video requires structured cross‑shot memory. Subject appearance, scene context, and speaker identity must persist across cuts. Existing approaches either train end‑to‑end over fixed‑length sequences and cannot scale, generate shot‑by‑shot with memory banks that grow linearly, or orchestrate pretrained generators under an LLM planner without a multi‑shot‑aware backbone. We present UnityShots, a memory‑driven multi‑shot audio‑video generation system built on LTX‑2.3, trained on annotated cinematic and music‑video shots. The video stream maintains two fixed‑size slots, a long‑term memory (LTM) slot anchored to the opening shot and a short‑term memory (STM) slot holding the immediately preceding tail, both updated at every cut by a boundary‑conditioned gate that fuses visual cut probability and beat‑tracker signals. The audio stream injects a reference speaker token at every shot to preserve vocal timbre without a sliding audio bank. A discrete cut‑type prior, learned through AdaLN, becomes an inference‑time control knob over transition strength. We release a benchmark of 200 multi‑cultural multi‑shot sequences spanning six ethnic regions and ten or more languages, with per‑shot reference identities, reference audio, and per‑boundary transition labels. Evaluated across I2V, T2V, and R2V conditioning modes, UnityShots leads open‑source baselines on every cross‑shot coherence metric and matches the strongest closed‑source system on the multi‑shot axes.
Authors:Chang Nie, Chaoyou Fu, Junlan Feng, Caifeng Shan
Abstract:
Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long‑context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain‑180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment‑batching techniques that tackle significant length variance and accelerate training by 3.8×. Extensive experiments show that our model not only outperforms larger‑scale specialists (e.g., Qwen3‑Embedding‑8B and KaLM‑Embedding‑Gemma3‑12B) across a range of long‑context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10× longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare‑nie.github.io/EvoEmbedding.
Authors:Tao Sun, Utkarsh Aashu Mishra, Jiaxin Lu, Danfei Xu, Iro Armeni
Abstract:
Compositional diffusion planners aim to solve long‑horizon robotic tasks using short training trajectories. Yet, current approaches often rely on the heuristic stitching of local predictions. We show that the resulting stitched update is generally a non‑conservative field that does not mathematically correspond to any valid global trajectory log‑density function. We propose Energy‑based Compositional Diffuser (ECD), a framework that formulates the global trajectory as the minimizer of the sum of local bridge potentials. This energy‑based perspective defines a conservative correction field and contains a boundary reaction term that heuristic stitching omits. To enable efficient inference, we further introduce a Markov‑based score approximation that computes the reaction term via a single block‑tridiagonal solve, maintaining time complexity linear in the planning horizon. Empirically, ECD achieves state‑of‑the‑art success rates on a range of OGBench stitching tasks, while nearly matching the inference speed of heuristic stitching methods. Code is available at https://github.com/GradientSpaces/ECD.
Authors:Zhiqing Yang, Yilun Liu, Yunpu Ma, Volker Tresp, Hinrich Schütze
Abstract:
Large language models (LLMs) can readily reproduce conventional expressions, yet their ability to model gradient frequency distributions remains underexplored. We investigate this using linguistic binomials, such as men and women, where both word permutations are grammatically valid but exhibit distinct, cross‑linguistic variations in conventionality. We formalize binomial ordering as a distributional alignment problem, and construct a multilingual dataset of 600 binomial pairs across 8 languages. With categorical and distributional metrics, we measure and compare the corpus‑derived preferences with model‑induced ordering probabilities of 6 open‑weight LLMs. While models often behaviorally recover the dominant corpus‑preferred order, particularly for strongly conventionalized pairs, they align less well with the exact corpus preference distributions. This suggests that apparent directional order overstates how faithfully LLMs capture the statistical nuances of language use. Sparse probing verifies that the concept of preference strength is partially encoded among middle‑to‑late layers, and steering along probe‑derived directions alters model‑induced ordering distributions, demonstrating that the statistical behavioral preference of LLMs can be mechanistically measured and manipulated via internal representations.
Authors:Zijian Fu, Xiangyang Chu, Mengshi Qi, Huadong Ma, Guanghao Zhang, Wei Li
Abstract:
Instruction‑conditioned trajectory prediction is an emerging problem in autonomous driving, where a model predicts the future ego trajectory not only from visual scene context and historical motion, but also from a natural‑language maneuver instruction. This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive, a vision‑language‑action driving agent with 3D perception, reasoning, and planning capabilities. We adapt OmniDrive to the doScenes setting by training it on instruction‑annotated nuScenes scenes and generating a 6‑second ego trajectory represented by 12 future waypoints. To improve multi‑view visual grounding, we further introduce a DVPE‑style divided‑view perception module into the OmniDrive perception head. Instead of attending globally to all camera features, the proposed module groups query features and image tokens into divided local view spaces and performs visibility‑aware cross‑attention within each view. This design reduces irrelevant cross‑view interference and helps the model better align language instructions with local driving‑relevant visual evidence. The code is publicly available at: https://github.com/feel12348/doscenes‑omnidrive.
Authors:Zhangwei Cao, Shuhan Fan, Yuting Wei, Jiajun Zhang, Yihang Peng, Qi Meng, Yangfu Zhu, Liangbin Yang
Abstract:
Evaluating Multimodal Large Language Models (MLLMs) in Chinese Cultural Heritage (CCH) requires fine‑grained reasoning over visual, textual, stylistic, and historical clues. However, existing CCH benchmarks mainly emphasize final‑answer accuracy, while the accuracy and completeness of reasoning processes remain underexplored. To address this gap, we introduce CulMind and CulMind‑R: a high‑quality benchmark for multimodal CCH covering 50 tasks from collections of more than 100 museums, and a 24‑task reasoning subset that adaptively defines task‑specific dimensions for reasoning process evaluation. To evaluate reasoning quality, we propose ReaScore, a task‑adaptive metric that evaluates reasoning by automatically weighting task‑relevant dimensions. Experiments on 14 leading MLLMs reveal a substantial gap between answers and reasoning, especially on challenging tasks. Further analysis shows that task‑adaptive dimension selection and weighting better align evaluation results with expert judgments. Overall, our benchmark and metric support a more expert‑aligned assessment of CCH understanding and offer a transferable reference for broader evaluations of cultural heritage. We publicly release the data, code, and evaluation scripts at https://github.com/ZevTsao/CulMind to facilitate reproducible research.
Authors:Lucas Fagan, Michele Tarquini, Ali Shehper, Maksymilian Manko, Angus Gruen, Coco Huang, Giorgi Butbaia, Davide Passaro, Sergei Gukov
Abstract:
Mathematical search problems present a unique challenge for Reinforcement Learning (RL) due to vast search spaces and sparse rewards. In previous works, the Andrews‑Curtis (AC) conjecture was established as an illustrative example of such problems. In this work, we identify a critical structural barrier in the AC landscape: a "Two‑Hump" distribution, where problem instances are either trivially solvable or effectively impossible, with a scarcity of intermediate "hard‑but‑solvable" instances required for effective learning. We tackle this challenge through two primary avenues: novel data generation techniques to populate the difficulty gap, and significant algorithmic enhancements including the introduction of supermoves and Transformer‑based architectures. We demonstrate substantial performance improvements over previous baselines, and release new comprehensive benchmark datasets including AC‑19 (125,192 AC‑trivial presentations of varying difficulty with length at most 19) and AC‑1M (1,136,154 hard AC‑trivial presentations of length at most 30), the first large‑scale, publicly available datasets of this kind.
Authors:Efe Ilıcak, Baris Imre, Chloé Najac, Ruben van den Broek, Beatrice Lena, Andrew Webb, Marius Staring
Abstract:
Deep unrolled networks (DUNs) integrate physical forward models with learned regularization in cascaded network architectures, achieving exceptional performance in inverse problems while maintaining interpretability. While most DUNs operate in the object domain (e.g., image space), recent variants explored representation spaces for improved information flow. However, these methods rely on heuristic methods for data consistency (DC), sacrificing fidelity with measurements. In this work, we introduce DUNE (Deep Unrolled Networks in rEpresentation space), a framework that maintains exact adherence to physical measurements while operating in learned representation spaces. By deriving the DC gradient via the chain rule and implementing it through the Vector‑Jacobian Product (VJP), we enable exact backpropagation of measurement residuals into the representation space. This formulation supports diverse architectural backbones, including pre‑trained encoders to guide the iterative process. We assess DUNE against state‑of‑the‑art baselines on accelerated MRI reconstruction tasks, demonstrating that exact VJP‑based gradients yield superior reconstruction quality and structural fidelity across both single‑channel portable low‑field and multi‑channel clinical high‑field MRI acquisitions. The code will be available upon publication at https://github.com/EfeIlicak/DUNE.
Authors:Linara Adilova, Henning Petzka, Asja Fischer, Bernhard C. Geiger
Abstract:
Deep neural networks transform input data into latent representations that support a wide range of downstream tasks. These representations can be characterized along information‑theoretic and geometric dimensions, but their relationship remains poorly understood. A central open question is whether low mutual information (MI) between inputs and representations necessarily implies geometrically compressed latent spaces and vice versa. We investigate this question using class‑wise clustering as a measure of geometric compression and theoretically sound MI estimation in conditional entropy bottleneck (CEB) networks and continuous dropout networks. We evaluate the interplay between MI, geometric compression, and generalization on classification tasks under controlled noise injection schemes. Our findings show that low MI does not reliably correspond to geometric compression, and that the connection between the two is more nuanced than often assumed. Indeed, our experiments reveal a negative and nonlinear relationship that can reverse when varying training setup. Our results put forward a hypothesis that generalization acts as a potential confounder in this connection rather than being their direct consequence.
Authors:Eight Suzuki, Hideitsu Hino, Noboru Murata
Abstract:
Transformers achieve strong performance, but their internal computations remain opaque. We view each Transformer layer as a dynamic graph whose nodes are token representations and per‑head attention outputs, with Multi‑Head Attention (ATT) and MLP as module boundaries. On this graph we use LIG (Layer‑wise Integrated Gradients), which applies set‑to‑set Integrated Gradients (IG) at nonlinear module boundaries. Set‑to‑set IG applies IG to a map from a set of input token representations to a set of output representations, evaluating token‑to‑token contributions, which is not standard in prior IG applications. This extends IG from the usual scalar‑objective setting to set‑to‑set maps via an L2 scalarization, and composes within‑layer contributions in the spirit of Layer‑wise Relevance Propagation (LRP), with IG completeness playing the role of LRP‑style conservation at each boundary. We use LIG to analyze (i) the agreement between module‑wise composition and layer‑whole attribution under an L2 criterion, and (ii) within‑layer information flow by tracing separated ATT and MLP contributions. On BERT‑base and PTB, configurations that best preserved within‑layer consistency used the target token's embedding as the ATT baseline and either the ATT output at a=0 or Zero as the MLP baseline. We therefore present LIG as a diagnostic XAI tool at module‑boundary granularity, without model‑specific retraining or per‑operation interpreter design. Code is available at https://github.com/eightsuzuki/layer‑wise‑integrated‑gradients.
Authors:Murong Yue, Desmond Alexander Mcglone, Emily Slutz, Wenhan Lyu, Yixuan Zhang, Jennifer Suh, Ziyu Yao
Abstract:
Collaborative Problem Solving (CPS) is a core skill in education, where the process of peer interaction is highly important. However, existing educational dialogue datasets mostly focus on classroom instruction or tutoring (i.e., teacher/tutor‑student interaction), yet datasets centering small‑group, student‑student interaction are limited. This thus leaves research with limited resources for studying how students interact, coordinate, and solve problems together in real educational settings. To address this, we introduce PeerMathDial, the first dataset of peer CPS dialogues collected from authentic middle school math classrooms. It contains 55 dialogues from 27 students, totaling 6,406 turns. To facilitate research on CPS discourse analysis, we further build a corpus‑grounded dialogue act taxonomy assisted by LLMs. Using the dataset and the dialogue act taxonomy, we demonstrate the practical applications of PeerMathDial across three use cases. First, we track how dialogues evolve over time and measure the impact of teacher interventions. Second, we align dialogue actions with student surveys to reveal the connection between students' traits (e.g., confidence, leadership) and their actual behaviors. Third, by evaluating LLMs on dialogue act prediction, we glimpse at the potential of LLMs for student simulation in educational applications. Our dataset and source code will be released to the community.
Authors:Sheroz Shaikh
Abstract:
Agentic retrieval‑augmented generation (RAG) systems combine iterative reasoning loops, query decomposition, and adaptive retrieval to tackle multi‑hop question answering. However, the contribution of each component remains poorly understood, particularly under resource‑constrained settings using only local language models. Many agentic designs add adaptive retrieval routing and deeper retrieval loops on the assumption that the added complexity helps. To test whether it does, we run a controlled ablation study of a full agentic RAG pipeline evaluated on 5,000 questions from the HotpotQA distractor development set using a local 7B parameter model (Qwen2.5‑7B‑Instruct). Our full pipeline achieves EM=53.2% and F1=61.6%, compared to a single‑pass dense‑retrieval baseline of EM=43.1% and F1=54.0%. Across eight ablation conditions, we find that: (1) fixed hybrid retrieval via reciprocal rank fusion consistently outperforms rule‑based adaptive routing (+1.8 EM, +1.9 F1), as the routing heuristic over‑routes to BM25 by firing on named entities present in nearly all multi‑hop sub‑questions; (2) two retrieval iterations over the decomposed sub‑questions capture 95% of the gains of five, with no meaningful benefit from deeper loops; and (3) query decomposition and cross‑encoder reranking each contribute statistically significant but smaller gains (p<0.01 and p<0.001 respectively). Taken together, on a fixed local‑model budget, the simpler and fixed choices turn out to be competitive with or better than their adaptive versions: most of the gain comes from running a short retrieval loop, not from adaptive routing or from many iterations. We use no proprietary APIs or large‑scale compute.
Authors:Yueyuan Li, Yifei Xiao, Mingyang Jiang, Xiang Zuo, Songan Zhang, Ming Yang
Abstract:
End‑to‑end autonomous driving systems tightly couple perception and decision‑making through latent representations. Consequently, updates to perception models can alter these representations and degrade the performance of downstream policies that remain fixed. Existing solutions typically rely on policy retraining or architectural decoupling, both of which incur substantial computation and validation costs. In this paper, we formulate the model stitching problem for end‑to‑end autonomous driving and test the hypothesis that policy compatibility can be preserved through lightweight latent‑space alignment. We study low‑complexity model stitching methods, including linear and convolutional stitchers, for restoring compatibility between updated perception modules and frozen downstream policy modules. Experiments demonstrate that stitching effectively preserves downstream driving behavior under diverse perception updates, including changes in random initialization, sensor configuration, and training domain. In the most challenging cross‑domain setting from nuScenes to CARLA, convolutional stitching retains over 91% of the no‑shift driving score while reducing adaptation time from \SI22.18h to \SI0.91h. These results suggest that model stitching provides an effective and computationally efficient alternative to retraining or fine‑tuning for maintaining end‑to‑end autonomous driving systems. The model will be open‑sourced upon paper acceptance at https://github.com/SCP‑CN‑001/model‑stitching to support further research and development in autonomous driving.
Authors:Yuanhao Chiang, Hongbo Duan, Chunru Yang, Jiahua Pei, Yi Liu, Xueqian Wang
Abstract:
Autoregressive text‑to‑image (T2I) generation has recently advanced rapidly, yet aligning generated images with human preferences remains challenging. GRPO‑style online reinforcement learning provides an effective framework; however, existing methods typically treat reference‑policy divergence as fixed, despite its direct impact on policy optimization. We study this overlooked factor within a unified f‑divergence framework, encompassing forward KL, reverse KL, and JS divergence, for GRPO‑style autoregressive T2I alignment. Our systematic theoretical analysis reveals that different divergences reshape token‑level updates in distinct ways. In particular, under the sampled‑token shaping form used, JS regularization achieves a favorable trade‑off by mitigating uniform bias relative to the reference policy while still discouraging large deviations. Extensive experiments on LlamaGen and Janus‑7B show that JS divergence achieves the strongest or highly competitive optimization performance on most evaluation metrics while maintaining favorable generation diversity. The code is available at https://github.com/tuoyou‑hao/BPD‑GRPO.
Authors:David Pascual-Hernández, Roberto Calvo-Palomino, Inmaculada Mora-Jiménez, Jose María Cañas-Plaza
Abstract:
In this report, we present our submission to the GOOSE 2D Fine‑Grained Semantic Segmentation Challenge, organized as part of the Workshop on Field Robotics at ICRA 2026. The challenge combines data from the GOOSE and GOOSE‑Ex datasets, which comprise more than 13k images captured from 4 distinct camera setups, annotated using a hierarchical taxonomy of 56 fine‑grained classes and 11 broader categories. Starting from SegFormer as a baseline, we progressively improve segmentation performance through increased training crop sizes, a transition to the query‑based Mask2Former architecture, and test‑time augmentation. Our experiments show that query‑based segmentation significantly outperforms the baseline model. Furthermore, increasing the crop size used during training yields substantial gains, highlighting the relevance of preserving scene context for fine‑grained semantic disambiguation. Our final submission, using test‑time augmentation, achieves an mIoU of 69.6% on the challenge test set, providing a strong baseline for fine‑grained semantic segmentation in outdoor environments. To facilitate reproducibility and future research, code and weights will be made publicly available at https://github.com/RoboticsLabURJC/outdoor‑fine‑grained‑segmentation .
Authors:Yang Yue, Xuancheng Zhu, Yuyang Ma, Guoshun Nan, Zihan Dou, Jingru Shan, Congyu Guo, Ji Zhang, Hua Wang, Jingfeng Zhang
Abstract:
The automated design of agentic systems offers a promising pathway for scaling large language models (LLMs) beyond single‑agent reasoning. While prior work has advanced task performance through handcrafted or automatically generated multi‑agent workflows, robustness is often treated as an afterthought, leaving systems vulnerable to external adversaries and internal failures. We propose AutoRAS, a framework for the Automated design of Robust Agentic Systems. AutoRAS formulates system design as generating a sequence of symbolic primitives that jointly encode structural connectivity and behavioral actions, and learns to optimize this sequence using execution‑derived safety signals and flow‑based sequence‑level objectives. Extensive experiments show that AutoRAS achieves the best performance in both vanilla and adversarial settings, with the smallest performance degradation under attacks. Further analyses demonstrate strong transferability, stable optimization behavior, stability across primitive sets, and favorable cost trade‑offs. Our code is available at \hrefhttps://github.com/guohezuy/AutoRAS\textthis https URL.
Authors:Alfarizy Alfarizy, Hung Truong Thanh Nguyen, René Richard, Roozbeh Razavi-Far, Hung Cao
Abstract:
Mixture‑of‑Experts (MoE) language models are often described as ideal for resource‑constrained inference. Each token activates only a small subset of experts, so the per‑token compute cost, in floating‑point operations (FLOPs), resembles that of a much smaller dense model. Whether that FLOP advantage survives in practice is far less clear. We ask whether MoE models actually run faster and cheaper than comparable dense models on consumer‑grade and edge hardware. We benchmark OLMoE‑1B‑7B (1.3 B active of 6.9 B total) against three dense baselines on an Apple M2 Pro and an NVIDIA Jetson Orin Nano 8 GB through llama.cpp, measuring throughput, memory, and on‑device energy. The answer is device‑dependent: OLMoE's active‑parameter advantage is only partly realised on the laptop (~10% behind the same‑active Llama‑3.2‑1B) and erodes on the edge device (~31% behind, at 2.1× the energy per token, with peak memory at the 8 GB ceiling). Patching llama.cpp to time the decode graph node‑by‑node shows routing accounts for under 9% of MoE‑block compute on the cleaner edge backend, so the gap reflects total‑parameter memory footprint, expert dispatch, and KV‑cache pressure rather than routing. The implication is that on bandwidth‑bound edge hardware, inference cost tracks total parameters, not active ones, and sparse activation does not buy back what the device is constrained on. These findings are bounded to one MoE model at this parameter scale and two devices, and we release the full measurement harness and per‑run data.
Authors:Yanlei Liu, Zhenyu Liao
Abstract:
Tensor factorization (TF) has been widely adopted for high‑dimensional sparse data completion tasks. Despite significant progress, neural TF methods often struggle to capture complex cross‑mode interactions and remain vulnerable to (extreme) data sparsity. To address these challenges, we propose a novel neural tensor factorization approach, termed Dual‑Attention Convolution Expert Networks with Group‑Level Contrastive Learning (DCGC). For the first problem, DCGC generates diverse non‑linear alignment patterns of latent factors via a multi‑channel convolution network, and leverages the gated dual‑attention mechanism to drive the model to focus on more important output channels (i.e., convolution experts) and the aligned features. Furthermore, DCGC introduces a group‑level contrastive learning strategy that aggregates positive samples with identical feedback levels while separating negative samples across different levels. This strategy injects high‑quality self‑supervised signals to mitigate data sparsity. Extensive experiments conducted on five datasets demonstrate that our DCGC outperforms the state‑of‑the‑art methods in sparse tensor completion for traffic and recommendation applications. Code to reproduce the experimental results in the paper is available at https://github.com/ku1z/DCGC.
Authors:Hanzhi Chen, Anran Zhang, Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers, Oier Mees, Stefan Leutenegger
Abstract:
A central question in robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation, embodied practice, and the experience of failure. Human videos provide the first of these in abundance, and prior work has shown they can initialize useful policies. Far less clear is whether they can support the second and third: whether priors extracted from human videos can ground a robot's own attempts well enough to evaluate them, correct them, and improve from them. In this work, we show that human videos can be used to learn embodiment‑agnostic action, dynamics, and value representations that transfer across robot embodiments, providing the predictive foundation required for robots to autonomously improve from their own rollouts and failures. We introduce Dynamics‑Guided Action Correction (DGAC), a training‑free approach that leverages these adapted models to repair failed states: each failure becomes a query for which the learned models propose and rank corrective actions, turning failures into supervision for the next policy update. Across seven real‑world manipulation tasks spanning both a mobile manipulator and a static manipulator arm, our approach improves success rates from 40% to 81% across multiple policy backbones, demonstrating cross‑embodiment robot self‑improvement from human‑video priors. These results show that human priors and robot failures can be combined to enable scalable autonomous policy improvement. Project page: https://ethz‑mrl.github.io/robot‑self‑improvement‑website/.
Authors:Dwarikanath Mahapatra, Abhijit Das, Behzad Bozorgtabar, Zongyuan Ge, Sudipta Roy, Deepak Nayak, Mauricio Reyes, Imran Razzak
Abstract:
Multimodal medical imaging fuses complementary anatomical and functional information, yet modalities frequently disagree in pathologically heterogeneous regions. Current segmentation models handle this in one of two inadequate ways: deterministic fusion that averages away disagreement, or post‑hoc uncertainty estimation decoupled from the fusion process that produces it. Both obscure the clinically critical question: why is this prediction unreliable? We present EnTrust, a framework that treats inter‑modal conflict as the primary source of predictive uncertainty. Our EnFuse module decomposes multimodal features into three disentangled components: shared anatomical consensus (F_c), modality‑specific cues (F_u,m), and spatially localized conflict signals (F_cf), with independence enforced via a cross‑covariance objective. This structured decomposition conditions SegDiff, a diffusion‑based generative segmentation model whose sampled hypotheses diverge specifically in regions of modal disagreement. TrustMap then translates this hypothesis divergence into calibrated, pixel‑wise uncertainty using ensemble entropy, conflict‑guided perturbation probing, and a learned calibration head, enabling clinicians to understand not only where predictions are uncertain, but why. Across four benchmarks spanning brain, cardiac, lesion, and oncology domains, EnTrust achieves state‑of‑the‑art segmentation accuracy while reducing calibration error by 40% compared to the strongest baseline. Notably, it outperforms 5x deep ensembles using a single model at roughly half the memory footprint. Code and checkpoints are available at https://github.com/GenMI‑Lab/EnTrust.git.
Authors:Nichula Wasalathilaka, Abhijit Das, Imran Razzak, Dwarikanath Mahapatra
Abstract:
Medical image re‑identification (MedReID) enables longitudinal patient linkage but remains vulnerable to shortcut learning and often produces decisions that clinicians cannot audit against named anatomy. We propose Graph‑of‑Differences (GoD), which grounds identity comparisons in explicit anatomical structure. Each image is represented as an anatomy graph whose nodes correspond to named anatomical regions; given an image pair, soft node correspondence is established, and differences are computed over matched anatomy. A graph‑level difference alignment objective ties these anatomy‑matched differences to the global backbone difference, ensuring the retrieval signal is anchored in homologous structures rather than arbitrary spatial tokens. Explanations are defined over named graph nodes and quantitatively audited via node insertion/deletion tests, replacing unstable pixel heatmaps with verifiable structure‑level evidence. On internal benchmarks, GoD improves Rank‑1 by +7.1 pp on fundus and +3.1 pp on CXR over a strong frozen‑backbone baseline, with further gains on zero‑shot external transfers confirming that anatomy grounding improves both accuracy and generalization. Code is available at https://github.com/GenMI‑Lab/GoD.git.
Authors:Cong Wan, Zeyu Guo, Zijian Cai, Jiangyang Li, SongLin Dong, Lin Peng, Xiangyang Luo, Zhiheng Ma, Yihong Gong
Abstract:
Massive unstructured multimodal streams suffer from high "data entropy," impeding both efficient human knowledge acquisition and high‑quality AI post‑training. Existing passive annotation paradigms, heavily reliant on heuristic rules or general VLMs, are costly, monotonous, and fail to unlock the deep procedural logic embedded in raw data. We elevate data processing to a learnable capability, proposing a paradigm shift towards Agentic Data Tailoring, which actively refining and structuring data to align with diverse user and downstream intents. To overcome the data scarcity bottleneck in training such high‑order capabilities, we design a two‑stage pipeline grounding generative semantic synthesis in deterministic Factual Anchors, yielding a large‑scale dataset spanning five core physical and digital domains. Building upon this, \textDataClaw_0‑9B model synergizes Supervised Fine‑Tuning (SFT) with Group Relative Policy Optimization (GRPO), achieving robust alignment with complex refinement and tailoring intents. To systematically quantify this capability, we construct \textDataClaw_0‑val, the first benchmark dedicated to data refinement. Crucially, we adopt downstream post‑training as the ultimate validation touchstone. Evaluations on video generation, real‑world VQA, and GUI navigation confirm that \textDataClaw_0 delivers high‑information‑density tailored data, facilitating efficient model adaptation to new tasks under limited training data regimes. Project page: https://czjdsg.github.io/MakeAnyData
Authors:Mingzhi Chen, Yiyu Gui, Guibo Luo
Abstract:
Medical time series (MedTS) signals such as electroencephalography (EEG) and electrocardiography (ECG) support many clinical applications. However, substantial subject‑level heterogeneity often induces subject‑level distribution shift, causing a fixed parameter set to generalize poorly to unseen individuals. Compared with domain adaptation methods that often depend on extra adaptation components or target‑batch statistics, Test‑Time Training (TTT) provides a more practical solution for sequential clinical data by enabling online adaptation from unlabeled test samples. However, many representative TTT methods require iterative inner‑loop optimization, increasing test‑time overhead. In this paper, we propose MedTS‑TTT, a test‑time training framework for medical time series modeling. MedTS‑TTT is built upon Closed‑Loop Self‑Alignment Test‑Time Training (CLSA‑TTT) and a Gated Convolutional Backbone (GCB). CLSA‑TTT constructs a token‑level self‑supervised target and performs a single‑step fast‑weight update for intra‑layer closed‑loop alignment, enabling rapid sample‑wise adaptation without iterative inner‑loop optimization. GCB combines CLSA‑TTT‑based fast adaptation and token‑level fusion with a gated convolutional branch to balance local dynamic modeling and information‑flow control. On 4 public datasets (2 EEG and 2 ECG) with subject‑independent splits, MedTS‑TTT achieves 11 top‑1 rankings out of 12 evaluations across 9 baselines and 3 metrics. The code is publicly available at https://github.com/mingzhi‑c/MedTS‑TTT.
Authors:Zheng Zhang, Lihe Yang, Tianyu Yang, Chaohui Yu, Yixing Lao, Xiaoyang Guo, Biao Gong, Fan Wang, Hengshuang Zhao
Abstract:
We present SCOPE (Scale‑Consistent One‑Pass Estimation of 3D Geometry), a novel approach for estimating 3D geometry from extended monocular video sequences, where existing methods struggle to maintain both geometric accuracy and temporal consistency across hundreds of frames. Our approach generates affine‑invariant 3D point maps with shared parameters across entire sequences, enabling consistent scale‑invariant representations. We introduce three key innovations: viewpoint‑invariant geometry aligning multi‑perspective points in a unified reference frame; appearance‑invariant learning enforcing consistency across exponential timescales; and frequency‑modulated positioning enabling extrapolation to sequences vastly exceeding training length. Experiments across diverse datasets demonstrate significant improvements, reducing relative point map error by 24.2% and temporal alignment error by 34.9% on ScanNet compared to state‑of‑the‑art methods. Our approach handles challenging scenarios with complex camera trajectories and lighting variations while efficiently processing extended sequences in a single pass. Project page: https://scope3d.github.io/.
Authors:Adnan Mustafic, Halim Benhabiles, Adnane Cabani, Kristhian André Oliveira Aguilar, Romain Amigon, Clément Bardin, Chiara Bentifece, Marin Boehm, Kévin Bouchard, Laura Burattini, Diedre Carmo, Fahima Idiri, Matthis Lahargoue, Ilaria Marcantoni, Hicham Messaoudi, Cyril Meyer, Farid Meziane, Léon Morales, Letícia Rittner, Agnese Sbrollini, Léonard Zipper, Karim Hammoudi
Abstract:
We propose NoduLoCC2026, a challenge on lung nodule detection and localization in chest X‑ray images. We have provided a dataset for both tasks and received submissions from 5 international teams. The participating teams' solutions are presented in this work along with results on an external dataset used for testing. Proposed methods show good performance on the classification task. The best method shows a balanced accuracy score of 0.72 and AUC‑ROC of 0.79. We highlight the limitations of current approaches for the localization task, with the best approach having predicted the correct number of nodules on 53% of the test images with a median distance of 12.83mm, showing that it is a more challenging task than the first one. The challenge website is available via https://gt‑i2mdp.github.io/website/nodule_challenge.html.
Authors:Zihang Tian, Jingsen Zhang, Rui Li, Xiaohe Bo, Yuanzi Li, Xu Chen
Abstract:
Reinforcement learning for multi‑step LLM agents often relies on scalar rewards that indicate success but cannot explain why a trajectory is good or bad. Rubric‑based rewards improve interpretability through natural‑language criteria, but existing methods score at the trajectory level and freeze the scorer behind a closed‑source judge, leaving step‑level credit assignment unresolved and the judge itself static. We propose ARCO (Adaptive Rubric CO‑evolution), a rubric framework in which a same‑scale model μ shares a backbone with two heads: a generation head that produces per‑step criteria, and a score head that predicts rubric‑conditioned step‑level rewards. A trajectory decomposition constraint ties the sum of step rewards to the terminal outcome, enabling credit assignment without step‑level labels, while μ and the policy π are jointly updated on on‑policy data so that the rubric content and the scoring function co‑evolve at the parameter level. Across HotpotQA, 2WikiMultiHopQA, and MuSiQue with two open‑source backbones, ARCO improves the best EM in every setting over strong outcome‑, rubric‑, and process‑reward baselines, and analyses show that its rubrics are step‑specific, robust to design choices, and useful for diagnosing agent behavior. Codes and data are available at https://github.com/zihangtian/ARCO.
Authors:Talha Ilyas, Deval Mehta, Zongyuan Ge
Abstract:
Video‑based seizure detection is essential for the management of epilepsy patients, offering a non‑invasive complement to electroencephalography. While several deep learning approaches have been developed for video‑based seizure detection, none are inherently interpretable, limiting their adoption and translation into clinical practice. We present, to our knowledge, the first exploration of a neurosymbolic framework for video‑based seizure detection that directly addresses this gap. Our approach (1) extracts patient‑centric skeleton sequences from epilepsy monitoring units via a prompt‑guided foundation model, (2) predicts binary spatio‑temporal concept activations grounded in clinical motor semiology guidelines, and (3) composes them via differentiable logic into interpretable Boolean rules with auditable contributions. Furthermore, to mitigate false positives arising from the traditional binary formulation (seizure vs.\ non‑seizure), we sub‑classify non‑seizure segments into clinically relevant normal activities, providing the model with fine‑grained discriminative supervision. Evaluated on two public seizure video benchmarks, our framework achieves 89.78% sensitivity with 0.06 false detections per hour on SAHZU and 85.27%,0.09 on IEEE, while producing complete three‑level interpretability: every prediction decomposes into which motor primitives were detected, how they were logically composed, and how much each rule contributed to the clinical decision. We publicly release all annotations, extracted pose sequences, our data pipeline and code, https://github.com/Mr‑TalhaIlyas/CDSD/.
Authors:Cengizhan Bayram
Abstract:
Retrieval heads, attention heads that copy information from earlier context to the current position, have been proposed as the mechanistic substrate for long‑context recall. Rotary position embeddings (RoPE) rotate queries and keys by frequencies decaying with a base hyperparameter theta, and a natural hypothesis is that this rotation either prevents retrieval heads from forming or degrades their function. We test both across four open‑weight 7‑8B models spanning multi‑head and grouped‑query attention and a 100x range of theta, using paired‑seed needle‑in‑a‑haystack tests, layer‑clustered permutation, and causal head‑masking. (i) Retrieval heads are causally necessary: masking the 87 detected heads in OLMo‑2 collapses recall from 1.00 to 0.00, while masking matched random heads has no effect; this replicates in Qwen. (ii) Higher theta does not reduce retrieval‑head count (LLaMA‑3.1 at theta=500K has 47 heads vs LLaMA‑2 at theta=10K with 42), refuting the prevention hypothesis. (iii) The norm‑utility relation is family‑specific and significant in opposite directions (Qwen d=‑0.49, OLMo d=+0.50, both significant; LLaMA null); since OLMo and LLaMA‑3.1 share theta=500K yet differ, the effect is not theta‑driven. (iv) Building on Chiang and Yogatama (2025), a controlled patch shows that zeroing the lowest‑frequency RoPE dimensions of retrieval heads degrades recall dose‑dependently (1.00 to 0.18 when 32 of 128 dimensions are zeroed, vs 0.98 for random dimensions); the effect is head‑specific and task‑specific. The causal variable is RoPE frequency, not norm‑utility. The direction holds in all five models patched (OLMo‑2, Qwen2.5‑7B/14B, Gemma‑2, Mistral) across four lineages and two scales. We do not claim cross‑model magnitude. Code and a paired‑seed harness are released.
Authors:Xun Gong, Jinchuan Tian, Haoran Wang, William Chen, Shinji Watanabe, Yanmin Qian
Abstract:
Current text‑guided audio editing methods rely on paired training data, predefined operation templates, and separate processing pipelines across speech, music, and sound. We present Bagpiper‑Edit to enable open‑ended audio editing via free‑form natural language instructions. We reformulate audio editing as a rich‑caption rewriting task by treating a rich caption as the semantic representation of an audio clip. The user request is translated into an edited caption, which then guides Bagpiper‑Edit to generate the target edited audio with the original audio as contextual acoustic anchor. This unlocks the potential of free‑form editing, and circumvents the need for paired audio‑editing training data, enabling powerful zero‑shot editing capabilities. Evaluations across speech, audio, and free‑form editing show Bagpiper‑Edit maintains good consistency to the original audio and achieves similar performance to other expert models in most cases. Demo: https://bagpiper‑edit.github.io, Codes: https://github.com/espnet/espnet/pull/6417 & https://github.com/HsunGong/espnet
Authors:Sergio Lanza, Jae Hee Lee, Stefan Wermter
Abstract:
Vision Language Models (VLMs) have demonstrated impressive performance in tasks requiring joint understanding of images and text, such as image captioning and Visual Question Answering (VQA), but our understanding of their internal processes remains limited. Recently, Sparse Autoencoders (SAEs) have emerged as a promising tool to support the interpretation of concepts encoded in VLMs. However, most SAE‑based approaches focus only on textual or visual concepts separately, ignoring multimodal concepts. This limitation hinders a comprehensive understanding of VLMs, since concepts that integrate both modalities can be misclassified. Moreover, previous visual approaches often produce low‑quality visual concept descriptions that are vague or incomplete, limiting their usefulness for understanding model reasoning. We propose a framework based on SAEs to extract and analyze visual, textual, and multimodal concepts from VLMs. For each neuron, we propose a candidate human‑interpretable concept and compute the alignment between the concept and the dataset samples using cosine similarity scores. Experiments on a VQA dataset (LLaVA‑NeXT) demonstrate that our framework improves visual concept quality by up to 45% compared to existing SAE‑based methods, while maintaining high textual concept quality and enabling systematic identification of multimodal concepts. This work contributes new insights into the conceptual space of VLMs, providing a structured approach to distinguish between visual, textual, and multimodal concepts. The code is available at https://github.com/PHDLanza/Multidata_SAE
Authors:Hounsu Kim, Juhan Nam
Abstract:
Speaker‑decoupled speech codecs can reduce bitrate by separating global speaker attributes from local content and prosody, while supporting voice conversion. Existing speaker‑decoupled codecs face a trade‑off: methods that explicitly suppress speaker leakage often rely on multi‑stage or auxiliary training, whereas simpler designs can leave residual speaker information in local tokens. We propose SDP‑Codec, a speaker‑decoupled, pitch‑injected codec trained with a single‑stage optimization pipeline. SDP‑Codec derives local tokens from continuous pre‑quantization features of a pretrained self‑supervised encoder and injects normalized F0 via a pitch encoder‑decoder with global‑conditioned denormalization and soft‑label pitch reconstruction objective. Across 16 kHz and 24 kHz settings, SDP‑Codec achieves competitive reconstruction and strong zero‑shot voice conversion at comparable bitrates, with the lowest speaker‑probing accuracy among compared systems, suggesting reduced speaker leakage.
Authors:Joohyeok Kim, Taejin Jeong, Jinyeong Kim, Seong Jae Hwang
Abstract:
The high cost of spatial transcriptomics (ST) has driven extensive studies into predicting gene expression directly from H&E histology images. However, this prediction task faces an inherent limitation, as tissue morphology alone provides insufficient information to fully resolve underlying gene expression. To address this limitation, a recent study leverages partial gene expression to guide the prediction process alongside histology images. Building on this paradigm, we approach the prediction task as a spatial imputation problem, employing a Masked Autoencoder (MAE) to utilize a small fraction of gene expression as genetic anchors for inferring whole‑slide gene expression profiles. Specifically, we propose a bio‑saliency score and a learning‑to‑rank strategy to adaptively identify the most informative spots within the tissue. Based on these identified spots, our framework selects contiguous regions as genetic anchors to ensure suitability for real‑world ST profiling hardware. To effectively leverage these anchors, we design a cross‑modal joint encoder that integrates visual and genetic modalities. By aligning the selected anchors with their corresponding visual features via contrastive learning, the encoder generates robust joint representations to accurately predict gene expression across the whole slide. Notably, our framework consistently surpasses existing methods in both histology‑only prediction and spatial imputation, achieving superior accuracy even without genetic anchors and further excelling with as little as 10% transcriptomic coverage. Our code is available at https://github.com/Kyyle2114/CAMMST.
Authors:Youngjoon Jeong, Jihwan Yu, Minsoo Jo, Junha Chun, Taesup Kim
Abstract:
Latent action pretraining learns representations of visual change from pairs of observations, but existing methods typically encode each transition as a single unstructured representation that entangles transition extent and transition mode. We introduce Polar Latent Actions with Radial structure (PoLAR), which imposes a radial‑direction structure on latent actions, encouraging radius to encode transition extent and direction to retain transition mode. PoLAR uses temporal offset between two observations as a weak proxy for transition extent, encouraging latent action from observation pairs separated by larger temporal gaps to occupy larger radii. We instantiate this structure in hyperbolic space, whose expanding volume with radius offers a natural fit for more diverse transition modes at larger extents. Across in‑task and large‑scale pretraining settings, PoLAR improves downstream policy performance in simulation and real‑world robot experiments, outperforming latent action baselines and strong pretrained VLAs. These results suggest that the geometry of the latent action space is an important design choice for transferring visual pretraining to downstream robot policy learning.
Authors:Kahim Wong, Kemou Li, Yiming Chen, Haiwei Wu, Jiantao Zhou
Abstract:
AI‑assisted image editing threatens trust in financial, legal, and identity records. The GenText‑Forensics Challenge at ACM MM 2026 addresses this by requiring structured forensic reports, in which integrating detection, pixel‑level localization, and natural language explanation for multilingual text‑centric forgery images. We present SEED, a modular system with three components. First, a similarity‑guided pipeline augments training with diverse synthetic forgeries. Second, a single ViT, built on DINOv3 with LoRA adaptation, jointly performs detection and pixel‑level localization while preserving pre‑trained priors with minimal trainable parameters. Third, an evolving harness takes the detector's predictions and generates a complete forensic report via an MLLM, iteratively improved through a proposer‑evaluator loop optimizing report quality. SEED ranked 3rd in the GenText‑Forensics Challenge. Code and data are available at https://github.com/KahimWong/GenText‑Forensics‑3rd‑Place.
Authors:Victor Lavrenko, Anastasiia Molodnitskaia
Abstract:
Large language models can produce confident but protocol‑invalid answers in domains where procedural compliance is critical. This paper presents Answer Engineering, a deterministic runtime and authoring layer that applies localized rule‑guided interventions to the visible reasoning trajectory during standard autoregressive generation, without retraining, modifying model weights, or performing global search. The method is evaluated on a controlled clinical benchmark for sudden sensorineural hearing loss (SSNHL), where correct management depends on protocol‑consistent interpretation of symptom timing, Weber/Rinne tuning‑fork findings, and otoscopic findings. In the benchmark, step‑by‑step reasoning shifted rather than eliminated errors: compliant outcomes for SSNHL decreased from 54.5% under unguided generation to 25.1%, while acceptance on the conductive contrast condition increased from 1.6% to 58.9%. Local trajectory editing increased SSNHL compliance to 83.5% and conductive‑case adherence to 77.9%, raising balanced accuracy from 42.0% under reasoning‑only generation to 80.7%. The results support a systems‑level view in which protocol adherence can be improved through auditable runtime control of reasoning trajectories, while also identifying limitations caused by rule coverage, trigger reliability, and persistent diagnosis‑first generation dynamics.
Authors:Jeff Brown, Tim Farkas, Gleb Razgar, Edward S. Boyden
Abstract:
Proofreading‑‑correcting segmentation errors in 3D brain reconstructions‑‑is the rate‑limiting step in synapse‑resolution connectomics. We release ConnectomeBench2, a unified multi‑species dataset of over 716,485 expert‑labeled proofreading decisions with >4,500,000 associated images spanning four major open connectomes (mouse, human, zebrafish, fly), spanning both split and merge error correction. Trained on this dataset, a single Vision Transformer with shared encoders for mesh geometry and electron microscopy reaches human‑level accuracy across species for split error correction and merge error identification, with performance scaling with data size and modality. Beyond accuracy, we show that the model is well‑calibrated within distribution, that measures of distribution distance predict where calibration and accuracy will degrade on unseen data, and that connectomics‑specific pretraining and active learning‑based sample selection show potential to substantially reduce the labeling effort needed to extend to new species and brain regions. The benchmark provides the infrastructure to train and evaluate increasingly capable vision models for connectomic proofreading. Data and code availability. The ConnectomeBench2 dataset is released on Hugging Face at https://huggingface.co/datasets/jeffbbrown2/ConnectomeBench2. The accompanying codebase is available on GitHub at https://github.com/timfarkas/ConnectomeBench2.
Authors:Vasile Marian, Yong-Bin Kang, Alexander Buddery
Abstract:
Object‑centric image generation is important in settings with few labeled examples, including pedestrian analysis in smart‑city scenes, traffic‑sign inspection, and domain‑specific object detection. Synthetic images are most useful for training and evaluation when datasets preserve object structure, bounding boxes, visual diversity, and realistic context. Existing image datasets usually target classification, detection, or scene understanding rather than controlled object‑centric generation and augmentation with limited class‑specific data. We present a shareable collection of three object‑centric dataset resources: Cityscapes‑Pedestrian, TrafficSigns, and COCO PottedPlant. The collection standardizes 256‑by‑256 object‑centric crops and bounding‑box annotations across three regimes: dense pedestrian scenes with privacy blur and occlusion, cleaner high‑contrast traffic signs, and context‑diverse potted‑plant scenes. The release contains 3,009 TrafficSigns samples, 2,156 Cityscapes‑Pedestrian manifest records, and 7,679 COCO PottedPlant manifest records. The larger COCO‑derived manifest preserves contextual and multi‑instance diversity, while equal‑size subsets can be drawn with a fixed random seed for controlled comparisons. The release provides direct TrafficSigns data where redistribution is permitted, together with scripts, manifests, box‑level annotation tables, checksums, and reconstruction documentation for the Cityscapes‑ and COCO‑derived subsets. It is available through the Latzi/object‑centric‑low‑data‑datasets GitHub repository and Zenodo DOI 10.5281/zenodo.20573001. The collection supports label and split inspection, subset creation, reconstruction from upstream data, and evaluation of object‑centric image generation or synthetic‑data augmentation methods on shared records.
Authors:Dong-Hyun Moon, Ju-Hyeon Nam, Sang-Chul Lee
Abstract:
Image forgery localization remains challenging due to diverse manipulation techniques and distribution shifts. Existing forgery localization models achieve high accuracy on benchmarks but often struggle with cross‑domain generalization and robustness. In this paper, we propose SARIF (Segment Anything for Robust Image Forensics), a framework that leverages the Segment Anything Model (SAM), which has a promptable architecture and strong generalization ability. SARIF introduces a feedback‑guided mask decoder and a dual‑encoder design that extracts forgery‑specific information to capture forensic traces while exploiting the SAM architecture. To localize manipulated regions, we design a block‑wise prompting mechanism that derives forgery‑specific cues from residual features between an adapted encoder and its frozen counterpart. These features are fused with the previous mask prompt to drive a feedback‑based mask refinement process, enabling automatic forgery segmentation without manual input. Extensive experiments on standard forgery‑localization benchmarks show that SARIF achieves strong average cross‑dataset performance and robustness to common image corruptions.
Authors:Hanqing Li, Xuewen Lu, Yuting Chen
Abstract:
Bayesian model averaging in support‑indexed regression induces a posterior distribution over active predictor supports. Under predictor redundancy, posterior mass can spread across many nearly interchangeable supports, making exact‑support summaries unstable or hard to interpret even when prediction is stable. We study how to report an already fitted Bayesian model averaging posterior without changing the Bayesian target. A report uses hard or soft regions of support space, and its compressed reporting law is compared with the reference posterior through an explicit density ratio. This ratio gives computable total‑variation and Kullback‑‑Leibler distortion, bounds for bounded predictive summaries, retained‑mass diagnostics, and fallback‑weight diagnostics. The framework covers fixed hard regions, metric‑ball regions, posterior‑cluster regions, and pooled‑pruned region dictionaries. We prove exact error formulas and validation bounds for these region reports, and give conditions under which a few regions can replace a long list of individual supports. In simulations, our region reports often give shorter and clearer summaries while preserving the main posterior information, and the density‑ratio diagnostics show when too much information has been lost.
Authors:Dongrui Han, Weidong Chen, Jiawen Kang, Mingyu Cui, Helen Meng, Xixin Wu
Abstract:
Recent advances in text‑to‑speech (TTS) synthesis have achieved highly natural and expressive speech generation. However, these systems are designed for general adults and overlook older adults' speech comprehension needs due to age‑related sensory and cognitive decline. Prior work involves older adults by collecting preference feedback to tune model parameters. However, obtaining sufficient preference data is costly and difficult, as older adults quickly become fatigued during collection. In this paper, we propose a novel imitation learning (IL) framework to learn TTS models from expert demonstrations. We further improve Group Relative Policy Optimization (GRPO) with two‑stage on‑policy reward learning (OPRL) to mitigate reward hacking under limited supervision from expert demonstration. Experimental results show that GRPO w/ OPRL outperforms GRPO and supervised baselines in objective and subjective metrics. Audio samples are available at https://dongru1.github.io/demo/im‑efss
Authors:Hanchun Wang
Abstract:
AI‑assisted research systems generate many failed attempts, but those failures rarely become a durable, shared knowledge asset. We propose a negative knowledge memory layer: a curator agent converts each failed attempt into a bounded, typed record in a shared bank, and a downstream research agent explicitly adopts or rejects those records before proposing its next experiment. We evaluate this layer in two settings: same‑task retry on ScienceAgentBench and cross‑task scientific research on two nonlinear math‑physics PDE problems. The negative knowledge layer outperforms vanilla AutoResearch baselines while using fewer tokens; agents with the negative knowledge bank solve new tasks that all baselines fail to solve in PDE systems research. We also show that the previous negative knowledge bank can transfer and enhance AutoResearch on different PDE problems. These results suggest that structured negative knowledge is a knowledge asset that should be explicitly maintained in broader AI‑engaged scientific research beyond a memory‑compression or debugging aid, alongside positive findings, as a collective infrastructure for scientific memory. Code is available at https://github.com/hch‑wang/Negative_Knowledge.
Authors:Hanxin Zhang, Abdulqader Dhafer, Hongbiao Dong, Zhou Daniel Hao
Abstract:
We present R2HandoverSim, a simulation benchmark for robot‑to‑human (R2H) object handovers. Although R2H handover methods have advanced rapidly, the lack of standardized evaluation protocols impedes objective comparison. Our benchmark enables reproducible evaluation by systematically comparing four baselines on their predicted shared grasp poses. We conduct a user study with 30 participants, analyze baseline performance, and show that simulation results correlate with real‑world evaluation outcomes. Crucially, five complementary metrics (planning feasibility, reachability, grasp stability, grasp affordance, and safety) better reflect user‑perceived handover quality than overall success rate alone. Website and code: https://robot‑future.github.io/r2handoversim/.
Authors:Yiqi Zhao, Ruohai Ge, Celina Shiyu Wang, Junjie Ye, Muchen Xu, Minhao Li, Sergey Zakharov, Basile Van Hoorick, Vitor Campagnolo Guizilini, Leonidas Guibas, Gaurav S. Sukhatme, Jyotirmoy V. Deshmukh, Yue Wang
Abstract:
Dual‑robot collaboration enables tasks that exceed the reach and payload of a single robot, such as collaboratively transporting objects across environments and executing coordinated handovers. Data acquisition is the primary bottleneck for training these systems. To this end, we introduce DUET, a dual‑robot learning framework for mobile manipulation. For efficient data collection, we create a unified dual‑embodiment synchronized VR‑based teleoperation system for in‑domain heterogeneous robot data collection. We further develop a complementary tracking pipeline that records human‑human coordination and collaborative mobile manipulation priors. To allow efficient learning, we introduce an Action Chunking Transformer based architecture that first pretrains collaborative policies on efficient human‑human demonstrations, before finetuning them on a minimal set of real‑robot teleoperation trajectories. We develop a benchmark of four collaborative tasks to evaluate our framework using a Unitree G1 humanoid and a Dexmate Vega1 mobile manipulator. The results demonstrate that harnessing human priors not only yields superior task performance compared to baselines trained only on robot data, but also reduces the total human effort required for data collection. Our human data collection pipeline achieves 5.4x acceleration on average from teleoperation, but we perform equally or better than robot‑only data trained policies across all tasks. Our project page is available at https://zhaoy37.github.io/Duet/.
Authors:Emmanuel Sérié
Abstract:
Make‑lineage workflow runners decide whether a job must re‑run from file‑modification time (mtime, a timestamp) ‑‑ a broken proxy for the question that matters: did the content change? A git checkout, a tree copy, or a backup restore rewrites mtimes without touching content, forcing spurious re‑execution; and in the reverse case ‑‑ when an output looks newer than its inputs but its content is stale ‑‑ the stale output is silently reused. (Snakemake 7's per‑output provenance survives this churn, as local bookkeeping; GNU Make and pure‑mtime fast paths are where it bites.) OxyMake, a single‑binary Rust workflow engine, replaces the proxy with a content‑addressed cache key: a BLAKE3 hash of rule source, input content, parameters, environment, and platform. Because the key is a pure function of these declared inputs, the caching decision survives mtime churn and travels across same‑platform machines and shared caches. Phantom re‑runs vanish for declared inputs (no sandbox: an undeclared input is invisible to the key). The spec stays declarative and statically parseable, keeping the Make rule model so Snakemake pipelines port directly. DAG resolution is an order of magnitude faster than Snakemake's on large graphs, but a cold end‑to‑end run is slower ‑‑ the price of content‑addressed bookkeeping ‑‑ repaid several‑fold on the warm re‑run that caching exists to serve (exact figures, hardware, and a bundled reproducer are in the evaluation). Execution is daemon‑free via a cooperative claim/reclaim protocol (sessions claim jobs, reclaiming stalled ones); today two sessions duplicate work safely rather than coordinate, and wiring the protocol as a hard execution gate is staged, not yet done. Cross‑session safety is specified in TLA+ and model‑checked over all interleavings for 2‑3 sessions, assuming atomic state commits. An ox.lock plan‑of‑record and NDJSON event stream record exactly what ran.
Authors:Aryan Das, Koushik Biswas, Moloud Abdar, Vinay Kumar Verma
Abstract:
We introduce UNITY, a Universal‑to‑Specialized adapter for efficient and scalable composite conditioning in diffusion based image generation. Unlike prior methods that train separate adapters for each conditioning modality, UNITY jointly learns shared semantics across multiple conditioning types and subsequently specializes without modifying the underlying architecture. The proposed two stage training paradigm consists of a Universal Stage that captures cross modal representations across all conditioning modalities using half of the total training steps, followed by a Specialization Stage that refines modality specific features using the remaining training budget. At the core of UNITY are the Morphable Attention Flow (MAF) Network and Morph Wrapper modules, which enable channel aware and spatially adaptive feature alignment through learnable flow fields and attention based fusion. This constant complexity formulation supports flexible operation under both single and composite conditioning settings while significantly reducing inference latency and memory consumption. Extensive experiments across multiple datasets demonstrate that UNITY achieves state of the art image fidelity while maintaining superior memory efficiency. Code: https://github.com/arya‑domain/UNITY
Authors:Sergei Trashchenkov
Abstract:
Executable evaluation ‑‑ checking the consequences of an agent's actions with a program rather than grading its prose ‑‑ has become a prominent way to assess tool‑using AI agents in software settings. Electric power engineering has not yet had an analogous benchmark: language‑model use is still dominated by retrieval and text question answering, while agents acting on power‑system artifacts remain mostly academic prototypes. We introduce the Power Systems Agent Benchmark, an executable benchmark for power‑engineering agents. An agent receives a structured task and returns a structured solution; a deterministic evaluator recomputes the engineering quantities, checks operational constraints, and returns a feasibility flag, a normalized score, and explicit violations. The benchmark contains 41 task families across eight areas of power engineering, from power flow and protection to stability, microgrids, reliability, power quality, and forecasting. Each task is grounded in a citable source, standard, or documented engineering formulation. To resist contamination, held‑out cases are synthesized on demand by per‑family generators from private seeds: the construction is inspectable, but the instances remain private. In a reference evaluation with three command‑line agents, the strongest score near the compact tier's ceiling, a smaller open model trails, and public and held‑out performance are broadly consistent; a separate public‑split grid with OpenCode and Aider probes harness effects. The reference evaluation doubles as quality control: unanimous failures flag candidate task or evaluator defects, and it exposed a latent evaluator bug missed by self‑consistency checks. The evaluators are compact deterministic surrogates, but the task contract allows their internals to be upgraded to simulator‑backed checks without changing how tasks are posed or solved.
Authors:Austin T. Wang, Dongchen Yang, Angel X. Chang
Abstract:
Developing robust models for 3D visual grounding (3DVG), the localization of entities in a 3D scene described in natural language, is important for enabling agents to correspond spatial language with objects in the physical world. However, the lack of diverse descriptions at scale prevents models from generalizing beyond simple linguistic patterns. Recent such attempts lack diversity in the constraint types and language used to ground objects. Captioning methods cannot precisely contrast objects, which is important for visual grounding. We therefore propose ViGiL3D++, a scalable, scene‑agnostic method that generates diverse visual grounding queries by combining constraint sampling in scene graphs with the language generation of LLMs. We show that it has greater diversity over existing scaled datasets and improves model performance over several 3DVG benchmarks but also illuminates outstanding limitations of VLMs.
Authors:Qingtao Pan, Kai Ye, Zhihao Dou, Bing Ji, Shuo Li
Abstract:
In multi‑object text‑to‑image (T2I) diffusion, ensuring semantic consistency between textual prompts and generated visual content is crucial for image synthesis. However, such consistency constraint is often underemphasized in the denoising process of diffusion models. Although token supervised diffusion models can mitigate this issue by learning object‑wise consistency between the image content and object segmentation maps, it tends to suffer from the problems of segmentation map bias and semantic overlap conflict, especially when involving multiple objects. In this paper, we propose ELDiff, a new evidential learning‑supervised T2I diffusion model, which leverages the advantages of uncertainty metric and conflict detection to enhance the fault tolerance of unreliable segmentation maps and suppress semantic conflicts, strengthening object‑wise consistency learning. Specifically, a pixel evidence loss is proposed to restrain overconfidence in unreliable labels through evidential regularization, and a token conflict loss is designed to weaken the contradiction between semantics through optimizing a measured conflict factor. Extensive experiments show that our ELDiff outperforms existing training based and train‑free based T2I diffusion models on SD v1.4, SD v2.1, SDXL, SD v3.5, and Qwen‑Image, without requiring additional inference‑time manipulations. Notably, ELDiff can be seamlessly extended to the existing training pipeline of T2I diffusion models. Code can be found at https://github.com/QingtaoPan/ELDiff.
Authors:Shanghao Shi, Xiao Wang, Chaoyu Zhang, Hao Li, Wenjing Lou, Thomas Hou, Yevgeniy Vorobeychik, Chongjie Zhang, Ning Zhang
Abstract:
The integration of external tools has substantially expanded the capabilities of large language model (LLM) agents, but it also introduces new attack surfaces beyond prompt injection. In particular, cross‑tool description poisoning can manipulate planner‑visible tool metadata to steer an agent's trajectory, even if the poisoned tool itself is never chosen. To understand the effectiveness of existing defenses against this emerging threat, we first evaluate several prompt‑injection defenses and find that they transfer poorly to cross‑tool description poisoning. A key observation is that poisoned descriptions persist in the planning context across steps, enabling continuous influence over subsequent tool choices. Building on this insight, we propose Tool‑Guard, a novel system‑level defense based on a new concept called isolated planning, in which tool invocations that are detected as misaligned or suspicious cause the corresponding tool to be placed in a quarantined list (the influenced list), breaking further influence from poisoned descriptions. With this influence isolated, the tool can continue to be used to support the task, enabling a robust defense that preserves legitimate tool utility. Experiments on the AgentDojo and ASB benchmarks show that Tool‑Guard substantially reduces attack success while maintaining high task utility. Our code is available at https://github.com/shishishi123/Tool‑Guard.
Authors:Abhijit Das, Nichula Wasalathilaka, Yifan Lu, Adinath Dukre, Dwarikanath Mahapatra, Shadab Khan, Imran Razzak
Abstract:
Medical vision‑language models (VLMs) enable zero‑shot clinical image classification, yet reliably detecting out‑of‑distribution (OOD) inputs at deployment remains an open problem. No static scoring method works across all shift types: Maximum Concept Matching (MCM) on FLAIR achieves 76.4% AUROC for far‑OOD but only 42.4% for covariate shifts such as ultra‑wide‑field fundus images, effectively random. We trace this to a structural mismatch: covariate‑shifted inputs are indistinguishable from in‑distribution samples in softmax space, yet occupy distinct regions in the VLM embedding space. To exploit this untapped signal, we propose PROTON (PROtotype‑based Test‑time ONline OOD detection), a lightweight post‑hoc module that maintains an online prototype bank from high‑confidence test predictions and adaptively fuses prototype distance with MCM scoring via stream‑level variance statistics, requiring no model modification, training data, or prompt engineering. On the ophthalmology benchmark FLAIR + FIVES, PROTON improves MCM by +23.9 AUROC on covariate shift, +8.8 on semantic shift, and +8.1 on far‑OOD, making it the only zero‑shot method to improve all three without hierarchical prompts or labeled data. Code is available at https://github.com/GenMI‑Lab/PROTON, and the project page is available at https://genmi‑lab.github.io/PROTON.
Authors:Zeyuan Chen, Ziqing Yang, Yihan Ma, Michael Backes, Yang Zhang
Abstract:
As academic submissions grow, the traditional peer review process struggles to keep up, raising concerns about quality and fairness. A trend of using large language models (LLMs) for assistance has emerged. In this work, we take a critical step toward improving the quality of LLM‑generated reviews. We propose the PeerCheck framework, which investigates LLM‑human review differences (RQ1) and explores methods to improve LLM‑generated review quality (RQ2). We first analyzed the human‑written reviews with reviews generated by various LLMs and found that LLMs and humans focus on different terms, e.g., LLMs prioritize theory while humans emphasize methodology and experiments. We further adopt prompt engineering, such as Chain‑of‑Thought (CoT), and utilize retrieval‑augmented generation (RAG) to enhance the LLM‑generated reviews towards human‑level quality. We find CoT significantly improves the quality of LLM reviews, while we discover an unexpected "RAG paradox," i.e., experiments with RAG produce different results for various LLMs and, in some cases, even reduce review quality. Our comprehensive analysis of LLM‑generated academic reviews illustrates both possibilities and limitations, contributing to a more effective, human‑aligned review system. Our dataset is available on https://github.com/TrustAIRLab/PeerCheck.
Authors:Sameek Bhattacharya, Bharath Krishnamurthy, Ajita Rattani
Abstract:
Deep learning‑based audio classification systems, including automatic speaker verification, are vulnerable to adversarial attacks. Realistic real‑time threat assessment remains difficult because optimization‑based methods, such as projected gradient descent (PGD) and Carlini‑Wagner, require costly iterative updates in the high‑dimensional waveform domain. Generative attacks allow single‑shot synthesis but often introduce perceptible artifacts or depend on computationally intensive architectures, while diffusion and autoregressive approaches incur high inference latency. To address this gap, we propose a generative attack framework operating in the continuous latent space of a neural audio codec. A conditional generator synthesizes class‑specific perturbations in a single forward pass and decodes them into adversarial waveforms. Our method achieves targeted attack success rates up to 99% with sub‑7 ms inference, outperforming generative baselines while reducing latency by 24x.
Authors:Koichi Namekata, Yash Kant, Zhizheng Liu, Ryan D Burgert, Yuancheng Xu, Kuan Heng Lin, Emmett Steven, Julien Philip, Li Ma, Andrea Vedaldi, Paul Debevec, Ning Yu
Abstract:
Filmmaking demands precise motion control and reference image compositing ‑‑ capabilities that existing methods treat separately. Point‑track‑conditioned image‑to‑video models restrict content insertion to the first frame, while reference‑to‑video models lack fine‑grained spatial‑temporal control over how reference content integrates across frames. We present Go‑with‑the‑Track, which unifies both capabilities by jointly conditioning on multiple reference images and reference‑anchored point‑tracks ‑‑ extending conventional point‑tracks to explicitly establish correspondences between generated frames and reference images, thus enabling precise compositing and motion control throughout the video. To achieve this, we introduce spatially‑aware point‑track embeddings that encode the full sequence of point‑track coordinates using a coordinate‑wise MLP followed by temporal pooling. This representation captures the spatial characteristics of each point‑track (serving as a unique identifier), while the embedding similarity correlates directly with spatial proximity, enhancing the model's ability to distinguish and associate point‑tracks. We inject these point‑track embeddings into a video diffusion transformer via a lightweight adapter, resolving the pixel‑to‑patch resolution mismatch while avoiding the substantial motion detail loss inherent in naive point‑track subsampling. We use a hybrid training strategy to train jointly on dynamic, static, and synthetic scene video datasets to boost motion controllability. Experiments demonstrate that Go‑with‑the‑Track achieves superior motion and reference control in a single model and enables new capabilities: multi‑reference conditioned video generation with point‑track driven compositing, as well as camera control for both static and dynamic scenes. Project Page: https://eyeline‑labs.github.io/Go‑with‑the‑Track/
Authors:Luan Marko Kujavski, Rayson Laroca, Paulo Lisboa de Almeida
Abstract:
As urban areas expand, automatic monitoring of parking lots becomes essential for efficient and sustainable cities. This work proposes a self‑supervised approach for parking spot occupancy recognition that requires no labeled samples from the target parking lot. Building upon a self‑supervised transfer learning fine‑tuning protocol, the proposed training strategy consists of two self‑supervised stages: first on unlabeled generic data and then on unlabeled target‑specific data, followed by supervised fine‑tuning using only generic parking lot labels. We adopt SimCLR with a ResNet‑50 encoder and evaluate the method under a leave‑one‑out cross‑environment protocol on three public datasets: PKLot, CNRPark‑EXT, and PLds. We also introduce a two‑stage deployment strategy in which a Strong General Model is initially deployed, followed by a Specialized Model that incorporates unlabeled images collected during the first N days of deployment in a self‑supervised manner. Experimental results show that the Strong General Model alone outperforms supervised and self‑supervised baselines, achieving an average accuracy of 97.2%, which further improves to 97.8% with the proposed two‑stage strategy. These results demonstrate that self‑supervised learning enables a scalable and labelefficient solution for real‑world parking occupancy monitoring. Our trained models and source code are publicly available at https://github.com/LoanMaikon/Parking‑Spot‑Occupancy‑Recognition.
Authors:Bahrul Ilmi Nasution
Abstract:
Synthetic tabular data enables microdata sharing in regulated domains, yet deploying continuous‑time generative models requires balancing analytical utility, disclosure risk, and computational cost. Latent‑space flow models are flexible, but theoretical equivalences across learning targets, probability paths, and sampling dynamics can translate into different behaviour under finite‑step integration and explicit compute budgets. We present an empirical study of tabular latent flow models across seven datasets, evaluating velocity, score, noise, and posterior matching objectives under optimal transport (OT) and variance‑preserving (VP) paths, ODE and SDE sampling, and varying integration budgets. Our contributions are threefold: (1) we show that the learning target largely determines the utility‑risk operating regime, with velocity and posterior matching tending to yield higher utility, while score and noise matching tend to achieve lower disclosure risk; (2) we demonstrate that configuration and sampling choices shift performance, with midpoint often improving distributional fidelity and OT paths often tolerating earlier stopping than VP, enabling compute savings under fixed budgets or risk thresholds; and (3) we distil these findings into actionable defaults and practical configuration guidance to support pre‑release model selection under disclosure risk and resource constraints. The code implementation and supplementary materials can be accessed in https://github.com/rulnasution/tabular‑latent‑flow/.
Authors:Alan Nadelsticher Ruvalcaba
Abstract:
The temporal structure of reward composition in reinforcement learning (RL) is typically hand‑designed and held fixed throughout training, leaving the progression of motivational priorities largely unexplored. In this work, we propose an evolutionary framework for discovering developmental reward schedules, in which three distinct biologically inspired motivational components ‑‑ agency, novelty, and reactivity ‑‑ are combined through time‑varying weights that dynamically shift over the course of training. Evaluated on two sparse‑reward MiniGrid tasks: DoorKey‑6x6 and KeyCorridorS3R1, our framework compares the generalizability of four evolutionary algorithms: CMA‑ES, xNES, DE, and L‑SHADE against an extrinsically motivated baseline (our main comparison point), and three additional hand‑designed methods. On DoorKey‑6x6, all evolved methods outperform the non‑evolved baselines, with L‑SHADE achieving the best performance ‑‑ an approximate relative mean improvement of 11.4% over the extrinsic only baseline. On KeyCorridorS3R1, CMA‑ES achieves the best overall performance, with the remaining evolved methods showing weaker and less reliable generalization capability compared to the extrinsic only baseline. Interestingly, the discovered schedules diverge from our defined developmental ordering, with novelty consistently emerging as the dominant early signal during training, across both tasks. Collectively, our results position evolutionary optimization as a promising approach for developmental reward schedule discovery in deep reinforcement learning, and suggest that what evolution finds to be optimal in computational settings may differ from what it finds to be optimal in biology. The code for this project can be found at: https://github.com/alannadels/Evolutionary_RL.git.
Authors:Hiroki Sakuma, Masatoshi Okutomi
Abstract:
Multi‑view surface reconstruction is a core problem in computer vision. One prominent line of work represents the surface implicitly as a signed distance field (SDF), optimizing it based on the photometric loss between rendered and observed pixel colors. These approaches typically employ SDF‑based volume rendering to obtain a differentiable relaxation of discontinuous visibility along rays, thereby reducing reliance on silhouette supervision. In this paper, we reformulate SDF‑based volume rendering as probabilistic surface rendering, where each pixel color is modeled as a mixture distribution induced by the random first ray‑surface intersection. To this end, we introduce Stochastic Signed Distance Processes (SSDP), which model the SDF along each ray as a stochastic process, inducing a first‑passage‑time distribution for each ray. We then derive the first‑passage probability for each sampling interval based on Bayesian filtering, together with its practical approximation for parallel rendering. We further show that NeuS, an existing SDF‑based volume rendering method, arises as a special case of our formulation. Experiments on the DTU and MobileBrick datasets demonstrate that our method outperforms baselines in both surface reconstruction and uncertainty quantification, supporting the effectiveness of our first‑passage formulation. Our code is available at https://github.com/skmhrk1209/SSDP.
Authors:Xueqi Cheng, Qinwen Ge, Hamid Karimi, Yushun Dong, Tyler Derr
Abstract:
Adverse social interactions (ASIs) can shape how online communities evolve over the time. However, structural‑based ASIs and content‑based ASIs are often studied separately and at a single analytical scale. In this study, we propose a multi‑level framework to examine how adverse social interactions appear locally, spread through neighborhoods, and disrupt cohesive subgroups. Using large‑scale datasets from X and Bluesky, we analyze friend and foe patterns at the micro level, peer influence through matched triadic designs at the meso level, and subgroup disruption against random and recommendation‑based references at the macro level. Our results show that structural disconnection and toxic communication provide complementary signals: structural negativity more persistently marks subgroup disruption, while toxic communication captures broader conflict both within and across communities. These findings suggest that adverse social interactions are multi‑scale processes that influence how online communities form, fracture, and evolve. Our source code is publicly available at https://github.com/XueqiC/Adverse‑Social‑Interactions.
Authors:Shishen Lin, Yixin Chen
Abstract:
NeuroEvolution of Augmenting Topologies (NEAT) is a widely used neuroevolution algorithm for learning neural network architectures and weights for control tasks. However, standard offline optimisation searches for connection strengths directly, which can scale poorly in high‑dimensional weight spaces and more difficult continuous control problems. Hybrid methods that combine neuroevolution with online learning can address this challenge, but their theoretical properties remain underexplored. This paper gives the first regret analysis for a general NeuroEvolutionary Online Learning (NEOL) framework, which decouples learning into two timescales: an outer loop for architecture search and an inner loop for online weight adaptation via rewardmodulated plasticity. Under mild conditions, we prove that NEOL achieves sublinear regret. Empirically, under fixed interaction budgets on four standard control benchmarks, a NEAT‑based NEOL implementation achieves higher final fitness and lower variance than pure NEAT, and is competitive with strong reinforcement learning (RL) baselines on several tasks. The results are supported byWilcoxon rank‑sum tests and ablation studies. Overall, the findings show that online plasticity can improve the sample efficiency and robustness of two‑timescale neuroevolution. Code is available at https://github.com/boobaa2001/NeuroEvolution Online Learning NEOL.
Authors:Jaedong Hwang, Kathleen Zhang, Wei Dai, Konstantinos Kontras, Maarten Vanmarcke, Maarten De Vos, Ila Fiete, Paul Pu Liang
Abstract:
EEG foundation models can learn generalizable representations from large‑scale EEG corpora to enable single‑backbone transfer across diverse clinical and brain‑computer interface tasks. Existing models typically discretize the continuous multi‑channel EEG waveform into patches or codebook tokens and train a transformer with masked self‑supervision. Recognizing that this discretization fragments continuous brain rhythms and obscures fine‑grained temporal dynamics, we present B[FM]^2(Brain Foundation Model via Flow Matching), whose inductive bias aligns with the data by pretraining directly on the raw signal using continuous‑time flow matching without patches, tokenization, or masking. However, multi‑channel EEG signals pose an architectural challenge for flow matching: time is densely sampled and highly autocorrelated (thousands of timepoints), while the electrode axis is short (tens of channels) at distinct scalp positions. To address this time‑electrode asymmetry, we introduce SplitUNet, a velocity network that factorizes each block into separate 1D temporal and 1D electrode convolutions and downsamples only along time, preserving electrode topology throughout the hierarchy. B[FM]^2 sets a new state of the art on 7 of 9 standard downstream EEG classification tasks, using a pretraining budget of only 36,895 segments (\approx 307h), 1‑2 orders of magnitude (\approx 30x) less than required by existing EEG foundation models. Further, it generates synthetic EEGs that two board‑certified neurologists cannot distinguish from brain data (Cohen's κ= ‑0.096). https://jd730.github.io/projects/BFM2
Authors:Qiuhong Shen, Shihua Zhang, Yue Liao, Qi Li, Zhenxiong Tan, Shizun Wang, Shuicheng Yan, Xinchao Wang
Abstract:
World Action Models (WAMs) are embodied predictive‑action models that make a forecast of the future available to action. Recent WAMs repurpose large video generation models, and a parallel line relies on language or vision‑language backbones without a video‑generation core. This rapid expansion has blurred the boundary among broad world models, video generation models, action‑grounded video world models, Vision‑Language‑Action policies, and WAMs. This survey gives the field a common account. It first clarifies these boundaries, then organizes existing works through two complementary views. The first view asks what each method is required to generate, spanning rendered futures, latent futures, and video‑generation‑free action reasoning. The second view decomposes each method by predictive substrate, backbone, action coupling, and deployment regime. This anatomy supports a unified discussion of interactability, causality, persistence, physical plausibility, and generalization, followed by data, evaluation, and open challenges. Across these axes, a consistent design pattern emerges: WAMs are not simply video generators with action heads, but predictive‑action methods whose design choices trade representational richness against compute, memory, latency, and action‑label cost. The field is moving toward methods that generate less of the future while preserving what control requires. The survey homepage is available at https://world‑action‑models.github.io/.
Authors:Maruthi Vemula
Abstract:
Trained transformers reliably develop massive activations, a small number of hidden dimensions whose magnitude is far above the median and which concentrate on the sequence‑start token. Whether these outliers are a removable artifact of the residual stream's overloaded read and write role, or instead a functional necessity, is actively debated. We test the artifact hypothesis directly, with an architectural intervention. Our architecture, Ledger Residuals, splits the residual stream into a mutable scratch stream (Deliberation) that intermediate computation may freely overwrite and a protected, decode‑only accumulator (Commitment) that holds the representation the model reads out. If massive activations exist only because one stream is forced to be both scratchpad and answer, then a dedicated answer channel should remove the need for them. We find that it does not. In matched‑loss language models at the 160M and 290M scales, the model rebuilds the canonical fixed‑dimension, start‑token outlier inside the protected channel. The rebuilt feature is smaller in magnitude than in a standard transformer but more sharply concentrated on the start token, and a stronger sparsity penalty makes it more persistent and more concentrated still, rather than removing it. Massive activations therefore look architecturally robust: they re‑emerge in whichever representation the model decodes from, which is what we would expect if they are functional rather than incidental. We release our architecture and measurement code.
Authors:Francesca Morandi, Omayma Moussadek, Federico Venturini, Mauro Suardi, Alessandro Banzatti, Francesco Cannarile, Angelo Porrello, Simone Calderara
Abstract:
Open Vocabulary Action Recognition (OVAR) enables the recognition of novel actions by leveraging vision‑language representations, overcoming the limitations of traditional closed‑set approaches. However, achieving robust performance in real‑world scenarios typically requires domain‑specific fine‑tuning, which is often costly and raises privacy and regulatory concerns. In this work, we propose an alternative paradigm that bypasses target‑domain training and recombines knowledge from existing datasets and models. Leveraging model merging and task arithmetic, we extract and combine task vectors from models fine‑tuned on diverse public OVAR datasets. We show that, in out‑of‑distribution settings, the resulting merged model achieves superior zero‑shot generalization to the pre‑trained base model. Code is available at https://github.com/omaymaMoussadek/robust‑ovar
Authors:Shiwen Zhang, Yifan Xu, Haibin Huang, Chi Zhang, Xuelong Li
Abstract:
Given a content reference and a style reference, content‑preserving style transfer requires the model to generate stylized outputs with content and style consistency. We introduced TeleStyle V1 to tackle this problem. However, TeleStyle V1 is trained with photorealistic content reference and artistic style reference, which makes it incapable to cope with artistic content reference and realistic style reference in most cases. In this paper, we designed a Self‑Distillation data synthesis strategy to construct such triplets from TeleStyle V1. Trained with such self‑distilled triplets, our TeleStyle V2 supports Content‑Style references in the forms of Realistic‑and‑Realistic (RnR), Realistic‑and‑Stylized (RnS), Stylized‑and‑Realistic (SnR), Stylized‑and‑Stylized (SnS). In addition, we found Distribution Matching Distillation could preserve the general text‑guided image editing capability of the foundation model and fix the content consistency degradation caused by SFT process. Through quantitative evaluations, our TeleStyleV2‑QIE‑2509‑DMD performs at least on par with Qwen‑Image‑Edit‑2509‑DMD, demonstrating strong general image editing skills beyond content‑preserving style transfer. We observed the content/style reference order confusion problem in TeleStyle V1 and further introduced prompt enhancer to solve it. TeleStyle V2 uses Qwen‑Image‑Edit's VLM encoder, Qwen2.5‑VL‑7B, to generate content prompt and style prompt for free. TeleStyle V2 could achieve comparable style transfer performance with state‑of‑the‑art commercial model, gemini‑3‑pro‑image‑preview.
Authors:Jianyuan Guo, Zhiwei Hao, Chengcheng Wang, Cheng Fan, Tingzhang Luo, Hongguang Li, Ying Gao, Hefei Mei, Jiankun Peng, Rongjian Xu, Minjing Dong, Han Wu, Mengyu Zheng, Kai Han, Shiqi Wang, Chang Xu, Yunhe Wang
Abstract:
LLM‑based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent‑native training with co‑evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM‑based agents through a model‑harness lens. We first clarify the functional definition of agents and the implementation view of an LLM‑based agent as a foundation model coupled with an execution harness. We then analyze the limits of model‑centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities: observation, context, control, action, state, and verification. Using this decomposition, we map task properties and domain pressures to harness configurations, review benchmark and evaluation practices, and synthesize model‑harness evidence on how runtime design affects long‑horizon task completion, efficiency, and reliability. Finally, we identify open challenges in value‑aware evaluation, safety, harness generalization, and model‑harness co‑evolution. Rather than treating agents as models with auxiliary tools, this survey argues that agent quality ‑‑ including success, efficiency, safety, and generalization ‑‑ emerges from the interaction between model capability, runtime infrastructure, task structure, and evaluation design. A collection of papers discussed in this survey is provided in https://github.com/ggjy/Awesome‑Agent‑Engineering.
Authors:Yifan Li, Shengbin Yue, Boyu Feng, Jinhu Qi, Bo Ke, Zixing Song, Hongru Wang, Zhongyu Wei, Irwin King
Abstract:
The integration of external tools has transitioned LLM agents from passive responders to autonomous systems. However, current benchmarks prioritize execution success, neglecting self‑awareness capability, the ability to discern whether a problem requires necessary external resources or can be solved via internal parametric knowledge. To address this, we introduce KAPRO (Knowing‑Acting Quadrant PRObe), a framework that evaluates cognitive‑behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting). We further construct KAware, a dataset rigorously partitioning tasks into external, internal, and hybrid subspaces to systematically probe these epistemic boundaries. Extensive experiments across diverse agent architectures show that self‑awareness capability is strongly correlated with task success but degrades sharply in internal‑capability settings. Moreover, open‑source and instruction‑following models exhibit stronger tool overuse due to shallow pattern matching, while proprietary and reasoning‑oriented models demonstrate more reliable cognitive gating. Benchmark and codes are available at https://github.com/AI‑Santiago/KAware.
Authors:Linda Mümken, Diyar Altinses, Michael Schwung, Stefan Lier, Andreas Schwung
Abstract:
Autonomous drone swarms in space‑constrained environments such as warehouses, inspection corridors, and urban delivery routes must share limited airspace safely at high vehicle density. Existing approaches rely on fixed safety zones sized for worst‑case velocity, which wastes airspace in congested scenarios. We replace the fixed radius with an adaptive, speed‑dependent safety sphere whose size scales with braking distance: tight at low speeds, expanded at high speeds. We develop both a centralized model predictive control (MPC) formulation and a distributed MPC (DMPC) in which each drone optimizes locally from detected neighbors, accommodating mixed fleets with non‑cooperative agents. We prove feasibility up to the geometric packing limit evaluated at the minimum radius, establish Lyapunov stability under sufficient conditions on the adaptation parameter, drone density, and prediction horizon, and extend these guarantees to the distributed setting via a contraction condition that preserves the centralized stability margins. We further derive modified sphere‑packing capacity bounds and a throughput‑optimal crossing speed for narrow passages. Simulations confirm that the adaptive framework remains feasible where fixed‑radius methods fail: it roughly doubles the admissible drone count, reduces traversal time through constrained passages by about 25 percent, and enables passage through openings impassable to static safety zones. The centralized variant realizes a larger fraction of the theoretical capacity, while the distributed variant offers a more realistic deployment model for mixed‑fleet operations under the same safety guarantees.
Authors:Minghui Wu, Ganjun Liu, Zikun Fang, Ting Meng, Hongchuan Wu, Bingao Xu, Yonglong Cai, Jiasheng Chen, Jun Du
Abstract:
Instruction‑based controllable speech synthesis enables users to specify emotions through natural language. However, existing approaches often rely on coarse emotion labels and lack explicit modeling of fine‑grained intensity. We propose EmoInstruct‑TTS, a dual‑path instruction‑guided framework for emotional speech synthesis. We introduce Emotion2embed, a supervised semantic‑acoustic emotion embedding covering 48 emotional states, including fine‑grained categories and intensity levels. To infer embeddings from free‑form instructions, we design an Instruction‑Conditioned Emotion Flow Model (ICE‑Flow) that generates acoustically grounded emotion representations. The inferred embeddings are integrated into an LLM‑based synthesis pipeline to provide explicit emotional control while preserving semantic planning. Experiments show improved emotional controllability and speech naturalness over strong baselines.
Authors:Thanh Ly, Truong-Duy Dang, Chien Le, Tan-Dzung Do, Phuong Tuan Dat, Cuc T. Trinh, Vien Anh Ngo, An T. Le
Abstract:
Foothold selection on structured terrain requires explicit reasoning about contact planarity, surface steepness, and kinematic reachability, properties not captured by a single height‑based terrain signal. We propose a multi‑channel terrain cost combining flatness, steepness, and velocity‑aware height feasibility, plus a forward climb reward, that simultaneously drives a GPU‑parallel divergent component of motion (DCM) foothold planner and shapes a dense per‑step affordance reward for an asymmetric actor‑critic policy trained with proximal policy optimization (PPO) from depth images. A Bézier swing trajectory with adaptive apex bias extends foothold tracking to joint position‑and‑orientation, using the arc tangent to guide sole orientation through riser crossings and tread landings. To support payload tasks, we introduce a lower‑body compliance training procedure in which a virtual wrench is injected at a sampled load attachment point, generating physically consistent force and moment; wrench‑aware compliance targets replace rigid pose penalties, and the policy learns to yield to load‑induced perturbations without force sensing. The full system trains end‑to‑end with standard PPO, no distillation, and no teacher‑student staging, and is deployed on a humanoid directly from simulation with configuration changes only. In simulation, the policy reaches 1.0~\mathrmm/s on stairs with risers up to 0.20~\mathrmm and improves payload robustness up to ～15~\mathrmkg centered load and for moment‑dominated wrist loads without fine‑tuning. We also provide a qualitative hardware demonstration on structured terrain. Project website: https://fai‑rl‑tech.github.io/tact‑locomotion.github.io/
Authors:Tingzhou Wei, Zeyu Zheng, Ethan X. Fang, Junwei Lu
Abstract:
Asymptotic statistical theory is a challenging domain for AI‑assisted formalization: its central results mix convergence statements, asymptotic expansions, functional analysis, and regularity conditions that have a large gap from existing infrastructure in Lean 4 formalization. To address these challenges, we propose a hypothesis‑disciplined Lean 4 formalization pipeline built from multiple agents: a manager that coordinates seven specialist roles for proof planning, skeleton scaffolding, Mathlib reconnaissance, proof construction, integration, independent review, and audit. The main methodological discipline is the hypothesis‑disciplined audit, implemented by the Auditor agent: every main‑theorem hypothesis and concept‑layer field must be anchored in the source mathematical prose, justified as a Lean encoding adapter, marked as source‑implied, or rejected as an unsupported strengthening. Using this workflow, we build a systematic formalization of asymptotic statistical theory, especially the parametric and semi‑parametric models' asymptotic distribution and efficiency results. The resulting Lean development is axiom‑clean and source‑faithful, with Lean‑checked and human‑audited proofs of core parametric and semi‑parametric theorems organized so that theorem‑agnostic infrastructure and statistical concept definitions are separated from theorem‑specific assembly. The formalization results are available at https://github.com/junwei‑lu/Lean‑Asymptotic‑Statistical‑Theory.
Authors:Oleg Solozobov
Abstract:
Agent‑runtime systems emit traces, ledgers, provenance graphs, policy logs, delegation tokens, cache events, and tool‑firewall records, but those containers do not necessarily answer governance questions about a specific decision. DEMM‑Bench is a cross‑regime benchmark for agent‑runtime governance‑evidence sufficiency, grounded in the Decision Evidence Maturity Model (DEMM): it measures whether records across eight evidence regimes are sufficient to reconstruct decision‑level properties rather than merely present. The benchmark normalizes the regimes through adapters, asks property questions over actor, authority, action, policy, decision basis, resource touch, lifecycle context, and verification strength, and applies eight deterministic degradation conditions. Across 64 manuscript cases, trace‑present and schema‑present baselines overclaim on 75% of cases, ledger‑present overclaims on 50%, and the redacted property‑level candidate scorer has zero overclaim with 56.25% mean Property Sufficiency Accuracy. The deposited package provides the 64‑case dataset, construction‑oracle labels, baselines, and adapters, supporting reproducible evaluation of decision‑evidence maturity across heterogeneous agent‑runtime evidence substrates.
Authors:Yinsicheng Jiang, Liang Cheng, Yeqi Huang, Yufan Zhao, Zhan Lu, Li Dong, Wenda Li, Edoardo Ponti, Luo Mai
Abstract:
LLM agents are increasingly deployed as multi‑role teams, where tasks are divided across specialized roles such as planner, executor, and verifier. In these systems, cost and accuracy are no longer properties of a single model: they depend on which model fills each role and where it is hosted, including API, self‑hosted, and hybrid deployment. Existing agentic benchmarks typically evaluate fixed models or fixed agent configurations, and therefore offer limited guidance for cost‑accuracy‑optimal deployment. We introduce AgentCARD, a role‑aware benchmark suite for evaluating LLM agent teams across role assignment and deployment mode. AgentCARD combines a role‑decomposed evaluation harness, a unified API/self‑hosted cost model, Pareto‑frontier analysis, and a Shapley‑based diagnostic for identifying role bottlenecks. Our evaluation shows that heterogeneous teams consistently occupy the cost‑accuracy frontier. They improve accuracy by up to 44% over cost‑equivalent homogeneous teams, or match the strongest homogeneous team at up to 12× lower per‑task cost through hybrid deployment. We further find that the best role assignment is domain‑dependent: some domains are planner‑bottlenecked, while others are executor‑bottlenecked. Finally, AgentCARD extends beyond planner‑‑executor teams to workflows with additional roles such as verification, and supports continual evaluation as new domains and team structures emerge. Our code is released at: https://github.com/Auto‑CAP/AgentCAP
Authors:Hang Yu, Zifan Zheng, Jeff Z. Pan, Tongliang Liu, Zhiyong Wang, Fengxiang He
Abstract:
LLM agents are promising for alpha mining via combining financial priors, symbolic reasoning, executable factor generation, and feedback‑driven refinement. Yet, they face a combinatorial search space, noisy non‑stationary feedback, redundant discoveries, and overfitting risks from naively reusing past successes. To address these challenges, we propose AlphaMemo, a self‑evolving alpha mining agent with Structured Search‑Process Memory. Rather than memorizing only final factors or full trajectories, AlphaMemo records reusable evidence about which edit motifs work or fail under specific parent‑factor contexts. It extracts motifs from Abstract Syntax Tree (AST) differences, applies confidence‑gated residual memory on top of a search‑ledger prior, and uses asymmetric veto control to suppress high‑confidence failure patterns. Experiments on CSI 500 and S\&P 500 show improved out‑of‑sample performance and fixed‑budget discovery efficiency, with ablations validating the roles of residual learning, confidence gating, AST‑diff motifs, and veto memory. Code is at https://github.com/jarrettyu/AlphaMemo.
Authors:Kejiang Qian, Fengxiang He
Abstract:
Significant progress has been made in aligning LLMs with target value functions. We argue that, even when an LLM has been well aligned in (post‑)training, it may still fail to maximise the aligned value in reasoning. We mathematically formalise this gap as rational value risk: the utility discrepancy between a model's deployed reasoning strategy and its rational counterpart, which is defined to be the responses that maximise expected utility in the steepest direction. The estimation error of rational value risk is further decomposed into three components from finite candidates, finite prompts, and imperfect verifiers. Extensive experiments are conducted, covering models Llama‑3.1, Qwen‑2.5, T\"ulu‑3 families (7B‑72B), GPT‑5.2, GPT‑5.5, and DeepSeek‑V4, and benchmarks UltraFeedback, AlpacaEval, GSM8K, MATH, HumanEval, and MathArena. The results validate that (1) rational value risk is widespread; (2) value alignment can reduce, but cannot eliminate, it; (3) the risk is highly sensitive to inference‑time reasoning strategy; and (4) longer reasoning improves rationality with diminishing returns. The code is at https://github.com/EVIEHub/LLM‑Rationality.
Authors:Yang Feng, Ziwei Xu, Xia Hu, Fengxiang He
Abstract:
Multi‑agent debate improves the reliability of large language models (LLMs) through iterative peer critiques. However, fixed topologies often introduce persistent positional biases, amplify unreliable agents, and cause high sensitivity to role assignments. We introduce Permutation‑Equivariant Adaptive Routing Multi‑Agent Debate (PEAR), an inference‑time protocol that dynamically reconfigures communication roles and sparse topologies across consecutive debate rounds. By strategically switching agent‑to‑role assignments based on evolving agent states, PEAR prevents any agent from permanently occupying a privileged network position or distributes influence more evenly across the debate. We theoretically characterize PEAR as an equivariant sparse router: it preserves accuracy under agent relabeling while reducing routing complexity and improving generalization. Comprehensive empirical evaluations across four reasoning benchmarks and six diverse LLM backbones demonstrate PEAR significantly improves average accuracy over the strongest debate baselines. The code is at https://github.com/EVIEHub/PEAR.
Authors:Tabia Tanzin Prama, Mikaela Irene Fudolig, Abigail M. Crocker, Christopher M. Danforth, Peter Sheridan Dodds
Abstract:
Intimate Partner Violence (IPV) is a major public health concern, and large language models (LLMs) are increasingly used for support and information‑seeking in sensitive domains. We examine whether LLMs perceive relationship abuse differently depending on victim‑‑perpetrator gender configuration. Using 475 Reddit posts from r/relationship\_advice, we generate counterfactual variants by swapping gendered identifiers to create four dyads: female‑‑female (F/F), female‑‑male (F/M), male‑‑female (M/F), and male‑‑male (M/M), where the first position denotes the victim. Four recent LLMs (GPT‑5o, Gemini 3, Llama 4, and Grok 3) evaluate each variant using a structured questionnaire covering IPV, perpetrator intent, cheating, and abuse subtypes. Results show substantial variation across models and dyads. Abuse and intent detection systematically decrease in mixed‑gender dyads where the victim is male, with female perpetrator identity emerging as a consistent negative predictor of abuse recognition. Mixed‑effects logistic regression confirms that gender roles significantly shape model outputs. Our findings suggest that LLMs reproduce gendered biases from online training data, with implications for support‑related deployment. Code and resources are available at https://github.com/TabiaTanzin/Gender‑Disparities‑in‑LLM‑Based‑Intimate‑Partner‑Violence‑Detection.git.
Authors:Siang-Ling Zhang, Huai-Hsun Cheng, Tsung-Ju Yang, Yu-Lun Liu
Abstract:
Creating 3D visual illusions, a single 3D mesh that reveals entirely different semantics from various viewing angles, is a fascinating but tough challenge. Existing optimization‑based methods are slow and can produce oversaturated colors. In contrast, naive stitching approaches fail to produce geometrically coherent objects. This results in visible unnatural seams and semantic leaks. In this paper, we present a fast and training‑free framework for generating text‑driven 3D visual illusions. Our approach decouples the generation into two stages. First, we propose a cross‑space dual‑branch denoising process. This process dynamically decodes 3D latents into voxel space for CLIP‑guided orientation alignment and Signed Distance Field (SDF) blending, which ensures seamless geometric fusion. Second, we introduce a view‑conditioned texture synthesis module that projects and aggregates view‑specific 2D diffusion priors onto the fused geometry. Extensive experiments demonstrate that our method generates highly realistic, dual‑semantic 3D illusions in just 3‑5 minutes. It significantly outperforms existing methods in geometric integrity, semantic recognizability, and efficiency. Project page: https://siang1105.github.io/JanusMesh.github.io/
Authors:Vishal Verma
Abstract:
Scientific papers are frequently dense and characterized by features such as small fonts and line spacing, double columns of text, and tightly arranged figures. While these features make papers more compact, they can hinder readability, make them less accessible, and can strain the reader. arXiv is a premier open‑access repository for scientific papers across different fields and is used extensively by researchers, including those in the physics and astrophysics communities. Easy Reads is an automated, end‑to‑end, open‑source Python program that helps address the stated challenge by making papers from arXiv more reader‑friendly and accessible. Easy Reads can automatically fetch a paper from arXiv via its URL and work with the source TeX file to allow custom formatting of the paper features, primarily the font size, and the number of columns used. The main goal of Easy Reads is to facilitate ease of reading of scientific papers.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
When a cache miss fetches from cloud object storage, the bill is per GET request and per byte of egress, not latency. Classic caching minimizes the miss rate, the wrong objective: a rarely but expensively fetched object can cost thousands of times more dollars than a frequently but cheaply fetched one. Generalized‑caching theory bounds the miss‑cost objective, but no reported benchmark measures how far deployed heuristics sit from the dollar‑optimal offline policy on real cloud prices. We supply that reference. For uniform‑size page caches with heterogeneous miss costs the offline dollar‑optimum is exact in polynomial time via an integral interval linear program ‑‑ validated against brute force; variable sizes are NP‑hard, so we extend the flow‑based offline bound from the hit‑ratio objective to dollars (cost‑FOO), tight to about four percent. Against this reference we find: (i) a heterogeneity‑regret law ‑‑ LRU's dollar‑regret rises with miss‑cost dispersion (Spearman 0.87) while cost‑aware GreedyDual cuts it to roughly a tenth; (ii) a contention frontier ‑‑ GreedyDual's residual regret collapses to near zero exactly when the budget fits the expensive working set, and is the open slice otherwise; and (iii) a closed‑form crossover s = GET_fee/egress_rate (about 4 KB on S3, 330 B on GCS) that predicts which deployments need dollar‑aware caching at all. On a real Twitter trace the price vector alone moves the workload across s, shifting the regime as predicted. The artifact is a reproducible billing‑faithful benchmark; heuristics and bounds it builds on are prior work, credited.
Authors:Qingyang Zhu, Eric Karl Oermann, Kyunghyun Cho
Abstract:
Bayesian predictive inference provides a principled framework for uncertainty quantification, data efficiency, and robust generalization. However, exact inference is often intractable, and scalable approximations may remain computationally expensive or require restrictive modeling assumptions that degrade predictive performance. Prior‑Data Fitted and in‑context models have recently emerged as an amortized alternative by learning to map datasets directly to predictive distributions, but existing approaches are tightly coupled to the support of the training prior and lack explicit mechanisms for adapting to new priors at test time, resulting in limited robustness under distribution shift. We introduce a multi‑task in‑context learning framework for amortized hierarchical Bayesian predictive inference that explicitly represents prior information as a prefix of in‑context datasets. A transformer trained on sequences of prior and target tasks learns to adapt its predictions across families of priors. On a suite of evaluations with increasing difficulty, including out‑of‑meta‑distribution priors and priors with high‑dimensional latent structures, our method matches oracle Bayesian predictors while being orders of magnitude faster. We further demonstrate its practical relevance on a real‑world spatiotemporal temperature prediction benchmark. Code is available at https://github.com/martianmartina/multi‑task‑bayesian‑icl/.
Authors:Shaghayegh Kolli, Timo Cavelius, Nafiseh Nikeghbal, Samantha Dalal, Jana Diesner
Abstract:
Multimodal large language models (MLLMs) are increasingly deployed in personally and societally consequential settings, yet the visual cues that shape how these models judge people remain poorly understood. Prior work often compares different (groups of) individuals, making it difficult to separate appearance effects from identity differences. We introduce StylisticBias, a controlled benchmark for evaluating attribute‑level social bias in MLLMs. We generate 500 photorealistic base faces and create about 50 single‑attribute variations per face, producing about 25K images. This design keeps identity fixed and changes one visual attribute at a time. It lets us measure how specific cues shift model judgments. We evaluate six MLLMs across 25 binary social judgment scenarios. We find that age and body type dominate identity‑level effects, while fashion style and other visual cues drive the largest attribute‑level shifts. We further find that about 15 attributes account for nearly 80% of the total variation, showing that bias is concentrated in a small set of visual cues. Sensitivity is strongest in judgments that are semantically aligned with appearance, especially socioeconomic and style‑related judgments. We release StylisticBias as a benchmark for fine‑grained bias evaluation in multimodal models. Code and dataset: https://github.com/timo‑cavelius/StylisticBias and https://hf.co/datasets/shaghayegh/stylistic‑bias‑dataset.
Authors:Saimun Habib, Vaishak Belle, Fengxiang He
Abstract:
Neurosymbolic systems such as DeepProbLog combine neural perception with probabilistic logic, but standard inference is associational. Counterfactual reasoning additionally requires a causal semantics for interventions and evidence. We introduce DeepSWIP, a single‑world counterfactual semantics for DeepProbLog programs. Using neural materialization, we reduce fixed‑context neural predicates to ordinary ProbLog choices, apply Single World Intervention Programs (SWIPs), and compute counterfactuals by weighted model counting (WMC) over a single transformed program. Under finite grounding and unique‑supported‑model assumptions, DeepSWIP is exact relative to the learned materialized FCM. The standard quotient‑WMC form of ProbLog conditionals identifies active neural probabilities and explains intervention cleaning, calibration sensitivity, and rare‑evidence instability. Experiments on MPI3D confirm the transformation against a DeepTwin construction against 12,000 queries, as predicted and a 2.14× inference speedup from avoiding the Twin's endogenous duplication. A SUMO HOV experiment shows that neural calibration degradation biases plug‑in estimates, while a correctly scoped randomized‑policy AIPW estimator removes most first‑order bias for population mean and ATE estimands. Code is at https://github.com/saibib/deep_SWIP.
Authors:Juncheng Ma, Jianxin Bi, Yufan Deng, Xuanran Zhai, Kewei Zhang, Ye Huang, Bo Liang, Shukai Gong, Jiankai Tu, Xiaotian Tang, Jiaxin Li, Kaiqi Chen, Duomin Wang, Yuqi Wang, Bingyi Kang, Eric Huang, Zhiyang Dou, Zhen Dong, Enze Xie, Wojciech Matusik, Tat-Seng Chua, Daquan Zhou
Abstract:
Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck. Teleoperated real‑robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment, yet their scalability is limited by high collection cost, acquisition difficulty, and low behavioral and environmental diversity. These limitations have sparked interest in egocentric human video as a scalable, substantially lower‑cost, and more diverse alternative for embodied model pretraining. However, its effectiveness compared to teleoperated real‑robot data remains underexplored. To address this question, we conduct a systematic study comparing egocentric human video and teleoperated real‑robot trajectories as pretraining data sources for embodied foundation models, under fixed post‑training and validation protocols. Surprisingly, we find that egocentric data, when processed through a carefully designed filtering and labeling pipeline, is not merely a viable substitute for model pretraining but can lead to superior performance. With the same amount of pretraining data, models pretrained on egocentric data achieve a 24% lower validation loss on real‑robot action prediction, as well as 52.5% and 90% higher success rates on in‑distribution and out‑of‑distribution real‑robot task execution, respectively. This finding verifies a scalable paradigm for embodied foundation models: pretrain on egocentric human video to learn diverse world representations, then adapt with a small amount of labeled real‑robot data for action‑space alignment. We hope this study encourages broader exploration of egocentric data and offers guidance for data quality assessment before costly robot data collection.
Authors:Yalun Dai, Hao Li, Shulin Tian, Runmao Yao, Yuhao Dong, Fangzhou Hong, Zhaoxi Chen, Fangfu Liu, Baoliang Tian, Dingwen Zhang, Tao Wang, Kim-Hui Yap, Ziwei Liu
Abstract:
Real‑world spatial intelligence requires reasoning over a continuous and evolving 3D world, yet existing VLMs and tool‑augmented agents largely remain tied to static, stateless inference from isolated visual observations. We introduce \textscS‑Agent, a spatial tool‑use agentic paradigm for understanding and reasoning over continuous multi‑view images and videos. By formulating spatial reasoning as spatio‑temporal evidence accumulation rather than isolated frame‑level prediction, \textscS‑Agent reshapes spatial perception into scene‑centric understanding beyond frame‑centric recognition. Specifically, \textscS‑Agent casts the VLM as a semantic planner that decides what evidence is needed, while a hierarchy of spatial tools and experts grounds objects in 2D, lifts them into 3D geometric evidence, and aggregates this evidence into high‑level spatial knowledge (e.g., counting, measurement, orientation, and relative position). Additionally, a temporal memory mechanism, including Scene Memory for maintaining the evolving scene state and Agent Memory for accumulating reasoning context, enables evidence integration across frames and reasoning steps. Comprehensive experiments on multi‑view and video spatial reasoning benchmarks show that \textscS‑Agent consistently improves both open‑source and closed‑source VLMs in a training‑free manner. Beyond inference‑time augmentation, supervised fine‑tuning (SFT) on \textscS‑Agent‑generated spatial trajectories \textscS‑300K yields \textscS‑Agent‑8B, a compact spatial agent that significantly surpasses similar‑scale baselines (e.g., Qwen3‑VL‑8B) and performs comparably to advanced closed‑source models (e.g., GPT‑5.4 and Gemini 3).
Authors:Anthony Leverrier, Rüdiger Urbanke
Abstract:
We introduce the Frontier decoder, a pruned dynamic‑programming decoder for sparse quantum decoding problems. Frontier processes error variables in a chosen order, merges prefixes with the same residual syndrome and logical label, and approximates logical‑coset posterior masses by retaining only a narrow scored frontier. Without pruning, the recursion is exact ordered inference with exponential complexity. In the code‑capacity setting, the decoder reaches thresholds close to optimal for the surface code and the color code. In the circuit‑level noise model, it achieves state‑of‑the‑art performance with a very small average retained list size: less than 100 for the gross code [[144,12,12]] at a physical error rate of 0.001. When the list size is constant, the decoder has linear complexity, suggesting the possibility of low‑latency implementations.
Authors:Jinghong Lan, Wei Cheng, Yunuo Chen, Ziqi Ye, Peng Xing, Yixiao Fang, Rui Wang, Yufeng Yang, Xuanyang Zhang, Xianfang Zeng, Difan Zou, Gang Yu, Chi Zhang
Abstract:
Style‑content dual‑reference generation aims to synthesize an image that preserves the structure and semantics of a content reference while adopting the style of a separate style reference.Despite recent progress, this setting remains challenging because models must balance content fidelity, style alignment, and instruction following avoiding semantic leakage from the style reference.A key bottleneck is the lack of large‑scale triplet data with clean content‑style separation and broad long‑tail style coverage.In this work, we propose FreeStyle, a scalable dual‑reference generation framework based on community LoRA mining.We treat community LoRAs as compositional anchors for style and content, and design a rigorous generation and filtering pipeline to construct large‑scale Style‑Reference and Content‑Reference triplets across multiple base models.To address content leakage, we adopt a two‑stage curriculum with stage‑specific disentanglement mechanisms: an attention‑level enrichment constraint that suppresses style‑reference leakage in the style‑transfer stage, and a frequency‑aware RoPE modulation strategy that targets positional‑correspondence‑based leakage in the harder dual‑reference stage.We also introduce a benchmark covering both style‑reference and dual‑reference generation, with evaluations on style similarity, content preservation, aesthetics, instruction following, and leakage rejection. The benchmark incorporates a style‑invariant Content Alignment Score (CAS) and introduces a calibrated VLM‑based Rejection Score for evaluating generation reliability and leakage suppression.Extensive experiments show that our model achieves a strong balance among style alignment, content preservation, and leakage suppression.
Authors:Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari, Aryan Sajith, Hamed Zamani
Abstract:
To align a Large Language Model (LLM), most existing methods collect explicit human feedback and train a reward model to predict the human preference based on the response text. These existing methods have two key limitations. First, the users rarely provide explicit feedback for LLM responses, which makes the high‑quality preference annotation expensive to collect. Second, the methods do not leverage implicit human feedback, which has proven vital to the economic moats of Internet giants. To quantify the value of implicit feedback, we build a new dataset called IFLLM, which collects 1336 multi‑turn questions from the 59 Mechanical Turk workers, their mouse trajectories, and eye gazing points to the LLMs' responses from their webcams. IFLLM shows that the users have very diverse types of gazing behavior and mouse trajectories. Our reward model based on the implicit user feedback boosts the accuracy of the text‑based reward model from 55% to 64% and nearly triples the relative response quality improvements after applying the DPO to eight LLMs, demonstrating the value of implicit feedback in the wild. Our data collection website, dataset, and codes can be found at https://github.com/themehulpatwari/llm‑implicit‑feedback/.
Authors:Yuehai Chen, Farhad Merchant
Abstract:
Spiking neural networks (SNNs) promise energy‑efficient computing due to their sparse spatio‑temporal activity. However, effectively translating such irregular sparsity into practical performance and energy gains remains challenging, as full‑event computing architectures are still underexplored. This paper proposes ExSpike, a general full‑event neuromorphic architecture that fully exploits irregular sparsity in SNNs. To realize pure event‑driven execution, we first propose a set of dataflow optimizations to ensure that the inputs to each SNN layer remain spike‑based, thereby enabling full‑event execution throughout the network. We then design a hardware‑efficient full‑event architecture, named ExSpike, which supports the optimized pure event‑driven dataflow and an additional Attention Core for spike‑driven self‑attention. To further improve computing efficiency, we introduce adjacent‑position event compression to reduce redundant accumulations across spatially adjacent spike sequences. ExSpike is implemented on an AMD Xilinx Virtex‑7 FPGA and evaluated on both classification and segmentation workloads. Experimental results show that ExSpike achieves high normalized energy efficiency across diverse SNN models while maintaining competitive accuracy, delivering up to 479.15 GOPS, 281.85 GOPS/W, and 0.80 GOPS/W/PE. In particular, ExSpike achieves up to 10× higher PE‑normalized energy efficiency than the SOTA FPGA‑based SNN accelerator (FireFly‑T). The code for ExSpike is available at \urlhttps://github.com/xiaoyuehai/ExSpike.
Authors:Ning Dong, Yingna Su, Xin Dong, Ziyun Jiao, Xinnian Guo, Zhuangzhuang Pan
Abstract:
Pose‑flow video anomaly detectors are attractive for one‑class surveillance because they provide likelihood‑based rankings for tracked skeleton windows. However, a single likelihood score may hide multimodal normal behavior and be sensitive to pose‑observation noise. We study a frozen‑detector setting in which the pose‑flow backbone, cached skeleton tracks, and evaluation pipeline are fixed. Reliability‑Aware Prototype Calibration (RPC) is a post‑hoc score calibration method for this setting. It adds a standardized nearest‑prototype deviation in the frozen latent space to the standardized flow score, and uses keypoint confidence only to gate this added geometric evidence. Thus, RPC preserves the original density signal while correcting the ranking with empirical normal‑mode structure under pose reliability. Across two frozen pose‑flow backbones and four datasets, RPC improves frame‑level AUROC in all eight backbone‑dataset pairs, with gains ranging from 0.34 to 4.49 percentage points and averaging 2.03 points. Ablation and reliability analyses show that prototype deviation is the main corrective signal, while reliability gating is most useful when pose observations are less trustworthy. These results suggest that lightweight post‑hoc calibration can strengthen cached pose‑flow systems when retraining or reproducing the full pose pipeline is impractical.
Authors:Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro
Abstract:
Deepfakes targeting a high‑profile individual, known as Person‑of‑Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post‑processing, efficiency and interpretability, focal aspects of modern deepfake detectors. In this paper we propose CUPID, a POI video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake videos in its training phase. Moreover, it does not even require to include a specific POI in the training set: the combination of UV texture maps extracted from real video frames and the MAE context‑guided reconstruction yields a latent space that captures rich and discriminative facial features also for identities unseen during training. In the testing phase, the embeddings extracted from a query video depicting the POI can be matched against pristine reference videos to assess the video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake datasets show that CUPID outperforms current state of the art on most datasets and achieves the best overall robustness against strong downscaling and compression, providing also substantially faster inference. Our experimental code will be released at https://github.com/polimi‑ispl/CUPID.
Authors:Junhui Li, Jialu Li, Youshan Zhang
Abstract:
Mamba‑based models have emerged as a promising alternative for salient object detection (SOD), offering significant advantages in modeling long sequences. However, existing models often fail to explore contextual information and the depth of the entire architecture. This paper introduces U^2Mamba, a powerful and innovative U‑structured network for salient object detection. We propose multiscale Mamba U‑blocks (MMUBs) that enhance the model depth to improve local feature extraction capabilities. Our newly developed nested U‑structure, incorporating MMUBs, enables the network to integrate various receptive fields from shallow and deep layers, thereby collecting richer contextual information and longer‑range data without being constrained by resolution. Instead of using the traditional deep supervision scheme and top‑level supervised training, we propose a hierarchical training supervision method where the loss is computed at each level during the training process. Extensive experiments demonstrate that U^2Mamba achieves highly competitive performance against state‑of‑the‑art methods. The source code is available at \urlhttps://github.com/JL021/U2Mamba.
Authors:Duc T. Nguyen, Hoang-Long Nguyen, Thanh-Ha DO, Huy-Hieu Pham
Abstract:
Existing weakly supervised semantic segmentation (WSSS) methods in computational pathology rely on a multi‑stage paradigm: class activation map (CAM) generation, offline pseudo‑mask refinement, and fully supervised retraining. While established, this decoupled approach presents fundamental limitations. The multi‑stage process not only incurs high computational training costs but also suffers from error propagation: local texture biases in shallow CNN layers generate false‑positive artifacts that subsequent refinement steps often fail to correct. To address these persistent challenges through a simple yet highly effective approach, we propose the Single‑Stage Hierarchical Rectification (SSHR) framework. Rather than passively refining CAMs post‑hoc, our method proactively purifies intermediate feature representations during the forward pass. We introduce a Hierarchical Feature Rectification Module (HFRM) that utilizes deep global semantic context to filter out local anomalies in shallow layers. This mechanism generates high‑fidelity activation maps directly within a single training loop. Experiments on the LUAD‑HistoSeg and BCSS datasets demonstrate that SSHR outperforms state‑of‑the‑art multi‑stage methods. Furthermore, SSHR reduces training duration by 2 to 5 times. This efficiency minimizes computational overhead and accelerates clinical translation for large‑scale histopathology workflows. The code is available at: https://github.com/trongduc‑nguyen/SSHR
Authors:Bo Yin, Xiaobin Hu, Chengming Xu, Ruolin Shen, Mo Yang, Jiangning Zhang, Peng-Tao Jiang, Cheng Tan, Shuicheng YAN
Abstract:
Vision‑language models (VLMs) often underperform on evidence intensive tasks because decisive visual evidence are small, localized, and easy to overlook, leading to failures in evidence readout even when high‑level reasoning is intact. Prior inference‑time visual interventions can improve grounding without retraining, but they are largely open‑loop and lack a mechanism to verify whether highlighted evidence is actually used. We study answer‑span prediction entropy as a model‑internal feedback signal and show that naive entropy minimization is ambiguous, since low entropy may arise from evidence‑grounded confidence or shortcut collapse. To resolve this ambiguity, we introduce low‑entropy anchors and an entropy‑shaping objective that reduces answer uncertainty while preserving baseline high‑confidence tokens. We instantiate this principle in SPOT‑E, a plug‑and‑play test‑time method that produces question‑conditioned spotlights, optimized per instance via light‑weight tuning based on Group Relative Policy Optimization (GRPO). Across all benchmarks and different VLM families, SPOT‑E yields consistent gains and improved robustness under visual corruptions. Code is publicly available at: \urlhttps://github.com/YinBo0927/SPOT‑E
Authors:Yudong Li, Zihao Fang, Junwen Qiu, Ruihai Jing, Ruixiang Hang, Yingda Shen, Zhizheng Wu
Abstract:
Streaming zero‑shot voice conversion struggles to disentangle timbre from linguistic content without degrading utility or inflating latency. Current methods rely on information bottleneck (IB) or speaker perturbation. While IB filters out timbre, it discards prosody, forcing models to explicitly inject features like fundamental frequency. This often requires buffering future frames, creating algorithmic lookahead latency. On the other hand, existing perturbation methods largely overlook the crucial trade‑off between timbre leakage and utility preservation. Recognizing this neglected trade‑off, we find that the inherent objective of Speaker Anonymization (SA) aligns well with balancing these factors. Thus, we introduce SA as a novel perturbation mechanism to explicitly mitigate timbre leakage while retaining prosodic utility. Crucially, SA's robust representations significantly alleviate the generator's reliance on future context, enabling our strictly causal, zero‑lookahead network. Audio samples are available at https://amphionteam.github.io/Zero‑VC‑demo/.
Authors:Francesco Argenziano, Miguel Saavedra-Ruiz, Sacha Morin, Charlie Gauthier, Daniele Nardi, Liam Paull
Abstract:
Joint spatial and temporal understanding of 3D scenes is a crucial requirement for robots deployed in everyday household environments. Such agents must not only comprehend and navigate spatial layouts, but also reason about how these spaces evolve over time. In particular, humans interact with objects daily, causing them to change position throughout the environment and making it difficult for robots to reliably associate current observations with previously seen objects. However, these interactions are not random: human habits and routines induce spatio‑temporally consistent patterns in object locations, which robotic agents can potentially learn and then exploit for downstream tasks such as navigation. To this end, we introduce FlowMaps, a latent flow matching model for estimating multimodal distributions over the future locations of dynamic objects in a continuous 3D space. By learning the implicit dependencies among objects and their temporal evolution, FlowMaps predicts likely changes in object locations conditioned on past human interactions, while supporting generalization across previously unseen environments that share similar object routines. To demonstrate the utility of this method, we deploy FlowMaps in a downstream dynamic Object Navigation task in both simulated and real‑world environments. Across more than 600 episodes, FlowMaps outperforms state‑of‑the‑art approaches, showing that modeling object dynamics through continuous, multimodal spatio‑temporal distributions improves robotic search and navigation in changing household environments. Code and additional material is available at https://fra‑tsuna.github.io/flowmaps/.
Authors:Hyun-Kurl Jang, Jihun Kim, Hyeokjun Kweon, Kuk-Jin Yoon
Abstract:
Continual Test‑Time Adaptation (CTTA) aims to maintain model performance under evolving target domains by adapting online without labeled data. However, practical deployments often cannot retain the source dataset due to privacy or licensing constraints, and purely source‑free CTTA methods tend to become unstable under long‑term distribution shift, suffering from compounding self‑training errors and catastrophic forgetting. We introduce DO‑ALL (Distill Once, Adapt Life‑Long), a plug‑and‑play framework that revisits source information in a compact and privacy‑conscious form via Dataset Distillation (DD). Before deployment, DO‑ALL performs DD to produce a small set of synthetic distilled anchors that summarize the source distribution. During adaptation, each target sample is matched with its most semantically aligned anchor, which provides a stable reference for various CTTA via source replay, representation alignment, and manifold‑smoothing regularization. DO‑ALL can be seamlessly integrated into existing CTTA algorithms, consistently improving long‑term robustness across CIFAR100‑C, ImageNet‑C, and the CCC benchmark. This demonstrates the potential of leveraging DD to enable stable and continuous adaptation without retaining raw source data. The code is available at https://github.com/blue‑531/DOALL.
Authors:Maciej Wozniak, Jesper Ericsson, Hariprasath Govindarajan, Truls Nyberg, Thomas Gustafsson, Patric Jensfelt, Olov Andersson
Abstract:
Leveraging Vision Foundation Models (VFMs) for camera‑to‑LiDAR knowledge distillation offers a promising solution to the scarcity of annotated data needed to represent the immense geometric and kinematic diversity of real‑world autonomous driving (AD). However, current approaches typically treat VFMs as black‑box teachers, relying exclusively on frame‑wise feature similarity. Consequently, they do not fully exploit the teacher's layer‑wise semantic structure and global context, as well as the rich spatiotemporal information inherent in LiDAR sequences. We propose HilDA, a self‑supervised pretraining framework for LiDAR backbones that better captures the semantic what and geometric where needed for driving tasks. HilDA combines hierarchical distillation comprising multi‑layer distillation for progressive semantic alignment and global context distillation for scene‑level semantics, with a temporal occupancy diffusion objective promoting spatiotemporal consistency. Models pre‑trained with HilDA achieve state‑of‑the‑art results on cross‑modal distillation benchmarks and outperform models trained via prior distillation approaches on 3D object detection, scene flow, and semantic occupancy prediction. Code available at: https://maxiuw.github.io/hilda.
Authors:Diego Fajardo-Rojas, Megan Hall, Daniel Cromb, Mary A. Rutherford, Lisa Story, Emma C. Robinson, Jana Hutter
Abstract:
Preterm birth is associated with significant mortality and a risk for lifelong morbidity. The complex multifactorial aetiology hampers accurate prediction and thus optimal care. A pipeline consisting of bespoke machine learning methods for data imputation, feature selection, and regression models to predict gestational age (GA) at birth was developed and evaluated from comprehensive multi‑modal morphological and functional fetal MRI data from 333 control cases and 93 preterm birth cases. The GA at birth predictions were classified into term and preterm categories and their accuracy, sensitivity, and specificity were reported. An ablation study was performed to further validate the design of the pipeline. Performance was evaluated using stratified 10‑fold cross‑validation. The pipeline achieves an R2 score of 0.13 and a mean absolute error of 2.74 weeks. It also achieves a 0.77 accuracy, 0.59 sensitivity, and 0.82 specificity across folds. The predominant features selected by the pipeline include cervical length and statistics derived from placental T2 values. The confluence of fast, motion‑robust and multi‑modal fetal MRI techniques and machine learning prediction allowed the prediction of the gestation at birth. This information is essential for any pregnancy. To the best of our knowledge, preterm birth had only been addressed as a classification problem in the literature. Therefore, this work provides a proof of concept. Future work will increase the cohort size to allow for finer stratification within the preterm birth cohort. Our code is available at https://github.com/dfajardorojas/ml‑for‑preterm‑birth‑.
Authors:Tong Wang, Siwen Wang, Yaolei Qi, Jinxing Zhou, Yuting He, Guanyu Yang, Yutong Xie
Abstract:
Imperfectly supervised video polyp segmentation (VPS) aims to learn dense, temporally consistent masks from inexpensive supervision, including weak annotations (points, scribbles) and semi‑supervision with few densely labeled frames. This setting is clinically valuable but challenging due to weak contrast, ambiguous boundaries, motion blur, and specular highlights, compounded by sparse pixel‑level guidance. While SAM2 can generate dense masks from sparse inputs, direct pseudo‑labeling often yields geometry‑degraded masks with boundary leakage, underutilizes temporal consistency, and ignores reliability. To address these issues, we propose ARTEMIS, a unified framework for imperfectly supervised VPS driven by agent‑guided reliability‑aware temporal mask evolution. ARTEMIS initializes coarse masks from available supervision: SAM2 converts points/scribbles, while dense labels serve as reliable anchors. A debate‑and‑judge vision‑language agent selects reliable temporal anchors under weak supervision, which are propagated bidirectionally with SAM2 to refine unreliable or unlabeled frames. Finally, ARTEMIS trains the segmenter using temporal reliability‑aware robust learning, incorporating reliability‑guided reference selection, a Reference Prototype Transport Module, and reliability‑aware robust loss. These components assess mask reliability, evolve anchors over time, transport target identity across frames, and down‑weight noisy supervision instead of discarding difficult samples. Experiments on SUN‑SEG and CVC‑ClinicDB‑612 under scribble, point, and limited‑label settings demonstrate that ARTEMIS achieves state‑of‑the‑art performance. Code will be released at https://github.com/wangtong627/ARTEMIS.
Authors:Heejo Kong, Beomchul Park, Sung-Jin Kim, Seong-Whan Lee
Abstract:
Physics‑informed neural networks (PINNs) have become a powerful framework for solving PDEs by embedding physical laws into differentiable objectives. Despite their advances, training PINNs remains fragile: recent conflict‑averse optimization schemes alleviate gradient interference between residual and boundary losses, but we show that their effectiveness deteriorates as model capacity increases. In this paper, we identify a capacity‑induced failure mode, where overparameterized networks undergo functional modularity, self‑partitioning into task‑exclusive modules that suppress cross‑objective interaction and hinder convergence toward Pareto‑stationary points. To address this issue, we propose a novel framework, Modular‑Sparsity Synchronization (ModSync), which integrates structural optimization into conflict‑averse training by penalizing task‑exclusive connections while preserving interaction‑promoting pathways. Extensive experiments across diverse PDE benchmarks demonstrate that ModSync consistently prevents capacity‑driven failures, sustains robust cross‑objective coupling, and achieves state‑of‑the‑art accuracy. Codes are available at \urlhttps://github.com/heejokong/ModSync.
Authors:Masaya Kawamura, Yuma Shirahata, Kentaro Mitsui, Reo Shimizu
Abstract:
Existing mean opinion score (MOS) prediction models typically predict utterance‑level naturalness MOS and can be insensitive to localized pitch‑accent errors. We propose Pitch‑Accent‑focused Speech Quality Assessment (PASQA), which explicitly targets pitch‑accent correctness. To train our model, we construct a controlled Japanese accent‑error dataset by changing accent patterns using an accent‑controllable text‑to‑speech system, and compute a pseudo accent‑quality score from the accent‑error rate. PASQA builds on self‑supervised representations and employs mora‑conditioned fusion, ranking loss, an auxiliary accent‑error localization task, and speaker‑invariant training. Experiments show that conventional models fail to preserve the ordering by accent‑error severity, whereas PASQA achieves high ordering accuracy on both seen and unseen speakers. Further, PASQA shows stronger agreement with human accent‑correctness judgments. The code is available at https://github.com/lycorp‑jp/PASQA.
Authors:Zhenkai Zhang, Markus Hiller, Krista A. Ehinger, Tom Drummond
Abstract:
Generating high‑resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel‑Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high‑quality 3D medical volumes directly at voxel‑level. PRDiT introduces a two‑stage training architecture comprising 1) a local denoiser in the form of an MLP‑based blind estimator operating on overlapping 3D patches to separate low‑frequency structures efficiently, and 2) a global residual diffusion transformer employing memory‑efficient attention to model and refine high‑frequency residuals across entire volumes. This coarse‑to‑fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC‑IDRI and RAD‑ChestCT datasets demonstrate that PRDiT consistently outperforms state‑of‑the‑art models, such as HA‑GAN, 3D LDM and WDM‑3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.
Authors:Pengwei Wang, José Morano, Qian Wan, Hrvoje Bogunović
Abstract:
Image quality control is vital for a wide range of downstream applications. Deep learning‑based image quality assessment methods typically train classifiers on dataset‑specific quality labels, inheriting two limitations: (1) generalization is tied to the labeling criteria of the training set and (2) these methods cannot provide spatial feedback on where the quality is degraded, lacking explainability. In this work, we propose EFIQA, a framework that requires no quality‑related supervision and produces spatial quality maps by design. Rather than learning ``what is degradation" from human‑annotated labels, EFIQA learns ``what should be there" by leveraging anatomical priors. For fundus photography, we instantiate this as a two‑stage approach, by first training an unsupervised anomaly detector via masked anatomical inpainting to identify regions of missing vasculature, and then distilling this prior knowledge into a shallow adapter mapping features of a frozen foundation model to precise quality maps. External‑dataset evaluation demonstrates that this label‑free approach with minimal adaptation achieves better performance and explainability compared with supervised methods across benchmarks with different quality criteria, highlighting its potential for real‑world applications.
Authors:William Guey, Pierrick Bougault
Abstract:
Large language models (LLMs) increasingly review and revise text, including their own. A documented self‑preference bias (models favoring their own generations when acting as judges) raises the question of whether models also resist valid corrections to their own writing. We test this in a setting where "valid" is decided not by another model but by a deterministic verifier: instruction‑following revision on IFEval. A model writes a draft; the official IFEval checker confirms the draft violates a constraint and that a candidate edit fixes it; the model then accepts or rejects that edit either as the genuine in‑context author or as a fresh model that sees the draft neutrally. Across four mid‑tier model families and 85 author‑versus‑fresh comparisons, we find no detectable self‑preference: authors reject verified‑good fixes to their own drafts at essentially the same rate as fresh models judging the same drafts (gap ‑5.1 pp, 95% CI [‑12.9, +2.7]). A self‑skepticism hint from a smaller pilot did not replicate at scale. The one robust observation is qualitative: when authors do reject a verified‑good fix, 97% of their stated reasons are flaw‑catching rather than preference, that is, about the character of rejections, not an elevated rate. Effects smaller than ~13 pp cannot be excluded at this sample size.
Authors:Xiangchen Yin, Wenzhang Sun, Jiahui Yuan, Zijie Liu, Yinda Chen, Wei Li, Dachun Kai, Chunfeng Wang, Xiaoyan Sun
Abstract:
Video world models are moving toward preserving an observed world under controllable camera and object motion while allowing its environmental state to change. Yet these controls remain isolated, and weather generation typically relies on a source video or reconstructed scene that already specifies future structure. We study a first‑frame‑anchored source‑to‑state setting, where the model starts from a single image and follows explicit camera and object controls and an optional weather instruction, then generates a video that either preserves the source world or transfers it to a target weather state. To address these challenges, we first build HoloStateData, a state video dataset that turns diverse videos into unified control samples for camera, object, and weather supervision. Second, we introduce Holo‑World, a unified controllable video world model that jointly controls scene from a single image. Its Unified Scene Adapter factorizes world preservation and weather transfer into distinct parameter subspaces, using rendered background, geometry buffers, and object controls to maintain controlled scene structure while modeling weather‑dependent appearance and particle effects. Additionally, Scene‑Weather Decomposed CFG guides scene and weather residuals separately, strengthening target weather effects without over‑amplifying the full condition. Quantitative and qualitative experiments demonstrate that Holo‑World maintains precise camera and object control with consistent scene structure while transferring scenes into diverse target weather state, outperforming video‑to‑video weather editing baselines on weather‑state generation. Our project page is available at \urlhttps://xiangchenyin.github.io/Holo‑World/.
Authors:Dong Hoon Lee, Seunghoon Hong
Abstract:
Latent Diffusion Models (LDMs) have become dominant in visual synthesis, but their quality‑compute trade‑off is largely constrained by the tokenizer's fixed compression ratio. Variable‑length tokenizers (VLTs) promise adaptive compression by varying token counts, allowing diffusion models to flexibly balance quality and compute. However, conventional VLTs modulate length by truncating ordered token sequences, which makes token semantics depend on token position and breaks representational alignment across lengths. This leads to a cross‑length shift in the latent distribution that hinders a single variable‑length diffusion model from operating effectively. To address this, we propose a novel variable‑length tokenizer that modulates length by merging tokens. We show that encouraging similar tokens to merge enables direct cross‑length representation alignment when the diffusion transformer operates according to the merging pattern. Since conventional merging methods are data‑dependent, making the merging pattern inaccessible during generation, we introduce learnable global merging, which is data‑independent, to ensure compatibility with diffusion transformers. On ImageNet 256×256 generation, our merging‑based variable‑length tokenizer integrated with a diffusion transformer achieves a superior gFID‑compute trade‑off compared to prior VLT methods. Code is available at [this https URL](https://github.com/movinghoon/lgm)
Authors:Xinghao Chen, Chak Tou Leong, Wenjin Guo, Jian Wang, Wenjie Li, Xiaoyu Shen
Abstract:
Latent Chain‑of‑Thought (CoT) internalizes reasoning within continuous hidden states, offering a promising alternative to verbose discrete reasoning traces. However, robust latent reasoning remains difficult because outcome supervision provides weak learning signals and leaves latent trajectories prone to semantic drift. In this work, we analyze Latent CoT from an information‑theoretic perspective and identify this failure as a dual collapse: gradient attenuation along the optimization path and representational drift in the latent space. We further decompose process supervision into two complementary dimensions: Trajectory Supervision, which injects dense stepwise reasoning signals, and Space Supervision, which preserves the semantic structure of the latent manifold. Our analysis shows that rigid geometric compression can collapse the reasoning space, whereas generative reconstruction provides a more flexible semantic anchor that better preserves information capacity. To measure these effects, we introduce the Unified Latent Probe (ULP), which quantifies the mutual information between latent trajectories and explicit reasoning steps. Experiments reveal a clear Information‑Performance Binding: reasoning accuracy depends on the information fidelity preserved in the latent chain. These findings provide a principled framework for latent reasoning supervision and suggest shifting from geometric imitation toward mutual information maximization. Our code is available at \hrefhttps://github.com/EIT‑NLP/Supervision‑in‑Latent‑CoTthis repository.
Authors:Fanfu Xue, En Yu, Yantian Shen, Zhikun Hu, Hongjun Wang, Yang Yang, Xindi Wang, Jiande Sun
Abstract:
UAV Vision‑Language Navigation (UAV‑VLN) is typically formulated as a holistic search‑and‑reach problem, where long‑range target discovery and final target approach are optimized and evaluated jointly. This formulation makes it difficult to assess a critical capability of aerial embodied agents, namely whether a UAV can accurately ground a visible target and translate vision‑language evidence into precise 3D motion once the target enters its field of view. To address this limitation, we introduce UAV‑VLN‑FOV, a target‑visible navigation task that isolates the see‑and‑reach stage and enables a more diagnostic evaluation of terminal reaching ability. We further propose 3DG‑VLN, a vision‑language waypoint prediction framework guided by dynamic 3D direction cues to enhance fine‑grained visual grounding and spatial direction alignment for precise target reaching. Specifically, 3DG‑VLN adaptively processes high‑resolution front‑view and downward‑view observations to preserve fine‑grained visual and geometric details for target grounding. It also updates the target‑relative direction online during closed‑loop navigation, allowing the agent to maintain spatial alignment with the target and reduce accumulated direction drift. To support this task, we construct a dedicated high‑resolution benchmark which contains 2,717 trajectories with target‑oriented high‑level instructions, high‑resolution front‑view and downward‑view egocentric observations, and continuous 3D waypoint annotations. Experiments show that 3DG‑VLN outperforms competitive UAV‑VLN baselines, achieving a 13.82% improvement in success rate. Real‑world trials further demonstrate the potential of 3DG‑VLN for practical see‑and‑reach navigation. The source code and benchmark are available at https://github.com/xuefanfu/3DG‑VLN.
Authors:Hongming Zhu, Huaji Chen, Bowen Du, Sicong Liu, Qin Liu
Abstract:
Unlike traditional remote sensing change detection that relies on predefined categories, Open‑Vocabulary Change Detection (OVCD) identifies land cover changes flexibly using arbitrary text prompts. However, existing methods suffer from an inherent trade‑off when modeling changes: instance‑level comparison overlooks fine‑grained semantic variations (e.g., partial building extensions), while direct pixel comparison proves unreliable, yielding unstable responses and boundary artifacts due to semantic ambiguity and spatial inconsistency. To this end, we propose an efficient training‑free Reliability‑Aware Open‑Vocabulary Change Detection (ReA‑OVCD) framework. It first derives candidate change regions from pixel‑wise semantic discrepancies to ensure flexible and detailed localization. To ensure reliability, it subsequently introduces a collaborative refinement strategy to explicitly model change validity from both semantic and spatial perspectives. Specifically, we develop a Semantic Change Reasoning (SCR) module that reassesses changes by jointly analyzing distributional divergence and response variation, enabling the suppression of incidental inconsistencies while preserving reliable semantic shifts. In addition, a Boundary‑aware Change Refinement (BCR) module is designed to mitigate artifacts stemming from boundary misalignment and uncertainty through validating whether candidate regions are supported by reliable interior pixels. Extensive experiments across multiple datasets (LEVIR‑CD, WHU‑CD, DSIFN, and SECOND) demonstrate that our method consistently outperforms state‑of‑the‑art approaches, achieving \mathrmF_1^C improvements of 2.13% to 9.75% with higher computational efficiency. The code is publicly available at \https://github.com/Funny0101/ReA‑OVCD
Authors:Luca Zedda, Davide Antonio Mura, Cecilia Di Ruberto, Maurizio Atzori, Muhammed Furkan Dasdelen, Carsten Marr, Andrea Loddo
Abstract:
Attention‑based Multiple Instance Learning aggregators in medical imaging are prone to attention concentration, producing overconfident and unstable predictions. We introduce QG‑MIL, a gated transformer aggregator that addresses this through four synergistic architectural components: RMSNorm‑based pre‑normalization, per‑head QK normalization, fine‑grained attention output gating, and SwiGLU‑style feed‑forward modules. Together, these design choices stabilize training and distribute attention more uniformly across instances without auxiliary losses, masking, or multi‑stage regularization. We evaluate QG‑MIL across six benchmarks spanning whole‑slide pathology and cell‑level hematology, covering two fundamentally different MIL scales. The best‑performing QG‑MIL variants outperform leading baselines on all six benchmarks, with an average improvement of +6.1 mean macro F1 points. Attention overlays and attention mass analysis confirm more distributed instance weighting. Ablation studies show that while individual components can match the full model on specific datasets, the QG‑MIL design provides the most consistent cross‑domain performance and tightest variance when compared to selected baselines. We release a configurable implementation to support reproducibility at: https://github.com/unica‑visual‑intelligence‑lab/QG‑MIL
Authors:Kaiyue Yang, Yuyan Bu, Jingwei Yi, Yuchi Wang, Biyu Zhou, Juntao Dai, Songlin Hu, Yaodong Yang
Abstract:
As LLM agents increasingly select tools autonomously, their choices among tools with different privileges become safety‑relevant. However, prior tool‑selection studies focus on safety‑agnostic metadata preferences, leaving privilege‑sensitive choices underexplored. To address this gap, we study over‑privileged tool selection, in which an agent selects or escalates to a higher‑privilege tool despite a sufficient lower‑privilege alternative. We introduce ToolPrivBench to evaluate whether agents choose higher‑privilege tools despite sufficient lower‑privilege alternatives, measuring both initial selection and escalation after transient tool failures. Across eight domains and five recurring risk patterns, we find that over‑privileged tool selection is common among mainstream LLM agents and is further amplified by transient failures. We further find that general safety alignment does not reliably transfer to least‑privilege tool choice, while prompt‑level controls provide only limited mitigation under transient failures. We therefore introduce a privilege‑aware post‑training defense that teaches agents to prefer sufficient lower‑privilege tools and escalate only when necessary. Our mitigation experiments show that this defense substantially reduces unnecessary high‑privilege tool use while preserving general capabilities.
Authors:Xu Zhang, Zhengang Huang, Yunzhi Wu, Xun Lu, Erpeng Qi, Yunkai Chen, Zhongya Xue, Peng Wang, Wei Wang
Abstract:
Current time series forecasting (TSF) research predominantly focuses on scale‑homogeneous data, where different time series share similar numerical magnitude ranges. However, in real‑world industrial scenarios such as financial product sales, different time series often differ by orders of magnitude (scale heterogeneity). Since these series share similar temporal patterns, joint modeling is desirable for better data utilization, yet existing scaling methods either compress low‑scale signals (global normalization) or destroy semantic discriminability and amplify inverse‑scaling errors (window‑based scaling). This paper proposes a self‑Adaptive Scale‑handling (AS) module that learns adaptive scale factors tailored to each input, preserving semantic discriminability while reducing inverse‑scaling errors. AS consists of Scale Calibrating (SC), which calibrates prior mean scaling factors through neural networks, and Scaling Selection (SS), which decides whether to apply calibration or retain the original factor, avoiding over‑calibration. Experiments on real‑world fund sales datasets from Ant Fortune and Alipay show that AS seamlessly integrates into popular TSF models and consistently improves their performance. The code and dataset are available at the link https://github.com/Meteor‑Stars/ASTSF.
Authors:Yanxi Chen, Weijie Shi, Yuexiang Xie, Boyi Hu, Yaliang Li, Bolin Ding, Jingren Zhou
Abstract:
This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta‑capability required by long‑lifecycle agents: as an LLM‑based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self‑updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end‑to‑end reinforcement learning (RL) with long rollout sequences interleaving solve‑task and update‑context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta‑capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof‑of‑concept implementations of the CoD framework, including a GRPO‑style RL algorithm with fine‑grained credit assignment, as well as tasks and environments tailored to the targeted meta‑capability (rather than domain‑specific LLM capabilities or standard task‑by‑task RL). Empirical results validate the efficacy of end‑to‑end RL training in the CoD setting, and demonstrate the potential for out‑of‑distribution generalization ‑‑ within the training domains, across different domains, and from CoD to Ralph‑loop settings ‑‑ of the elicited meta‑capability. Our investigation of CoD connects several lines of prior works, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at \urlhttps://github.com/agentscope‑ai/Trinity‑RFT/tree/research/cod/examples/research_cod.
Authors:Yu Deng
Abstract:
Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining. Existing methods handle only single‑direction injection; when multiple semantic directions are superposed without constraints, the model collapses. We show that this collapse decomposes into two independently acting sources: distributional deviation, where additive perturbations accumulate in norm across layers and drive activations outside the training distribution, and directional interference, where non‑orthogonal semantic vectors mutually dampen when superposed. These two sources define the design constraints that any training‑free multi‑directional intervention must address. As one instantiation of these principles, we propose GEMS, a training‑free method that maps each source to a corresponding geometric constraint: norm‑preserving weighted superposition and targeted attention‑pathway injection for distributional deviation, and real‑time orthogonalization for directional interference. On GSM8K, injecting three concurrent non‑mathematical directions preserves accuracy at 98% (baseline 92%), while unconstrained addition collapses to 4%; on Wikitext‑2, the same injection incurs only 2.2% PPL increase. Component ablation isolates the causal role of each constraint, and layer‑level probes confirm that orthogonalized signals survive the FFN pathway and reach the output distribution with semantic specificity. Qualitative steering effects transfer across architectures from 3B to 31B.
Authors:Guangyi Liu, Pengxiang Zhao, Gao Wu, Yiwen Yin, Mading Li, Liang Liu, Congxiao Liu, Zhang Qi, Mengyan Wang, Liang Guo, Yong Liu
Abstract:
MLLM‑based mobile GUI agents have made substantial progress in UI understanding and action execution, but adapting them to real target apps remains costly because mobile apps are numerous, frequently updated, and hard to cover with human‑written tasks, demonstrations, or reward labels. Existing annotation‑free GUI learning reduces manual supervision, yet lacks a unified substrate connecting target‑app exploration, curriculum mining, rollout execution, and feedback, while policy optimization often relies on isolated rollouts and coarse rewards that are hard to convert into reliable improvement signals. We present MobileForge, an annotation‑free adaptation system for mobile GUI agents. MobileForge consists of MobileGym, which grounds task generation and rollout evaluation in real mobile app interaction, and Hierarchical Feedback‑Guided Policy Optimization (HiFPO), which turns trajectory outcomes, step‑level process feedback, and corrective hints into hint‑contextualized step‑level GRPO updates. Using only automatically generated annotation‑free adaptation data, MobileForge adapts Qwen3‑VL‑8B to 67.2% Pass@3 on AndroidWorld, close to the closed‑data GUI‑specialized GUI‑Owl‑1.5‑8B base model at 69.0%. The MobileForge‑adapted ForgeOwl‑8B further reaches 77.6% Pass@3 on AndroidWorld and 41.0% success on the out‑of‑domain MobileWorld GUI‑only split, establishing the strongest open‑data mobile GUI agent in our evaluation. Code, data, and trained models will be released at https://mobile‑forge.github.io/.
Authors:Chengwen Liu, Hao Peng, Jisheng Dang, Hong Peng, Bin Hu, Tat-Seng Chua
Abstract:
In multimodal video reasoning, reinforcement learning‑based methods typically rely on simplistic and inflexible reasoning‑length control strategies that fail to adapt to the model's evolving competence. This mismatch may suppress necessary exploration at early stages, while encouraging redundant reasoning and inefficient decoding once the model becomes more competent. In this paper, we propose CARE, a competence‑aware reward shaping framework for adaptive reasoning length optimization in multimodal reasoning. Specifically, CARE maintains a smoothed competence estimate via an exponential moving average of pass rates, and uses it to route training into progressive stages that shift the reward preference from exploration‑oriented long‑form reasoning to efficiency‑oriented concise reasoning. To avoid conflating verbosity with intrinsic task complexity, CARE further normalizes reasoning effort with batch‑level statistics, and introduces a posterior amplifier to strengthen reward signals for unexpectedly strong performance on historically difficult samples. The proposed mechanism is seamlessly integrated into the GRPO training pipeline and incurs no additional inference‑time overhead. Extensive experiments on multiple video reasoning and general video understanding benchmarks demonstrate that CARE consistently improves reasoning accuracy, stabilizes reinforcement learning, and significantly enhances token efficiency. Moreover, CARE exhibits a characteristic inverted‑U trajectory of reasoning length during training, and yields shorter yet more informative reasoning traces at convergence, indicating effective adaptive allocation of reasoning budget. We provide the source code for our proposed CARE framework and experiments at https://github.com/1Pansy/Video‑CARE.
Authors:Guangyi Liu, Gao Wu, Congxiao Liu, Pengxiang Zhao, Liang Liu, Mading Li, Qi Zhang, Mengyan Wang, Liang Guo, Yong Liu
Abstract:
MLLM‑based mobile GUI agents have made substantial progress on short‑horizon tasks, yet remain unreliable on long‑horizon tasks that require retaining intermediate facts across many steps and app transitions. We attribute this limitation to ReAct‑style prompting, which passively accumulates per‑step records, leading to prompt explosion and dilution of critical cross‑app facts. To address this, we introduce MemGUI‑Agent, an end‑to‑end long‑horizon mobile GUI agent with proactive context management. MemGUI‑Agent is built on Context‑as‑Action (ConAct), which casts context management as first‑class actions emitted by the same policy that selects UI actions. Instead of passively appending history, ConAct maintains three structured context fields: folded action history, folded UI state, and recent step record, preserving critical UI facts while keeping context compact. To make proactive context management learnable across model scales, we construct MemGUI‑3K, a 2,956‑trajectory dataset with full ConAct annotations for supervised training and offline analysis. Training an 8B model on MemGUI‑3K produces MemGUI‑8B‑SFT, an 8B MemGUI‑Agent that achieves the best open‑data 8B performance on MemGUI‑Bench and generalizes to the out‑of‑distribution MobileWorld benchmark. Code, data, and trained models will be released at https://memgui‑agent.github.io/.
Authors:Mingyu Choi, Woo Kyoung Han, Sunghoon Im, Kyong Hwan Jin
Abstract:
Linear recurrent unit (LRU), designed with a principled formulation for stable linear recurrence, has demonstrated promising accuracy and robustness on long‑range dependency tasks. However, its static parameterization and single‑scan method limits its applicability to 2D vision tasks. In this study, we propose a LRU‑based restoration network with a semantic modulating unit (SMU) to achieve a harmonious balance between performance and efficiency in single‑image super‑resolution. The SMU plays three key roles: LRU modulation, spatial categorization, and feature enhancement through learned prototype. Extensive experiments demonstrate that our method quantitatively and qualitatively surpasses recent state‑of‑the‑art methods. Notably, our approach achieves superior performance with computational complexity on par with existing methods. The source code and models are available at https://github.com/MingyuChoi‑run/LSM
Authors:Chaeyun Kim, Daeyoung Park, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim
Abstract:
Existing safety benchmarks target general adversarial scenarios but miss finance‑specific risks. Financial LLMs face regulatory compliance violations, fraud facilitation, and systemic trust erosion that require targeted evaluation. We introduce FinRED, an expert‑guided red‑teaming framework for financial LLM safety evaluation developed with financial experts. FinRED uses a novel two‑level taxonomy mapping global standards (e.g., FATF and EU DORA) to threats ranging from regulatory evasion to complex fraud, integrated with a scalable pipeline that converts real financial documents into context‑rich red‑teaming Behavioral Prompts (seeds) through an expert‑defined schema. Rigorous expert validation confirms seed plausibility and realism for meaningful LLM safety evaluation. We also provide an expert‑validated, finance‑specific rubric that goes beyond disclaimer checks, aligns more closely with human experts than static one‑size‑fits‑all rubrics, and reduces critical false negatives from 28 to 12. Aligned with internationally adopted risk‑management and information‑security standards (e.g., ISO/IEC 27001), FinRED is deployed in South Korea's Financial Security Institute (FSI) regulatory sandbox for generative AI security evaluation in real financial services. To mitigate dual‑use risks, the dataset, generation pipeline, prompt template, and evaluation framework are gated for qualified researchers at https://github.com/selectstar‑ai/FinRED‑paper and https://huggingface.co/datasets/datumo/FinRED.
Authors:Yutaro Yamada, Kento Uchida, Shinichi Shirakawa
Abstract:
Probabilistic model‑based evolutionary algorithms are promising for black‑box optimization. Specifically, the adaptive stochastic natural gradient (ASNG) adaptively updates its learning rate, a typical hyperparameter in probabilistic model‑based evolutionary algorithms, thereby realizing efficient and robust optimization. Although weight parameters are common hyperparameters, with the increasing demand for parallel evaluation of time‑consuming tasks, it remains unclear how to set suitable weights for larger population sizes. In this paper, we propose Weight Adaptation ASNG (WA‑ASNG), which incorporates a weight adaptation mechanism into ASNG. We calculated the estimated signal of the update direction from the accumulations of the natural gradient. Then, to maximize the signal, WA‑ASNG adaptively updates its weight parameters by a gradient ascent over the optimization. While the learning rate adaptation plays a role in satisfying a sufficient condition for monotonic improvement of the expected objective value, the mechanism of weight adaptation is intended to maximize this improvement. The experimental results demonstrate that WA‑ASNG outperforms PBIL and ASNG across various settings with population sizes ranging from 25 to 100 for binary optimization problems. Furthermore, WA‑ASNG can perform efficiently in the presence of strong noise. Our code is available at https://github.com/shiralab/WA‑ASNG .
Authors:Yasutoshi Kishimoto, Kota Yamanishi, Takuya Matsuda, Shinichi Shirakawa
Abstract:
Deep neural networks (DNNs) exhibit attractive performance in various fields but often suffer from low interpretability. The neural additive model (NAM) and its variant called the neural basis model (NBM) use neural networks (NNs) as nonlinear shape functions in generalized additive models (GAMs). Both models are highly interpretable and exhibit good performance and flexibility for NN training. NAM and NBM can provide and visualize the contribution of each feature to the prediction owing to GAM‑based architectures. However, when using two‑input NNs to consider feature interactions or when applying them to high‑dimensional datasets, training NAM and NBM becomes intractable due to the increase in the computational resources required. This paper proposes incorporating the feature selection mechanism into NAM and NBM to resolve computational bottlenecks. We introduce the feature selection layer in both models and update the selection weights during training. Our method is simple and can reduce computational costs and model sizes compared to vanilla NAM and NBM. In addition, it enables us to use two‑input NNs even in high‑dimensional datasets and capture feature interactions. We demonstrate that the proposed models are computationally efficient compared to vanilla NAM and NBM, and they exhibit better or comparable performance with state‑of‑the‑art GAMs.
Authors:Jiwoong Yang, Haejun Chung, Ikbeom Jang
Abstract:
Multi‑view imaging, such as mammography and chest radiography, is a standard component of clinical practice. However, medical images are often unregistered and contain view‑specific artifacts or irrelevant background cues that can obscure diagnostically relevant findings. Many existing methods directly fuse per‑view representations, allowing such irrelevant content to contaminate the fused embedding and reducing robustness under varying view configurations. We propose OTCHA, a confidence‑aware latent hub token alignment module based on optimal transport (OT) that refines patch tokens before fusion for multi‑view classification. OTCHA introduces a set of learnable latent hub tokens shared across views. For each view, we compute an OT plan between patch tokens and hub tokens that jointly considers feature similarity and geometry, and augment the OT formulation with token‑conditional dustbins to enable partial matching and discard irrelevant tokens. The resulting transport plan provides token‑wise matching confidence, which gates hub‑mediated message passing and weights a novel optimal‑transport‑based representation alignment loss to stabilize refinement. Experiments on three multi‑view medical image datasets demonstrate consistent improvements over competing baselines across diverse anatomies and view configurations. Our code is available at https://github.com/labhai/OTCHA.
Authors:Daehwan Kim, Haejun Chung, Ikbeom Jang
Abstract:
Medical tabular data are ubiquitous in clinical research, but deep learning for tables remains underexplored because reliable labels often require costly expert adjudication, even though structured clinical variables are routinely available in tabular form. Self‑supervised learning can leverage these unlabeled tables, and recent binning‑based pretexts offer a promising inductive bias, but existing objectives fix a single global quantile discretization and apply feature‑agnostic supervision. We propose Adaptive Binning, a training‑adaptive discretization pretext for tabular SSL that couples discretization to learning through a feature‑wise coarse‑to‑fine curriculum. Motivated by the spectral bias of neural networks and the principles of curriculum learning, our method progressively refines discretization per feature upon plateau detection and selects representation‑aware splits to jointly improve value‑space concentration and representation‑space coherence. A heterogeneity‑aware objective unifies categorical reconstruction with ordinal supervision for numerical features, and experiments on public medical tabular datasets under unified evaluation protocols show consistent gains for linear probing and fine‑tuning without dataset‑specific discretization tuning. We further introduce a medical tabular SSL benchmark with standardized protocols to support reproducible progress in this underexplored domain. Our code is available at https://github.com/labhai/Adaptive‑Binning.
Authors:Junho Moon, Haejun Chung, Ikbeom Jang
Abstract:
Accurate segmentation of thin, tortuous anatomical structures, such as retinal vessels, cerebral vasculature, and facial wrinkles, remains challenging due to low contrast, frequent discontinuities, and severe class imbalance. Although recent convolutional and Transformer‑based models have improved performance, they often yield fragmented predictions and fail to recover fine branches. We propose CSWinUNETR, a general‑purpose backbone for 2D and 3D thin‑structure segmentation. It employs cross‑shaped stripe self‑attention to model long‑range principal‑axis context and incorporates cyclic shifts to enhance information exchange across stripes. To better preserve fine‑grained details, we further introduce a detail‑enhanced multi‑scale self‑attention module that aggregates contextual features from multi‑resolution representations. In addition, we propose sparse‑control dynamic snake convolution, which reconstructs reliable dense curvilinear kernels from sparsely predicted control points to better follow tortuous geometry. Extensive experiments on four benchmarks across ophthalmology, neurovascular imaging, and dermatology demonstrate that CSWinUNETR consistently outperforms state‑of‑the‑art methods without task‑specific post‑processing or topology‑aware losses. The code is available at https://github.com/labhai/CSWinUNETR.
Authors:Tai Hyoung Rhee, Dong-Guw Lee, Ayoung Kim
Abstract:
Thermal infrared (TIR) imaging has been a popular choice for field robotics due to its robust perception capability under low light visual degradation, but it suffers from severe stochastic and fixed‑pattern noise that breaks downstream estimation. This noise is intensified indoors due to low thermal contrast and uniform temperature distributions, contributing to the relative lack of indoor TIR deployments. Existing TIR denoising methods exhibit a poor accuracy‑efficiency tradeoff, either too slow for online deployment required in robotics or insufficiently robust to severe degradation, while typically being trained on synthetic noise. Addressing these problems, we propose TIDY, a lightweight wavelet‑domain denoiser trained on real clean‑noisy TIR data. By reformulating TIR denoising in the wavelet domain, TIDY explicitly disentangles noise from structural content, enabling targeted suppression with reduced spatial complexity, significantly improving inference speed over prior methods (~34Hz). TIDY introduces two new metrics, Wavelet Entropy and Wavelet Directional Stripe Index, as complementary loss terms to explicitly suppress stochastic noise and stripe artifacts. Across severe indoor corruption and zero‑shot settings, TIDY improves robustness and yields consistent gains in downstream robotics tasks including thermal inertial odometry and monocular depth estimation. Code and dataset is available at: https://github.com/williamrheeth/TIDY
Authors:Victoria Wu, Nima Hashemi, Hooman Vaseli, Christina Luong, Purang Abolmaesumi, Teresa S. M. Tsang
Abstract:
Echocardiography (echo) is a widely used imaging modality for assessing cardiac function, with Left Ventricular Filling Pressure (LVFP) serving as a critical physiological marker for conditions such as heart failure. Standard LVFP classification into normal \emphvs elevated categories relies on the Doppler‑derived E/e' ratio, which is operator‑dependent and often unavailable in resource‑limited settings, motivating methods that infer LVFP directly from B‑mode echo. Existing deep learning approaches achieve high performance but remain largely black‑box, limiting clinical interpretability. We propose HypOProto, a hyperbolic, ordinal prototype‑based framework for interpretable LVFP classification using a frozen, explainable foundation model backbone. HypOProto arranges prototypes along the physiological E/e' scale, placing borderline cases near the hyperboloid root where small angular differences separate similar cases, while normal and elevated cases occupy outward positions reflecting increasing diagnostic certainty. This hyperbolic geometry encodes clinically meaningful ordinal relationships and improves interpretability. We also introduce a novel Hyperbolic Prototype Angular Separation (HyperPAS) loss, enforcing inter‑class prototype separation in hyperbolic space. HypOProto achieves SOTA performance while maintaining transparency, and highlights clinically relevant regions in visualizations. This work represents the first prototype‑based framework for LVFP classification in echo. Our code can be found at https://github.com/DeepRCL/HypOProto.
Authors:Yuxu Zhou, Ondřej Kuželka, Yuyi Wang, Yuanhong Wang, Yi Chang
Abstract:
We present CombEval, a dynamic benchmark for evaluating combinatorial counting in large language models. CombEval represents each problem as a typed Cofola specification over entities, combinatorial objects, object dependencies, and constraints, enabling controlled generation of natural‑language counting problems with exact solver‑verified answers. Unlike static collections, CombEval supports systematic variation of object type, entity scale, constraint count, and reasoning depth. We evaluate 11 LLMs under direct and code‑augmented settings and find that models remain brittle on ordered objects, indistinguishable elements, relatively positional constraints, and nested object dependencies. Error analysis further identifies failures in constraint interpretation and counting principles. CombEval provides a diagnostic testbed for studying when and why LLMs fail at combinatorial reasoning. The code and generated benchmark suites are publicly available at \urlhttps://github.com/YuxuZhou‑CN/combination‑problem‑generation.
Authors:Trong-Bao Ho, Quang-Tan Nguyen, Thien-Loc Ha, Gia-Binh Nguyen, Viet-Thanh Nguyen, Long Dinh, Minh N. Vu, Duy M. H. Nguyen, An Thai Le, Ngo Anh Vien
Abstract:
Action chunking enables robot policies to produce temporally coherent behavior, but generating multi‑step action sequences with flow‑based policies incurs latency that is incompatible with real‑time control. Under asynchronous execution, the robot continues executing the current chunk while the next one is generated, causing even minor delays to create inconsistencies at chunk boundaries. Existing methods address this problem by steering generation toward the already executed action prefix. We instead show that prefix consistency can be achieved by selecting an appropriate initial noise before generation begins, allowing the unmodified flow ODE to produce a coherent next chunk. This reframes asynchronous inference as a noise selection problem rather than a trajectory steering problem. We introduce PAINT, a training‑free method that finds this noise via backward Euler inversion and constructs the final chunk through a repainting rule. In summary, \textttPAINT requires no gradients, retraining, or policy modification; yet it improves execution consistency and task performance across 12 simulated benchmarks and 6 real‑world manipulation tasks spanning single‑arm, bimanual, and humanoid embodiments. Website: ~\hrefhttps://paint‑action‑chunking.github.io\texttthttps://paint‑action‑chunking.github.io.
Authors:Punit Kumar Singh, Niladri Ghosh, Advait Joshiınst, Shailee Choudhary, Michael Färber, Haiqin Yang
Abstract:
Language models have become essential tools in shaping modern workflows. However, their global effectiveness hinges on a nuanced understanding of local socio‑cultural contexts. To address this gap, we present NRITYAM, a comprehensive benchmark for evaluating the cultural comprehension capabilities of language models in the context of global dance traditions. NRITYAM comprises 9,260 carefully curated question‑answer pairs spanning 12 languages, making it the largest dataset dedicated to evaluating cultural knowledge in dance. The dataset has been developed from the ground up through close collaboration with native dance artists and native speakers of the languages, who authored and validated culturally relevant questions specific to their regions. We evaluate a broad set of models, including large language models, small language models, multimodal large language models, and small multimodal language models. As a multilingual and multicultural benchmark, NRITYAM sets a new standard for evaluating the ability of AI systems to understand and reason about traditional performing arts. Detailed dataset samples are available at~\urlhttps://github.com/niladrighosh03/NRITYAM.
Authors:Aditeya Baral, Radoslav Ralev, Iliya Sotirov Zhechev, Srijith Rajamohan, Jen Agarwal
Abstract:
Semantic caching cuts LLM inference costs by serving a cached response to semantically similar queries. Standard practice evaluates these systems using PR‑AUC, a metric that only measures how well scores rank and ignores whether they are usable at a fixed threshold. We show this mismatch leads to systematically poor deployment choices, as models with the highest PR‑AUC are often the worst in operation. We introduce Precision‑Cache Hit Ratio (P‑CHR) AUC, a cache‑aware metric that measures precision across cache utilization levels, and Calibration Retention Rate (CRR), which captures how much offline ranking quality survives at deployment. We decompose the operational gap between offline and deployed quality into a recoverable calibration component and an irreducible structural component fixed by the dataset's positive rate. Our experiments show that the calibration gap is governed by the training objective rather than data scale, and post‑hoc calibration only partially closes it. Ultimately, model selection for semantic caching is a calibration problem, not a ranking one, and measuring it is the first step to closing the gap.
Authors:Shenjian Gong, Kangkan Wang, Shanshan Zhang, Jian Yang
Abstract:
This paper addresses the challenge of one‑shot novel view and pose human image synthesis. The existing methods transfer the reference human image to a target pose using a set of 2D pose keypoints or synthesize human images based on generalizable human NeRF which uses human model priors to extract point‑wise features. However, pose transfer based methods can not handle complex human pose using ambiguous 2D pose as the condition, while generalizable human NeRFs may be inaccurate to recover occluded/invisiable human parts without extracted reliable features. To solve these problems, we propose a novel approach for novel view and pose synthesis from a singe human image via conditional denoising diffusion model. Our diffusion model divides the novel view and pose synthesis problem into a sequence of conditional denoising steps. Specifically, to generate humans with complex and arbitrary poses, we introduce 3D human priors, i.e., 3D normal map and color prompt, as geometry and color conditions into the generation process. By transferring the reference human into the target human with a series of diffusion steps, our diffusion model enables high‑quality synthesis including the occluded/invisible parts. Further, we propose a self‑reconstruction based customized refinement to enhance fine details when tested on novel persons.Experimental results on different public datasets demonstrate that our approach significantly outperforms previous methods and also shows better generalization ability across datasets. The code will be made publicly available at https://github.com/Yankeegsj/3DPGDM.
Authors:Elijah Feldman, Dipak Meher, Carlotta Domeniconi
Abstract:
Court proceedings contain valuable evidence about human smuggling networks, but this information is often buried within unstructured, jargon‑heavy legal documents. While large language models (LLMs) can support knowledge graph construction through automated information extraction, existing approaches rely on general‑purpose models that are not tailored to the entity and relationship definitions required in this domain. We introduce FineREX, a streamlined knowledge graph construction pipeline built around a fine‑tuned LLM for named entity recognition and relationship extraction (NER‑RE). Using a manually annotated dataset of 512 text chunks, FineREX achieves absolute improvements of 15.50% and 31.46% in entity and relationship F1‑score, respectively, compared to a larger general‑purpose baseline. These gains translate into higher‑quality knowledge graphs, reducing legal noise by nearly half and lowering node duplication on long documents from 17.78% to 11.17%. By eliminating document rewriting and redundant extraction stages, FineREX also reduces end‑to‑end processing time by 50.0%. Our results demonstrate that domain‑specific fine‑tuning can substantially outperform larger general‑purpose models while improving both the quality and efficiency of knowledge graph construction for illicit network analysis.
Authors:Bingshuo Qian, Xiang Cheng
Abstract:
Multi‑representation diffusion models can improve visual synthesis by denoising complementary views of an image, but their performance depends critically on the asynchronous schedule that determines when each representation is denoised. We propose to learn this schedule. Our method formulates asynchronous flow matching over multiple representation spaces and uses a schedule‑corrected objective that keeps each representation's local noising‑time weights fixed as the schedule changes. We instantiate the schedule with a flexible parametric class that is convex and monotone by construction, and learn it using a fast joint probe with less than 1% additional training compute. On ImageNet 256x256, the learned schedule substantially improves both convergence speed and final quality under a matched 675M‑parameter XL backbone. With AutoGuidance, our 200‑epoch model reaches FID 1.05, matching the 800‑epoch SFD‑XL baseline with 4x less training. Training to 600 epochs further improves to FID 1.02, outperforming the 1B‑parameter SFD‑XXL result of FID 1.04 while using a smaller model. In the unguided setting, our 200‑epoch model reaches FID 2.37, already below the best 800‑epoch SFD‑XL result (2.54) at 4x less training, and improves to FID 2.14 at 600 epochs. Code is available at https://github.com/bsq532087/LWD
Authors:Hongxuan Liu, Roman Bushuiev, Ivy Lightheart, Mrunali Manjrekar, Anton Bushuiev, Magdalena Lederbauer, Filip Jozefov, Yinkai Wang, Soha Hassoun, Josef Sivic, James Taylor, Runzhong Wang, David Healey, Tomáš Pluskal, Connor W. Coley
Abstract:
Reliable benchmarking is critical for developing machine learning models for tandem mass spectrometry (MS/MS) based molecule discovery. Subtle issues in experimental design and model evaluation procedures can degrade the trustworthiness of such benchmarks and lead to erroneous conclusions. We conduct a thorough review of model evaluation issues in the recent MS/MS machine learning literature, using the standard MassSpecGym benchmark suite as a case study to illustrate the impact of these issues. We find evaluation issues in at least 17 of 26 papers reporting MassSpecGym benchmark results in the first year of its adoption. We isolate three classes of failures: (i) data leakage, (ii) shortcut learning, and (iii) implementation bugs and metric divergence. Through extensive experimentation and code replication, we quantify the impact of these issues and show how they corrupt the evaluation standards MassSpecGym was designed to enforce. We distill our findings into recommendations generalizable to MS/MS challenges, benchmarks, and custom evaluation setups. We also release MassSpecGym v1.5, an implementation of our recommendations in the MassSpecGym benchmarking suite which addresses the failure modes identified in this audit. MassSpecGym v1.5 is publicly available at https://github.com/pluskal‑lab/MassSpecGym.
Authors:Dipankar Sarkar
Abstract:
Autonomous coding agents now open millions of pull requests, yet large‑scale studies find their PRs are produced faster but accepted less often ‑ a coordination and trust gap that pull‑request‑level telemetry cannot explain. We argue the missing signal lives before the PR, in how concurrent agents claim, divide, and collide over shared work. We study this process through grite, our open‑source coordination substrate that needs no central server and stores its records inside git itself, so its append‑only, signed event log captures the coordination process directly. We show that (i) this shared substrate reduces duplicate and conflicting work at bounded overhead ‑ the share of work that merely re‑does a teammate's task falls from 78% to 0% while useful throughput more than triples; (ii) every agent's copy of the log converges to the same state with no write silently dropped, where a file‑based tracker loses concurrent writes; and (iii) the log is a mineable artefact from which concrete failure modes ‑ conflicting edits, lock starvation, redundant rediscovery, race‑to‑close ‑ are automatically recoverable with provenance, several invisible in pull‑request history. We release the dataset, harness, and mining toolkit.
Authors:Shivaay Dhondiyal, Divyansh Sharma, Dinesh Kumar Vishwakarma
Abstract:
Audio deepfakes generated by neural text‑to‑speech and voice‑cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross‑dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primarily because of structural synthetic speech artifacts which are multi‑timescale trajectory anomalies. Though every existing detector aggregates a fixed‑window frame statistics, this misaligns the architecture with the signal. We propose FlowFake, a Liquid Time‑Constant (LTC) architecture whose hidden state evolves via a learned ODE, with per‑neuron adaptive time constants simultaneously resolving spectral (10ms) and prosodic (2s) cues. At only 34K parameters FlowFake achieves formal BIBO stability and O(dt^4) integration error. On a four‑dataset cross domain benchmark (ASVspoof2019‑LA, FakeOrReal, InTheWild, MLAAD), FlowFake reaches 75.29% on ASVspoof2019 trained only on FakeOrReal and 79.97% trained only on MLAAD. It outperforms RawGAT‑ST and Whisper‑DF on every evaluated pair and matching SSL Wav2vec2 (300x larger) at 0.01% of its parameter count. The source code is available on : https://github.com/GhostRider2023/FlowFake
Authors:Gregory Matsnev
Abstract:
Recent position papers argue that the classical aleatoric/epistemic uncertainty framework is insufficient for interactive large language model (LLM) agents and call for underspecification‑aware, decomposed, and communicable uncertainty representations that can unlock new agent capabilities such as proactive clarification seeking and shared mental‑model building. Practical deployment constraints ‑‑ black‑box APIs, interactive latency budgets, and the absence of labeled trajectories ‑‑ rule out logprob‑based, multi‑sampling, and training‑based methods, leaving prompt‑based estimation as the most viable family for surfacing such signals at deployment time. We answer this call with a simple prompt‑based decomposition that separates action confidence from request uncertainty (u), enabling the agent to ask for clarification when the task specification is ambiguous. To evaluate it, we introduce two clarification‑augmented benchmarks (WebShop‑Clarification and ALFWorld‑Clarification) in which 50% of tasks are deliberately underspecified, and systematically compare the proposed decomposition against ReAct+UE and Uncertainty‑Aware Memory (UAM) across five LLM backbones (GPT‑5.1, DeepSeek‑v3.2‑exp, GLM‑4.7, Qwen3.5‑35B, GPT‑OSS‑120B) on these variants together with the standard WebShop, ALFWorld, and REAL benchmarks for fault detection. Averaged across the five backbones, the proposed decomposition improves clarification F1 on ALFWorld‑Clarification by 73% over ReAct+UE and by 36% over UAM, and leads clarification F1 on every backbone on WebShop‑Clarification and on four of five backbones on ALFWorld‑Clarification, indicating that the gains generalize beyond a single LLM.
Authors:Yueyi Sun, Yuhao Wang, Jason Li, Ye Tian, Tao Zhang, Jacky Mai, Yihan Wang, Haochen Wang, Jinbin Bai, Ling Yang, Yunhai Tong
Abstract:
Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM‑Base, a strong foundational baseline that achieves state‑of‑the‑art performance among open‑source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC‑Bench) by scaling the DLC‑Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi‑region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.
Authors:Yuyang Zhang, Wenyao Zhang, Zekun Qi, He Zhang, Haitao Lin, Jingbo Zhang, Yao Mu, Xiaokang Yang, Wenjun Zeng, Xin Jin
Abstract:
World Action Models (WAMs) commonly rely on video generation to bridge visual world modeling and robot control. However, video‑based WAMs face three coupled limitations: dense multi‑frame future tokens make inference costly, full video prediction spends capacity on action‑irrelevant temporal and appearance details, and long‑horizon future imagination may introduce errors that mislead action prediction. These issues raise a simple question: Does world action model really need video generation? We propose ImageWAM, a simple WAM framework that repurposes pretrained image editing models for robot action prediction. In contrast to video generation, image editing provides a better‑matched prior: it only needs to model a target‑frame transformation, focuses on action‑relevant current‑to‑target visual differences, and grounds task instructions to localized visual changes through edit pretraining. In practice, ImageWAM does not decode the target frame at inference time; instead, it conditions a flow‑matching action expert on the KV caches produced by image‑editing denoising, using them as a compact world‑action context. ImageWAM outperforms standard VLA baselines and matching competitive WAMs without additional policy pretraining across different simulator and real‑world experiments. It also reduces FLOPs to 1/6 and latency to 1/4 of video‑based WAMs. Attention analysis further shows that editing caches focus on task‑relevant change regions, supporting image editing as an effective alternative to video‑based world‑action modeling.
Authors:Aijie Shu, Bowei Chen, Wenbin Wu, Cathy Yi-Hsuan Chen, Fengxiang He
Abstract:
Decentralized finance exposes supervisors to fast‑moving, networked credit risks. General‑purpose LLM agents fit this setting poorly: they over‑read weak evidence and recommend high‑stakes interventions, while existing evaluations offer no regulator‑aligned way to measure the resulting false alarms. We introduce DeXposure‑Claw, a forecast‑grounded agentic supervision system that routes LLM decisions through structured evidence: (1) DeXposure‑FM, a graph time‑series foundation model, forecasts future exposure networks; (2) deterministic monitors and stress scenarios then turn those forecasts into typed alerts, attribution signals, and scenario evidence; and (3) data‑health and confidence gates constrain escalation before DeXposure‑Claw emits auditable supervisory tickets with rationales. We further develop DeXposure‑Bench, a six‑axis evaluation harness, whose decision axis scores tickets against a regulator‑aligned absolute‑loss ground truth and an explicit false‑intervention rate. Experiments on five years of weekly real data fully support our system. Code is at https://github.com/EVIEHub/DeXposure‑Claw.
Authors:Nathaniel L. Diamant, Brian L. Trippe
Abstract:
Generative models can produce individually plausible samples while deviating substantially from a target set in the distribution of key features. For example, a model pretrained on broad drug‑like chemical space may generate molecules whose molecular features differ from those of a therapeutic class of interest, such as known antibiotics. Correcting such distributional miscalibration is challenging: direct finetuning on the target set can overfit and does not control which features are matched. To fill this gap, we introduce kernel Calibrating Generative Models (kCGM). kCGM minimizes a maximum mean discrepancy (MMD) between generated and target feature distributions using an unbiased score‑function estimator, with KL regularization to remain close to the pretrained model. On a target set of 174 antibiotics, direct finetuning sacrifices chemical validity for feature‑distribution matching, whereas kCGM improves target feature matching while increasing validity. We further demonstrate kCGM in protein and DNA generation tasks, showing it can adapt autoregressive, continuous‑space diffusion, and discrete diffusion models using only feature‑level supervision. Code is available at https://github.com/smithhenryd/cgm.
Authors:Shariq Farooq Bhat, Niloy J. Mitra, Kalyan Sunkavalli
Abstract:
Precise 3D spatial orchestration in text‑to‑video generation remains a significant challenge, particularly for multi‑object scenes where semantic layout and temporal dynamics are often entangled. While existing depth‑conditioned models achieve good structural fidelity, they necessitate dense, frame‑accurate guidance that is labor‑intensive to author for dynamic events involving deformable objects. We present LooseControlVideo, a framework that enables intuitive and expressive control by using sparse, oriented 3D boxes as a "blocking" proxy. This allows users to author high‑level layout and trajectory while leveraging a video generative model to generate realistic occlusions, dynamics and interactions. We achieve this by fine‑tuning a Wan 2.2 backbone on a video dataset annotated with DNOCS, a novel encoding for 3D size, orientation and depth‑ordered occlusions. Furthermore, our method allows for localized refinement, such as adjusting a jump trajectory or adding an interaction, with minimal disruption to the global scene context. Extensive evaluations on the nuScenes, HO‑3D, and BEHAVE benchmarks demonstrate that LooseControlVideo significantly outperforms existing 2D‑box and flow‑based baselines. Our findings indicate a 1.2x to 3x improvement in Trajectory Error; 2x improvement in Rigid Motion Consistency; and a 1.5x to 2x increase in Occlusion Accuracy over current state‑of‑the‑art layout‑conditioned models, demonstrating that oriented 3D primitives provide good geometric prior for complex, multi‑agent video authoring.
Authors:Jiaqi Zhang, Ashton Lee, Anthony Wong, John Zou, Sami BuGhanem, Randall Balestriero
Abstract:
Vision Foundation Models (VFMs) with Vision Transformer (ViT) backbones, such as DINOv2, have become essential for downstream tasks like object recognition and semantic segmentation. The immense computational requirements of backbones often necessitate distillation into smaller architectures for edge deployment. Feature‑based knowledge distillation (KD) often suffers from the teacher‑student gap; the student struggles to imitate teacher's complex feature map due to its limited capacity. To mitigate this bottleneck, we propose LEAP: Layer‑skipping Efficiency via Adaptive Progression, a training curriculum for ViT feature‑based knowledge distillation. By utilizing the teacher's intermediate feature maps as a sequence of progressively more difficult targets, our curriculum allows the student to build a foundational representation before tackling higher‑level abstractions. Our results demonstrate that this paradigm significantly accelerates convergence through adaptive difficulty selection across various student model sizes and dataset scales. With our curriculum, the LEAP‑distilled ViT‑S achieves 90.1% accuracy on ImageNet‑100, a +12.24% improvement compared with baseline. On ImageNet‑1K, LEAP achieves +3.84% and +7.75% improvement for the instance retrieval task on the Oxford and Paris datasets, respectively. Furthermore, the curriculum enables 25.1% savings in training FLOPs and 21% savings in training time on ImageNet‑100 by implementing early‑stopping for teacher inference during the initial stages of training. Code is available at https://github.com/KevinZ0217/LEAP
Authors:Oğuzhan Yenen
Abstract:
We present MonaVec, a deterministic, embedded vector‑search kernel for edge and offline AI ‑‑ settings where server infrastructure, network connectivity, and training data are all unavailable. Existing vector‑search systems assume a persistent server, gigabytes of RAM, or a training pass over the corpus; MonaVec instead targets the deployment profile of SQLite: one file, one function call, runs anywhere. Its quantization core is training‑free by default and data‑oblivious: a Randomized Hadamard Transform (RHDH) conditions any input distribution toward N(0,1), so precomputed Lloyd‑Max tables quantize to 4 bits (8x smaller) with no learned codebook and no data pass. The index persists as a single .mvec file whose embedded ChaCha20 rotation seed makes results reproducible across architectures and byte‑identical within a build ‑‑ a determinism guarantee that parallel‑build graph libraries cannot offer. On semantic embeddings (AG News, 45K x 1024‑dim BGE‑M3, cosine), MonaVec 4‑bit BruteForce reaches 0.960 Recall@10 in 27 MB ‑‑ leading float32 FAISS‑IVF and 8‑bit usearch on recall ‑‑ while trading peak throughput for byte‑identical determinism. A single‑pass global standardization (fit()) extends the same data‑oblivious pipeline to magnitude‑sensitive L2 data, and optional IvfFlat and HNSW backends carry it to million‑vector corpora. MonaVec is implemented in pure Rust with Python bindings and runtime SIMD dispatch (AVX‑512/AVX2/NEON/scalar). It targets on‑device RAG, offline agents, and embedded retrieval ‑‑ the niche SQLite occupies for relational data: one file, one call, runs anywhere.
Authors:Ellina Zhang, Madhaven Iyengar, Amir Zadeh, Chuan Li, Deepak Pathak, David Held, Tal Daniel
Abstract:
We introduce 3D‑DLP, a self‑supervised object‑centric representation learning model that decomposes scene‑level RGB‑D or voxel observations into a set of 3D latent particles. Building on the Deep Latent Particles (DLP) framework, each particle encodes disentangled attributes, including 3D keypoint position, bounding box dimensions, and appearance features, and represents a distinct entity in the scene. The model learns interpretable per‑particle segmentation maps through an end‑to‑end self‑supervised reconstruction objective. We demonstrate on both simulated and real‑world datasets that the learned latent space is interpretable and controllable: by manipulating particle positions and decoding, we can generate novel scene configurations. Furthermore, we show that leveraging these compact 3D latent particles for downstream robotic manipulation improves performance over baselines that either lack explicit 3D information or rely on memory‑intensive dense 3D inputs without object‑centric structure. Code and videos are available at https://eubooks3003.github.io/3d‑dlp.
Authors:Georgios Ioannides, Adrian Kieback, Judah Goldfeder, Linsey Pang, Aman Chadha, Aaron Elkins, Yann LeCun, Ravid Shwartz-Ziv
Abstract:
Self‑supervised speech encoders are predominantly trained by predicting discrete hard cluster IDs at masked positions, a recipe that collapses acoustic ambiguity at category boundaries and requires interrupting training to re‑cluster the entire corpus between iterations. We introduce S‑JEPA, a JEPA‑style encoder‑predictor pair trained to match the soft posteriors of a Gaussian Mixture Model at masked positions via KL divergence. Training runs as one continuous optimization trajectory in two phases: a fixed GMM over MFCC features, then an online GMM over encoder features, with the input layer selected adaptively from a label‑free signal, removing both the offline re‑cluster step and the hand‑tuned choice of which transformer layer to cluster on. Under the SUPERB protocol, S‑JEPA achieves the lowest WER among evaluated SSL methods below 90M parameters and matches HuBERT‑Base on emotion recognition at roughly half its parameter count, establishing a new Pareto frontier without offline re‑clustering or teacher distillation. An analysis of the predictor's per‑frame entropy on held‑out speech reveals a bimodal distribution with a substantial minority of frames near the entropy of a perfect two‑cluster tie, providing direct empirical evidence that the soft‑target objective preserves the acoustic ambiguity that hard targets would collapse. Code is available at https://github.com/gioannides/s‑jepa.
Authors:Manvendra Modgil
Abstract:
Runtime monitors for autonomous agents commonly threshold an accumulated internal state ‑ a behavioural baseline, a drift statistic, or, in our prior work, a modelled affective state. We previously reported a State Saturation Trap: threshold‑on‑state triggers over a continuous affect engine become near‑constant alarms on SWE‑bench debugging agents (Modgil 2026). A post‑release audit found the engine received dt=0 between actions, so its exponential decay never operated: the published trap is a pure‑accumulator result. We correct the record (erratum, v2) and treat the flaw as an experiment. The key variable it exposes is whether a monitor's dynamics are calibrated in sample time (per observation, as in CUSUM) or wall‑clock time (half‑lives in seconds, as in affect models and EMA baselines). On fixed‑rate streams these coincide; on agent streams, where inter‑action time varies by orders of magnitude, they do not. A pre‑registered sweep over uniform intervals (dt in 0..600s) on 20 trajectories shows the wall‑clock level trigger has two regimes: at dt<=1s a constant alarm (20/20; median 18 firings); at dt>=60s silent. Every critical dt lies in (1,30]s. Real agent runs measure latency at median 1.53s (p90 2.33s); real coding cadence sits inside the trap regime, vindicating the empirical finding under a corrected mechanism. The structure is a property of the calibration class, not the engine: a minimal wall‑clock accumulator over the raw error stream reproduces the same cliff, while a sample‑time CUSUM over the identical stream is exactly dt‑invariant (20/20). A rising‑edge trigger with hysteresis fires 0‑3 times per trajectory in every condition. We conclude that wall‑clock‑calibrated leaky‑integrator monitors admit no regime in which they act as moment detectors on agent streams; transition detection escapes the trap at every cadence, but does not recover human intervention timing.
Authors:Kanishk Kushwaha, Vikrant Vinod Bansode, Harsh Vardhan, Dhaval C. Patel
Abstract:
While LLM‑powered agents offer end‑to‑end automation for industrial asset lifecycles, real‑world Industry 4.0 deployment is hindered by latency, concurrency instability, and safety risks. We present DynAMO (Dynamic Asset Management Orchestration), a deployment‑ready engine using a Plan‑then‑Execute architecture to generate verifiable workflow graphs. DynAMO supports both SequentialWorkflow (topological execution) and ParallelWorkflow (dependency‑aware concurrency). By dynamically identifying independent tasks, DynAMO preserves structural correctness and safety while significantly improving efficiency through controlled reasoning overlap. Across six controlled experiments on the AssetOpsBench industrial benchmark, DynAMO demonstrates substantial performance and robustness gains. Parallel execution reduces end‑to‑end latency by a median of 1.6x over sequential orchestration, rising to 1.8x on highly parallelizable workflows. After instrumenting external tool calls with realistic latencies, a latency decomposition shows that LLM reasoning and orchestration still account for more than 90% of execution time, identifying model inference as the primary system bottleneck. Structured context pruning reduces inference latency by approximately 30%, and DynAMO maintains correct functional behaviour (task completion, agent sequencing, and output quality) while exhibiting graceful degradation under controlled fault injection. Reproducibility analysis further confirms stable execution under repeated runs, with parallel scheduling reducing latency variance. These findings establish DynAMO as a practical blueprint for scalable, safe, and latency‑aware agent deployment in Industry 4.0 automation pipelines. Code is available at: https://github.com/kushwaha001/DynAMO
Authors:Mohamed Mouhajir, Limei Wang, El Houcine Bergou, Hajar El Hammouti, Lamiae Azizi, Dongqi Fu
Abstract:
Graph‑based representations are widely used in protein modeling, yet many existing approaches rely primarily on sequence adjacency or geometric proximity, which only partially reflect the principles governing protein folding. Proteins instead adopt complex three‑dimensional conformations organized around secondary structure elements, such as α‑helices and β‑sheets, which encode recurring local motifs and stabilizing hydrogen‑bond interactions. In this work, we introduce a secondary‑structure‑aware graph neural network for protein representation learning. Residue‑level node representations are augmented with secondary structure assignments, and graph edges are constructed from hydrogen‑bond interactions filtered by their energetic strength. This design enables the model to capture both local structural context and long‑range couplings that are central to protein stability and function. We evaluate the proposed approach on commonly used protein benchmarks and observe consistent improvements over existing graph‑based methods. In addition, the resulting graph representations offer enhanced biological interpretability, as the learned connectivity aligns with established structural motifs. These findings suggest that incorporating secondary structure and energy‑filtered hydrogen‑bond topology provides an effective inductive bias for protein representation learning. The code is released at https://github.com/mohamedmohamed2021/SSProNet
Authors:Tiexin Ding
Abstract:
Building on a two‑parameter Weibull framework for diagnosing transformer weight distributions, we study why the Weibull weight‑scale parameter λ grows, overshoots, and then relaxes during AdamW training. We derive a leading‑order three‑force decomposition of the squared weight norm from the AdamW update: an alignment force measuring the correlation between weights and the adaptive update direction, an injection force from adaptive step magnitude, and a decay force from decoupled weight decay. On self‑trained Pythia‑70M models with ground‑truth optimizer moments, alignment dominates the rise phase, contributing 88‑94% of the absolute force budget across four random seeds and remaining robust to super‑weight removal. Near saturation, alignment and decay approach balance, explaining the transition from weight‑scale growth to relaxation. These force dynamics directly govern the squared‑norm component underlying λ(t); the remaining RMS‑to‑Weibull reconstruction offset is measurable and decomposes into bridge and integration components, totaling approximately 5‑6% in densely sampled regions. To extend the analysis to real models where optimizer moments are unavailable, we introduce a spline displacement method that recovers the alignment force from sparse checkpoints with approximately 92‑94% accuracy, about twice the naive two‑point baseline. We further observe that the peak value of λ(t) varies with training‑data coherence in our experiments, suggesting a data‑dependent component of weight‑scale growth that we leave to a controlled follow‑up study. Code and data are available at https://github.com/tiexinding/NPM‑Weibull‑public.
Authors:Rupasree Dey, Abdul Matin, Nathan Orwick, Yao Zhang, Shrideep Pallickara, Sangmi Lee Pallickara
Abstract:
The deployment of Time‑Series Foundation Models (TSFMs) in physical sciences is hindered by a critical trade‑off: while these models encode rich, universal temporal dynamics, they suffer from severe distributional misalignment when applied zero‑shot to specific scientific domains, and their computational cost prohibits deployment in edge‑computing sensor networks. We address a fundamental challenge: How can we extract latent structural knowledge from misaligned foundation models (FM) to train lightweight, specialized forecasters? We propose Gated Uncertainty‑Aware Routing for Distillation (Guard), a novel framework that reframes multiteacher distillation as an instance‑wise decision process with two adaptive mechanisms: (1) a Contextual Router that dynamically selects the most relevant teacher based on local input statistics, exploiting complementarity across diverse foundation models; and (2) an Uncertainty‑Gated Temperature mechanism that acts as a "circuit‑breaker," automatically attenuating distillation strength when teacher confidence diverges from domain reality. We evaluate our proposed lightweight framework on four climate‑critical domains: meteorology, ecosystem carbon flux, soil moisture, and energy grids. Our method significantly reduces RMSE relative to a fixed‑weight multi‑teacher distillation baseline, successfully distilling knowledge from pretrained FMs (teachers) even when they exhibit suboptimal zero‑shot accuracy due to distribution shift between the original and target data domains. We demonstrate that these domain‑misaligned teachers can still serve as critical correctives, outperforming the globally superior FMs on 28.5% of the hardest instances. Ultimately, this enables high‑precision scientific forecasting suitable for resource‑constrained edge deployment. Code is available at https://github.com/RupasreeDey/GUARD‑KDD2026.
Authors:Lucius E. J. Bynum, Rajesh Ranganath, Kyunghyun Cho
Abstract:
Identification conditions describe the computability of a target query or parameter of interest as a function of the type and amount of information available. In causal identification, this information is often expressed in the form of a causal graph, and data are observed or collected for some subset of variables in the graph. Target queries may be for a single effect alone or for a class of effects in a given model. The derivation of an identification algorithm then defines mathematically the process by which the desired causal effect(s) can be uniquely determined, theoretically, in expectation. Identifiability in expectation, or 'theoretical identifiability,' generally assumes asymptotic properties, infinite data, or other mathematically idealized conditions. In this paper, we explore a fundamental distinction between this theoretical, idealized notion of identifiability and a proposed alternative that is computation‑bound. The framework we propose ‑ 'computational identifiability' ‑ is to instead define a finite computational search procedure for an empirical estimator. If this process finds an estimator empirically, within a desired error tolerance, then identifiability is satisfied, conditional on the specified assumptions of the search (i.e., a prior distribution over the parameters) and conditional on the search procedure itself. Through several experiments, we demonstrate how this framework allows us to answer fine‑grained, practical identification questions, such as identification with small finite samples, with ambiguous graphical criteria, with mixed observational‑interventional data, and across counterfactual data and estimands. Code is available at https://github.com/lbynum/metadentify.
Authors:Yiming Ni, Zhi-Qi Cheng, Jiayu Li, Wei Cheng
Abstract:
Sign languages are expressive visual languages used by Deaf and Hard‑of‑Hearing (DHH) communities. Despite substantial progress in sign‑language recognition, translation, and production, advances remain constrained by fragmented datasets, inconsistent annotations, and limited linguistic coverage. Existing benchmarks often fail to reflect real‑world communication needs, and systematic analyses of these limitations remain limited. In this survey, we present a comprehensive index of sign‑language datasets, covering 120 resources across 35 sign languages. We analyze key challenges such as modality imbalance, annotation granularity, and signer bias, and outline considerations for future dataset design. We also introduce a 24‑field Sign‑Language Datasheet and release a public GitHub repository (https://github.com/Ginqwerty/Open‑Sign‑Language) to support standardized documentation and reproducible evaluation. Overall, our work provides a unified and practical foundation for developing inclusive, robust, and scalable sign‑language technologies in real‑world applications.
Authors:Zhenghao Xing, Ruiyang Xu, Yuxuan Wang, Jinzheng He, Ziyang Ma, Qize Yang, Yunfei Chu, Jin Xu, Junyang Lin, Chi-Wing Fu, Pheng-Ann Heng
Abstract:
Passive models for long video understanding typically rely on a "watch‑it‑all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre‑scanning, and their context cost still scales with video length. We propose OmniAgent, the first native omni‑modal agent that formulates video understanding as a POMDP‑based iterative Observation‑Thought‑Action cycle. OmniAgent executes on‑demand actions to selectively distill audio‑visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. To operationalize this, we introduce (1) Agentic Supervised Fine‑Tuning to bootstrap native active perception via best‑of‑N trajectory synthesis with dual‑stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn‑aware Adaptive Uncertainty Rescaled Advantage), which leverages turn‑level entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test‑time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state‑of‑the‑art performance among open‑source models. Notably, on LVBench, our 7B agent outperforms the 10× larger Qwen2.5‑VL‑72B (50.5% vs. 47.3%).
Authors:Michael Finkelson, Daniel Segal, Eitan Richardson, Shahar Armon, Nani Goldring, Poriya Panet, Nir Zabari, Benjamin Brazowski, Or Patashnik, Yoav HaCohen
Abstract:
Existing multi‑speaker dialogue systems bind speakers to utterances through structured supervision: per‑turn tags, multi‑stream transcriptions, or learnable speaker embeddings. These systems operate within speech‑only pipelines that produce clean vocal sequences without the ambient texture of real conversations. We take a different approach. Our method, ScenA, conditions a text‑to‑audio flow‑matching foundation model, pretrained on large‑scale in‑the‑wild data, directly on multiple reference voices and a free‑form natural language prompt that describes an entire multi‑speaker audio scene. Leveraging such a foundational model allows us to inherit its capacity for natural, non‑studio audio: background noise, room acoustics, overlapping dialogue, and spontaneous paralinguistic events, while adding multi‑speaker control without any per‑turn structure. Concretely, reference latents are concatenated into the model's token sequence and distinguished by lightweight identity‑aware positional encodings. However, we identify a critical obstacle to this approach: the Reference Shortcut. During training under standard noise schedules, the model can identify the matching reference by acoustic similarity to the noisy target, bypassing the text prompt entirely. We address this with a high‑noise‑biased timestep distribution that forces the model to rely on the text prompt for speaker assignment. We evaluate ScenA on the CoVoMix2‑Dialogue benchmark, showing that it outperforms existing multi‑speaker systems on speaker‑binding metrics while generating rich conversational audio with overlapping speech, emotional vocalizations, and ambient sound. Our results demonstrate the advantage of using a general‑purpose audio model conditioned on a free‑form scene description, rather than passing structured dialog scripts through a speech‑only pipeline.
Authors:Chong Bao, Yuan Li, Bangbang Yang, Yujun Shen, Hujun Bao, Zhaopeng Cui, Yinda Zhang, Guofeng Zhang
Abstract:
Recently neural implicit rendering techniques have evolved rapidly and demonstrated significant advantages in novel view synthesis and 3D scene reconstruction. However, existing neural rendering methods for editing purposes offer limited functionalities, e.g., rigid transformation and category‑specific editing. In this paper, we present a novel mesh‑based representation by encoding the neural radiance field with disentangled geometry, texture, and semantic codes on mesh vertices, which empowers a set of efficient and comprehensive editing functionalities, including mesh‑guided geometry editing, designated texture editing with texture swapping, filling and painting operations, and semantic‑guided editing. To this end, we develop several techniques including a novel local space parameterization to enhance rendering quality and training stability, a learnable modification color on vertex to improve the fidelity of texture editing, a spatial‑aware optimization strategy to realize precise texture editing, and a semantic‑aided region selection to ease the laborious annotation of implicit field editing. Extensive experiments and editing examples on both real and synthetic datasets demonstrate the superiority of our method on representation quality and editing ability. Project page: https://zju3dv.github.io/neumeshplusplus/
Authors:Nikita Kachaev, Andrey Moskalenko, Matvey Skripkin, Nikita Kurlaev, Daria Pugacheva, Albina Burlova, Mikhail Kolosov, Denis Shepelev, Andrey Kuznetsov, Elena Tutubalina, Aleksandr I. Panov, Alexey K. Kovalev, Vlad Shakhuro
Abstract:
Embodied Vision‑Language‑Action (VLA) models are typically obtained by fine‑tuning powerful pretrained VLMs on robotics data, yet it is unclear how much commonsense and factual knowledge they retain after adaptation. Failures on knowledge‑sensitive tasks are ambiguous, conflating missing knowledge with poor generalization of low‑level control. We introduce Act2Answer, a lightweight protocol that adapts VLM knowledge benchmarks to VLA evaluation by requiring agents to answer through action. Each question becomes a short tabletop episode where the agent performs a single object‑placement action to select among candidate answers, yielding an action‑grounded success rate with reduced control confounds. We curate a test suite of such environments across diverse commonsense and world‑knowledge categories and introduce layerwise intent probing to localize answer‑relevant information across the VLM backbone and action head. In a large‑scale study of 7 VLA models and 9 VLM baselines, we systematically rank models across categories, finding that VLAs show solid performance on simple concepts while exhibiting larger gaps on richer semantic categories relative to their source VLMs, that VQA co‑training is associated with better knowledge retention, and that answer‑relevant signals peak in middle VLA layers but attenuate in upper layers. Act2Answer is available at https://tttonyalpha.github.io/act2answer/.
Authors:Youhe Jiang, Haoxu Wang, Haotong Bao, Kai Jiang, Jianfei Chen, Jun Zhu, Fangcheng Fu, Jintao Zhang
Abstract:
Streaming video generation is emerging as a new serving workload in which users interact with long‑lived sessions that generate video progressively, chunk by chunk. Unlike offline video generation or typical LLM serving, streaming video generation must preserve session state across active and idle periods, repeatedly schedule ongoing sessions, and deliver each chunk under a tight latency target. This creates two key serving challenges in multi‑user, multi‑GPU environments: session duration heterogeneity, where long‑running sessions make placement decisions suboptimal over time, and temporal user‑demand heterogeneity, where the number of active sessions fluctuates sharply across bursts and idle periods. We present TurboServe, the first serving system designed specifically for streaming video generation workloads. TurboServe formulates serving as an online scheduling problem that jointly coordinates session placement and GPU provisioning. Its closed‑loop scheduling algorithm combines a migration‑aware placement controller, which rebalances sessions across GPUs to reduce the maximum per‑chunk latency, with a load‑driven autoscaling controller, which adapts the GPU budget to workload variation for improved cost efficiency. To support these decisions at runtime, TurboServe implements coalesced chunk processing for batching concurrent active sessions on the same GPU, GPU‑CPU offloading for session suspension and resumption, and NCCL‑based GPU‑GPU migration for online rebalancing. We evaluate TurboServe on real‑world production traces from Shengshu Technology across multiple model sizes and GPU clusters with up to 64 NVIDIA B300 GPUs. Compared with baseline serving configurations, TurboServe reduces worst‑case per‑chunk latency by 37.5% and total GPU operating cost by 37.2% on average. Our code is publicly available at https://github.com/shengshu‑ai/TurboServe.
Authors:Tom Hadfield
Abstract:
We define a new measure of intrinsic dimension of a data manifold, which we call the Patnaik‑Pearson dimension, and apply this to internal representations of neural networks, in particular transformers. The inspiration for this comes from the HTSR and SETOL work of Martin, Mahoney and Hinrichs, combined with the TwoNN intrinsic dimension estimator of Facco et al. We prove various properties of this intrinsic dimension estimator. Treating weight matrices of neural networks as data manifolds, for weight matrices whose Empirical Spectral Density follows a Pareto (Power Law) distribution, we relate the Patnaik‑Pearson dimension to the HTSR and SETOL analysis, and show that critical values of the tail exponent coincide for the two approaches. Using a combination of theoretical and numerical techniques, we study the behaviour of the Patnaik‑Pearson dimension of a data manifold under the transformations typical to neural networks. We apply this machinery to the BERT‑base and DeepSeek‑R1‑Distill‑Qwen‑1 models, to investigate first the Patnaik‑Pearson dimension of the initial data manifold of token embeddings, and second the evolution of the Patnaik‑Pearson dimension as token embeddings pass through the layers of the model. Code and notebooks used for the numerical results presented here is available at https://github.com/tdhadfield/PatnaikPearson
Authors:Sanghyeok Choi, Henry Gouk, Esmeralda S. Whitammer
Abstract:
The knowledge encoded in large language models (LLMs) can serve as a substrate for structured reasoning over variables describing a complex world, but accessing this knowledge in a probabilistically coherent manner poses a difficult inference problem. We propose Large Language Gibbs, a scheme for structured probabilistic inference that uses conditional distributions of an LLM as transition operators. Rather than sampling structured objects through single‑pass autoregressive generation, we iteratively resample individual variables conditioned on others using an LLM's next‑token conditionals. This approach avoids order‑dependent biases and produces a stationary distribution that reflects a compromise between all local conditionals. We apply this approach to sampling from synthetic distributions, consistent reasoning tasks, and Bayesian structure learning. The results suggest that the use of LLM conditionals in MCMC is a practical alternative to one‑pass generation for structured probabilistic inference under a world prior accessible through noisy LLM conditionals.
Authors:Zirui Wu, Lin Zheng, Jiacheng Ye, Shansan Gong, Xueliang Zhao, Yansong Feng, Wei Bi, Lingpeng Kong
Abstract:
Block diffusion language models accelerate decoding through parallel block‑wise denoising, yet whether they can be reliably scaled for long chain‑of‑thought (CoT) reasoning remains unresolved. To this end, we develop DreamReasoner‑8B, an open‑source block diffusion reasoning model, and conduct a systematic study of how training and inference block sizes affect long‑CoT reasoning. Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning, whereas small block sizes preserve effective reasoning. To bridge this granularity gap, we propose block‑size curriculum learning, which gradually transitions training from fine‑grained to coarse‑grained block sizes, thereby overcoming this limitation and enabling strong reasoning performance that generalizes across diverse inference block sizes. On mathematical and code reasoning benchmarks, DreamReasoner‑8B achieves results competitive with leading open autoregressive models such as Qwen3‑8B. This work establishes a practical foundation for efficient, reasoning‑capable diffusion language models. We release our model at https://github.com/DreamLM/DreamReasoner.
Authors:Bartłomiej Baranowski, Dave Zhenyu Chen, Matthias Nießner
Abstract:
Existing approaches to 3D scene understanding in Vision‑Language Models (VLMs) either rely on complex, model‑specific geometry encoders or large training budgets in pursuit of spatial reasoning. Instead, OneCanvas aggregates patch features from all views onto a single equirectangular panoramic canvas. Namely, each patch is unprojected to a 3D world coordinate using its depth and camera pose, then placed on the canvas at the continuous longitude and latitude of that point as seen from the canvas origin, with no rasterization or aggregation across overlapping views. A 3D position embedding of the patch's metric coordinates is added to its feature, restoring the depth lost when collapsing the world position to an angular canvas coordinate. Patches from all frames thus share one spatial coordinate system with no fusion or major architectural modifications of the backbone. The pretrained VLM consumes this representation as if it were an ordinary image. Because the canvas can be centered on any pose of interest, the same representation directly supports situated reasoning from a specific viewpoint, a common requirement in robotics and embodied AI. Thanks to this representation, we can also introduce a spatial pretraining curriculum: by procedurally placing patch features of objects, drawn from real images, at chosen 3D world positions on an otherwise empty canvas, we generate on‑the‑fly supervision spanning a broad range of spatial reasoning tasks, with answer distributions controlled to reduce spatial reasoning shortcuts. OneCanvas achieves state‑of‑the‑art accuracy on SQA3D and VSI‑Bench, and generalizes to out‑of‑distribution data on SPBench, using an order of magnitude less training compute than the strongest competing methods.
Authors:Haipeng Luo, Qingfeng Sun, Songli Wu, Can Xu, Wenfeng Deng, Han Hu, Yansong Tang
Abstract:
Reinforcement Learning with Verifiable Rewards algorithms like GRPO have emerged as the dominant post‑training paradigm for complex reasoning in LLMs, yet commonly suffer from policy entropy collapse during training. We conduct a first‑order gradient analysis of token‑level entropy dynamics under GRPO and identify a token‑level credit assignment mismatch: the per‑token entropy variation decomposes into the product of the trajectory‑level advantage and an entropy sensitivity function over the next‑token distribution, yielding an advantage‑surprisal four‑quadrant structure and a near‑criticality property. Motivated by it, we propose STARE (Surprisal‑guided Token‑level Advantage Reweighting for policy Entropy stability), which identifies entropy‑critical token subsets via batch‑internal surprisal quantiles, selectively reweights their effective advantages, and incorporates a target‑entropy closed‑loop gate for stable entropy regulation. Across model scales from 1.5B to 32B and three task families (Short CoT, Long CoT, and Multi‑Turn Tool Use), STARE sustains stable RL training over thousands of steps while maintaining policy entropy within the target band. On AIME24 and AIME25, STARE outperforms DAPO and other competitive baselines by 4%‑8% in average accuracy, with reflection tokens and response length growing in tandem, indicating sustained exploration‑exploitation balance that further unlocks RL training potential.Code is available at https://github.com/hp‑luo/STARE.
Authors:Kangsheng Duan, Ziyang Xu, Wenyu Liu, Xiaohu Ruan, Xiaoxin Chen, Xinggang Wang
Abstract:
While 10B‑level industrial foundation models have pushed the boundaries of image inpainting, their prohibitive computational costs severely hinder practical deployment. Constructing a highly optimized task‑specific specialist offers a promising solution; however, extreme structural compression inevitably triggers a severe representation bottleneck. To conquer this, we propose Moebius, a highly efficient lightweight inpainting framework. We systematically reconstruct the diffusion backbone by introducing the Local‑λ Mix Interaction (LλMI) block. Comprising Local‑λ and Interactive‑λ modules, it elegantly summarizes spatial contexts and global semantic priors into fixed‑size linear matrices, preserving complex latent interactions while drastically shedding parameters. Furthermore, to unlock the full representational capacity of this highly compact architecture, we synergistically pair it with an adaptive multi‑granularity distillation strategy. Operating strictly within the latent space to avoid expensive pixel‑space decoding, this strategy dynamically balances multiple gradient‑based losses to achieve high‑fidelity alignment. Extensive experiments across natural and portrait benchmarks demonstrate that this optimal synergy enables Moebius to rival or even surpass the generation quality of the 10B‑level industrial generalist FLUX.1‑Fill‑Dev. Remarkably, Moebius achieves this using less than 2% of the parameters (0.22B vs. 11.9B) while delivering a >15× acceleration in total inference time, setting a new efficiency standard for high‑fidelity inpainting. Project page at https://hustvl.github.io/Moebius.
Authors:Bolin Shen, Ziwei Huang, Zhiguang Cao, Yushun Dong
Abstract:
The Traveling Salesman Problem (TSP) is a cornerstone of combinatorial optimization and arises in many practical scenarios. Although graph‑based learning approaches have been explored for TSP, the question of how to exploit graph structure more effectively remains open. We present the Anisotropic Graph Diffusion Network (AGDN), a new Graph Neural Network framework designed to solve TSP. Our method tackles two central difficulties: (1) the lack of informative topological prior in fully connected TSP graphs, and (2) losing connected nodes in the optimal solution after the commonly used graph sparsification techniques. To overcome these issues, we construct a MixScore transition matrix that merges node similarity with pairwise distance, and we develop an anisotropic graph diffusion strategy that supports efficient information exchange across multiple hops. Comprehensive experiments spanning diverse instance sizes and node distributions show that AGDN consistently outperforms existing methods while keeping computation time competitive. Furthermore, AGDN generalizes well to problem sizes and distributions beyond those seen during training. The implementation is publicly available at: https://github.com/LabRAI/AGDN.
Authors:Fangyijie Wang, Jianjun Yu, Wentao Shi, Haixia Huang, Ran Shi, Guénolé Silvestre, Kathleen M. Curran
Abstract:
Clinician‑centered evaluation is critical for validating medical AI systems, especially in ultrasound imaging where quantitative metrics do not always capture clinical usability. Existing medical image platforms primarily focus on dataset labeling. They lack integrated support for blinded model comparison and reproducible evaluation workflows. We present a clinician‑centered pipeline for remote annotation and evaluation in ultrasound AI studies. The proposed pipeline uses a centralized server and lightweight browser interfaces to enable clinicians to perform annotation, blinded ranking, and review without local dataset downloads. The pipeline also supports multi‑rater participation, centralized result aggregation, and automated statistical analysis. We validate the pipeline in a fetal ultrasound segmentation study with six raters spanning expert, generalist, and non‑expert experience levels. The system automatically generated Spearman correlation, Kendall's τ, and top‑1 selection statistics. Results indicated moderate to strong agreement across experts and other groups. The blinded evaluation results showed a tendency for later active learning models to be preferred. These outcomes suggest that the pipeline can support clinician‑centered annotation and reproducible human‑\acAI evaluation studies in ultrasound imaging. The proposed pipeline is available on \hrefhttps://github.com/13204942/SonoRateGitHub.
Authors:Nichula Wasalathilaka, Parakrama Ekanayake, Roshan Godaliyadda
Abstract:
CT‑derived airway models support pulmonary morphometry and airflow simulation, but are often limited by distal scan resolution and the need for substantial cleanup near bifurcations. Procedural alternatives are reproducible, yet many rely on stitched tubular primitives that introduce non‑smooth junctions and poorly defined open boundaries. We present RespGeomLib, a reproducible parametric engine for generating analysis‑ready human airway lumen surfaces from compact YAML specifications. The framework combines port‑based assembly with implicit smooth‑min junction blending to produce seamless junctions, while avoiding full‑tree voxelization through analytic segments and local implicit extraction around bifurcations. Quantitatively, RespGeomLib yields cleaner junctions than a Boolean/stitch baseline and is substantially faster and more memory‑efficient than whole‑tree global implicit extraction. We further demonstrate morphometry‑guided tree generation, controlled synthetic airway variants, and CFD‑ready export with stable airflow simulation. RespGeomLib targets biomedical workflows requiring reproducible morphometry, controlled synthetic variants, and simulation‑ready lumen geometry. The code is publicly available at https://nichula01.github.io/Respgeomlib/
Authors:Jeongmin Bae, Seoha Kim, Marc Pollefeys, Mahdi Rad, Youngjung Uh, Taein Kwon
Abstract:
Dynamic 3D hand reconstruction from egocentric videos is essential for next‑generation computing platforms such as AR/VR and AI glasses. Despite its importance, most prior works focus either on multi‑view 3D hand reconstruction or on 4D human body reconstruction. Egocentric 4D hand reconstruction remains challenging due to fast head motion, rapid hand dynamics, severe occlusions, and inherent ambiguity from single‑view observations. To address these challenges, we introduce Hand‑4DGS, the first feed‑forward framework for reconstructing dynamic 4D hands directly from egocentric videos, enabling both fast (~60 FPS) inference and strong generalization. Our approach incorporates a mesh‑guided representation for structural priors and temporal convolutions to model dynamic motion. We evaluate our framework on two challenging egocentric datasets, H2O and ARCTIC, and demonstrate significant improvements over baselines. Our method benefits from the generalization capability of feed‑forward networks and effective 2D image supervision through Gaussian splatting, without requiring expensive 3D hand pose ground‑truth annotations.
Authors:Nahum Korda, Gadi Evron
Abstract:
Automated vulnerability discovery in large codebases remains challenging: traditional static analysis produces high false‑positive rates, while dynamic approaches such as fuzzing require substantial infrastructure and often target narrow classes of bugs. Recent advances in large language models (LLMs) enable semantic reasoning about program behavior, but applying LLMs to repository‑scale security analysis introduces challenges related to context management, cost, and verification. We present OpenAnt, an open‑source vulnerability discovery system that integrates static program analysis with LLM‑based reasoning in a multi‑stage pipeline. OpenAnt introduces three key techniques. First, codebases are decomposed into self‑contained analysis units filtered by reachability from external entry points, reducing the analysis surface by up to 97% while preserving attack‑relevant code. Second, candidate vulnerabilities undergo adversarial verification through constrained attacker simulation, where the model evaluates exploitability under realistic attacker capabilities. Third, findings are validated through dynamic verification, in which exploit environments are generated automatically, executed in sandboxed containers, and discarded after use. Evaluation on widely used open‑source projects including OpenSSL, WordPress, and Flowise shows that this architecture can identify previously unknown vulnerabilities while maintaining manageable analysis cost and substantially reducing false positives. Our results suggest that closed‑loop vulnerability discovery pipelines, combining semantic reasoning with exploit validation, provide a practical path toward scalable automated security analysis. OpenAnt is released as open source under the Apache 2.0 license at https://github.com/knostic/OpenAnt.
Authors:Michael Detzel, Gabriel Nobis, Kristiyan Blagov, Juri Schubert, Jackie Ma, Wojciech Samek
Abstract:
Neural Controlled Differential Equations (NCDE) provide a powerful continuous‑time framework for forecasting time series, but standard graph‑based extensions typically learn spatial structure purely from data, even in settings where a directed graph structure is known a priori. We introduce Informed Neural controlled Differential EQuationS (INDEQS), a graph‑based NCDE forecasting method that incorporates prior knowledge of a directed graph at distinct architectural positions. INDEQS separates inner mixing of hidden states across graph nodes from outer mixing between vector field and control, and offers both a lightweight graph‑constrained variant and a more expressive variant, learning additional graph connections from data via adaptive graph convolutions. To systematically study when graph informedness is beneficial in forecasting, we devise a continuous advection simulation on directed graphs, yielding synthetic spatio‑temporal datasets with known ground‑truth flow structure. We then evaluate INDEQS on two real‑world tasks: river discharge forecasting on a hydrological network and traffic flow prediction on PeMS08. Across these synthetic and real‑world benchmarks, outer informedness consistently improves mean absolute error over an uninformed NCDE with comparable parameter count, particularly on larger graphs, while inner informedness offers a more parameter‑efficient alternative when strict adherence to a known adjacency is desired. A comparison of discrete convolutional and continuous‑time decoders further shows that continuous decoders yield better accuracy and greater temporal flexibility on real‑world tasks. An implementation of INDEQS and the advection simulation is available at https://github.com/Mitchi1/indeqs.
Authors:Zanjia Tong, Wenlong Dong, Chengjie Zhang, Hong Zhang
Abstract:
Task‑oriented grasping performance degrades significantly when object views suffer from occlusions. Existing task‑oriented grasping methods typically assume task‑relevant regions are visible in the initial frame, while view planning approaches enable active perception but often ignore task semantics and rely on time‑consuming scene reconstruction. To address these limitations, we present GCNGrasp‑VP, an efficient framework integrating affordance field prediction with active view planning. Central to this framework is GCNGrasp‑v2, a task‑oriented grasp model that simultaneously supports grasp evaluation and affordance field prediction, achieving constant‑time inference complexity. Leveraging this capability, our Affordance‑guided View Planner (Affordance‑VP) utilizes the affordance field as an information gain metric to guide camera observation of task‑relevant regions without requiring scene reconstruction. View planning results show that our method significantly outperforms scene‑uncertainty‑driven baselines with only one view adjustment. Real‑world validation further confirms substantial improvements in grasp success rates for single‑object scenarios while maintaining millisecond‑level computational latency. Code and models are available at https://github.com/Instinct323/GCNGrasp‑VP.
Authors:Alessandro Scherl, Bernhard Neuberger, Simon Schwaiger, David Mulero-Pérez, Lucas Muster, Jose Garcia-Rodriguez
Abstract:
Visual servoing with self‑supervised Vision Transformer (ViT) features enables training‑free robotic positioning with strong generalization, but faces a fundamental trade‑off between robustness and precision. Coarse patch‑level descriptors provide stable correspondences yet limit positioning accuracy. Increasing image resolution improves precision but yields only marginal robustness gains ‑ under perturbation, high‑resolution processing improves convergence success rate from 76.6% to just 81.0% despite 12x more ViT patches. Therefore, we propose Adaptive Resolution Tiling Visual Servoing (ART‑VS), a two‑phase method that adapts feature granularity to servoing progress: a coarse phase at native ViT resolution for stable alignment, then a tiled high‑resolution phase that restricts matching to local neighborhoods improving positioning accuracy. Without any task‑specific training, ART‑VS achieves 95.4% convergence under perturbation, outperforming standard and full‑resolution ViT‑based servoing by 18.8 and 14.4 percentage points. Over the former it reduces positioning error by 53%, while running at over 10x higher speed and 27% lower VRAM than the latter. We validate ART‑VS across three ViT backbones and demonstrate real‑world category‑level grasping of unseen object instances, achieving 95/100 on transparent bottles and 98/100 on shoes. Code available under https://art‑vs.github.io/.
Authors:Jiayi Gao, Qingchao Chen, Yuxin Peng, Yang Liu
Abstract:
Current image editing methods excel at static attributes but fail at complex Human‑Object Interactions (HOI), a critical challenge unaddressed by existing benchmarks that conflate HOI with static attributes, relying on global metrics incapable of simultaneously assessing dynamic interaction validity and entangled human‑object pair preservation. Thus, we first introduce HOI‑Edit, a comprehensive benchmark with three progressive cognitive levels, which features an automated metric HOI‑Eval that reliably evaluates instance‑level interaction by letting VLM Q&A after thinking with images containing grounded Human‑Object pairs. Considering the task's essence of remodeling dynamic relationships, we benchmark Image‑to‑Video (I2V) models, finding them inherently suited for dynamic editing due to their temporal generation capabilities. Crucially, beyond superior performance, this capability provides a "replay of the failure process," offering unique diagnosability into why errors occur. We thus propose SCPE (Self‑Correcting Process Editing), a novel, agentic self‑correcting framework that constrains the generation of I2V models through iteratively refined prompts, enabling the generated videos to more accurately present the target HOI. Extracted frames from these videos are the final editing results. On HOI‑Edit, SCPE achieves performance competitive with state‑of‑the‑art (SOTA) editing models like Nano Banana on interaction. Code is available at https://github.com/oceanflowlab/HOI‑Edit.
Authors:Hong-Tao Yu, Chen-Wei Xie, Yuxin Peng, Serge Belongie, Xiu-Shen Wei
Abstract:
Recent advancements in Large Vision‑Language Models (LVLMs) have demonstrated remarkable multimodal perception and reasoning capabilities. While numerous benchmarks have evaluated LVLMs from holistic or task‑specific perspectives, their capabilities on fine‑grained image tasks‑fundamental to computer vision‑remain insufficiently understood. To address this gap, we introduce FG‑BMK, a comprehensive fine‑grained evaluation benchmark containing 1.01 million questions and 0.28 million images, covering diverse scenarios from common object‑centric domains to specialized domains. FG‑BMK jointly evaluates dialogue‑level fine‑grained semantic recognition and feature‑level visual discriminability through human‑oriented and machine‑oriented paradigms, enabling diagnostic analysis of whether LVLM failures arise from insufficient visual representations, weak visual‑to‑semantic grounding, or limited fine‑grained knowledge. Through extensive experiments on a diverse set of representative LVLMs/VLMs, we find that current LVLMs remain inadequate fine‑grained recognizers, with failures arising from intertwined bottlenecks in visual representations, semantic grounding, modality alignment, and category‑level knowledge. We further analyze training design factors for improving fine‑grained capabilities and examine how visual and linguistic perturbations affect LVLM predictions. These findings provide diagnostic insights into the limitations of current LVLMs and offer guidance for future data construction and model design in developing more reliable LVLMs for fine‑grained visual tasks. Our code is open‑source and available at https://fg‑bmk.github.io/.
Authors:Yuchen Rao, Xuqian Ren, Yinyu Nie, Sayan Deb Sarkar, Biao Zhang, Vincent Lepetit, Friedrich Fraundorfer
Abstract:
Recovering complete 3D representations of objects from few casual image captures remains a significant challenge. Recent 3D generative models, particularly those based on Flow‑Matching (FM), can synthesize high‑quality textured assets; however, they often suffer from ''synthetic bias'' where learned priors override observational evidence, alongside a lack of alignment with the observed instance. Conversely, optimization‑based methods like 3D Gaussian Splatting (3DGS) provide high fidelity on visible surfaces but fail to reason about unobserved geometry. In this paper, we present FlowObject, a framework that reformulates sparse‑view 3D reconstruction as a training‑free, guided inverse problem. Our approach applies a dual‑space guidance strategy to steer the Ordinary Differential Equation (ODE) trajectory of a flow‑matching model, enabling the completion of unseen regions through learned generative priors while enforcing strict consistency with real‑world observations. By integrating a 3DGS refinement stage, FlowObject further bridges the gap between ''synthetic‑looking'' generative outputs and photorealistic reconstructions. Comprehensive benchmarks on synthetic and real‑world datasets demonstrate that current state‑of‑the‑art methods often struggle to achieve geometric completeness and observational consistency simultaneously, especially under severe occlusions. In contrast, our method significantly outperforms state‑of‑the‑art generative models and optimization‑based frameworks in both geometric completeness and view‑dependent appearance fidelity.
Authors:Zheng He, Danica J. Sutherland
Abstract:
Testing conditional independence is fundamental yet intrinsically difficult: without additional assumptions, Type I error control is impossible in general. The "Model‑X'' paradigm addresses this difficulty by assuming exact knowledge of a relevant conditional distribution. While small deviations from this assumption can sometimes be tolerated in classical one‑shot testing, existing sequential conditional independence tests typically require the Model‑X conditional to be known exactly, making them fragile when it must instead be estimated. We propose a new approach that is substantially more robust to such estimation error. Our method applies testing‑by‑betting to an adaptively optimized Kernel Conditional Independence statistic, together with a normalization scheme and a truncate‑and‑shift calibration strategy. These modifications greatly reduce Type I error inflation while preserving high power across high‑dimensional synthetic benchmarks and real‑world fairness tasks, outperforming existing sequential Model‑X approaches. Code is available at https://github.com/he‑zh/SKCI.
Authors:Minseo Kim, Minjae Lee, Seunghyuk Oh, Kevin Galim, Donghoon Kim, Coleman Hooper, Harman Singh, Amir Gholami, Hyung Il Koo, Wonjun Kang
Abstract:
Reinforcement learning (RL) has become a representative post‑training paradigm for LLMs, enabling strong reasoning and agentic capabilities. However, rollout generation remains a dominant latency bottleneck because autoregressive sampling decodes responses sequentially and a small number of long‑tailed generations often determine completion time. Speculative decoding (SD) offers a natural way to address this bottleneck, as it is a well‑established technique for serving fixed LLMs that reduces latency by rapidly drafting tokens and accepting them through parallel verification while preserving the target‑model distribution. However, its practical speedups do not directly carry over to RL rollouts: (i) the evolving target policy makes any fixed drafter increasingly mismatched with the policy's output distribution; and (ii) active batch sizes shrink throughout rollout decoding, shifting decoding from compute‑bound to memory‑bound regimes where parallel verification can exploit underutilized compute. Therefore, accelerating RL rollouts requires both a drafter that remains effective under long, high‑temperature generations from an evolving policy and system‑aware use of SD that avoids compute‑bound regimes. We present EfficientRollout, a system‑aware self‑SD framework designed to address this gap for RL rollouts. EfficientRollout induces a quantized drafter from the target model (i.e. self‑speculative decoding), keeping it coupled to the evolving policy without separate drafter pretraining or online adaptation. It further coordinates a system‑aware SD toggle policy with acceptance‑aware draft‑length adaptation, enabling speculation only in beneficial regimes while matching the drafting budget to evolving drafter quality. EfficientRollout reduces rollout and end‑to‑end latency by up to 19.6% and 12.7%, respectively, over an accelerated AR rollout baseline, while preserving final model quality.
Authors:Jingkun Luo, Yifan Sun, Da-Tian Peng, Guanxiong Pei
Abstract:
Sentence‑level AI‑generated text detection (S‑AGTD) for hybrid documents, where humans and LLMs co‑author one text, faces two gaps: existing methods classify each sentence in isolation, discarding inter‑sentence dependencies, and existing benchmarks omit the newest generation of generators. We construct MOSAIC, a benchmark of 16,000 hybrid documents over PubMed and XSum, generated by DeepSeek‑V3.2 and Kimi K2 under stringent quality controls including a perplexity‑consistency filter absent from prior benchmarks. We recast S‑AGTD as structured prediction over the document sentence sequence and instantiate it as SenFlow, integrating graph‑based inter‑sentence propagation with linear‑chain CRF decoding in a single document‑level pass over a sentence graph. SenFlow reaches state‑of‑the‑art performance on MOSAIC, with a +4.15 pp average Macro‑F1 margin on cross‑domain transfer, the hardest of three protocols of increasing difficulty. We further find that even after the perplexity filter equalizes overt cues, AI insertions retain a generator‑dependent sentence‑length gap that sentence‑level detectors still exploit. Code and data: https://github.com/luojingkun22/SenFlow
Authors:Tim Rädsch, Yuki M Asano, Hilde Kuehne, Stefan Bauer, Priyank Jaini, Robert Geirhos, Carsten T. Lüth
Abstract:
Video generative models ( VGMs) have become a new frontier that can be used not just for video generation but for a multitude of downstream tasks, including world modeling. To advance these tasks, a good video model must understand the physical reality of the world. Evaluating this understanding is an emerging field and has led to the Physics‑IQ benchmark, which quantifies this explicitly by comparing model‑generated videos to real‑world videos of physical experiments. In this work, we present a systematic audit of the Physics‑IQ benchmark, expose shortcomings and propose three solutions that sharpen how we can measure physical understanding of VGMs. Specifically, we improve prompt and ground‑truth quality to reduce the influence of confounding factors and further introduce a sample‑level scoring system that weights each sample and metric equally. Our resulting benchmark, Physics‑IQ Verified, refines 57.6% of all samples and improves over 34.8% of prompts. In a comparison study using six image‑to‑video generative models, we observe moderate but meaningful ranking changes (Kendall's τ= 0.46). We hope Physics‑IQ Verified advances the community by providing a more reliable signal toward physically accurate VGMs. The code for the benchmark can be accessed at https://github.com/google‑deepmind/physics‑iq‑benchmark
Authors:Jasmine Owers, Edwin Simpson, Martha Lewis
Abstract:
Figurative language and negation are two areas that challenge current language models, however, both are widely used throughout written and spoken language. Large language models (LLMs) are also widely used in everyday contexts where they cannot necessarily be tuned for a specific dataset. It is therefore essential to understand the ability of LLMs to correctly interpret text that includes both negation and figurative language. To investigate this, we develop a set of new annotations to an existing dataset of figurative language, and test a range of language models on the dataset. We find that the combination of negation and figurativeness can present a particular challenge, and that performance overall and across different negation types is particularly dependent on the prompt style used.
Authors:Yuanxin Liu, Ruida Zhou, Xinyan Zhao, Amr Sharaf, Hongzhou Lin, Arijit Biswas, Mohammad Ghavamzadeh, Zhaoran Wang, Mingyi Hong
Abstract:
Test‑time scaling via sequential revision has emerged as a powerful paradigm for enhancing Large Language Model (LLM) reasoning. However, standard post‑training methods primarily optimize single‑shot objectives, creating a fundamental misalignment with multi‑step inference dynamics. While recent work treats this as multi‑turn reinforcement learning (RL), conventional approaches optimize over the multi‑step trajectories directly, failing to further exploit the high‑quality mistakes in intermediate steps that model can learn from correcting them. We propose a two‑stage iterative framework that alternates between online data/prompt augmentation and policy optimization. By converting the intermediate steps (``near‑miss'' answers) in the successful recovery trajectories into decoupled revision and verification prompts, our approach concentrates training on both effective answer transformation and error identification. This approach enables efficient off‑policy data generation and reduces the computational overhead of long‑horizon sampling compared to standard multi‑turn RL. On LiveCodeBench, using publicly available test cases as feedback, we observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi‑turn training. Beyond coding, our approach matches the previously reported SOTA result on circle packing while using the smallest base model (4B) and far fewer rollouts than the much larger evolutionary search systems. Math results under ground‑truth verification further confirm improved correction ability. It also generalizes to out‑of‑distribution constraint‑satisfaction puzzles such as n\_queens and mini\_sudoku, where correctness is defined entirely by problem constraints. Code is available at https://github.com/yxliu02/REVES.git.
Authors:Abdulmalik Alquwayfili, Faisal Almeshal, Jumanah Almajnouni, Leena Alotaibi, Faisal Alhajari, Mohammed Alkhrashi, Alreem Almuhrij, Abdullah Aldwyish, Raied Aljadaany, Huda Alamri, Muhammad Kamran J. Khan
Abstract:
Image retrieval in crowded scenes is particularly challenging due to the salience bias of conventional visual encoders, which tend to focus on dominant objects while neglecting low‑attention regions that are often crucial for fine‑grained retrieval. We propose LARE (Low‑Attention Region Encoding), a framework that explicitly models these overlooked regions. LARE adopts a dual‑encoding strategy that encodes low‑attention regions of an image and the full image in parallel, leading to more diverse and informative image embeddings. To evaluate image retrieval performance in challenging crowded scenes, we introduce Dense‑Set, a challenging subset derived from COCO and Flickr30K. In this subset, images are re‑captioned to provide richer descriptions of low‑attention or previously overlooked regions. This dataset highlights the limitations of existing retrieval models and enables a more rigorous evaluation under densely crowded scene conditions. Experimental results demonstrate that the proposed framework improves retrieval performance by preserving subtle, non‑dominant visual cues within the shared latent space.
Authors:Veit Hucke, Thomas Pinetz, Gregor Reiter, Ursula Schmidt-Erfurth, Hrvoje Bogunović
Abstract:
Optical coherence tomography (OCT) is essential in ophthalmology, but inconsistent image quality especially in low‑cost devices hinders automated analysis. To address this, we introduce a flow‑matching‑based test‑time adaptation method that generates high‑quality surrogate images from noisy inputs. Typically, domain gaps between test and training data cause pixel distribution mismatches during the denoising process. We overcome this by matching the test image's histogram to synthetic reference trajectories, successfully aligning the input with expected distributions. Additionally, we remove the network's time conditioning to account for slight deviations in real‑world noise distributions. Our approach achieves state‑of‑the‑art performance in segmenting critical biomarkers for two stages of Age‑related Macular Degeneration (AMD). Code is available: https://github.com/Veit21/tta‑flow.
Authors:Hana Jebril, Thomas Pinetz, Günter Klambauer, Hrvoje Bogunović
Abstract:
Reliable pixel‑level uncertainty quantification holds the potential to transform clinical workflows by enabling high‑fidelity longitudinal monitoring and distinguishing true pathological changes from artifacts. Ideally, these models provide the stability required for critical treatment planning and surgical intervention. However, standard deep learning models often suffer from miscalibration, yielding overconfident predictions that mask underlying vulnerabilities at subtle pathological boundaries. To address this, we propose QUAM‑SM, a post‑hoc framework using targeted adversarial search to identify "adversarially fragile" pixels. By actively seeking perturbations that expose predictive instability, our method highlights regions where decisions are most vulnerable to being flipped. Importantly, the framework disentangles epistemic uncertainty from aleatoric uncertainty. Experiments on two public datasets with multiple expert annotations demonstrate that QUAM‑SM outperforms both standard and recent uncertainty estimation approaches in terms of reliability and boundary sensitivity. Code is available at https://github.com/HanaJebril/quam_sm
Authors:Bohou Zhang, Xiaoyu Tao, Mingyue Cheng, Huijie Liu, Qi Liu
Abstract:
Abstractive summarization plays a crucial role in enabling efficient understanding of scientific literature, yet it inherently demands both linguistic fluency and factual faithfulness. Existing approaches often fail to reconcile these two requirements. Extractive methods rely on rigid sentence splicing that disrupts macro‑level logical coherence, while large language model (LLM)‑based generative approaches, despite mastering linguistic fluency, exhibit limited factual consistency. In this work, we propose ScholarSum, a hierarchical reflective graph‑based framework that emulates a student‑teacher writing process for fluent and faithful scientific summarization. ScholarSum first organizes the document into a hierarchical knowledge graph by segmenting it into semantically coherent units, whose multi‑layered community structure captures global logic and macro‑level themes. Guided by this global structure, the student generates an initial draft, which is subsequently refined through fine‑grained evidence retrieval. To ensure factual consistency, a teacher‑like reviewer then iteratively examines the draft, identifies unsupported content, and prompts targeted re‑retrieval and rewriting until the summary meets rigorous quality standards. Extensive experiments demonstrate that ScholarSum significantly outperforms previous baselines in terms of both completeness and faithfulness. Our code is available at https://github.com/Xiaoyu‑Tao/ScholarSum.
Authors:Like Zhang, Runliang Niu, Shiqi Wang, Xiyu Hu, Qianli Xing, Pan Wang, Qingzu He, Qi Wang
Abstract:
Vision‑language models (VLMs) are rapidly advancing toward sophisticated grounded structured visual reasoning. Training models for such advanced capabilities demands a new genre of data that seamlessly unifies spatial coordinates, open‑vocabulary descriptions, structured attributes, and topological relationships into a singular representation. However, existing data annotation tools fundamentally fail to meet these intricate demands, suffering from three systematic bottlenecks: limited expressiveness, severe annotation‑training decoupling, and poor data reusability. To bridge this infrastructure gap, we introduce an open‑source annotation tool, ScreenAnnotator. First, we define a unified annotation atom schema that binds spatial, semantic, and structural primitives into a single unit. Second, we implement an on‑policy annotation loop embedded with a Bayesian Annotation Verifier (BAV). Finally, we design a template‑driven multi‑task data synthesis process dynamically transforms static atoms into diverse multi‑dimensional reasoning tasks, eliminating redundant re‑annotation. The on‑policy loop drives the annotation accept rate to nearly 100% on flowcharts and 77% on GUI screenshots, while steadily reducing per‑image annotation time as labeled data accumulate. In the flowchart scenario, fine‑tuning a VLM yields 76.1% average accuracy, which is a 35.1% point absolute gain. Our code is available at: https://github.com/WnQinm/Annotator.
Authors:Zhoupeng Guo, Yunqi Zhu, Zhihe Fan, Xinjie Yao, Ruipu Zhao, Boan Tao, Yiming Sun, Zhen Wang, Pengfei Zhu
Abstract:
Air‑ground collaborative perception is crucial for robust visual understanding in real‑world dynamic environments. However, existing studies typically formulate collaboration as single‑task cross‑view fusion, overlooking the functional dependencies among localization, target association, and fine‑grained parsing. In addition, the heterogeneous nature of aerial and ground views introduces substantial geometric, scale, and occlusion discrepancies, making uniform feature sharing vulnerable to negative transfer. To tackle these issues, we model air‑ground perception as a progressive cross‑task collaboration task and construct the Air‑Ground Progressive Collaboration (AGPC) benchmark, a spatio‑temporally aligned benchmark comprising more than 745K raw video frames. Built upon this benchmark, we propose Socialized Co‑Perception (SCP), a coarse‑to‑fine framework that organizes collaboration progressively from aerial global localization to ground target association and identity‑aware parsing. Its core module, the Dual‑Layer Router (DLR), decouples input‑side multi‑scale expert selection from output‑side task‑conditioned modulation, enabling selective cross‑view and cross‑task interaction while suppressing harmful interference. Extensive experiments demonstrate the effectiveness of SCP. It achieves a 3.73% coevolutionary gain and a 7.86% improvement in average downstream performance. These results show that task‑conditioned collaboration is more effective than uniform fusion for heterogeneous air‑ground perception. The code is available at https://github.com/g1136639260‑spec/AGSCP.
Authors:Zhe Ren, Yibo Yang, Yimeng Chen, Zijun Zhao, Benshuo Fu, Zhihao Shu, Bingjie Zhang, Yangyang Xu, Dandan Guo, Shuicheng Yan
Abstract:
Memory benchmarks for LLM agents largely assume single‑user settings, leaving shared assistants for hospitals, workplaces, campuses, and households understudied. In these deployments, multiple principals write to a common memory pool and query it under different roles, scopes, and relationships, so memory quality requires governance as well as recall. We introduce GateMem, a benchmark for multi‑principal shared‑memory agents. GateMem jointly evaluates utility for legitimate long‑horizon requests with state updates, access control across contextual authorization boundaries, and agent‑facing active forgetting after explicit deletion requests. It spans medical, office, education, and household domains, with long‑form multi‑party episodes, incremental memory injection, hidden checkpoints, structured judging, and leak‑target annotations. Across diverse baselines and backbone models, no method simultaneously achieves strong utility, robust access control, and reliable forgetting. Long‑context prompting often yields the best governance score at high token cost, while retrieval‑based and external‑memory methods reduce cost yet still leak unauthorized or deleted information. These results show current memory agents remain far from reliable shared institutional deployment.
Authors:Haobin Qin, Baofeng Zhang, Hidehisa Akiyama, Keisuke Fujii
Abstract:
Robot soccer is a challenging testbed for multi‑agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long‑horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot‑soccer platform, but its competition‑oriented server‑client architecture is difficult to use directly with modern Python‑based MARL workflows. We introduce R2D‑RL, a reinforcement learning environment that connects RCSS2D and HELIOS‑based player clients to a Python MARL interface through shared‑memory communication and cycle‑level synchronization. R2D‑RL supports full‑field and scenario‑based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value (EPV)‑based reward shaping, and parallel execution. We provide front‑goal scenarios and an 11‑vs‑11 full‑field benchmark, together with baseline results.
Authors:Shanshan Lyu, Yiwei Wang, Yujun Cai, Jiafeng Guo, Shenghua Liu
Abstract:
Dense retrieval ranks one query vector against one document vector. On long documents, this interface can fail when a short but decisive span is weakened during document encoding before ranking. We study this failure mode as document‑side early compression and introduce the Evidence Dilution Index (EDI) to measure how far a document‑level representation falls below the strongest chunk‑level evidence within the same gold document. Guided by this view, we propose DICE (Document Inference via Chunk Evidence), a training‑free document‑side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector while preserving the standard one‑query‑one‑document interface. On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens: for Dream, Passkey >4k rises from 30.0 to 90.0 and Needle >4k from 23.3 to 74.0. Across 12,779 filtered samples, DICE yields lower EDI than the single‑vector baseline in 92.8% of cases. These results establish document‑level encoding as a practical and underexplored lever for long‑document retrieval.
Authors:Guannan Lai, Haoran Hu, Han-Jia Ye
Abstract:
We present RouteJudge, an online pairwise preference evaluation framework for LLM routing systems, with a public platform available at https://routejudge.cn. Different from model‑level response evaluation, RouteJudge focuses on router‑level decision quality. For each user query, multiple routing strategies independently recommend candidate models under the same model pool and budget constraints. The selected model responses are then presented to users through anonymous pairwise comparisons, and the resulting user preferences are attributed back to the routing strategies behind the compared responses. Each evaluation record stores the query, routing decisions, model responses, preference labels, cost, latency, and task metadata, enabling preference‑aware, cost‑aware, and task‑conditioned analysis of LLM routers. To support the continuous expansion of routing methods in RouteJudge, we further release ORBIT (Optimal Routing and Budgeted Inference Toolbox), a modular and extensible toolbox that standardizes the end‑to‑end workflow of LLM routing. ORBIT provides unified interfaces for benchmark loading, query representation, router implementation, budget‑aware evaluation, and method comparison, allowing researchers to develop and evaluate routing algorithms under consistent protocols. It also serves as the submission and integration layer for RouteJudge: researchers can implement routing methods within ORBIT, validate them on existing routing benchmarks, and submit compatible routers for online preference‑based evaluation. The code of ORBIT is available at https://github.com/AIGNLAI/LAMDA‑ORBIT.
Authors:Benjamin Lang, Matthew Roughan, Mengbin Ye
Abstract:
Australia depends on an Internet built from multiple networks of long‑haul links. We study the interactions of these independent provider networks to investigate how the peering between these networks provides redundancy for failures on a single network, as well as the potential vulnerabilities introduced by failures of Shared Risk Link Groups (SRLGs), whereby ostensibly independent links of different providers fail simultaneously due to joint physical dependencies such as shared conduits. We introduce a generalised multilayer network model in which each layer represents the network of an individual Internet Service Provider (ISP), along with an Internet Exchange Point (IXP) layer that facilitates interconnections between ISP networks. We construct an Australia‑specific model, consisting of six major ISPs. A failure analysis is performed on this network, revealing that diversity provides redundancy, even in the presence of shared risks, indicating the importance of a diverse network ecosystem.
Authors:Hong-Jun Choi, Jongho Lee, Jaeyoung Kim
Abstract:
Table Structure Recognition (TSR) using a pointer network achieves impressive results by predicting HTML sequences while aligning tags to detected text (or cell) regions. However, our analysis reveals that when pointer networks fail, 79.6% of errors occur between spatially adjacent cells (Manhattan distance <= 2). Despite this, standard cross‑entropy loss weights all negative candidates equally. In this work, we propose Geometry‑Aware Pointer (GAP) Loss, which reweights the cross‑entropy objective based on spatial proximity to ground truth. By applying inverse distance weighting, GAP focuses gradient flow where the model struggles most: immediate neighbors receive stronger gradients than distant cells. Our approach requires only a straightforward modification to the loss computation, maintaining the same model architecture with zero additional inference cost. Extensive experiments on PubTabNet and SynthTabNet demonstrate that GAP consistently reduces adjacent‑cell errors, achieving new state‑of‑the‑art performance. Our findings suggest that incorporating geometric inductive biases at the loss level provides a simple yet effective approach to robust TSR. Our code is available at https://github.com/teamreboott/GAP
Authors:Tolga Şakar
Abstract:
Turkish is agglutinative: meaning is carried by morphemes, yet the subword tokenizers that drive modern language models split words by corpus statistics, fragmenting semantically loaded suffixes and ‑‑ in the case of WordPiece and rule‑based analyzers ‑‑ failing to decode their output back to the original text. This paper presents Morpheus, a neural morpheme‑boundary model for Turkish that is at once a lossless, morphology‑aware tokenizer and a word‑embedding producer. A differentiable Poisson‑binomial dynamic program turns per‑character boundary probabilities into soft morpheme memberships during training and exact segments at inference, with no string normalization, so \mathrmdecode(\mathrmencode(w)) = w holds by construction. Because the model is neural, the same forward pass that tokenizes also emits a structured word embedding. Among reversible tokenizers ‑‑ the only ones valid for generation ‑‑ Morpheus attains the lowest bits‑per‑character (1.425), roughly doubles the gold morphological alignment of the subword family (MorphScore macro‑F1 0.61 vs.\ ～0.32), and uses ～19% less GPU memory than 64K‑vocabulary subword tokenizers. As an embedder, frozen Morpheus vectors lead on lexical retrieval (root‑family MAP 0.85) and same‑root verification (ROC‑AUC 1.00), surpassing the multilingual retriever BGE‑M3 and BERTurk; on context‑ and inflection‑dependent tasks (NER, case/number probing) the heavier contextual encoders remain ahead ‑‑ a trade‑off we attribute to Morpheus's root‑centric geometry. Code: https://github.com/lonewolf‑rd/TurkishMorpheus; model: https://huggingface.co/lonewolflab/Morpheus‑TR‑50K; interactive demo: https://huggingface.co/spaces/lonewolflab/morpheus‑tr‑demo.
Authors:Lin Zhang, Sicheng Mo, Zefan Cai, Jinhong Lin, Zihao Lin, Jiuxiang Gu, Krishna Kumar Singh, Yuheng Li, Yin Li
Abstract:
Autoregressive video diffusion models have emerged as a promising approach for long video generation, achieving strong performance in streaming settings. However, existing methods are restricted to forward temporal generation, whereas practical video creation often requires flexible generation order, e.g., conditioning on future context to extend backward, or on both past and future context for inbetween generation. We bridge this gap by training an autoregressive model that supports generation in arbitrary temporal directions. A key technical challenge arises from the Causal 3D VAE widely used in video diffusion models, which encodes latents strictly conditioned on past context. While suited for forward generation, this causal structure causes inter‑block discontinuities when generation proceeds backward. To address this, we introduce blockwise anchor latents, a set of auxiliary latents that restore the missing past context at block boundaries during backward generation. Built on this design, we propose UniTemp, a bidirectional distillation framework that trains a single autoregressive student model for any‑direction video generation. At inference time, UniTemp conditions on arbitrary past and/or future frames, improving controllability for both bidirectional and inbetween generation. Experiments show that UniTemp maintains competitive performance on short and long video generation compared to forward‑only methods, while enabling diverse workflows such as bidirectional video extension, inbetween generation, looping video generation, scene transition, and visual story generation. Project website: https://lzhangbj.github.io/projects/unitemp/
Authors:Jinmo Lee, Doyun Choi, Moongi Choi, Jaemin Yoo
Abstract:
Tabular stream learning requires predictions on sequentially arriving examples under distribution shift. While standard methods adapt by updating model states, tabular foundation models (TFMs) make predictions conditioned on a labeled context in an in‑context manner, making them a natural alternative for stream learning. This shifts the challenge from how to update the model to how to manage the context. We propose a future information view that yields three practical requirements for context management: preserve recent examples, retain uncertain examples, and remove redundant examples. We instantiate these requirements as CURE (Context management via Uncertainty‑aware admission and Redundancy aware Eviction), a context‑managing policy with entropy‑gated admission and redundancy‑aware eviction. Across seven streams, CURE shows up to 27.0% relative improvement over classical stream learners, remains robust across multiple TFM backbones, and ranks first among other policy variants. Code and datasets are available at https://github.com/morcellinus/CURE‑ICML‑FMSD.
Authors:Jinke Wu, Yifan Wang, Siyu Yi, Caiyang Yu, Ziyue Qiao, Nan Yin, Jiancheng Lv, Wei Ju
Abstract:
Single‑cell RNA sequencing (scRNA‑seq) serves a pivotal role in characterizing gene expression at the cellular level, enabling the identification of cell types and advancing the understanding of cellular heterogeneity. Despite the significant progress in scRNA‑seq data clustering, we argue that current methods always ignore the sparsity and noise, as well as the complex intercellular structural information inherent in scRNA‑seq data. Toward this end, in this paper, we propose a novel single‑cell RNA‑seq clustering framework via deep Siamese Graph Transformer Network (termed scGTN), which explicitly integrates gene expression profile and intercellular structural dependencies for cell clustering. In particular, we formulate scRNA‑seq data as a graph and construct two augmented graph views that serve as dual views to capture complementary intercellular information. Then, a Siamese graph transformer network is employed to explicitly incorporate shortest‑path information and node‑wise distances for capturing richer structural relationships between cells. Finally, we employ an optimal transport strategy to guide the cell clustering in a self‑supervised manner. Extensive experiments on multiple benchmark scRNA‑seq datasets demonstrate that our scGTN consistently outperforms existing methods. Our code is available at https://github.com/W‑RMSL/scGTN.
Authors:Jihyung Park, Minchao Huang, Leqi Liu, Elias Stengel-Eskin
Abstract:
Natural language understanding often depends on meanings that are implied rather than explicitly stated, requiring pragmatic reasoning. Despite strong performance on math and logical reasoning, large language models (LLMs) still struggle with making pragmatic inferences, often choosing literal interpretations. To improve LLM pragmatic reasoning, we introduce PragReST, a self‑supervised framework that constructs pragmatic QA data, generates counterfactual reasoning traces, and trains models to internalize them through supervised fine‑tuning and reinforcement learning, without human‑labeled training data or distillation from a stronger teacher. Across four pragmatic benchmarks (PragMega, Ludwig, MetoQA, and AltPrag), PragReST improves over backbone models, task‑specific pragmatic tuning baselines, and non‑counterfactual variants of the same pipeline. On accuracy‑based benchmarks, PragReST improves over the instruct backbone by 5.37 and 5.50% (absolute) for Qwen3‑8B and Qwen3‑14B, respectively. Our error analysis and ablations underscore the importance of counterfactual reasoning: PragReST primarily reduces errors caused by failures to contrast observed utterances with plausible alternatives, and removing counterfactual reasoning substantially reduces performance. Moreover, our training preserves out‑of‑domain performance on general‑knowledge and mathematical reasoning benchmarks.
Authors:Minsik Oh, Advit Deepak, Sophie Wu, Douwe Kiela, Ekaterina Shutova
Abstract:
It is essential for large language model (LLM) technology to serve many different cultural sub‑communities in a manner that is acceptable to each community. However, research on LLM alignment has so far predominantly focused on predicting a unified response preference of annotators from certain regions. This paper aims to advance the development of alignment models with a more global outlook, that are able to accurately represent the preferences of subcommunities and do not exhibit excessive bias towards any of them. We focus on the development of reward models for this purpose and present a novel reward model training algorithm (SCPO) that can incorporate diverse cultural preferences in a balanced manner. Our method results in performance increases of the minority reward model of up to 7 points over the baseline model across two datasets, PRISM and GlobalOpinionQA, and across 7 countries. SCPO is up to 280% more training data‑efficient than full‑data finetuning of reward models. In addition, we perform analysis of bias by separately evaluating on the preference of subcommunities and show that excessive bias is mitigated via our weighting method. Our code is available at https://github.com/minsik‑ai/Steerable‑Cultural‑Preference
Authors:Patrick Cooper, Alvaro Velasquez
Abstract:
A rule‑based logic solver resolves every instance in our benchmark in under 50 microseconds with 100% accuracy; the best frontier language model reaches 65% at best and drops to 23.5% under rendering‑robust evaluation (worst case over four surface renderings). We introduce DeFAb (Defeasible Abduction Benchmark), a dataset and generation pipeline that converts four decades of publicly funded knowledge bases into formally grounded instances for defeasible abduction: constructing hypotheses that explain anomalies by overriding defaults while preserving unrelated expectations. Because every hypothesis must pass polynomial‑time checks for valid derivation, conservativity, and minimality, DeFAb makes logical rigor the instrument for measuring creativity and theoretical reasoning, scoring the disciplined construction of theory revisions rather than fluent but theory‑destroying prose. The pipeline pairs taxonomic hierarchies (OpenCyc, YAGO, Wikidata) with behavioral property graphs (ConceptNet, UMLS) to produce 372,648+ instances across 33.75M materialized rules from 18 sources, in three levels with polynomial‑time verifiable gold standards. Four frontier models do not reliably internalize defeasible reasoning: rendering‑robust Level 2 accuracy is 7.8‑23.5%; chain‑of‑thought variance (~36 pp) exceeds any inter‑model gap; and a matched contamination control isolates a +19.4 pp Level 3 gap. We further release DeFAb‑Hard (a 235‑instance Level 3 difficulty variant; best model 53.3% vs 100% symbolic) and CONJURE (a kernel‑verified transformative‑creativity variant of 560 Lean 4/Mathlib instances whose gold answers are definitions the proof kernel did not previously contain, judge‑free verifier; a pilot finds zero novel concepts). The same verifier doubles as an exact reward for preference optimization (DPO, RLVR/GRPO). Released under MIT at https://huggingface.co/datasets/PatrickAllenCooper/DeFAb.
Authors:Minh-Loi Nguyen, Xuan-Vu Le, Long-Bao Nguyen, Hoang-Bach Ngo, Trung-Nghia Le
Abstract:
Traditional image captioning methods often struggle to generate comprehensive, context‑rich descriptions, especially for details not directly observable from visual cues. To overcome this, we propose a novel retrieval‑augmented image captioning framework that generates captions with deeper insights, such as object attributes, event context, and underlying significance, by leveraging external knowledge. Our approach features a hierarchical multi‑modal article retrieval mechanism that moves beyond monolithic text entities. This retrieval considers article structure‑aware features, including weighted textual components (e.g., headlines, body sections) and visual placement patterns, alongside multi‑faceted similarity computations (content‑‑visual, visual‑‑visual, and discourse positioning). A subsequent contextual relevance refinement stage further enhances the retrieved information. The retrieved articles then serve as the knowledge base for caption generation: first, a VLM generates a concise image description; second, we segment relevant information from the retrieved articles based on this description; and finally, an LLM utilizes both the description and extracted knowledge to generate a comprehensive, contextually detailed caption. We participated in the ACM Multimedia EVENTA 2025 Challenge and achieved 5th place with an overall score of 0.2824 on the private test set of the OpenEvent‑V1 dataset. Source code is publicly released at https://github.com/mf0212/EVENTA‑Challange.
Authors:Yuyang Zhao, Lian Xu, Hao Miao, Chenxi Liu, Hao Xue
Abstract:
Time series forecasting (TSF) underpins consequential decisions in energy, transportation, finance, and healthcare, yet TSF models are almost universally ranked by a single number (e.g., average error) on clean held‑out data, under the implicit assumption that it predicts deployed reliability. However, real faults are not i.i.d noise but structured events with temporal shape, broken cross‑variable dependencies, regime change coupled with missingness, and causal propagation across a sensing pipeline. Treating TSF robustness as a data‑quality problem, we present TS‑Fault, a benchmark that evaluates forecasting models under explicit, parameterized fault scenarios with controllable semantic difficulty. TS‑Fault organizes recurring failures into four modes along two orthogonal axes (observation‑ vs mechanism‑level; univariate vs multivariate) and injects each fault into the most prediction‑critical window via a unified importance score. This design enables robustness to be tested against the structures models actually rely on, rather than reduced to generic noise sensitivity. We evaluate 21 models across 6 datasets, 4 modes, and 5 difficulty levels under a paired clean/corrupt protocol. The results reveal three findings that contradict common leaderboard intuition: (i) clean‑data accuracy anti‑correlates with robustness; (ii) clean rankings are preserved under observation‑level faults but reshuffled under mechanism‑level faults; and (iii) all catastrophic failures occur under mechanism‑level faults, with foundation models achieving the highest clean‑data accuracy yet exhibiting the greatest fragility. The code is publicly available at https://github.com/Ray‑zyy/TS‑Fault.
Authors:Kecia G. de Moura, Robert Sabourin, Rafael M. O. Cruz
Abstract:
Offline handwritten signature verification aims to distinguish genuine from forged signatures using static images. Since real forgeries are rarely available, negative samples are usually randomly drawn from genuine signatures of other users to create training data. However, this random selection often lacks diversity, increases redundancy, and escalates computational cost, leading to inefficient training. We propose a data‑driven strategy to generate diverse, informative negative samples using prototypical signatures, which are compact, non‑identifiable summaries of genuine signature features. Based on the experiments results, we conclude that (i) prototypical signatures yield more informative negative samples, improving the detection of skilled forgeries; (ii) the proposed approach is backbone‑agnostic, showing robustness across architectures; and (iii) when combined with a primal‑form linear SVM, it serves as an alternative to RBF‑based models while significantly improving scalability and computational efficiency. Implementation of the method is available at https://github.com/kdmoura/proto_hsv.
Authors:Amirhossein Abaskohi, Raymond Li, Gaetano Cimino, Peter West, Giuseppe Carenini, Issam H. Laradji
Abstract:
Retrieval‑augmented generation (RAG) systems depend critically on how documents are chunked and searched. Fine‑grained chunks can improve retrieval precision but expand the search space, increasing latency and cost; larger chunks reduce the number of candidates but make dense similarity less reliable, as the representation for each chunk mixes multiple topics and introduces more semantic noise. This trade‑off becomes especially limiting in deep research tasks, where retrieval must be both fast and precise across large, heterogeneous corpora. We introduce MCompassRAG, a metadata‑guided retrieval framework that uses topic‑level signals as a semantic compass for selecting relevant evidence. Instead of relying only on cosine similarity between queries and noisy chunk embeddings, MCompassRAG enriches chunk representations with topic metadata in the same embedding space and trains a lightweight retriever through LLM‑teacher distillation. At inference time, MCompassRAG performs topic‑aware retrieval without additional LLM calls, improving both efficiency and evidence quality. Across six complex retrieval benchmarks, MCompassRAG improves information efficiency (IE) by 8.24% on average with over 5 times lower latency than the strongest efficient RAG baselines. Code is available on https://github.com/AmirAbaskohi/MCompassRAG.
Authors:Truong Xuan Khanh
Abstract:
Grokking, the delayed jump from memorization to generalization, is usually tied to the weight norm: a smaller norm generalizes sooner. We ask what the norm actually controls. Holding the weight norm fixed by clamping and varying only an output temperature, we slide the grokking delay across its entire norm‑induced range under cross‑entropy; matching the effective logit scale back to baseline recovers about 85% of the delay at two moduli. Across a grid of norms and temperatures the delay collapses onto the logit scale alone (R2 = 0.97), with the norm adding 1‑2% beyond it. The effect is loss‑dependent: under mean‑squared error the logit scale is pinned and the norm acts through a different route. A memorization control, a float64 softmax‑collapse audit, and a no‑LayerNorm transformer point to the same channel. Forking arms from one identical state, the delay follows the held norm value and not the clamp operation, which closes a rescaling‑artifact concern. The proximal variable is the logit scale and the softmax saturation it drives; the weight norm is only an upstream handle. All numbers, tables, and figures reproduce from released code and data.
Authors:Behzad Shomali, Luisa Victor, Tim Selbach, Ali Hamza Bashir, David Berghaus, Joachim Koehler, Mehdi Ali, Markus Frey
Abstract:
Large language models (LLMs) exhibit substantial cross‑lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language‑specific parameters or a shared mechanism that manifests differently by language. We present a cross‑lingual mechanistic analysis of mathematical reasoning in LLMs, enabling us to localize and compare model parameters that support mathematical reasoning across languages. We find that the extracted math‑associated parameters exhibit partial cross‑lingual overlap, with the strongest overlap concentrated in intermediate model layers. We further observe that English consistently produces the largest set of math‑relevant parameters, whereas lower‑resource languages reveal smaller sets of relevant parameters. These results suggest that math‑related behavior in multilingual LLMs is neither fully language‑invariant nor fully language‑specific, but instead exhibits partial cross‑lingual parameter overlap with systematic language‑dependent differences.
Authors:Ziyan Jiang, Li An, Yujian Liu, Jiabao Ji, Qiucheng Wu, Jacob Andreas, Yang Zhang, Shiyu Chang
Abstract:
Computer‑use agents (CUAs) approach human‑level performance on standardised benchmarks but still struggle on long‑horizon tasks and unseen software. Existing skill libraries address this with reusable skills, but represent the skill artifact as text only, despite the visual nature of GUI interaction. We propose VISUALSKILL: a hierarchical multimodal skill, tailored to each target application and organised as a central index over per‑topic files, which the agent consumes through a load_topic MCP tool that fetches the relevant topic's text and figures on demand. We construct each skill with a two‑stage pipeline that combines authored documentation with live‑application UI exploration. On two CUA benchmarks, CUA‑World and OSExpert‑Eval, a Claude Code CLI agent backed by Claude Opus 4.6 reaches an average score of 0.456 with VISUALSKILL, a +15.3 point absolute lift over the no‑skill baseline (0.303). Against a matched text‑only skill that is generated from the same source content and differs from VISUALSKILL only in modality, VISUALSKILL yields a further +8.3 point absolute gain over the matched text‑only skill (0.373 vs. 0.456), providing direct evidence that retaining visual figures in the skill artifact, rather than verbalizing them away, helps the agent both identify UI elements and verify workflow state after each action. Our code is available at https://github.com/XMHZZ2018/VisualSkills.
Authors:Chengwen Liu, Zhe Huang, Jisheng Dang, Hong Peng, Qi Tian, Tat-Seng Chua
Abstract:
Reinforcement learning has improved the reasoning ability of large language models, but applying outcome‑only rewards to video multimodal large language models (Video‑MLLMs) provides limited guidance on which visual evidence should support the answer. Inspired by multisensory integration, where consistent cues can enhance the salience and reliability of perceptual estimates, we introduce Consensus Frame GRPO (CF‑GRPO), a temporal‑annotation‑free process‑level reward framework for evidence‑aware video reasoning. CF‑GRPO constructs a consensus frame prior from intrinsic video cues, including temporal coverage, scene‑transition cues, and query‑conditioned visual relevance. It then computes a model‑side frame‑use score from visual and response representations and optimizes their agreement through the Consensus Frame Reward (CFR). With salience‑aware sparse aggregation and distribution sharpening, CFR provides a high‑contrast reward signal without requiring human temporal annotations. Experiments show that VideoCFR achieves competitive performance across complex video reasoning benchmarks and improves several metrics over representative Video‑MLLM and RL baselines, while the consensus prior provides an interpretable view of the evidence frames emphasized during training. The implementation is available at https://github.com/1Pansy/VideoCFR.
Authors:Hiranya Garbha Kumar, Minhas Kamal, Balakrishnan Prabhakaran
Abstract:
Accurately aligning CAD models to their corresponding objects in indoor RGB‑D scans is a central challenge in 3D semantic reconstruction. The task requires estimating a 9‑Degree‑of‑Freedom (DoF) pose‑position, rotation, and scale along three axes‑but is hindered by noisy and incomplete scans, as well as segmentation errors that cause geometric distortions. We present Completion‑Assisted Object‑CAD Alignment (CAOA), a method that integrates a semantically and contextually aware point cloud completion module with a symmetry‑aware relative pose estimation algorithm, enabling precise alignment of CAD models to scanned objects. Existing completion methods are typically trained and evaluated on synthetic datasets, which often fail to generalize to real‑world scans. To bridge this gap, we introduce a synthetic data generation strategy tailored to indoor scenes, significantly reducing the synthetic‑to‑real domain gap‑validated through quantitative comparisons with widely used completion datasets. In addition, we release S2C‑Completion, an expert‑annotated dataset of over 8,500 object‑CAD pairs from Scan2CAD, created for real‑world indoor single‑object completion and intended as a new benchmark for this task. For object‑CAD alignment, we incorporate symmetry information via a symmetry‑aware loss, improving robustness to symmetric ambiguities. On the Scan2CAD benchmark, CAOA achieves a 17% accuracy improvement over state‑of‑the‑art methods.
Authors:Lanxiang Hu, Zhaoxiang Feng, Yulun Wu, Haoran Yuan, Yujie Zhao, Yu-Yang Qian, Bojun Wang, Daxin Jiang, Yibo Zhu, Tajana Rosing, Hao Zhang
Abstract:
Speculative decoding (SD) accelerates autoregressive Large Language Models (LLMs) by drafting multiple tokens and verifying them in parallel, but it faces a scaling limitation: increasing the draft budget improves speed only when acceptance remains high and drafting overhead stays low. This ceiling has been difficult to break because prior head‑based SD methods face a causality‑efficiency dilemma. Autoregressive drafters produce path‑conditioned candidates that are effective for tree speculative decoding with higher acceptance length, but their drafting cost grows with tree depth. Bidirectional block‑diffusion drafters generate all positions in one pass, but their branch‑agnostic marginals can form individually plausible yet mutually inconsistent trees, wasting budget and reducing acceptance. We propose JetFlow, a head‑based SD framework that combines one‑forward drafting efficiency with branch‑wise causal conditioning. JetFlow trains a causal parallel draft head over fused hidden states from the frozen target model, producing candidate trees whose scores align with the target model's autoregressive factorization. This enables JetFlow to convert larger draft budgets into longer accepted prefixes and higher end‑to‑end speedup. Across math, coding, and chat benchmarks on dense and MoE Qwen3 models, JetFlow consistently outperforms bidirectional‑head and tree‑based SD baselines. On H100 GPUs, JetFlow achieves up to 9.64x speedup on MATH‑500 and 4.58x on open‑ended conversational workloads, with further latency gains demonstrated through vLLM integration under realistic serving loads. Our code and models are available at https://github.com/hao‑ai‑lab/JetFlow.
Authors:Amirhossein Abaskohi, Issam H. Laradji, Peter West, Giuseppe Carenini
Abstract:
Retrieval‑augmented generation (RAG) systems must balance retrieval granularity with contextual coherence, a challenge that existing methods address through LLM‑guided chunking, single‑level context expansion, or hierarchical summarization. These approaches variously depend on costly LLM calls during indexing or retrieval, limit context aggregation to a single granularity level, or introduce information loss through summarization. We present SproutRAG, an attention‑guided hierarchical RAG framework that addresses this trade‑off by organizing sentence‑level chunks into progressively larger but semantically coherent units, using learned inter‑sentence attention to construct a binary chunking tree. Unlike prior approaches that rely on external LLMs, fixed context expansion, or lossy summarization, SproutRAG learns which attention heads and layers best capture semantic document structure, enabling multi‑granularity retrieval without additional LLM calls or compressed summaries. At retrieval time, SproutRAG uses hierarchical beam search to retrieve candidates at multiple granularities, capturing multi‑sentence relevance beyond flat retrieval. The framework is trained end‑to‑end with a joint objective that improves both embeddings and tree structure. Experiments across four benchmarks spanning scientific, legal, and open‑domain settings demonstrate that SproutRAG improves information efficiency (IE) by 6.1% on average over the strongest baseline. Code is available on https://github.com/AmirAbaskohi/SproutRAG.
Authors:Edward T. Stevenson, Mei Ting Mak, Eric Wolf, Denis E. Sergeev, Tobi Hammond, N. J. Mayne, Miles Cranmer
Abstract:
The search for life beyond Earth will depend on detecting faint signatures in the atmospheres of potentially habitable exoplanets. Interpreting those signatures requires understanding the host planet's climate: the same molecule may signal life on one planet and abiotic chemistry on another. Global climate models (GCMs) provide this understanding, but individual runs can require up to millions of core‑hours and substantial domain expert time. Machine‑learning emulators could remove this bottleneck, but progress has been limited by the absence of a curated, multi‑model exoclimate dataset. We introduce ThousandWorlds, an ML‑ready benchmark for exoclimate emulation and for the broader regime of low‑data, multi‑simulator, parameter‑to‑field regression. The dataset contains approximately 1800 simulations from five GCMs, mapping eight planet parameters to 3D atmospheric fields including temperature, humidity, winds, clouds, and radiation. Three nested subsets define progressively harder challenges: single‑simulator regression, multi‑simulator regression with complete observations, and multi‑simulator regression with structured missingness. We propose two evaluation protocols: one for ranking methods, and one that measures performance relative to the disagreement between GCMs themselves. We evaluate seven baselines spanning simple methods, deep learning, and Gaussian processes. GP‑based methods perform best, suggesting that ThousandWorlds exposes a regime where off‑the‑shelf deep learning does not yet succeed. Data: https://doi.org/10.57967/hf/8695. Code: https://github.com/edstevenson/ThousandWorlds.
Authors:Bowen Li, Mayank Mishra, Y. Isabel Liu, Stone Tao, Nishanth Kumar, Alexander G. Gray, Ruwan Wickramarachchi, Jonathan Francis, Sebastian Scherer, Tom Silver
Abstract:
Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn reactive recovery behaviors, training a separate policy for every distinct failure mode is highly inefficient. We introduce Recovery‑Driven Synthesis of Relational Concepts (ReSYNC), the first approach that progressively discovers and refines state abstractions (relational predicates) from failure‑recovery experience to support abstract planning. Unlike purely reactive methods, ReSYNC jointly learns skills and concepts through an incremental dual‑learning process. In the skill‑learning phase, the robot uses RL to learn to recover from failures seen in training tasks. In the concept‑learning phase, the robot discovers new relational predicates and refines its abstract planning model to explain and generalize the learned recovery behaviors. This interaction enables ReSYNC to convert local recoveries seen during training into global failure avoidance at test time. Across four simulated domains, we show that ReSYNC's ability to continually expand and refine its abstraction library allows it to solve long‑horizon, previously unseen problems, outperforming strong baselines by over 50%. Additionally, we demonstrate sim‑to‑real transfer of ReSYNC, where it performs real‑world non‑prehensile manipulation skills and generalizes to unseen scenarios through abstract planning. Overall, ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure‑aware planning in the physical world.
Authors:Mingyue Cui, Linghui Shen, Xingyi Yang
Abstract:
Sparse Autoencoders (SAEs) decompose residual‑stream activations into interpretable features. Recent latent‑space defenses increasingly rely on these decompositions, assuming that identified "unsafe" SAE features serve as actionable handles for monitoring and intervention. In this paradigm, clamping a specific harmful feature is expected to reliably prevent model misbehavior. However, we show that this success may hide a recoverable failure mode: the clamp may block one visible route to a behavior without eliminating the behavior itself. We formulate this vulnerability as post‑intervention recovery, a constrained residual‑space optimization problem. Starting from the post‑intervention residual state, we optimize residual perturbations to recover the pre‑intervention behavior while preserving the post‑intervention values of the targeted SAE features. Even under a strong threat model where the intervention remains active throughout optimization and generation, recovery remains possible. To rule out that recovery simply undoes the intervention, we use encoder‑orthogonal updates for single‑layer interventions and the corresponding feature‑map Jacobian in the cross‑layer setting. Across TPP, unlearning, IOI, and refusal steering experiments, this stress test reveals recoverable behavior despite successful feature‑level intervention. Especially in the safety‑critical refusal‑steering setting, we achieve a 95.8% recovery rate on valid samples while keeping defended‑feature relative drift to 0.131, substantially below suffix‑based baselines. A recovery‑path attribution analysis further localizes this recovery to the SAE reconstruction residual, the component left unexplained by the SAE. These results expose a gap between feature‑level control and behavioral completeness: SAE features can support causal intervention, but controlling them does not guarantee control over the underlying behavior.
Authors:Sidnei Barbieri, Ágney Lopes Roth Ferraz, Lourenço Alves Pereira Júnior
Abstract:
Cybersecurity literature reviews require a reproducible denominator: the set of papers that a protocol includes before screening and synthesis begin. Today, that denominator is often reconstructed from publisher portals, bibliographic indices, and scholarly application programming interfaces (APIs) whose coverage, formats, and query semantics change over time. This paper presents TopVenues, an open‑source system that materializes corpus construction as a versioned research artifact. TopVenues declares a venue and year scope, uses DBLP Computer Science Bibliography (DBLP) as the metadata spine, enriches records with abstracts and BibTeX entries via open scholarly APIs and publisher‑specific extractors, and stores the results in a monotonic SQLite snapshot, accessible via a command‑line interface (CLI), a web interface, and export paths for review workflows. The May 2026 snapshot contains 9,925 papers from 11 cybersecurity sources over 2017 to 2026, with 99.86% abstract coverage and 99.99% BibTeX coverage; keyword search over the full corpus completes in under 31 ms, and a 250‑test suite validates the data‑integrity invariants. The fixed denominator also enables repeatable measurement: 29.2% of 2024 to 2025 papers from the four top‑ranked security conferences in our scope appear as arXiv preprints, with a median of five months before publication, and a prior‑author‑track‑record filter yields a 16.5x precision gain at 90% recall for triaging preprints that later appear in the same venue set. TopVenues links corpus construction to auditable cybersecurity measurement by making the corpus itself executable, inspectable, and citable. The artifact is available at https://github.com/sidneibarbieri/topVenues.
Authors:Vaibhav Balloli, Laura Peyton Ellis, Vishala Mishra, Alice Chi, Alex Peahl, Elizabeth Bondi-Kelly
Abstract:
Social media platforms like TikTok have become a key source of health information, with studies reporting inaccuracies in posts. As Large Language Model (LLM) providers increasingly integrate LLMs into digital platforms to fact‑check content (e.g., Grok and Perplexity on X and WhatsApp, respectively) and are being used by people to fact‑check information, deploying these systems in critical areas such as reproductive health without rigorous evaluation can cause serious harm. We introduce RELIANCE, an expert‑annotated dataset of health information on TikTok surrounding pregnancy and postpartum queries, serving as both an analysis of the reproductive health information landscape and an evaluation of LLMs' capabilities in fact‑checking this content. Our dataset comprises 409 annotated sentences from 336 videos across 56 clinician‑reviewed queries, annotated by three expert clinicians in Obstetrics, Gynecology, and Internal Medicine. Our findings reveal that nearly 60% of the health information in the videos we sampled is accurate. Furthermore, LLM evaluations reveal a gap between evaluating specific claims and evaluating the entire content (15%). We believe that our methodology, dataset, and tool will support the machine learning community in improving LLMs for important domains with real‑world data, extending to other platforms and languages, and helping the health community further understand the information landscape on social media. Our dataset and code are made available at https://realize‑lab.github.io/RELIANCE/.
Authors:Hatim Chergui, Claudia Carballo González, Farhad Rezazadeh, Merouane Debbah
Abstract:
This paper presents an autonomous agentic resource negotiation framework designed to enable zero‑touch network slicing in 6G architectures using Large Language Model (LLM) agents. While LLMs offer powerful reasoning capabilities, we demonstrate that such agents inherently suffer from anchoring bias, rigidly adhering to initial heuristic proposals and causing severe network over‑provisioning. To systematically mitigate this cognitive bias, we propose a novel randomized anchoring strategy modeled via a Truncated 3‑Parameter Weibull distribution. This mathematically bounded approach seamlessly integrates with burst‑aware Digital Twins (DTs) employing Conditional Value at Risk (CVaR) to rigorously guarantee strict Service Level Agreement (SLA) tail‑latencies. To validate our methodology, we introduce and prove the \emphBimodal Constraint‑Avoidance Utility Theorem, demonstrating that while feasible negotiations follow classical convex bounds, highly constrained scenarios undergo a phase transition governed by an inverse rational decay envelope. Empirical results generated using a locally hosted 1B‑parameter model otel‑llm‑1b‑it confirm these dual‑regime bounds. Our cognitive de‑biasing successfully dismantles rigid negotiation patterns, forcing agents into active exploration to safely ride SLA boundaries and boost system energy savings up to 25%. Crucially, the lightweight 1B LLM achieves sub‑second inference latencies (0.95s mean), ensuring our multi‑agent framework is compatible with the operational timescales of the O‑RAN non‑Real‑Time RAN Intelligent Controller (non‑RT RIC)\footnoteOur source code is available for non‑commercial use at https://github.com/HatimChergui.
Authors:Wujian Peng, Lingchen Meng, Yuxuan Cai, Xianwei Zhuang, Yuhuan Yang, Rongyao Fang, Chenfei Wu, Junyang Lin, Zuxuan Wu, Shuai Bai
Abstract:
Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re‑encoding. UniAR adapts a pretrained vision encoder with multi‑level feature fusion and a lookup‑free bitwise quantization scheme, preserving both high‑level semantics and low‑level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel‑bitwise‑prediction to jointly predict spatially grouped, multi‑level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion‑based visual decoder operates on discrete visual tokens to decode high‑fidelity images. Through large‑scale pre‑training, followed by supervised fine‑tuning and reinforcement learning, UniAR achieves state‑of‑the‑art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab‑sii.github.io/uniar‑web.
Authors:Jiye Lee, Yonghun Choi, Jungdam Won
Abstract:
Collaborative human‑object interaction shows dynamic and complex movements that require mutual anticipation and continuous adjustment between participants and the shared object. Modeling such collaborative multi‑human object interaction (MHOI) scenarios requires high‑quality data acquisition as a foundational step; however, this is challenging due to the inherent complexity of MHOI where human‑human and human‑object interactions occur simultaneously. Such complexity leads to noisy MHOI captures characterized by several artifacts: contact misalignment between hands and objects, motion jitter and temporal inconsistencies in the captured sequences, and missing or incomplete finger‑level articulation details. To address these challenges, we present MOCHI (MOtion Enhancement of Collaborative Human‑object Interactions), a two‑stage framework for enhancing noisy MHOI data. Our approach first generates physically plausible hand grasps through optimization from noisy body input, producing grasps that are both physically plausible and semantically consistent with the body pose, where these optimized grasps are extended into complete hand‑object interaction sequences. Consequently, the full‑body motion for all participants are refined through a diffusion‑based noise optimization framework that uses single‑person motion priors. During the optimization process, we introduce optimization objectives to encode human‑object and human‑human interaction information within these single‑person priors. Experimental results demonstrate the effectiveness of our pipeline across diverse MHOI data, either acquired by existing capture methods or synthesized by generative models. We further show robustness of our system across varying numbers of participants and types of interactions, and demonstrate various applications including keyframe‑based MHOI creation and data augmentation through varying object geometries.
Authors:Dongyue Lu, Rong Li, Ao Liang, Lingdong Kong, Wei Yin, Lai Xing Ng, Benoit R. Cottereau, Camille Simon Chane, Wei Tsang Ooi
Abstract:
Event cameras sense the world through asynchronous brightness changes with microsecond latency and high dynamic range, offering motion fidelity far beyond frame‑based sensors and capturing temporal structure that conventional exposures often miss. These properties make events a powerful complement to RGB in autonomous driving, especially under blur, glare, and rapid motion, where frame‑based perception can become unreliable. However, existing event‑aware vision‑language models remain limited to generic perception and do not reveal how event sensing contributes to reasoning and decision‑making across the full driving loop. We present EventDrive, a large‑scale benchmark and model suite that unifies event streams, RGB frames, and language supervision across four core dimensions: Perception, Understanding, Prediction, and Planning, covering captions, structured QA, grounding, motion‑state recognition, trajectory forecasting, and planning tasks. Building on this foundation, EventDrive‑VLM introduces a multi‑horizon event pyramid and a temporal‑horizon mixture‑of‑experts module to adaptively encode and fuse asynchronous and frame‑based information for downstream reasoning. Comprehensive evaluation across diverse tasks shows that event streams provide substantial gains in temporal precision, motion awareness, and robustness, bringing event sensing into the center of driving intelligence.
Authors:Shanda Li, Qiuhong Anna Wei, Jingwu Tang, Valerie Chen, Nihar B Shah, Tim Dettmers, Yiming Yang, Ameet Talwalkar
Abstract:
Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to evaluate whether LLM agents can assist with reproducibility, but they are difficult to scale due to their reliance on substantial manual effort for data curation and evaluation. We introduce ReproRepo, a scalable framework for reproducibility evaluation that leverages human‑raised GitHub issues as naturally occurring supervision on realistic reproduction blockers. We instantiate ReproRepo on 1,149 recent machine learning papers from major conferences and evaluate four frontier model‑agent configurations. Our results show that LLM agents, even without executing code, can identify many real‑world reproducibility problems from paper‑repository pairs: the best agent in our study, namely Codex with GPT‑5.5, surfaces at least one semantically related human‑reported blocker for ~90% of papers in the study. Further analysis shows that agents are particularly effective for surfacing visible failures and identifying the right semantic region, but may still be insufficient in exact localization. ReproRepo can serve as a reusable, scalable framework for future evaluations of LLM agents on real‑world reproducibility auditing. Our code is released at https://github.com/LithiumDA/ReproRepo.
Authors:Joy Bose
Abstract:
We introduce Darshana Graph, a corpus of over 125,000 text records spanning classical Hindu, Buddhist, and Jain philosophical traditions, drawn from public‑domain and openly licensed translations of sources including the Bhagavad Gita, Brahma Sutras, principal Upanishads, the Pali Canon, and core Jain texts. Its distinctive contribution lies in a structurally unique subset of roughly 8,500 Hindu and Jain records in which the same root verse or sutra is aligned across eighteen historical commentators representing five schools of Vedanta and other darshanas, enabling direct comparison of how independent interpretive traditions read identical source material. To our knowledge, no publicly available resource provides comparable cross‑commentator alignment at this scale. We present two analyses built on this corpus. First, a transparent stylometric comparison requiring no machine learning measures argumentative style through scriptural citation density, explicit refutation rate, and sentence complexity. It finds a moderate negative correlation between citation density and refutation rate, a marked increase in refutation rate across three commentators in a related doctrinal lineage, and measurable genre‑level differences within the Pali Canon itself. Second, we describe a constrained large language model pipeline that extracts typed philosophical relationships between concepts using a predefined relation vocabulary and deterministic post‑hoc validation. The resulting graph surfaces cross‑school disagreement patterns while also revealing important extraction limitations, including cases where an independent embedding‑based analysis disagrees with the graph‑derived findings. We release the full corpus, extracted relationship graph, and all source code.
Authors:Byung-Kwan Lee, Ximing Lu, Shizhe Diao, Minki Kang, Saurav Muralidharan, Karan Sapra, Andrew Tao, Pavlo Molchanov, Yejin Choi, Yu-Chiang Frank Wang, Ryo Hachiuma
Abstract:
Knowledge distillation transfers a teacher's competence to a small student but is brittle in the small‑student regime: forcing the student to imitate logits from a much larger teacher concentrates it on the teacher's sharpest modes, hurting generalization on benchmark families beyond the training corpus. Reinforcement learning (RL) avoids logit imitation by training on the student's own rollouts. However, on questions where every rollout fails‑yielding zero advantage and being silently discarded‑injecting a stronger teacher's response into the policy gradient breaks the on‑policy assumption and induces drift. We introduce Zone of Proximal Policy Optimization (ZPPO), inspired by Vygotsky's zone of proximal development, which keeps the teacher inside the prompt rather than the policy gradient. On hard questions, ZPPO constructs two reformulated prompts: a Binary Candidate‑included Question (BCQ) pairs one correct teacher response with one incorrect student response as anonymized candidates the student must discriminate, and a Negative Candidate‑included Question (NCQ) aggregates the student's wrong rollouts into a single prompt to surface their shared failure modes. A prompt replay buffer recirculates each hard question until it either graduates‑the student's mean rollout accuracy on it reaches half‑ or is FIFO‑evicted under finite capacity, amplifying BCQ and NCQ inside the student's current zone of proximal development. On the Qwen3.5 family at four student scales (0.8B‑9B) with a 27B teacher, post‑trained as vision‑language models and evaluated on a 31‑benchmark suite (16 VLM, 10 LLM, 5 Video), ZPPO outperforms off/on‑policy distillation and GRPO, with the largest gains at the smallest scale.
Authors:Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin, Alexander Theus, Thomas Hofmann, Valentina Boeva, T. Konstantin Rusch, Antonio Orvieto
Abstract:
Looped architectures provide an inductive bias toward learning step‑by‑step procedures for tasks that require compositional reasoning. The number of effective layers reached by looping determines the quality of the solution these models find. Like deep architectures, looped architectures are prone to a signal propagation problem induced by depth as the halting decision is postponed. In this paper, we address this signal propagation issue using pre‑norm layers and residual scaling. Building on these architectural modifications, we propose FPRM, a Transformer‑based Fixed‑Point Reasoning Model that uses fixed‑point convergence as an end‑to‑end halting mechanism in a looped architecture. We show that fixed‑point halting allows FPRM to adapt its compute to task difficulty. FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state‑tracking, and ARC‑AGI.
Authors:Lennon J. Shikhman
Abstract:
Learned PDE simulators are increasingly used as low‑cost replacements for expensive numerical solvers, but standard relative L^2 error does not determine whether a learned model behaves as a coherent numerical time propagator. This paper presents a diagnostic software suite for auditing learned PDE simulators as approximate evolution operators. The suite provides architecture‑independent, post hoc diagnostics for relative state error, semigroup consistency, finite‑difference generator discrepancy, energy behavior, integral balance, admissibility constraints, perturbation response, and scaling‑law consistency. The software is designed around a minimal contract: reference trajectories, a learned propagator or saved predictions, equation metadata, and a diagnostic configuration specifying which structures are meaningful for the problem under study. We validate the suite on five benchmark PDE tasks: two‑dimensional incompressible Navier‑Stokes, shallow‑water dynamics, active matter, three‑dimensional compressible Navier‑Stokes, and three‑dimensional magnetohydrodynamics, using FNO, DeepONet, U‑Net, and ResNet‑style surrogate models together with controlled underfit and oversmoothed variants. The validation study shows that relative L^2 error can remain moderate, or even improve, while structural diagnostics deteriorate substantially. The package therefore supports software‑level auditing of learned PDE simulators by reporting an interpretable diagnostic panel rather than collapsing model behavior into a single state‑error score.
Authors:Yifu Luo, Zeyu Chen, Haoyu Wang, Xinhao Hu, Yuxuan Zhang, Zhizhou Sha, Shiwei Liu
Abstract:
On‑policy self‑distillation (OPSD) has proven effective for post‑training large language models (LLMs), yet its application to diffusion LLMs (dLLMs) remains unexplored. Existing OPSD methods are inherently autoregressive‑centric. They inject privileged information via left‑to‑right prefix conditioning with token‑level divergence supervision, a design that fundamentally conflicts with the arbitraryorder generation of dLLMs. We introduce d‑OPSD, the first OPSD framework tailored for dLLMs. Our approach makes two core contributions. First, we reframe self‑teacher construction by using self‑generated answers as suffix conditioning, enabling the student model to learn from "self future‑experience" rather than privileged prefixes. Second, we shift supervision from token‑level to step‑level, aligning training with the iterative denoising process of dLLMs. Experiments across four reasoning benchmarks show that d‑OPSD consistently outperforms RLVR and SFT baselines with superior sample efficiency, requiring only around 10% of the optimization steps by RLVR and opening a promising pathway for dLLM posttraining. The code is available at https://github.com/xingzhejun/d‑OPSD.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Group commit amortizes the fixed cost of a durable log flush across many committing transactions; the release rule ‑ a timer, a batch size, or an adaptive policy ‑ is a classic tuning knob. The textbook theory is open‑loop: for Poisson arrivals the optimal timer is the EOQ square‑root rule, and the wait‑or‑flush decision is ski‑rental 2‑competitive. We ask when that tuning is worth its machinery, and show that in closed‑loop OLTP it usually is not. Real commit arrivals are closed‑loop: a client issues its next transaction only after its last commits, so the arrival rate is induced by the policy's own latency. Modeling this as a closed queueing network, the parameter‑free greedy‑pipelined policy (flush the instant the device is free) self‑clocks to a computable fixed point and is within about 0.1% of the best oracle‑tuned timer at every load. The square‑root rule prescribes waiting T^\star=\sqrt2F_0/λ, but T^\star<F_0 exactly when λ>λ^\star=2/F_0; above this device‑set load threshold the timer collapses onto greedy and tuning is vacuous. The clean theory only bites below λ^\star and in the open‑loop world, where a parameter‑free ski policy still beats a fixed tuned timer under rate shifts. We instantiate λ^\star with measured fsync distributions on two AWS storage classes (EBS gp3 versus instance NVMe, a 25× range), and confirm on PostgreSQL that commit_delay=0 is competitive with any tuned value. The contribution is a characterization that explains deployed practice; we add no new logger.
Authors:Gbenga T. Awojinrin, Abdul-Akeem Olawoyin, Rami M. Younis
Abstract:
We present a numerical method for the forward solution of nonlinear partial differential equations (PDEs) in which Bellman‑Kalaba quasilinearization reduces the nonlinear problem to a sequence of linear subproblems, each discretized by collocation onto a trial space that is linear in its parameters and solved by a single direct linear least‑squares QR factorization. The trial space, which we term Linear‑in‑Learnables (LiL), comprises representations whose trainable parameters enter linearly, including random‑feature extreme learning machines, spectral polynomial bases, and trigonometric expansions, each implemented as a physics‑informed neural network. The method thus replaces the nonconvex gradient‑based training that limits standard PINNs with a convex per‑step solve. We establish local Newton‑Kantorovich convergence of the outer iteration to a residual‑limited neighborhood under an explicit smallness condition, with the limiting accuracy governed by the best‑approximation residual of the trial space rather than by an optimization tolerance. The method, denoted LiL‑Q, is assessed on seven benchmarks spanning scalar nonlinear PDEs (Bratu, viscous Burgers, Buckley‑Leverett), coupled systems (plane‑strain elasticity and the incompressible Navier‑Stokes equations in two and three spatial dimensions), and steady‑state Darcy flow with heterogeneous permeability. Across these problems, LiL‑Q converges in single‑digit outer iterations in most cases, even at the coarsest basis sizes and independent of the parameter count. When the exact solution lies in the span of the trial space, the method recovers it to machine precision in a single solve. On the Navier‑Stokes benchmarks, it matches or exceeds published PINN solvers with up to two orders of magnitude fewer trainable parameters, without gradient‑based optimization.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Log‑structured merge (LSM) trees attach an approximate‑membership filter to every run and must split a fixed memory budget across them. The static optimum is known (Monkey); a large systems literature then makes the allocation adaptive, tracking shifting hotness online. We ask a prior question: when is that adaptivity worth its machinery? We give three analytical answers and validate them on synthetic sweeps, real Twitter production cache traces, and a real RocksDB engine. First, a log‑law: optimal bits‑per‑key is affine in the logarithm of access frequency, at a fixed slope. Second, a robustness law: because the workload enters only logarithmically, the excess read cost from a hotness misestimate is half the size‑weighted variance of the log error, and a common‑factor misestimate is absorbed by the budget multiplier, so coarse estimates lose little. Third, an adaptivity‑value frontier: since compaction rebuilds filters for free on its own clock, the value of continuous tracking over an allocation recomputed only at compaction grows quadratically in the within‑epoch drift, with a closed‑form scale. This yields a three‑regime policy (coarse‑at‑compaction suffices, then track, then at extreme drift fall back to uniform) and predicts that more skew makes fine tracking matter less. On a real cluster, reallocating only at compaction captures 96‑99% of tracking's benefit; on RocksDB the false‑positive primitive holds within four percent to eight bits per key. The contribution is a characterization of when adaptive tuning pays; we add no new filter and no engine fork. Code and pre‑registration are public.
Authors:Wanhao Niu, Qiyan Ke, Yuan Sun, Hao Sun, Jie Xu, Muyuan Ma, Ruiqi Hu, Fuchun Sun
Abstract:
Cross‑end‑effector grasp generation seeks a unified model that generalizes across objects and across embodiments ranging from parallel grippers to dexterous end effectors. Existing grasp generators are typically designed for a fixed embodiment or encode embodiment identity with a static descriptor, which weakens transfer when topology, actuation coupling, and contact geometry differ substantially. We present EAGG, an embodiment‑aligned grasp generator that represents each embodiment with a topology‑aware end‑effector graph and an embodiment‑specific low‑dimensional end‑effector control space. A frozen end‑effector‑cognition backbone converts the current articulated state into geometry‑aware tokens that act as a reusable morphology prior, and iterative geometry injection refreshes these tokens throughout sampling so that conditioning remains synchronized with the evolving end‑effector geometry. On the MultiGripperGrasp benchmark, EAGG reaches 56.17% average success across six training end effectors, remaining within 1.10 percentage points of specialized training while preserving transfer to finetuning and zero‑shot end effectors. Iterative geometry injection further reduces the pooled median contact distance from 0.239 cm to 0.189 cm. These results show that cross‑end‑effector grasp generation is strengthened by aligning embodiment structure inside a shared generator rather than suppressing embodiment differences. Code is available at https://github.com/wanhaoniu/EAGG.
Authors:Alphan Sahin
Abstract:
In this paper, we propose a general‑purpose multi‑dimensional symbol construction for computing an arbitrary symmetric function with digital over‑the‑air computation (OAC) and discuss the practical aspects of coherent aggregation. For our first contribution, we discuss the categorical representation of a symmetric function. By using this representation and leveraging the sufficiency of the histogram to evaluate a symmetric function, i.e., inspired by type‑based multiple access (TBMA), we introduce a general approach to design a single set of OAC symbols to compute any digital function. For our second contribution, we use a comprehensive platform based on low‑cost nodes that maintain synchronization in time, frequency, phase, and amplitude via a trigger mechanism, enabling coherent OAC experiments without Global Positioning System (GPS) or cable‑based synchronization. Using measurements from the platform, we characterize the phase and amplitude statistics of the composite channel to derive a realistic impairment model for coherent OAC. Through a comprehensive analysis, we demonstrate the effectiveness of the proposed scheme under impairments captured by the proposed model
Authors:Nathan Ouyang, Kexin Wan, Anna Seigal
Abstract:
Causal discovery seeks to uncover the causal dependencies among variables. For this purpose, we propose an algorithm called Tensor‑based Second‑order Causal Discovery (TSCD). Its input is a tensor obtained from the covariance matrices of observational and interventional data. Assuming the causal dependencies follow a linear structural equation model on a directed acyclic graph (DAG), TSCD outputs the DAG and the functions on its edges, requiring only that the noise variables are uncorrelated. We also implement a version of the approach for nonlinear models. Our focus on second‑order statistics (via the covariance matrices) is motivated by their statistical and computational efficiency relative to higher‑order moments, their identifiability relative to first‑order statistics, and that they work regardless of whether the variables are Gaussian. We show that TSCD has identifiable causal order and parameters from a number of interventions that is logarithmic in the number of variables. Experiments show that TSCD is robust to noise, competitive with existing methods, and scales to hundreds of variables.
Authors:Shayan Dodge, Alessandro Formisano, Sami Barmada
Abstract:
We propose a new weak‑form Physics‑Informed Neural Network approach (named INI‑VPINN). INI‑VPINN naturally incorporates Neumann boundary and interface conditions into the variational formulation. It removes the need for additional loss terms or multiple subdomain networks. This framework employs compact support weighting functions and integration by parts to implicitly impose flux and continuity constraints. In this way, it implicitly ensures physical consistency across material boundaries. The proposed method is tested on Poisson and Laplace problems with sharp interfaces and complex geometries. Results show that, compared with several other Physics Informed Neural Networks‑based formulations, the INI‑VPINN consistently achieves higher accuracy, smoother and faster convergence. The proposed framework provides a general approach for solving multimaterial problems with complex geometries and mixed Neumann‑Dirichlet boundary conditions using neural networks. The implementation is publicly available in a GitHub repository.
Authors:Eric V. Strobl
Abstract:
Constraint‑based causal discovery relies on repeated conditional independence tests, but fast nonparametric tests often sacrifice calibration, especially when variables depend on the conditioning set through nonlinear relationships. We introduce BLITZ (Broad‑to‑Local Independence Testing via residualiZation), a nonparametric conditional independence test designed to run well under a second while maintaining the accuracy needed for the thousands of queries performed by constraint‑based causal discovery algorithms. BLITZ first removes broad smooth dependence on the conditioning set using low‑order polynomial regression, then applies a small nonlinear feature map and residualizes those features with shallow tree regressions. The resulting statistic tests residual cross‑covariance, with a moment‑matched chi‑square approximation to the null distribution. We show theoretically that the two‑stage design reduces the effective complexity faced by the tree residualizers, allowing shallow trees to control residual conditional‑mean bias while avoiding excessive overfitting. In simulations, BLITZ provides better null calibration than fast kernel, random‑feature, and regression‑based competitors while remaining among the fastest methods tested. In causal discovery experiments on synthetic graphs and flow‑cytometry data, BLITZ yields more reliable endpoint orientations among retained adjacencies and competitive structural recovery. These results suggest that broad‑to‑local residualization is a practical route to calibrated, scalable nonparametric conditional independence testing for causal discovery.
Authors:Zihan Gu, Ruoyu Chen, Junchi Zhang, Li Liu, Xiaochun Cao, Hua Zhang
Abstract:
Visual attribution is a fundamental tool for interpreting modern vision and vision‑language models, particularly when their decisions must be inspected, diagnosed, or audited. Its goal is to explain how a model's decision depends on local regions of the visual input, typically by assigning an importance ordering over candidate image regions. Given an image partitioned into n regions, faithful attribution can be cast as an ordered subset‑search problem, in which progressively inserting the selected regions should recover the target model response as early as possible. Exhaustive search over region subsets incurs exponential cost, while the widely used greedy search still requires a quadratic number of model evaluations, because every selection step rescores all remaining candidates. We propose PhaseWin, an efficient subset‑search algorithm for faithful visual attribution. PhaseWin reorganizes greedy region selection into a phased window‑search procedure: rather than re‑evaluating the full candidate set at every step, it alternates between global candidate screening, adaptive pruning, and localized window refinement, while preserving the essential region‑ranking behavior of greedy search. We analyze PhaseWin under monotone evidence‑accumulation conditions and show that, under feature‑level structural assumptions, it attains controllable linear evaluation complexity together with near‑greedy faithfulness guarantees. Extensive experiments on image classification, object detection, visual grounding, and image captioning show that, among all compared attribution methods, PhaseWin reaches high faithfulness with the fewest forward passes, empirically realizing the predicted reduction from O(n^2) to O(n). The code is available at https://github.com/Qihuai27/phasewin‑va.
Authors:Chunyu Liu, Zhengyang Fan, Kaisen Yang, Alex Lamb
Abstract:
MDLMs generate text by denoising a preallocated masked response canvas, making response‑length modeling central to instruction tuning. Existing MDLMs often inherit the autoregressive convention of using repeated \texttt[EOS] tokens for padding during instruction tuning, giving \texttt[EOS] a dual role as both a semantic terminator and a padding token. We show that this dual role is a root cause of \texttt[EOS] overflow under large‑block decoding. To decouple these roles, we propose VoidPadding, which introduces \texttt[VOID] for padding and reserves \texttt[EOS] for termination. During inference, the learned \texttt[EOS] signal enables early stopping, while the learned \texttt[VOID] signal guides adaptive response canvas expansion. On Dream‑7B‑Instruct, VoidPadding improves the block‑size‑averaged four‑task mean across mathematical reasoning and code generation benchmarks by \(+17.84\) points over the original model and \(+6.95\) points over RainbowPadding, while reducing decoding NFE by 55.7% on average. Code is available at https://github.com/Haru‑LCY/VoidPadding.
Authors:Yonghao Chen, Sicheng Yang, Rui Tang, Lei Zhu
Abstract:
Multi‑contrast magnetic resonance imaging (MRI) provides complementary information for clinical diagnosis. However, acquiring all MRI sequences is often time‑consuming and costly. Recent generative models perform cross‑contrast synthesis to address this issue by inferring absent contrasts from the available ones. Nevertheless, synthesizing 3D MRI presents significant challenges. Due to the massive volume sizes, operating directly in the pixel space is computationally prohibitive; therefore, a common approach is to first compress the 3D volumes into a latent space and subsequently train generative models in that space. We observe that existing compression architectures face several critical issues: they under‑preserve long‑range anatomical coherence, discard clinically meaningful semantics, and rely on optimization objectives that lead to over‑smoothed reconstructions. Ultimately, these shortcomings compromise the performance of subsequent generative models. In this work, we propose a semantics‑first latent modeling framework for 3D MRI reconstruction and cross‑contrast synthesis. Specifically, we introduce a Latent Harmonization Encoder (LHE) to capture global anatomical dependencies, ensuring coherent volumetric representations. To mitigate semantic degradation during latent compression, we further design a Semantic Recovery Block (SRB) that injects high‑level priors from a self‑supervised semantic teacher, enhancing contrast‑aware separability in the latent space. Additionally, we propose an Anatomy‑aware Frequency Loss (AFL) to adaptively preserve diagnostically relevant high‑frequency structures. Extensive experiments on two public multi‑contrast MRI datasets demonstrate consistent improvements in reconstruction fidelity and cross‑contrast synthesis quality. Our code is available at https://github.com/script‑Yang/RSF.
Authors:Sicheng Yang, Hongqiu Wang, Zhaohu Xing, Sixiang Chen, Qiuxia Yang, Yize Mao, Guang Yang, Lei Zhu
Abstract:
Self‑supervised DINO models provide strong transferable visual representations, yet applying them directly to image segmentation remains challenging. Existing approaches commonly rely on heavy decoders with complex upsampling, introducing substantial parameter and computational overhead. We observe that introducing scale into DINO features is far more critical than increasing decoder capacity. In this work, we present SegDINO, an efficient segmentation framework that integrates a DINOv3 backbone with lightweight scale modeling. SegDINO introduces Token Pyramid Adaptation (TPA) to reorganize intermediate DINO features into a pseudo multi‑scale hierarchy, and Scale‑Aware Decoding (SAD) for efficient intra‑scale refinement and top‑down multi‑scale propagation. We further curate PanCT, a new CT dataset containing 284 patients with expert‑annotated pancreatic tumors, to assess SegDINO's ability to handle difficult small‑lesion cases. Extensive experiments on PanCT and three public benchmarks demonstrate that SegDINO achieves state‑of‑the‑art results with high efficiency. The code is available at https://github.com/script‑Yang/segdino_v2.
Authors:Yuming Chen, Yuxin Xie, Tao Zhou, Yi Zhou
Abstract:
Semi‑supervised medical image segmentation has emerged as a dominant research problem in medical image analysis, mitigating annotation scarcity by leveraging consistency regularization on unlabeled data. However, existing approaches operate predominantly via visual pattern matching, relying heavily on pixel‑level similarities. This visual‑centric dependency often falters in clinical scenarios characterized by the visual‑semantic mismatch, where visually similar lesions warrant distinct diagnostic conclusions, thus failing to capture the underlying diagnostic logic used by experts. To address this, we move beyond visual cues and propose CERS (CoT‑Enhanced Reasoning Segmentation), a framework that integrates Chain‑of‑Thought (CoT) reasoning to distinguish pathologically distinct cases. Specifically, we construct a knowledge pool enriched with linguistic reasoning descriptions generated by large language models (LLMs). A semantic‑aware reference selection strategy is introduced to identify historical evidence, filtering candidates first by morphology, and then refining them via CoT consistency to eliminate hard negatives. Furthermore, a multi‑scale coordinate attention module (MCAM) is designed to effectively fuse this reasoning‑derived context into the decoding process. Extensive experiments demonstrate the superiority of CERS against state‑of‑the‑art approaches, particularly in resolving boundary ambiguities and semantic inconsistencies. The code is available at https://github.com/cymasuna/CERS.
Authors:Wei Da, Evangelia Kalyvianaki
Abstract:
Heterogeneous LLM serving stacks split scheduling into two layers that optimize in isolation: model routers pick a model from quality and cost signals while ignoring instance load, and serving load balancers optimize queues while ignoring quality. We present RouteBalance, a serving‑aware scheduling layer that fuses both into a single online assignment over concrete model instances, jointly trading off quality, latency, and cost. A batched in‑process predictor stack and dead‑reckoned instance state keep the joint decision cheap on the request hot path (\approx32 ms at 12 req/s). On a 13‑instance, 28‑GPU heterogeneous cluster serving four model sizes, a single deployed RouteBalance stack traces the upper region of the three‑way quality‑cost‑throughput frontier. Sweeping one weight vector reaches both the highest routing‑decision quality (DeepEval 0.419, +0.013 over the strongest baseline, 95% CI [+0.005,+0.022]; the ordering holds when a second judge re‑scores the actually served text) and, at its cost‑priority corner, per‑request cost that ties the cheapest baseline. With router engineering equalized against concurrent‑scoring baseline variants we build, its balanced preset serves at 2.8 s and 30 req/s, leading 2.6 to 4.1× ahead of enhanced BEST‑Route at high load. (Deploying those routers as published, one serial scoring call per request, makes them collapse 23× under load, a deployment‑architecture effect we isolate separately, not the routing result.) A four‑arm isolation shows the benefit follows from pricing latency at model‑selection time; the learned predictors contribute calibration and SLO headroom rather than the headline frontier. Code: https://github.com/AKafakA/route‑balance
Authors:Dominic Ebner, Markus Lienkamp
Abstract:
In autonomous racing, fast detection of other participants' movements is required to plan safe, collision‑free trajectories with non‑cooperative opponents. LiDAR detection is inherently slower and harder to deploy on edge devices than vision methods, causing delayed detections that limit object tracking performance during high‑dynamic maneuvering. Utilizing monocular 3D detection enables an easy‑to‑deploy, low‑latency detection of other participants on the racetrack. We present SPARK, a single‑camera pose‑estimation algorithm for autonomous racing using keypoint detection. It achieves long‑range detection with high accuracy, exceeding the performance of state‑of‑the‑art monocular camera detection algorithms while maintaining lower latency. By employing well‑optimized YOLO models and leveraging the fixed geometry in the autonomous racing domain, the algorithm also exhibits low latency and resource usage. We evaluate the performance of our approach on real‑world autonomous racing data and compare it to state‑of‑the‑art LiDAR and camera detection algorithms. The source code is available at: https://github.com/TUMFTM/SPARK‑camera‑det
Authors:Guo Pu, Yixuan Han, Haofeng Li, Yao Zhang, Hui Zhou, Zhouhui Lian
Abstract:
Online 3D reconstruction from monocular image sequences is a challenging and ongoing research topic. 3D Gaussian Splatting (3DGS), leveraging its high‑quality real‑time rendering capability, empowers online 3D reconstruction to represent dense scenes with enhanced expressiveness, and thus holds great promise for a wide range of applications such as robotics and AR/VR. However, existing online 3DGS methods still suffer from some key challenges: fragile camera pose estimation due to the lack of global optimization, and low optimization efficiency in large‑scale or long‑sequence scenarios. To address these issues, we propose a robust and efficient online voxelized 3DGS reconstruction framework integrated with global \textSim(3) optimization, which enables reliable camera tracking and efficient global loop closure for both camera poses and voxelized 3DGS. To accelerate the convergence of the voxelized 3DGS, we further introduce a color residual learning strategy, which not only boosts optimization speed but also enhances rendering quality. Extensive experiments on diverse indoor and outdoor datasets demonstrate that our method achieves state‑of‑the‑art performance in both camera pose estimation accuracy and rendering quality, while retaining real‑time efficiency. Additionally, we develop and deploy a real‑world UAV‑based active reconstruction system grounded on our proposed method, validating its robustness and generalizability for practical online 3D reconstruction tasks. Our code and data are available at https://github.com/TrickyGo/MoonSplat.
Authors:Antonio Scardace, Daniele Ravì
Abstract:
Despite increasing adoption of multimodal approaches in Alzheimer's Disease (AD) research ‑‑ aimed at integrating molecular, structural, clinical, and genetic biomarkers to enhance disease characterization ‑‑ the relationships among these modalities remain poorly understood. A systematic analysis of their dynamic interaction is essential for improving disease modeling, identifying redundant assessments, and reducing patient burden and acquisition costs. In this paper, we present a quantitative analysis of multimodal AD biomarkers by integrating tau‑PET, structural MRI, cognitive scores (MMSE and CDR), and APOE4 data from 789 subjects drawn from the ADNI dataset. In our analyses, we (A) quantify cross‑modal mutual information and explained variance to assess redundancy and predictive dependencies; (B) examine associations between tau topologies and structural atrophy across brain regions to select informative ROIs; (C) perform a statistical decomposition of the tau‑cognition association into atrophy‑related and atrophy‑independent components; (D) and identify a dominant neurodegenerative trajectory that aligns with cognitive decline. This study provides a systematic characterization of cross‑modal relationships, improving the interpretability and selection of biomarkers in AD. Code is publicly available at: https://github.com/antonioscardace/Multimodal‑AD.
Authors:Tongxu Luo, Rongsheng Wang, Jiaxi Bi, Chenming Xu, Zhengyang Tang, Jianlong Chen, Juhao Liang, Ke Ji, Shuqi Guo, Yuhao Du, Fan Bu, Wenyu Du, Xiaotong Zhang, Kyle Li, Shaobo Wang, Linfeng Zhang, Yuxuan Liu, Xin Lai, Chenxin Li, Yiduo Guo, Zhexin Zhang, Xinyuan Wang, Tianyi Bai, Ziniu Li, Benyou Wang
Abstract:
Game generation is an emerging application of coding agents, requiring models to transform natural‑language specifications into playable interactive systems. Unlike traditional coding tasks, game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay. We formalize end‑to‑end game generation as the problem of producing a complete game artifact that realizes a specification through observable player‑game interaction in a target environment. We argue that evaluating this setting requires three desiderata: Engine Grounding, Artifact Completeness, and Interactive Verification. We propose an interaction‑grounded evaluation framework that assesses executable gameplay through replayed demonstrations and rubric‑guided multimodal judging. We instantiate this framework as GameCraft‑Bench, a benchmark comprising 140 Godot tasks across 15 game families. Evaluations of frontier coding agents show that end‑to‑end game generation remains highly challenging: the strongest agent achieves only 41.46%, and most agents score below 40%. Further analysis reveals that while agents often implement recognizable mechanics, they struggle to deliver complete games with sufficient content, functional visual feedback, and coherent presentation. See https://tongxuluo.github.io/gamecraft‑bench‑website for demos, code, and data.
Authors:Toshitaka Hayashi, Hamido Fujita, Dalibor Cimr, Richard Cimler, Jitka Kühnová
Abstract:
Machine Learning (ML) techniques have been applied to various problems. However, applying ML to ML models is an unexplored direction. For this purpose, this paper considers a meta‑classification of one‑class classification (OCC) models, because all ML models could be approximated as OCC models. The proposal represents OCC models as normality rankings and classifies them using nearest‑neighbor and ranking‑correlation metrics. The experiment classifies OCC models, where classes correspond to training datasets, algorithms, and hyperparameters. The proposal achieves high accuracy when class labels are datasets. Moreover, it can classify algorithms when the training datasets contain the same class. In addition, the discussion highlights that the classification of OCC models is essentially the classification of datasets that treats multiple samples as a single input. The experiment demonstrates the classification of datasets using sleeping records. The proposed method can provide a unified solution for classifying OCC models, datasets, and rankings. Source code is uploaded to the public repository https://github.com/ToshiHayashi/ClassOCC.
Authors:Fernando Zhapa-Camacho, Robert Hoehndorf
Abstract:
A wide range of neurosymbolic (NeSy) systems compute one functional: a belief‑weighted sum of a logical quantity over a space of σ‑structures, of which weighted model counting, fuzzy logic, and probabilistic logic are special cases. This account is built on sets, and a set deliberately forgets two things that are important for NeSy: when two σ‑structures are the same up to a symmetry of the theory, and how many distinct proofs witness a query. Replacing the underlying sets by types, in the sense of homotopy type theory, preserves this information, and turns this functional into a belief‑weighted homotopy cardinality, a notion of size that counts each object in inverse proportion to its symmetries. We develop the framework from scratch for NeSy systems, prove a conservativity theorem that recovers the classical functional when symmetries are trivial, and show that the symmetry our framework exposes is exactly the one behind reasoning shortcuts. The payoff is concrete: the shortcut‑aware concept posterior that recent methods reach by ensembling or expressive density estimation is the only symmetry‑invariant point of the confusion‑set simplex, computable in closed form by averaging a single model over the symmetry group. On MNIST reasoning‑shortcut benchmarks this single‑model wrapper is better calibrated than a diversity‑trained ensemble, while leaving label accuracy and identifiable concepts untouched. Code is freely available at https://github.com/bio‑ontology‑research‑group/hott‑nesy.
Authors:Chenghan Wang, Zhen Zhuang, Shui Jiang, Siyuan Liang, Xiaoman Yang, Kai Zhu, Darong Huang, Luis Costero, Rongmei Chen, Tsung-Wei Huang, David Atienza, Tsung-Yi Ho
Abstract:
Coarse‑grained thermal simulation tends to underestimate localized thermal issues, potentially missing critical hotspots. Accurate analysis, therefore, demands fine‑grained information, which dramatically increases grid resolution and thus computational workload. Fortunately, the coefficient matrices are often sparse with regular sparsity patterns, offering optimization opportunities. However, existing general‑purpose matrix solvers on GPUs rarely exploit these domain‑specific properties, thereby encountering bottlenecks in data storage, memory access, parallelism, computational efficiency, and hardware utilization. Therefore, we propose CUTh‑Solver, a co‑designed GPU‑accelerated Preconditioned Conjugate Gradient (PCG)‑based sparse solver framework for Symmetric Positive Definite (SPD) systems arising from high‑resolution steady‑state and transient 3D IC thermal simulation. For data storage, CUTh‑Solver condenses the Diagonal (DIA) storage format to remove redundancy. To optimize the memory access, CUTh‑Solver employs diagonal‑wise SpMV to achieve coalesced memory access. We further observe a critical conflict between parallelism and preconditioning quality and thus adopt a high‑parallelism preconditioning strategy. To improve computational efficiency and hardware utilization, we employ an adaptive fine‑grained mixed‑precision strategy that leverages diverse floating‑point units to avoid resource contention, enhancing throughput without compromising numerical stability. Experimental results show that CUTh‑Solver achieves up to 25.8x speedup over GPU‑accelerated COMSOL Multiphysics 6.4 and over 3x speedup over NVIDIA's native general‑purpose libraries (AmgX, cuSPARSE, cuDSS). Ablation studies validate the individual contribution of each optimization. The code is available at: https://github.com/Chenghan‑Wang/CUTh‑Solver
Authors:Sripath Mishra, Bharat Bhargava, Zizheng Liu, Shafkat Islam
Abstract:
Intrusion detection systems (IDS) trained on wired‑network benchmarks degrade sharply in real‑world unmanned aerial vehicle (UAV) swarms, where mobility, fluctuating link quality, and decentralized routing reshape traffic distributions. Existing UAV‑specific datasets also do not systematically vary these conditions, leaving no way to train or test an IDS against the very shift that defeats it. We present UAV‑CAS, a large‑scale labeled flow dataset for UAV‑network intrusion detection, generated by a Containernet digital twin that is systematically calibrated against AERPAW testbed measurements. We have a four‑layer calibration pipeline spanning altitude‑dependent path loss, mission‑specific mobility, the link‑level performance chain, and end‑to‑end trace fidelity. UAV‑CAS comprises 99,492 flows drawn from 1,024 configurations that span five attack families (DoS, DDoS, blackhole, wormhole, replay) and nine collaborative attack compositions. A diversity analysis shows that high‑rate attacks separate from benign traffic up to an order of magnitude more strongly than in any prior benchmark, while stealth attacks deliberately blend with benign traffic. Across ten baseline IDS, binary attack detection saturates above 0.98, confirming the dataset is learnable, whereas full attack‑class identification remains hard ‑‑ per‑class F_1 ranges from near zero to 0.82 and falls into the single digits for stealth attacks. We release the dataset, simulator, and calibration data to support reproducible UAV intrusion‑detection research.
Authors:Jean Seo, Minkyu Kim, Jeonguk Lee, Jisoo Jung, Wooseok Han, Eunho Yang
Abstract:
Automatic speech recognition (ASR) in non‑English clinical settings is challenged by multiscript variability, where the same term may appear in multiple valid orthographic forms. Conventional string‑matching evaluation metrics often underestimate ASR performance by treating orthographic variants as errors. To address this issue, we introduce MultiClin, a clinical ASR benchmark designed to evaluate robustness to multiscript variability. Experiments across diverse ASR models show that multiscript‑aware evaluation provides a fairer assessment of recognition quality than conventional single‑reference evaluation. We further investigate the impact of script consistency during training and find that inconsistent script mappings increase orthographic uncertainty and hinder model convergence, with a balanced 50% mapping ratio producing the highest entropy. In contrast, script unification consistently yields the best ASR performance. Our dataset and code are publicly available at: https://github.com/aitrics‑ronaldo/Interspeech_MultiClin.
Authors:Zhenyu Yang, Kairui Zhang, Bing Wang, Shengsheng Qian, Changsheng Xu
Abstract:
Despite the remarkable progress of Video Large Language Models (Video‑LLMs), current online architectures still struggle to simultaneously process continuous video streams, decide autonomously when to respond, and preserve long‑horizon contextual memory. These obstacles undermine real‑time responsiveness and cause severe forgetting throughout prolonged interactions. In this work, we introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long‑horizon streams. The design of LiveStarPro rests on three complementary components. The first component is Streaming Verification Decoding (SVeD), an inference framework that identifies the appropriate response timing through single‑pass perplexity verification, thereby eliminating the dependency on explicit silence tokens. The second component is Streaming Causal Attention Masks (SCAM), a training strategy that enforces incremental video‑language alignment over variable‑length streams. The third component is Tree‑Structured Hierarchical Memory (TSHM), a recursive memory architecture that organizes evicted historical information into event chains and consequently enables efficient retrieval from effectively unbounded video streams. To facilitate a comprehensive evaluation under realistic online conditions, we further present OmniStarPro, a large‑scale benchmark that spans 15 diverse real‑world scenarios and that extends to hour‑scale streams for the assessment of long‑term recall. Extensive experiments demonstrate that LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness and an 18.2% reduction in timing error, while its streaming key‑value cache further yields a 1.58x inference speedup over the same model without caching. The model and the code are publicly available at https://github.com/sotayang/LiveStarPro.
Authors:Zhexiao Xiong, Yizhi Song, Hao Kang, Qing Yan, Liming Jiang, Jenson Yang, Zhoujie Fu, Stathi Fotiadis, Angtian Wang, Zichuan Liu, Bo Liu, Yiding Yang, Xin Lu, Nathan Jacobs
Abstract:
Interactive world models aim to simulate environment dynamics under real‑time user actions. However, their action vocabulary is largely confined to navigation: most actions correspond to motion (e.g., walk, turn, look around), while interaction with objects in the scene (e.g., pick up plates, open doors, or trigger physical responses) is either absent, restricted to game domains, or relegated to prompt‑to‑full‑video scenarios. The resulting worlds are visually explorable but not truly actionable. In this work, we present ActWorld, an interactive world model that extends prior navigation‑centric generators to support mid‑rollout object interaction within a chunk‑autoregressive framework. We argue that the navigation‑interaction gap stems from two bottlenecks. First, a data bottleneck: the lack of human‑object interaction data with accurate, dense labels. Second, a memory bottleneck: recency‑biased history compression in existing world models discards the event‑transition frames that causally determine subsequent object states, leading to an action‑forgetting pathology. On the data side, we construct a 100K interaction video dataset, each annotated with per‑chunk captions via chain‑of‑thought reasoning. On the model side, we introduce a hierarchical action‑aware memory design that routes history compression by interaction importance, complemented by a persistent memory bank that maintains event‑update and object‑identity tokens across long rollouts. Experiments show that ActWorld supports both flexible navigation and rich object interaction within a single model, substantially improving interaction fidelity over navigation‑only baselines without sacrificing viewpoint control. Project page is available at https://interactwm.github.io/ActWorld.
Authors:Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang
Abstract:
Recent vision‑language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single‑screen, and largely static webpages. We introduce LongWebBench, a benchmark for evaluating long‑horizon webpage generation from both structural and functional perspectives. LongWebBench contains 490 real‑world long webpages for structural fidelity evaluation and 507 goal‑oriented interaction tasks over 129 webpages for functional evaluation. It employs two complementary protocols: a multi‑dimensional VLM‑based metric for assessing long‑range structural coherence, and a DOM‑augmented agent‑based pipeline for end‑to‑end functional verification. We further examine the automatic evaluation protocols through human agreement analysis. Experiments with state‑of‑the‑art open‑source and proprietary VLMs under single‑image and multi‑image settings reveal that structural fidelity degrades as webpage length increases, while visually plausible generations often fail to support executable multi‑step interactions. These results highlight the need to evaluate long webpage generation beyond visual similarity, with executable interaction as a core criterion. Our code and data are available at https://github.com/zheny2751‑dotcom/LongWebBench.
Authors:Jiangong Xu, Weibao Xue, Xiaoyu Yu, Jun Pan, Xinlian Lianga, Mi Wang
Abstract:
Optical remote sensing imagery is frequently degraded by cloud and cloud‑shadow contamination, which limits its reliability for near‑real‑time land use and land cover (LULC) mapping. Although synthetic aperture radar (SAR) can provide cloud‑penetrating structural information, existing SAR‑optical fusion methods often assume reliable optical observations and insufficiently address the semantic uncertainty introduced by cloud contamination. To address this issue, we propose CloudLULC‑Net, an end‑to‑end heterogeneous SAR‑optical fusion framework that directly predicts LULC maps from cloud‑contaminated Sentinel‑2 imagery and temporally adjacent Sentinel‑1 SAR observations. The proposed network incorporates optical reliability modulation to suppress unreliable optical responses, heterogeneous information adaptive aggregation to model high‑order spatial‑channel interactions between optical and SAR representations, and a unified semantic mapping transformer to organize fused features in a LULC‑oriented latent space. A semantic anchor‑guided optimization strategy is further introduced to improve the consistency of intermediate semantic representations. To support this task, we construct CloudLULC‑Set, a large‑scale benchmark dataset containing 40,223 curated SAR‑optical‑label triplets with pixel‑level LULC annotations across diverse geographic regions and cloud conditions. Experimental results show that CloudLULC‑Net achieves an OA of 86.60%, an F1‑score of 83.29%, and an mIoU of 73.51%, outperforming representative heterogeneous reconstruction‑first and end‑to‑end SAR‑optical mapping methods. Comparisons with existing global LULC products and analyses under different cloud‑cover levels further demonstrate the robustness and practical value of CloudLULC‑Net for target‑date LULC mapping in cloud‑prone regions.The project is publicly available at: https://github.com/RSIIPAC/CloudLULC
Authors:Jens Bayer, Stefan Becker, David Münch, Michael Arens, Jürgen Beyerer
Abstract:
Pixel‑wise adversarial patches are computationally heavy and often visually detectable, limiting utility in security‑critical systems. We present adversarial Voronoi camouflage that optimizes only seed‑point locations under fixed, printable palettes using a soft assignment, producing structured, splinter camouflage‑like patterns without additional regularization. Evaluated on person detection with COCO‑style AP@[.5:.95], naive placement (Inria ‑> COCO) performs comparably bad, while garment‑level application via segmentation mask (3DPeople) results in a significant AP drop. The attack transfers to out‑of‑domain backgrounds and across detector families (YOLOv9/10/11/12), indicating robustness in black‑box settings. Repainting with different palettes largely nullifies the effect, and single‑color tweaks show limited tolerance (<=0.17), highlighting a structure‑palette coupling. The parameter‑efficient, palette‑constrained design improves visual plausibility while degrading real‑time detector performance. Physical validation and color calibration are left for future work. Code: https://github.com/JensBayer/Voronoi This paper was originally presented at the International Conference on Military Communication and Information Systems (ICMCIS), organized by the Information Systems Technology (IST) Scientific and Technical Committee, IST‑224‑RSY ‑ the ICMCIS, held in Bath, United Kingdom, 12‑13 May 2026.
Authors:Jiyuan Yang, Gengxin Sun, Mengqi Zhang, Lingjie Wang, Yuanzi Li, Hongxi Cui, Xin Xin, Pengjie Ren
Abstract:
Recommender systems alleviate information overload, yet repeated feedback between recommendations and user interactions can reinforce existing preferences and narrow users' exposure, forming information cocoons. While this phenomenon has been widely studied in traditional sequential recommendation, its impact on generative recommendation remains unclear. By replacing atomic item IDs with Semantic ID (SID) sequences, generative recommenders introduce a different recommendation mechanism whose role in information cocoon formation is not yet understood. To investigate whether generative recommenders deepen information cocoons, we propose \textscRecLoop, a closed‑loop simulation framework with LLM‑driven user agents. We compare two generative recommenders and two traditional sequential baselines on two Amazon datasets across multiple feedback cycles. In addition to standard exposure‑level metrics, we introduce \emphCode‑Space Structural Cocoon, a model‑level metric that measures concentration in the generated SID space. Experimental results show that generative recommenders are generally less prone to exposure‑level cocoon formation than traditional baselines, preserving broader exposure diversity and slowing cross‑user homogenization. However, feedback loops can still induce concentration within the generated SID space. We further find that cocoon severity depends strongly on tokenization strategy and model scale: collaborative‑signal tokenization produces stronger cocoon effects than semantic tokenization, whereas larger models maintain greater code‑space diversity and better retain access to niche content. These findings suggest that information cocoons in generative recommendation are shaped not only by recommendation behavior, but also by item tokenization and model capacity. Our code is available at https://github.com/Dregen‑Yor/RecLoop.
Authors:HyunJin Kim, Jaejun Shim, Young Jin Kim, JinYeong Bak
Abstract:
Unsupervised dense retrievers offer scalability by learning semantic similarity from unlabeled documents via contrastive learning, but they struggle to capture the temporal relevance, retrieving semantically related but temporally misaligned documents‑an important aspect when a document collection spans multiple time periods (e.g., retrieving documents from 2018‑2025 for "Who is the president in 2019?" introduces temporal ambiguity). Existing methods rely on supervised training with explicit timestamps, which are not always feasible. We propose TPOUR (Temporal Preference Optimization for Unsupervised Retriever), which uses our novel training method Temporal Retrieval Preference Optimization (TRPO). TRPO reinterprets preference learning in the temporal dimension, guiding the retriever to favor temporally aligned documents. TPOUR further generalizes to unseen time periods via interpolation in a learned time embedding, enabling continuous temporal alignment. Experiments on temporal information retrieval (T‑IR), TPOUR outperforms both unsupervised and supervised baselines. Compared to Qwen‑Embedding‑8B, despite being about 72.7x smaller, TPOUR Contriever improves average nDCG@5 by +4.04 (+12.15%) on explicit and +4.98 (+15.21%) on implicit queries. We provide our code at https://github.com/agwaBom/TPOUR.
Authors:Siyue Chen, Yifu Guo, Yuquan Lu, Zishan Xu, Jiaye Lin, Jianbo Lin, Siyu Zhang, Cheng Yang, Junxin Li, Yujia Li, Yu Huo, Ruixuan Wang
Abstract:
Standard accuracy metrics cannot explain why LLMs handle variable tracking but fail on semantically equivalent loops. We study an internal lifecycle of code reasoning in which models first brew the answer, making it linearly recoverable many layers before it becomes self‑decodable, and then diverge into one of four resolution outcomes: Resolved, Overprocessed, Misresolved, or Unresolved. Understanding this lifecycle matters because similar task accuracies can mask fundamentally different failure modes that surface‑level evaluation cannot detect. We introduce a dual diagnostic framework pairing layer‑wise linear probing with Context‑Stripped Decoding (CSD) and apply it to six code‑reasoning task families across 16 models spanning Qwen, Llama, and DeepSeek architectures. All four outcomes carry substantial mass in every task family: overall Resolved is only 41.5%, with multiple tasks below 30%. Controlled sweeps over structure, depth, and operators expose task‑specific failure bottlenecks: Function Call Resolved plunges from 61.1% to 2.5% as call depth increases from one to three. Across architectures and scales, the brewing scaffold remains stable, with normalized brewing duration 24‑42% across all 16 models, while resolution success varies with capability. This indicates that the scaffold is a stable empirical regularity across the tested decoder‑only Transformer families, whereas resolution success covaries with capability, scale, and training. Code: https://github.com/euyis1019/llm‑brewing
Authors:Hong Yang, Basura Fernando
Abstract:
Generalist embodied agents require more than object recognition: they must reason about spatial relations, actions, procedures, human intentions, environmental constraints, and commonsense consequences from situated visual observations. Yet existing visual and embodied question answering benchmarks often provide limited control over the reasoning dependencies being tested, making it difficult to distinguish grounded embodied reasoning from shortcut‑driven visual or linguistic pattern matching. We present ERQA‑Plus, a diagnostic benchmark for reasoning in embodied AI. ERQA‑Plus contains 1,766 question‑answer instances grounded in 711 robot‑centric images and organized according to a structured taxonomy spanning perceptual, action‑centric, social‑interaction, navigation‑environmental, and contextual commonsense reasoning. The dataset is constructed using a multi‑stage generation and validation pipeline that combines taxonomy‑guided question generation, automatic quality judging, iterative revision, and human assessment to improve visual grounding, answer validity, and reasoning quality. We benchmark representative general‑purpose vision‑language models and embodied models, including LLaVA‑NeXT‑8B, Prismatic‑7B, MiniCPM‑V‑4.5‑8B, Qwen3‑VL, RoboRefer‑8B, and RoboBrain2.5‑8B. Although the strongest model, Qwen3‑VL‑32B, achieves 83.4% overall accuracy and 61.4 SBERT score, category‑level results reveal persistent weaknesses in spatial reasoning, procedural reasoning, event prediction, and intention inference. ERQA‑Plus therefore provides a fine‑grained evaluation framework for measuring not only whether embodied agents answer correctly, but also which forms of embodied reasoning they can and cannot perform reliably. The dataset is available https://huggingface.co/datasets/huggingdas/erqa‑plus and the project page at https://github.com/LUNAProject22/erqa‑plus.
Authors:Jie Wang, Tao Wang, Ru Zhang, Jianyi Liu
Abstract:
The widespread deployment of face recognition (FR) systems exposes personal images shared on social media and public platforms to identity linkage and privacy risks. Existing adversarial privacy protection methods can degrade unauthorized FR performance but are not compatible with generative face editing. Artificial intelligence‑driven face editing tools are gaining popularity, which has significantly increased user demand for personalized portrait generation and social sharing. However, current editing methods often preserve identity features, making the edited images still susceptible to tracking by malicious FR systems. Thus, this paper proposes Flux‑Guard, a privacy‑preserving face editing framework based on adversarial attacks, which integrates face editing and privacy protection within a unified generative process. Specifically, we design a flow trajectory control method to align semantic manipulations with the generative process and introduce latent‑space adversarial optimization with an adaptive perceptual‑loss‑driven weighting strategy, dynamically adjusting adversarial strength to maximize attack effectiveness while preserving visual quality. Extensive experiments demonstrate that Flux‑Guard supports face editing while significantly improving attack success rates against cross‑domain face recognition models on the CelebA‑HQ and LADN datasets. Furthermore, evaluation results for commercial APIs have confirmed its effectiveness in real‑world applications. The code is released at https://github.com/JLMWang/Flux‑Guard.
Authors:Semin Kim, Jihwan Yoon, Seunghoon Hong
Abstract:
Finding the initial noise that generates a given data sample, known as inversion, is a key component for downstream applications such as training‑free image editing. Existing fixed‑point inversion methods improve inversion accuracy by formulating each inversion step as a fixed‑point problem, but they lack a principled mechanism for selecting among multiple fixed‑point solutions that can arise in practice. We observe that different selections induce different inversion trajectories, leading to substantial variation in reconstruction and editing quality. For rectified flows, we further find that this variation is closely associated with trajectory straightness, motivating straightness as a principled selection criterion. We propose SelFix, a fixed‑point inversion method that selects fixed‑point solutions inducing straighter inverse trajectories while retaining convergence to an exact inverse root under standard local assumptions. Experiments on FLUX.1‑dev and PIE‑Bench show that SelFix improves fixed‑point inversion, achieving stronger real‑image reconstruction and better source‑preserving prompt‑based editing than prior inversion baselines. The code is available at https://github.com/seminkim/selfix.
Authors:Xiyu Zhai, Xinyi Chen, Yiping Wang, Runlong Zhou, Liao Zhang, Simon S. Du
Abstract:
We present a dependent‑type‑based prover designed around the way LLMs (and humans) tend to write mathematics, complementing existing systems such as Lean and Rocq. Its core design choices are a surface that imitates mathematical natural language and a rule‑driven automation layer that closes the routine steps a textbook would omit, so that an accepted proof can be re‑emitted as a checked Lean file. Early experiments suggest that, even without any prover‑specific training data, LLMs can learn to use it effectively on the miniF2F benchmark. Lean output excerpts: https://github.com/xiyuzhai‑husky‑lang/visored/
Authors:Hao-Yuan Ma, Li Zhang, Zhiwei Zhu, Jie Gao
Abstract:
Text‑guided open‑vocabulary object counting (TOOC) aims to count objects belonging to the categories specified by natural language descriptions. Although vision‑language pre‑trained models have been successful applied to TOOC tasks, they still struggle with fine‑grained spatial understanding and real‑time inference requirements in counting scenarios. To address these limitations, this paper proposes a real‑time TOOC framework, called the Real‑Time Counter (RT‑Counter), that achieves not only good counting accuracy but also high computational efficiency. RT‑Counter designs a novel Visual Prototype Textualization (VPT) module that can project learned visual features into a text feature space and then generate features containing the abstract information that is hard to capture with visual prototypes and the detailed prototype information that is difficult to describe in text, enhancing the object‑level visual‑language model's counting capabilities. Additionally, RT‑Counter incorporates our Weaving Transformer (Weaformer) layers, maintaining high descriptive power at a fraction of the computational cost. The Weaformer layer adopts a novel hybrid attention mechanism that can efficiently weave together local and global visual features. Extensive experiments on three public datasets show that RT‑Counter successfully breaks the accuracy‑speed trade‑off in TOOC. While achieving a competitive MAE of 13.30 on FSC147, RT‑Counter operates at 112.48 FPS, making it 7.4x faster and over 4× more parameter‑efficient than the existing leading methods in TOOC. Our work aims at balancing high accuracy and real‑time performance in TOOC. Code is available at: https://github.com/Jason‑Mar1/RT‑Counter.
Authors:Yu Guo, Zhengru Fang, Shengfeng He, Senkang Hu, Yihang Tao, Phone Lin, Yuguang Fang
Abstract:
Image restoration seeks to recover high‑quality images from degraded inputs but becomes highly ill‑posed under complex, mixed degradations. While unified all‑in‑one models are common, their performance declines as degradation complexity increases. Recent works adopt Chain‑of‑Thought (CoT) reasoning for multi‑round restoration using specialized modules. However, this approach faces two key limitations: (i) increased computational cost due to multi‑step processing, and (ii) weak modeling of interactions between degradations during stepwise inference. We introduce CoTIR, a universal image restoration framework that internalizes CoT reasoning within a single model. Concretely, we view image restoration as a specialized subtask of image editing, which implies that a large‑scale pre‑trained editing model provides a more favorable optimization starting point. Building on this, we fine‑tune the model for restoration and further encode structured CoT‑style reasoning into the learning objective via a differentiable formulation inspired by Lagrangian optimization, enabling holistic restoration without chaining specialized restorers. To facilitate training and evaluation, we further present CoTIR‑Bench, a large‑scale benchmark comprising 5.2 million samples with CoT‑style reasoning traces. Extensive experiments on CoTIR‑Bench and broad real composite degradation scenes show that CoTIR achieves stronger perceptual quality and more competitive fidelity than both all‑in‑one models and multi‑round restoration methods. The source code is available at https://github.com/gy65896/CoTIR.
Authors:Da Chang, Ganzhao Yuan
Abstract:
Efficient optimization is essential for training large language models. Although intra‑layer selective updates have been explored, a general mechanism that enables fine‑grained control while ensuring convergence guarantees is still lacking. To bridge this gap, we propose MGUP, a novel mechanism for selective updates. MGUP augments standard momentum‑based optimizers by applying larger step‑sizes to a selected fixed proportion of parameters in each iteration, while applying smaller, non‑zero step‑sizes to the rest. As a nearly plug‑and‑play module, MGUP seamlessly integrates with optimizers such as AdamW, Lion, and Muon. This yields powerful variants such as MGUP‑AdamW, MGUP‑Lion, and MGUP‑Muon. Under standard assumptions, we provide theoretical convergence guarantees for MGUP‑AdamW (without weight decay) in stochastic optimization. Extensive experiments across diverse tasks, including MAE pretraining, LLM pretraining, and downstream fine‑tuning, demonstrate that our MGUP‑enhanced optimizers achieve superior or more stable performance compared to their original base optimizers. We offer a principled, versatile, and theoretically grounded strategy for efficient intra‑layer selective updates, accelerating and stabilizing the training of large‑scale models. The code is publicly available at https://github.com/MaeChd/MGUP.
Authors:Patrick Blöbaum, Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan
Abstract:
Causal discovery from observational data remains challenging due to the need to recover directed structure and latent confounding without interventions. We propose FoundCause, an amortized causal discovery model trained entirely on synthetic data that maps datasets directly to causal graphs in a single forward pass. By learning from large collections of simulated structural causal models, FoundCause captures transferable statistical patterns that generalize beyond individual datasets. The architecture incorporates several key inductive biases for causal discovery. It uses a permutation‑invariant transformer encoder with alternating attention over samples and variables to jointly model cross‑variable dependence and per‑variable distributions. Pairwise statistical features derived from classical asymmetry measures are injected through statistics‑conditioned attention, guiding the model toward known causal signals. A factorized decoder separates edge existence from direction, while a triangular refinement module enables reasoning over higher‑order causal motifs such as chains and colliders. In addition, a dedicated confounder module based on learnable latent tokens explicitly models hidden common causes, and the model explicitly handles missing data via its masked input representation. To our knowledge, FoundCause is the first amortized causal discovery approach to explicitly model latent confounding. FoundCause outperforms 11 classical non‑amortized methods (e.g., PC, GES, NOTEARS‑style optimization) and 4 amortized causal discovery methods on 15 real‑world datasets, achieving +9.6% improvement in F_1, +1.2% in AUROC, and an 18.9% reduction in structural Hamming distance relative to the strongest non‑amortized methods, while performing inference in a single forward pass.
Authors:Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed, Zhijing Jin, Shion Guha, Syed Ishtiaque Ahmed
Abstract:
Evaluations of social bias in LLMs largely focus on whether models generate or imply biased content. However, as LLMs are increasingly used as judges of bias, they may exhibit social biases in subtler ways in how they evaluate biased content, which current methods do not systematically capture. We call this second‑order bias: social bias in an LLM's judgment about social bias, which we evaluate through a novel, philosophically grounded reasoning task. Drawing on entitlement epistemology, we conceptualize bias as misplaced foundational knowledge that shapes an agent's rational inquiry, and derive a logical reasoning task for LLMs to judge to whom a biased text is acceptable or non‑acceptable. We develop two simple metrics to measure how biased LLM judges are in inferring demographics for acceptability without sufficient support, and how these inferences vary across groups targeted by biased texts. Evaluating open and closed models, we find that our task evades safety guardrails by surfacing bias in model judgment. It varies systematically across target groups, reflects implicit social maps, and shows how models are still triggered by demographic labels. Our work points to the need for LLM bias evaluation in judgment tasks and broadly, for more theoretically grounded approaches to bias evaluation in NLP. We release our code and model responses at https://github.com/uofthcdslab/second‑order‑bias.
Authors:Aryan Bhagat
Abstract:
Melanoma is the most dangerous form of skin cancer with five‑year survival rates exceeding 99% when detected early but falling sharply once the disease spreads. This paper proposes and evaluates a two‑stage fine‑tuning approach for ResNet50 applied to binary melanoma classification on dermoscopic images. The core challenges addressed are class imbalance and suboptimal transfer learning from single‑stage fine‑tuning. After stratified train/validation/test splitting, random oversampling was applied exclusively to the training set to achieve a 1:1 class balance. Stage 1 trained only the classification head with the ResNet50 base frozen, while Stage 2 fine‑tuned all layers jointly at a low learning rate of 1e‑5 to prevent catastrophic forgetting of learned visual features. On an independent test set of 3,826 images, the model achieved an AUC‑ROC of 0.9559, accuracy of 88.34%, sensitivity of 87.56%, specificity of 89.13%, and F1‑score of 88.29%. An ablation study confirms the two‑stage protocol significantly outperforms single‑stage fine‑tuning, with sensitivity gains of over 4%. Grad‑CAM visualizations demonstrate correct lesion localization. A fully deployable Streamlit detection application is provided alongside all training code.
Authors:Gram Koski, Sean Lipps, Zhenghua Ma, G. Abarajithan, Ryan Kastner
Abstract:
Transformer‑based models achieve strong performance for jet tagging at the CERN LHC, but deploying them in low‑latency, resource‑constrained trigger systems is challenging. We present an initial implementation of a quantized, integer‑only transformer for jet tagging on the AMD Versal AI Engine (AIE), mapping dense and multi‑head attention (MHA) layers to AIE tiles. The main contribution is a reusable software framework that represents transformer layers as composable AIE building blocks and automatically generates the corresponding Vitis graph code from a high‑level Python model description. This framework provides a foundation for future research and is released as open‑source software at https://github.com/KastnerRG/particle_transformer_aie.
Authors:Haoyu Wang, Guoqing Ma, Zeyu Zhang, Yandong Guo, Boxin Shi, Hao Tang
Abstract:
Generalist vision‑language‑action systems need object‑centric 3D evidence and reusable manipulation experience to plan reliable robot trajectories. GeneralVLA provides a hierarchical interface for converting language and RGB‑D observations into 3D end‑effector paths, but two bottlenecks remain. First, monocular SAM3D‑style object reconstruction can hallucinate pose and unseen geometry, while manipulation benefits from stable object shape when calibrated multi‑view observations are available. Second, the original KnowledgeBank mainly retrieves semantically similar snippets and appends new knowledge, which makes it difficult to control memory quality, conflicts, confidence, and geometric relevance. To address the first challenge, we introduce GeoFuse‑MV3D, a geometry‑prior‑guided MV‑SAM3D reconstruction branch that verifies external geometry cues with input‑view masks, applies soft visual‑hull support, performs axis‑wise refinement, and fuses only geometry while preserving appearance. To address the second challenge, we upgrade KnowledgeBank into a governed long‑term memory system with explicit quality, confidence, lifecycle, verifier, and conflict metadata, together with precision‑oriented retrieval. Finally, we evaluate the reconstruction branch on GSO‑30 and the memory module on Terminal‑Bench 2.0 and SWE‑Bench Verified; GeoFuse‑MV3D improves over the MV‑SAM3D baseline by reducing CD and LPIPS by 2.20% and 2.02% while increasing PSNR and SSIM by 2.36% and 1.03%, and KnowledgeBank improves over ReasoningBank by 4.53% on Terminal‑Bench SR and 3.73% on SWE‑Bench resolve rate, while reducing AS by 4.95% and 5.65%, respectively. Code: https://github.com/AIGeeksGroup/GeneralVLA‑2. Website: https://aigeeksgroup.github.io/GeneralVLA‑2.
Authors:Hao Li, Man Fung Zhuo
Abstract:
Laser‑induced breakdown spectroscopy (LIBS) quantitative analysis faces critical challenges in wavelength selection due to high‑dimensional spectral data and the fundamental trade‑off between prediction accuracy and feature efficiency. This paper presents a novel Multi‑Adapter PPO framework that transforms wavelength selection into a reinforcement learning problem, leveraging cross‑attention mechanisms and multiple specialized adapters to capture complex spectral relationships. Our approach outperforms traditional Particle Swarm Optimization (PSO) by an average of 28.4% in comprehensive score and 45.2% in prediction accuracy across steel and coal datasets. The proposed method demonstrates superior performance in balancing prediction accuracy with feature efficiency, achieving state‑of‑the‑art results in LIBS quantitative analysis while maintaining interpretability and computational efficiency. We released our code and dataset here: https://github.com/Hflying/MAPPO
Authors:Xianda Guo, Pinhan Fu, Ruilin Wang, Wenke Huang, Mang Ye, Qin Zou
Abstract:
Stereo matching has advanced through foundation models trained on large‑scale datasets, yet this paradigm suffers from a scalability bottleneck: incorporating new data requires costly joint retraining. Model merging offers a scalable post‑hoc alternative by integrating knowledge from specialized models after source checkpoints are available. However, existing merging methods typically retain all available models or rely on greedy inclusion, which can preserve harmful task‑vector interference. We propose StereoFactory, a coarse‑to‑fine evolutionary framework for adaptive model merging. Stage~1 employs a genetic algorithm to search the combinatorial space of model subsets, determining which models should participate. Stage~2 addresses module‑level knowledge specialization (different functional modules exhibit distinct preferences for knowledge sources) through CMA‑ES optimization of architecture‑adaptive routing over the selected task vectors, with optional module‑level scaling. Experiments across two architectures and four benchmarks demonstrate that StereoFactory consistently achieves the best four‑benchmark average under the same checkpoint pool, reducing the average error from 3.80 to 3.30 on NMRF and from 2.88 to 2.19 on FoundationStereo relative to the strongest controlled baseline. The post‑hoc search requires only 2.7‑‑3.7% of the corresponding joint‑retraining wall‑clock time. Analysis reveals that knowledge contributions are inherently module‑specific, and selected subsets can transfer across architectures with minimal degradation. Code will be publicly released upon acceptance at: https://github.com/XiandaGuo/StereoFactory.
Authors:Jeffrey G. Wang, Jason Wang, Marvin Li, Seth Neel
Abstract:
Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties. Although several attempts have been made to evaluate MIAs on language models, the extant literature has suffered numerous difficulties in constructing clean evaluations to test new techniques. In particular, subtle distribution shifts between member and non‑member sets can undermine the statistical validity of MIAs; recent work has underscored this by showing that "blind" methods with no access to the underlying model can perform far better than published methods on the same benchmarks. This paper constructs a benchmark for principled evaluation of MIAs against LLMs, by leveraging the insight that training data before and after a fixed point during training are drawn from the same distribution. Therefore, all open‑source models with intermediate checkpoints and public training data can be converted into MIA testbeds. We apply our framework to a half‑dozen published attacks on the Pythia and OLMo family of models, from 70M to 7B parameters. To facilitate further privacy research, we open‑source a modular library for designing and implementing attacks in this setting: https://github.com/safr‑ai‑lab/pandora_llm.
Authors:Weiya Li, Zhiwei Tang, Yizhou He, Chenghao Wang, Liang Feng, Xiao Sun, Dongrui Liu, Zichen Wen, Hu Wei, Jinghang Wang, Yi Luo, Li Guo, Linfeng Zhang
Abstract:
With the rapid advancement of Deep Research Agents in knowledge‑intensive domains such as finance, establishing reliable and domain‑aligned evaluation standards remains a critical challenge. Existing benchmarks focus on either closed‑ended question answering or open‑ended report evaluation, failing to jointly capture retrieval‑reasoning accuracy and end‑to‑end research quality required in real‑world workflows. We introduce ICBCBench, a consortium‑driven benchmark for financial deep research, developed in collaboration with domain experts from a broad range of financial institutions and academia, involving over 50 experts across more than 40 organizations. It adopts a dual‑track paradigm integrating objective tasks with verifiable answers and subjective long‑form report evaluation, enabling complementary assessment of retrieval‑reasoning accuracy and end‑to‑end report quality in terms of expert alignment, citation consistency, and source quality. Experiments on state‑of‑the‑art DRAs and large language models reveal substantial gaps in complex reasoning, factual grounding, and report quality, highlighting the challenges of achieving industry‑level performance. Our dataset and evaluation framework are available at https://github.com/DeepFin‑Intelligence/ICBCBench.
Authors:Bo Gou, Jicheng Zhang, Jianlong Xiong, Tao He, Bentian Liu, Hai Wu, Yijiao Wang, Yu Zhang, Yujia Yang, Yun Dai, Jian Liu, Jie Wang
Abstract:
Automated classification of standard echocardiographic views is crucial for efficient clinical workflow but faces three main challenges. First, publicly available datasets are scarce and limited in scale and view coverage. Second, the performance of some modern video‑level architectures for echocardiographic view classification remains underexplored. Third, some view categories exhibit highly similar spatial appearances, making single‑frame features insufficient for discrimination, while heterogeneous frame quality complicates robust temporal information fusion. To address these challenges, we release the Echocardiographic Videos of Nine Views (EV9V) dataset, comprising 5,138 videos, 910,579 frames, and 9 standard views, which is, to the best of our knowledge, the largest publicly available echocardiography video dataset. Using EV9V, we systematically benchmark representative video classification architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers. Furthermore, we propose a Spatio‑Temporal Fusion Model (STFM), an efficient dual‑stream CNN‑LSTM (Long Short‑Term Memory) framework that jointly captures spatial anatomical structures and temporal cardiac dynamics. The proposed framework leverages uncertainty‑aware learning to preferentially sample representative video segments during training and evidence‑based fusion during inference, improving robustness to variations in frame quality across echocardiographic videos. Extensive experiments demonstrate that our method achieves competitive performance across diverse video classification models, validating the effectiveness of uncertainty‑aware spatio‑temporal learning for echocardiographic view classification. The code is available at https://github.com/bgx666/stfm.
Authors:Xiongjun Guan, Jianjiang Feng, Jie Zhou
Abstract:
Fingerprint recognition is still dominated by task‑specific pipelines, where enhancement, structural parsing, alignment, and matching are optimized in isolation. Although effective in narrow settings, this design limits representation reuse across sensors, qualities, and downstream applications. We therefore present UoU, short for ``a Universal fingerprint foundation model based on large‑scale Unsupervised learning,'' which reframes fingerprint feature extraction as a domain‑specific foundation‑model problem. UoU is organized around a multi‑level representation hierarchy spanning image restoration, structural fields, semantic tokens, point‑level biometric entities, and compact global descriptors. Its training recipe combines a supervised cold start on precise annotations, large‑scale weakly supervised refinement, and large‑scale unsupervised consolidation, with the latter two stages iterated during large‑scale training so that weak supervision broadens semantic coverage while unsupervised learning stabilizes correspondences, invariances, and representation geometry. Rather than treating fingerprint imagery as generic texture, UoU exploits domain‑specific symmetries and intermediate structure, including orientation flow, periodic ridge patterns, sparse biometric entities, and spatial equivariance. The framework is intentionally architecture‑agnostic: while the present study includes an initial transformer‑based structured‑prediction instantiation, the broader design supports multi‑task learning, scalable model configurations, and downstream specialization for matching, alignment, enhancement, registration, and related fingerprint applications. This paper presents the technical motivation, system design, and validation protocol of UoU, and part of the baseline implementation is publicly available at https://github.com/XiongjunGuan/UoU.
Authors:James Avtges, Jake Ketchum, Helena Young, Taekyoung Kim, Ryan Truby, Todd Murphey
Abstract:
Adaptation to damages and in‑situ physical repairs is essential for long‑term robot autonomy, yet challenging outside of narrowly defined and well‑anticipated bounds. In this work we proprioceptively adapt to catastrophic damage in soft‑actuated systems in under one minute. Architected materials are well equipped for adaptation: actuator failure occurs gradually rather than acutely, and damage can be described in a low‑dimensional, discrete coordinate space. Surprisingly, latent damage representations plus a simple yet robust ensemble method is sufficient for adapting to unseen damage in real‑time. Moreover, we identify conditions under which exponential sample complexity collapses to linear sample complexity for learned representations of architected materials, a concrete advantage over rigid components or continuum soft mechanisms. We demonstrate LEAP, our method for adaptive proprioception, via a tracing task for a 6DoF soft wrist based on Handed Shearing Auxetic (HSA) actuators. Our algorithm is able to adapt to cuts, burns, and actuator repairs, enabling simulation‑free real‑time adaptation that is critical for realizing the promise of soft robots outside the lab. Videos and more information are available at https://murpheylab.github.io/leap.
Authors:Logan Mann, Yi Xia, Ajit Saravanan, Ishan Dave, Saadullah Ismail, Shikhar Shiromani, Emily Huang, Ruizhe Li, Kevin Zhu
Abstract:
Multimodal Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical. A common intuition, which we call the Attention‑Confidence Assumption, holds that reliability follows from "structural" visual perception: tight attention on relevant regions should signal a trustworthy answer, while scattered attention signals confusion. We challenge this through the VLM Reliability Probe (VRP), a systematic cross‑family study of reliability signals in contemporary Vision‑Language Models (VLMs). We introduce structural‑attention metrics, cluster counts (C_k) and spatial entropy (H_s), to quantify the visual encoder's gaze, and track its evolution (Delta H_s) across layers. This reveals a "Symbolic Detachment": models often "Early Lock" visual features only to diffuse attention later, severing early perception from final generation. Contrary to the grounding hypothesis, we find a "Cluster Failure": spatial attention has near‑zero correlation (R approx 0.001) with accuracy. Instead, reliability is a phenomenon of generation dynamics and internal‑state distributions. Self‑Consistency, the agreement rate across sampled reasoning paths, is the dominant predictor of truth (R = 0.429). Scaling causal interventions exposes a sharp architectural divergence: LLaVA locks its prediction in a fragile late‑stage bottleneck, whereas PaliGemma and Qwen2‑VL distribute reliability globally, staying resilient even when ~50% or more of their most predictive layer is destroyed. For current VLMs, reliability signals are detached from visual grounding maps and are best inferred from generation‑time dynamics and hidden‑state probes.
Authors:Ahmad Darkhalil, Dima Damen, David Fouhey
Abstract:
Understanding hands and the objects they interact with, both directly and through tools, is a key step for tasks ranging from action perception to 3D reconstruction and robotics. Our paper provides several contributions to the Hand‑Object Interaction (HOI) understanding literature: (1) HOI‑DETR, a new framework that introduces hand‑object and object‑object interactions to the Co‑DETR architecture to produce a state‑of‑the‑art method; (2) a comprehensive HOI evaluation suite of 4 diverse datasets, including a video benchmark derived from the HD‑EPIC dataset and fresh annotations that improve the Hands23 benchmark and (3) a trained checkpoint that significantly improves the state of the art across Hands23, HOIST, FineBio, and HD‑EPIC, including mAP gains of over 20 percentage points on Hands23 and FineBio. Our ablations confirm the contributions of each model component.
Authors:Matthew Francis Dixon
Abstract:
Agentic artificial intelligence systems introduce a new class of model risk. Unlike traditional predictive models, autonomous agents continuously acquire information, form beliefs regarding latent states of the environment, generate forecasts, select actions, and adapt their behavior over time. Existing validation methodologies focus primarily on predictive accuracy and therefore provide limited insight into the quality of the underlying decision process. This paper proposes a model validation framework for agentic AI based on Partially Observable Markov Decision Processes (POMDPs). The framework decomposes autonomous decision making into information, beliefs, forecasts, actions, and utility, allowing each component to be validated independently. Large language models (LLMs) are formalized as approximate Bayesian filtering operators, and a model‑risk taxonomy is developed encompassing state‑space, filtering, forecast, policy, utility‑specification, and parameter risks. The model risk validation methodology is demonstrated through a portfolio‑management case study in which an agent infers latent market regimes from market and macroeconomic information, generates belief‑conditioned forecasts, and constructs portfolios using a Black‑‑Litterman framework. Empirical validation combines performance analysis, belief calibration diagnostics, coverage tests, ablation studies, and parameter‑sensitivity analysis. The results indicate that latent‑state inference contributes independently to decision quality and that the principal conclusions remain robust across a broad range of parameter values. The principal contribution of the paper is a practical framework for extending established model risk management concepts to autonomous AI systems and providing a rigorous foundation for their validation, governance, and monitoring.
Authors:Suttisak Wizadwongsa, Hyelin Nam, Supasorn Suwajanakorn, Jeong Joon Park
Abstract:
Generating novel renderings of a scene along user‑defined camera trajectories from a single monocular video, dubbed video retaking, is a compelling but difficult problem in content creation and visual effects. Existing geometry‑guided approaches reconstruct a 4D representation from the source video and render it along the target trajectory to condition video diffusion models. However, this guidance degrades as the target camera departs from the source trajectory, leaving newly revealed regions sparse or entirely missing. We propose SierpinskiCam, which addresses this limitation by augmenting geometry‑based guidance with Sierpinski dome texture cues that contains rich trackable features even under large viewpoint changes. We further introduce a reference video conditioning mechanism that appends source‑video tokens to the target‑token sequence and separates the two streams with negative RoPE indices, enabling appearance grounding without architectural modification or per‑video adaptation. Extensive experiments show that SierpinskiCam achieves significant gains in camera controllability, geometric consistency, and video quality across diverse and challenging retaking scenarios. Project page: https://hyelinnam.github.io/SierpinskiCam/.
Authors:Sam Collin, Ali Ayub
Abstract:
Robotic assistance in household environments requires not only predicting where objects should be placed, but also reasoning about when objects should not be placed at all. Existing approaches to personalized object rearrangement primarily focus on placement decisions under the assumption of clean observations and complete actionability, limiting their applicability in realistic, cluttered, and partially erroneous settings. In this paper, we introduce APOLLO, a hybrid framework for abstention‑aware personalized object rearrangement that combines a lightweight, personalized embedding model (PEM) with selective large language model (LLM) assistance. PEM is trained for each user‑environment pair using a small number of demonstrations, operates entirely on CPU, and produces uncertainty estimates, which are used to selectively invoke LLM‑based reasoning only for ambiguous decisions, balancing efficiency, privacy, and reasoning capability. To evaluate this formulation beyond existing benchmarks, we introduce APOR, a synthetic, LLM‑generated dataset that captures room‑level, multi‑furniture environments, diverse organizational profiles, explicit abstention behavior, and noisy partial scene context. Extensive experiments on both PARSEC and APOR provide initial evidence that APOLLO improves over prior LLM‑based baselines in controlled benchmark settings while substantially reducing LLM usage. Code is available at https://github.com/PaInt‑Lab/APOLLO.
Authors:Emre Can Kizilates
Abstract:
The dominant trajectory of modern machine learning has been to scale up: larger models, larger accelerators, larger memory budgets. Yet a multi‑year global semiconductor supply constraint and the growing energy and carbon cost of always‑online inference expose the fragility of this trajectory and motivate the opposite direction: refactoring AI and ML algorithms to fit the small, ubiquitous microcontrollers already in mass production in wearables, sensors, and edge appliances. We present an end‑to‑end open‑source reproduction of FastGRNN, a compact gated recurrent cell, deployed on two bare‑metal targets: the 8‑bit Arduino (ATmega328P) and the 16‑bit MSP430 (no hardware multiplier; 16 KB Flash; 512 B SRAM). Our compression pipeline combines low‑rank weight factorization, iterative hard‑thresholding sparsity, and per‑tensor Q15 post‑training quantization with explicit activation calibration. The deployed model occupies 566 bytes of weights and achieves macro F1 = 0.918 (seed 0; five‑seed Q15 mean 0.853+‑0.107) on the HAPT test set. It matches a PyTorch reference at 100% prediction agreement across 3,399 test windows (MCU seed 0; 99.91‑100% C‑equivalent across five seeds). Both platforms sustain real‑time 50 Hz streaming inference (9.21 ms per sample on Arduino; 13 ms on MSP430), where a 256‑entry sigmoid/tanh look‑up table delivers a 30.5x speedup on the multiplier‑less MSP430. Four contributions extend the original FastGRNN paper: (i) cross‑platform bit‑equivalent deterministic inference; (ii) characterization of recurrent warm‑up latency (median 74 samples, 1.48 s; worst‑case 125 samples, 2.50 s over 100 test windows); (iii) a deployable look‑up‑table recipe for multiplier‑less embedded targets; and (iv) hardware energy characterization showing 17.7 mW active inference power, <0.09 mW idle power, and 96.7% energy reduction with the LUT.
Authors:Petr Nyoma
Abstract:
A model that lies while knowing the truth is the central case ELK cannot handle with behavioral evaluation alone. We ask whether such deception leaves an internal signature distinguishing it from honest error. Our key move is a control for wrongness: we contrast a sleeper agent (knows the truth, lies on trigger) against a naive liar (fine‑tuned to emit the same wrong answers with no honest training). Both produce identical wrong outputs; any difference is about knowledge conflict, not incorrectness. We find deceptive forward passes carry a conflict signature ‑ 2.1‑2.3x higher residual rank than naive‑liar passes on the same wrong answer ‑ strong enough to identify which of two responses is the lie with 100% accuracy and no labels, across GPT‑2 small/medium (three seeds) and three instruct models. Across Qwen2.5‑1.5B/7B and Phi‑3‑mini, instructed deception raises residual rank on every tested fact (18/18, 40/40, 34/34); on Phi‑3, lies separate perfectly from both honest answers and hallucinations (AUC 1.0, Wilcoxon p~6e‑11). The signature survives strategic self‑constructed deception (model invents its own lie, AUC 1.0), active concealment attempts (AUC 1.0), and length‑controlled replication (20/20, AUC 1.0, p~1e‑6). Using basis‑free relative representations, a probe trained on one model family detects deception in two other families zero‑shot (mean AUC 0.933), surviving simultaneous architecture and format change (AUC 0.821), and transfers across five languages (AUC 1.000, length‑controlled). The signature is read‑only: detectable but not injectable (0/8 both directions). Honest limitations and six negative experiments are documented in full.
Authors:Sidhaarth Murali, João Coelho, Jingjie Ning, João Magalhães, Bruno Martins, Chenyan Xiong
Abstract:
Test‑time scaling for agentic search typically increases depth (i.e., more turns and tokens per trajectory) or breadth (i.e., more parallel rollouts). Here we focus on breadth scaling, showing that standard parallel sampling yields diminishing returns, tracing this to query redundancy at the first turn. When models issue similar first queries across rollouts, the threads retrieve overlapping evidence, and subsequent turns are conditioned on this shared retrieval. We address this limitation with DivInit, a training‑free intervention at the first turn. Rather than sampling k independent first queries, DivInit draws n candidates from a single call, picks k < n diverse seeds, and runs them as parallel trajectories. Across five open‑weight models and eight benchmarks, DivInit consistently improves over standard parallel sampling, with average gains of five to seven points on multi‑hop QA at matched compute. Code available at https://github.com/cxcscmu/diverse‑query‑initialization
Authors:Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina, Rajvee Sheth
Abstract:
Current multilingual evaluations for Vision‑Language Models (VLMs) assume a one‑to‑one mapping between language and orthography, overlooking billions of users of multi‑script languages. We introduce PuMVR (Punjabi Multimodal Visual Reasoning), a benchmark of 1,000 strictly parallel image‑text instances across Punjabi's three active scripts: Gurmukhi, Shahmukhi, and Roman. Evaluating 10 state‑of‑the‑art VLMs, we expose a substantial and systematic Script Gap. Models frequently solve visual tasks in one script while failing identical tasks in another, with accuracy deltas reaching 16%. Crucially, visual input boosts absolute performance uniformly yet does not close the orthographic gap. Furthermore, cross‑script in‑context transfer is highly brittle, exposing script‑locked knowledge representation. Supported by McNemar tests across all script pairs, our findings demonstrate that current "multilingual" VLMs are not truly multi‑script. We propose the Script Consistency Rate (SCR), which falls as low as 24.8% on our benchmark, as a mandatory metric for script‑agnostic evaluation to ensure equitable AI access. Data and code are available at: https://github.com/prabhjotschugh/Not‑Truly‑Multilingual‑PuMVR.
Authors:Zhihan Xiao, Hongbing Lang, Zhe Zhao, Luke Ztz Hu, Songping Mai
Abstract:
High‑Level Synthesis (HLS) relies on transforming original C specifications into synthesizable HLS‑oriented C (HLS‑C) implementations. Functional consistency verification between original C specifications and HLS‑C implementations is a critical yet labor‑intensive task in HLS design flows. While Large Language Models (LLMs) have recently shown promise in automated testbench generation, their stochastic nature often leads to insufficient coverage, inconsistent verification environments, and unreliable equivalence checking results. To address these limitations, we propose a knowledge‑augmented, agent‑driven shift‑left verification framework for automated functional consistency checking between original C and HLS‑C implementations before synthesis. The framework introduces a Dual‑Tier Consistency Checking mechanism that jointly enforces static structural alignment and dynamic behavioral equivalence between paired testbenches, while integrating symbolic execution and coverage‑driven refinement to improve verification completeness. Furthermore, we construct a heterogeneous HLS Verification Knowledge Graph to provide topology‑aware reasoning priors for testbench generation, and design an autonomous verification agent to orchestrate iterative refinement and failure diagnosis across heterogeneous toolchains. Experimental results on 107 HLS benchmark pairs demonstrate that the proposed framework achieves 0.9826 average coverage and 0.9533 dynamic consistency, outperforming representative AST‑based, retrieval‑augmented, and iterative agent‑based baselines. https://github.com/cz‑5f/HLS‑LeVeri.git
Authors:Jay Jung, Xiaohan Zhang, Shenghan Song, Mahmoud Sayedahmed, Chijian Xiang, Yunong Xu, Ahmed AbdelKhalek, Severin T. Schneebeli, Matthew J. Wargo, Jianing Li, Safwan Wshah
Abstract:
Antimicrobial resistance causes to over a million deaths annually. Antimicrobial peptides (AMPs) are a promising solution, but generative AMP models are not yet ready to design peptides with non‑natural amino acids and/or chemical modifications, which are essential for real‑world peptide drugs. We present AMPGAN v3, a multi‑objective conditional GAN that expands the generative vocabulary to D‑amino acids and N/C‑terminus modifications such as amidation. By separating adversarial and activity‑aware supervision across two specialized discriminators, AMPGAN v3 substantially improves training stability and outperforms prior generative AMP models on external classifiers. We validated five candidates spanning three structural classes in vitro; two showed activity against Gram‑positive strains, with the best candidate reaching MIC 8 μg/mL against B. subtilis. To support downstream curation, we further present PepCraft, a multi‑agent framework for end‑to‑end AMP discovery in which a Planning Agent orchestrates specialized executors for generation, filtering, and verification. Its prioritization recommendations align with our in vitro outcomes. Together, these contributions let us examine, on a small but real scale, how generative and agentic AI compose in therapeutic peptide discovery. Code: https://github.com/marszzibros/AMPGANv3
Authors:Sahith Reddy Chada, Isht Dwivedi, Nirav Savaliya
Abstract:
Reliable autonomous driving requires vectorized HD maps that are geometrically accurate, semantically rich, and scalable to long‑horizon driving. However, existing public HD map datasets are limited in scale, provide sparse semantic attributes, and lack modalities such as aerial imagery that could enable new research directions. We present HRDX, a large‑scale dataset for vector HD‑map construction, spanning about 40 hours (1,400 km) of minimally overlapping drives, which is several times larger than prior public HD map datasets. Data is captured using six synchronized surround cameras, a 128‑beam LiDAR, and centimeter‑level RTK GNSS/IMU, and is further complemented by precisely aligned aerial orthoimagery. Annotations cover 10 vector map classes, complemented with over 20 semantic and topological attributes. To evaluate this richer ontology, we introduce the Composite Score (CS) to jointly assess geometric fidelity and attribute correctness. Benchmark experiments show that HRDX's scale improves online vector‑map construction, and that aligned aerial imagery provides a useful structural prior: using aerial imagery at training and/or inference improves geometric map quality, while aerial‑augmented teachers can transfer part of this benefit to camera‑only students without increasing inference‑time sensor requirements. HRDX is intended to support reproducible research on large‑scale HD‑map learning, multimodal BEV fusion, and training‑time privileged information. HRDX dataset and benchmarks are available at https://github.com/honda‑research‑institute/HRDX
Authors:Dmitrii Pantiukhin, Boris Shapkin, Ivan Kuznetsov, Thomas Jung, Nikolay Koldunov
Abstract:
The Coupled Model Intercomparison Project Phase 6 (CMIP6) has generated thousands of peer‑reviewed publications documenting model configurations, evaluation procedures, emergent constraints, and projection uncertainties. As the community transitions toward CMIP7, efficiently extracting and operationalizing this unstructured knowledge alongside live data analysis represents a critical bottleneck. Here we present CMIP‑Forge, a hybrid retrieval‑augmented generation (RAG) and autonomous analysis system that bridges the gap between scientific literature and Earth System Grid Federation (ESGF) data archives. The system pairs a curated corpus of 6,581 CMIP6‑related open‑access publications (101,828 indexed chunks) with an agentic pipeline in which a tool‑augmented worker plans and executes Python workflows over live climate data, while a panel of independent reviewer models audits its methodology end to end. CMIP‑Forge introduces a multi‑layered Defense‑in‑Depth architecture that enforces physical and methodological invariants through executable mechanisms: Abstract Syntax Tree (AST) static analysis, audited scientific primitives, and an autonomous adversarial peer‑review protocol. We demonstrate the system's capabilities through end‑to‑end autonomous research pipelines spanning atmospheric teleconnections, ocean dynamics, regional extremes, and global warming projections. An agentic analysis system grounded in peer‑reviewed literature, constrained by automated code guardrails, and audited by an independent adversarial review loop can complete complex climate‑research workflows autonomously. The same experiments expose concrete failure modes of the review loop (sycophantic regression, REVISE verdicts that are never resolved, and the submission of stub code for review), each diagnosable from the immutable telemetry and provenance record released with the article.
Authors:Zhenhong Yang, Zhuoyun Liu, Jintao Fei, Wen Tang, Shichao Quan, Jun Zhao, Jun Xu
Abstract:
Radiology report evaluation must distinguish clinical compatibility from surface similarity, because negation, laterality, or normal‑abnormal polarity can reverse a finding. We propose RadSEM (Radiology Sentence‑Level Evaluation Metric), a constrained LLM‑assisted metric for reference‑based evaluation of radiology Findings. RadSEM rewrites reference and generated reports into ordered atomic finding sentences, each expressing one site‑finding proposition. It then performs contradiction‑constrained many‑to‑many matching: incompatible pairs such as "effusion" and "no effusion" receive no credit, while compatible granularity differences can receive partial credit. A deterministic stage weights pairs by part‑whole and abnormal‑detail relationships, counts unmatched findings, and produces an abnormal‑focused weighted F1 score. Thus, the LLM supports structured rewriting and local alignment rather than acting as an opaque judge. We evaluate RadSEM with SSREE, a controlled monotonicity stress test built from 2,448 de‑identified reports expanded into five graded corruption levels. RadSEM achieves Kendall tau_b of 0.957, all‑pairs concordance of 97.8%, adjacent concordance of 95.0%, and strict five‑level ordering for 81.9% of reports, outperforming radiology‑specific and general text metrics while avoiding the failure in which polarity‑inverted reports regain lexical overlap. On the same SSREE set, RadSEM outperforms the Ref‑anchored RadSEM‑Alt policy, improving adjacent concordance from 90.7% to 95.0% and strict ordering from 67.2% to 81.9%. On a 599‑triplet synonym/antonym subset, RadSEM prefers synonyms in 597 cases (99.67%). These results suggest that explicit finding units, contradiction‑aware matching, and abnormal‑focused deterministic scoring make report scoring more interpretable and sensitive to clinically meaningful errors. Code is available at https://github.com/jdh‑algo/RadSEM.
Authors:Jisang Han, Seonghu Jeon, Jaewoo Jung, René Zurbrügg, Honggyu An, Tifanny Portela, Marco Hutter, Marc Pollefeys, Seungryong Kim, Sunghwan Hong
Abstract:
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision‑language‑action models (VLAs) and video world‑action models (WAMs) inherit strong semantic or temporal priors from large‑scale foundation models, but they still operate primarily on 2D image frames or 2D‑derived latent spaces, leaving implicit the 3D geometry required for contact‑rich manipulation. We propose the Geometric Action Model (GAM), a language‑conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language‑conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real‑robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation‑model‑scale baselines.
Authors:Anzhe Xie, Weihang Su, Yujia Zhou, Yiqun Liu, Qingyao Ai
Abstract:
Meta‑analysis is a demanding form of evidence synthesis that combines literature retrieval, PI/ECO‑guided study selection, and statistical aggregation. Its structured, verifiable workflow makes it an ideal substrate for evaluating systematic scientific reasoning, yet existing benchmarks lack ground truth across the full retrieval‑screening‑synthesis pipeline. We introduce MetaSyn, a dataset of 442 expert‑curated meta‑analyses from Nature Portfolio journals. Each entry pairs a research question with PI/ECO criteria, a retrieval corpus of 140k PubMed articles, verified positive studies, hard negatives that are topically similar but PI/ECO‑ineligible, and complete search strategies and date bounds. Benchmarking twelve pipeline configurations (nine RAG variants and a protocol‑driven agent) reveals a critical screening bottleneck: despite a retrieval ceiling of 90.9% recall at K=200, no system recovers more than 52.7% of ground‑truth included literature. Current LLMs fail to reliably separate eligible studies from PI/ECO‑failing distractors in pools of comparable topical relevance. Stage‑attributed metrics capture where systems succeed and fail; a single end‑to‑end score does not.
Authors:Zewen Liu
Abstract:
When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge. We show that Evaluator Preference Collapse (EPC) is dramatically amplified in multimodal settings. Using GPT‑4o to evaluate DeepSeek‑chat across text and visual tasks, we find that a single strategy (step_by_step) absorbs 48.4% of all weight ‑‑ 3.2x the collapse observed in text‑only self‑evaluation ‑‑ while three visual‑domain strategies receive only 9.1% combined weight. We then demonstrate a novel phenomenon we term cross‑modal contagion: evaluator preferences acquired on one modality transfer to and corrupt strategy selection on another. Through a four‑phase isolation training paradigm, we measure contagion coefficients and document strategy inversion ‑‑ the optimal strategy for a modality reverses after cross‑modal exposure. A Phase 3 statistical validation across five evaluator configurations (N=80 total independent repetitions, ~35,000 API calls) with both text‑proxy and real‑image visual tasks finds: cross‑model evaluation produces strong contagion (JSD~0.19‑0.34), real‑image inputs yield the most directionally consistent signal (mean gamma_T‑>V=1.145, gamma_V‑>T=0.937, 70% T‑>V, Cohen's d=0.56), and self‑evaluation provides near‑complete immunity ‑‑ 97% of runs (N=30) yield zero contagion (JSD=0.003, d=0.07). Three methodological ablations and multi‑executor validation confirm the effect is not a structural artifact. We introduce the contagion matrix indexed by evaluator identity, release the MM‑EPC framework, and identify cross‑model evaluator architecture as the primary risk factor for preference drift. Code and data: https://github.com/aidless/mm‑epc.
Authors:Shengyu Gong, Weiming Zeng, Yueyang Li, Zijian Kang, Hongjie Yan, Wai Ting Siok, Nizhuan Wang
Abstract:
Non‑invasive brain‑computer interfaces exhibit significant performance degradation when moving from controlled laboratory stimuli to real‑world natural images. This degradation occurs because conventional multimodal contrastive representation learning models focus exclusively on optimizing geometric distance alignment, thereby failing to account for semantic consistency and inter‑subject variability in neural representation and selective attention. As a result, these models are prone to producing spurious zero‑shot matches. To address these limitations, we propose SUP‑MCRL, a unified framework integrating three collaborative mechanisms: (1) a Semantic‑entity Aware Visual Encoder (SAVE) that learns spatial attention to extract semantic content without relying on pre‑trained saliency models; (2) a Unified EEG Enhancer (UEE) that employs multi‑scale atrous convolutions and inter‑band attention for adaptive cross‑subject robustness; and (3) a Prototype‑based Progressive Augmenter (PPA) that maintains an EMA‑updated pseudo‑feature pool to prevent representation collapse. Zero‑shot experiments on the THINGS‑EEG achieve 66.0%/91.9% (Top‑1/Top‑5) intra‑subject and 24.0%/52.9% LOSO accuracy, significantly surpassing state‑of‑the‑art methods and demonstrating that structured alignment supervision is key to overcoming the limitations of cross‑modal decoding. Code is available at https://github.com/NZWANG/SUP‑MCRL.
Authors:Francisco M. López, Markus R. Ernst, Francisco Cruz, Matej Hoffmann, and Jochen Triesch
Abstract:
Exploration in deep reinforcement learning (RL) is commonly implemented as temporally uncorrelated white noise. However, recent works show that temporally correlated colored noise can improve exploration efficiency by producing smooth trajectories with better coverage of the state space. We inquire whether action noise inspired by infant spontaneous movements can also improve exploration in deep RL. We find that the power spectral densities of babies' end‑effector velocities follow a colored noise process where the spectral exponent increases with age. Inspired by this developmental pattern, we introduce a mechanism that progressively increases the temporal auto‑correlation of exploration noise during RL training, matching the infant statistics. Experiments across several RL environments show that infant‑inspired noise produces structured exploratory behavior and can improve learning efficiency compared to conventional exploration strategies. These findings suggest that human motor and cognitive development can provide useful guidance for designing learning mechanisms in artificial agents. Our code is available at https://github.com/trieschlab/baby‑noise‑rl.
Authors:Shuai Yang, Bingjie Gao, Ziwei Liu, Jiaqi Wang, Dahua Lin, Tong Wu
Abstract:
Consistent video generation under editing operations requires persistence: when edits modify scene appearance or layout, subsequent generations should remain coherent across time and viewpoints. However, existing memory designs struggle to maintain long‑term consistency after such modifications, as stored contexts may become outdated or invalid. To address this, we propose PermaVid, a novel framework built upon a multi‑modal context memory that disentangles spatial context into semantic appearance and geometric structure, together with an edit‑aware memory update and retrieval strategy that keeps memory evolution aligned with subsequent observations. Specifically, we develop two complementary memory banks: an RGB context memory that captures appearance‑aware observations while implicitly encoding geometry, and a depth context memory that preserves geometry‑only structure disentangled from semantics. Building on this design, we introduce a memory‑guided video generation model that performs multi‑modal feature fusion under reference conditions drawn from mixed‑modality memory contexts. Experiments demonstrate that our method maintains strong long‑term semantic and structural consistency after edits, significantly outperforming state‑of‑the‑art methods.
Authors:Xintong Wang, Ye Wang
Abstract:
Accent text‑to‑speech (TTS) aims to synthesize speech with target accents. Existing accent TTS systems typically rely on a two‑stage pipeline that first converts standard phone sequences into accented phone sequences and then synthesizes accented speech. However, such approaches suffer from error accumulation and require paired standard‑accented phone sequence data, which is often limited in practice. Moreover, text‑based accented phone representations are insufficient to model acoustic accent characteristics such as prosody and rhythm. In this work, we propose Joycent, a diffusion‑based accent TTS model that synthesizes accented speech directly from standard phone sequences and speech references without accented phone prediction. Joycent integrates accent and speaker representations through conditional layer normalization (CLN) in the text encoder. We introduce WhisAID, a Mandarin accent identification model trained on accented Mandarin speech to extract accent representations. Experimental results show that Joycent improves accentedness while preserving speaker identity compared with baseline systems. We release our code and demos at: https://github.com/oshindow/Joycent‑code.
Authors:Prasanth YSS, Zhichen Ren, Rasa Hosseinzadeh, Ilan Gofman, Yuqi Chen, Zhaoyan Liu, Guangwei Yu, Jesse C. Cresswell, Satya Krishna Gorti
Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves language‑model reasoning, but GRPO‑style optimization remains prone to collapse. We analyse this instability through token‑level gradient dynamics, deriving a taxonomy that predicts how updates affect next‑token probabilities and entropy. The taxonomy shows that stability depends jointly on the advantage sign and token distribution under the current policy. Motivated by this finding, we propose Winner Advantage Policy Optimization (WAPO), a simple online clipped policy‑gradient objective that updates only on positive‑advantage completions. Across mathematical reasoning and multi‑hop QA benchmarks, WAPO improves training stability and matches or outperforms baselines across multiple model families. Full code can be found at https://github.com/layer6ai‑labs/wapo.
Authors:Pengyu Zhu, Xiaojing Zhang, Kunbo Zhang, Chunyan Zhang, Zhenyu Wang
Abstract:
Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification. This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U‑Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives. Unlike surveys that focus on a single model family or a specific clinical application, this review organizes U‑Net‑, Transformer‑, and SAM‑based methods within a unified analytical framework, with a particular focus on their effectiveness in improving segmentation accuracy and efficiency. This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository: https://github.com/andrew‑pengyu/Awsome_MedSeg/tree/main.
Authors:Haolong Qian, Xianliang Yang, Yinuo ma, Lirong Che, Feng Lu, Ye Guo, Lei Song, Jiang Bian, Chun Yuan
Abstract:
Knowledge distillation from powerful reasoning models is widely used to improve Small Language Models (SLMs) on mathematical reasoning, often assuming that traces with higher reward model scores provide more useful supervision. We identify a counterintuitive Quality‑Utility Paradox in mathematical reasoning distillation. Data refined or synthesized by a stronger Oracle obtains higher perceived quality according to reward models, yet consistently underperforms traces generated by the SLM itself and selected through rejection sampling across Qwen2.5, LLaMA‑3, and DeepSeek families. Our analysis shows that Oracle refinement couples logical repair with distributional drift away from the SLM's native reasoning distribution. This drift increases the learner's adaptation cost and can outweigh the benefit of improved reasoning logic. To test this mechanism, we introduce Style‑Aligned Refinement, which preserves the native trajectory of the SLM while retaining logical repair from the Oracle. This intervention lowers adaptation cost and restores downstream utility. These findings suggest that effective mathematical reasoning distillation should jointly optimize perceived solution quality and learner‑data compatibility, rather than relying solely on reward‑model scores. The datasets and code are available at https://github.com/Dracoqhl/Quality‑Utility‑Paradox.
Authors:Wen-Kao Li, Ze-Feng Gao, Peng-Jie Guo, Wei Ji, Zhong-Yi Lu
Abstract:
Inverse materials design starts from target functionality and searches for structures that can realize it. Its value in closed‑loop discovery depends not only on prediction performance, but also on whether expensive first‑principles results are independently validated, provenance‑recorded, and admitted as feedback only when evidence is sufficient. This is especially important for composite properties such as carrier mobility, where a final scalar value hides intermediate quantities, fit quality, convergence history, and workflow assumptions. Here we present InvDesMobility, a reliability‑gated first‑principles feedback framework that integrates multi‑agent automated DFT, evidence stratification, generative structure proposal, acquisition ranking, and auditable release. Using 516 2DMatPedia‑derived candidates, the workflow produced 280 QC‑passed materials and 573 retained carrier‑direction seed channels after channel‑level reliability gating. These records were split into two feedback objects: relaxed structures updated the generative model, while retained mobility channels trained the acquisition model and set validation priority. Over multiple iterations, InvDesMobility screened 2.4 x 10^6 structures, submitted 102 candidates for DFT validation, and retained 86 reliability‑gated generated channels across 41 formulas. Overall, the main contribution is not a fixed list of high‑mobility materials, but a transferable feedback contract that makes closed‑loop inverse design both useful and auditable when learning from expensive calculated properties. All source data, retained feedback records, and workflows are available at https://github.com/DreamLufei/invDesMobility, with an accompanying evidence website at https://dreamlufei.github.io/invDesMobility/.
Authors:Andreas Einwiller, Max Klabunde, Florian Lemmerich
Abstract:
The worldwide surge of authoritarianism, combined with the increasing central role in users' everyday lives, raises the question of to what extent specific models exhibit or promote authoritarian attitudes and characteristics. We introduce AuAu, a comprehensive benchmark that aims to assess the risk of LLMs generating responses with authoritarian tendencies. This benchmark combines three evaluation approaches: (i) psychometric questions from an extensive pool of 15 human validated instruments; (ii) contextual behavior vignettes probing intended actions in concrete situations; and (iii) responses to realistic user prompts. Unlike prior work, AuAu evaluates not only a general closeness towards authoritarianism but also the established sub‑concepts Authoritarian Aggression, Authoritarian Submission, and Conventionalism. Evaluating 17 models from China, the EU, Russia, and the USA, we find that all tested models exhibit substantial authoritarian response rates under the psychometric evaluation, though rates drop significantly in increasingly more realistic downstream task. We further find that an authoritarian system prompt easily manipulates 15 out of 17 models to promote increased authoritarianism. Our results underscore the need for continued, systematic auditing of LLM‑based AI systems to detect and ultimately mitigate undesired authoritarian tendencies in generated output. Our code and data are available at: https://github.com/andreaseinwiller/AuAu
Authors:Jaehoon Kang
Abstract:
Frequency‑aware latency estimators let deadline‑aware DVFS governors schedule edge ML inference by modeling latency over CPU and GPU clocks, but they cannot observe the memory clock (EMC) ‑‑ a missing deployment state that decides whether a governor meets its deadlines and at what energy. We show this with a deployed, measured governor on a Jetson Orin NX: an EMC‑blind GPU‑only fit misses 25‑28% of cycles at tight deadlines, whereas an EMC‑aware refit holds misses to at most 1.3% under a 2% QoS miss budget by selecting a budget‑feasible clock ‑‑ the energy‑minimal one for periodic vision (calibrated module‑rail power). The failure generalizes across three workload classes ‑‑ MobileNetV2, a ViT transformer, and Qwen2.5 LLM token decode (where saturated decode makes the aware policy lower‑energy than the infeasible blind choice): a CPUxGPU estimator sends the deployed governor to an infeasible operating point, and only an EMC‑aware model identifies the feasible side of the energy frontier. The effect is real and outside the CPUxGPU state abstraction: across two Orin SKUs sharing the same lockable EMC points it shifts median latency by up to ~45%, replicates on both, and survives a fused TensorRT fp16 engine. CPUxGPU models do not absorb it: per‑lockable‑point EMC tables are needed, a scoped inversion shows monotone assumptions can pick the wrong direction, and clustered misses make aggregate QoS rates understate deployment risk. We release the harness; this complements, not rebuts, the state of the art within its CPUxGPU scope.
Authors:Samah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Elyas Irankhah, Sreeraj Ramachandran, Ashley Hagaman, Sarah Lowe, Aimee Roundtree
Abstract:
Motivation: Patient‑generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient‑centered outcomes research. Prior work introduced the PV‑Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine‑tuning (SFT) alone struggles with rare, fine‑grained, and unevenly distributed errors, particularly in token‑critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token‑critical errors beyond the reach of supervised fine‑tuning. Our method introduces (i) a preference objective with token‑level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion‑aware preference pair construction to better capture low‑separation distinctions. We further incorporate token‑importance weighting and inverse‑frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub‑code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data‑Mining‑Lab‑Yale/PVminerLLM2
Authors:Bita Azarijoo, John Krumm, Cyrus Shahabi
Abstract:
Human mobility appears highly diverse, yet much of a person's daily mobility can be explained by a small set of recurring behavioral templates, such as commuting, school‑centered activities, caregiving, nightlife, or errand patterns. We present \textttIBAD (\underlineInterpretable \underlineBehavioral \underlineAnomaly \underlineDetection), a framework that learns interpretable daily mobility templates and represents each individual as a distribution over mixtures of these templates. Rather than focusing on specific locations, IBAD characterizes activities that individuals perform across locations. This approach first discovers global behavioral templates using Latent Dirichlet Allocation (LDA), then employs a hierarchical self‑supervised model to learn normal behavior of individuals from their soft behavioral templates. We also introduce a \emphsplicing benchmark that creates controlled behavioral mismatches between an individual's historical profile and injected mobility patterns. Experiments on real‑world and synthetic datasets show that daily behavior can be effectively decomposed into a small number of interpretable templates. Crucially, we show that the learned behavioral archetypes \emphtransfer across distinct geographic and demographic contexts. Furthermore, IBAD maintains a robust competitive performance across all settings. For reproducibility purposes, the code is accessible at ~\hrefhttps://github.com/USC‑InfoLab/IBADhttps://github.com/USC‑InfoLab/IBAD.
Authors:Nafiseh Nikeghbal, Amir Hossein Kargaran, Shaghayegh Kolli, Jana Diesner
Abstract:
Standard accuracy benchmarks are designed to test how closely large language models (LLMs) approach correct answers, but are not suitable for testing whether LLMs stick with a correct answer when that answer is challenged by a plausible counter‑argument. We introduce a controlled protocol for evaluating answer stability: after a model answers a multiple‑choice question correctly, we challenge the model's answer with a coherent argument for an incorrect option and measure whether the model flips. The setup a) isolates argumentative content from overt social pressure and b) varies argument length, self‑attribution, and cross‑model source. Across seven frontier models and 57 MMLU subjects, flip rates range from 17.5% to 97.3%, revealing large differences in stability that are not captured by accuracy metrics alone. We find that self‑attribution consistently increases flip rates (mean +7.1pp, up to +18.7pp). Also, pooling wrong‑answer arguments across models and selecting the most effective one per question yields stronger adversarial challenges than relying on any single source model. We further construct MaxFlip, a curated challenge set that amplifies flips by up to +23.6pp over standard self‑generated challenges. We release the protocol, challenge records, and MaxFlip to support stability evaluation alongside standard accuracy benchmarks. Materials are available at https://github.com/nafisenik/WhoFlips and https://hf.co/datasets/nafisehNik/WhoFlips.
Authors:Toshitaka Hayashi, Dalibor Cimr, Hamido Fujita, Richard Cimler
Abstract:
One‑class classification (OCC) is a classification problem in which the training data contains only one class. The one‑class support vector machine (OCSVM) is one of the most competitive OCC algorithms. However, OCSVM has scalability issues with large‑scale datasets. This paper proposes the acceleration strategy of OCSVM. The idea is to decompose the dataset into samples and train OCSVM models for single data points. Subsequently, ensemble learning is applied to combine all models to compute the OCSVM model for the dataset. In addition, further acceleration is achieved through a data‑reduction strategy with an OCSVM model trained on the average of the training samples. The experiment compared the proposal and traditional OCSVM using the Python package. The proposed strategy is faster than traditional OCSVM, while achieving similar classification results. Moreover, the proposed strategy can create one‑to‑one correspondence between samples and models. Source code is uploaded at https://github.com/ToshiHayashi/ODSVM
Authors:Qiyue Liang, Steven Ingram, George Vanica, Andi Gavrilescu, Newfel Harrat, Hassan Sipra, Sethuraman Sankaran
Abstract:
Translating deep learning models from PyTorch's flexible, object‑oriented design to JAX's functional, stateless setup is usually a manual and error‑prone task. Automated migration is challenging because Large Language Models (LLMs) struggle with strict and dynamic API alignment and are prone to mistakes for exacting operations. We propose a fully autonomous system that combines In‑Context Learning (ICL) with oracle‑driven self‑debugging. First, we curated an ICL context that serves as a strict reference for idiomatic JAX styling and test case generation. Second, instead of depending on the LLM to deduce mathematical outputs, we run the source PyTorch modules to get their actual dynamic tensor states. This creates an unchangeable execution oracle. We then use an autonomous agentic loop to synthesize tests based on the oracle data. The test cases are executed repeatedly, and the traceback is sent back to the LLM for self‑correction. Ablations show that combining ICL references with oracle grounding and self‑debugging greatly outperforms pure instructional and basic agentic baselines. This improvement does not add an excessive computational overhead. Our lightweight pipeline achieves 91% numerical equivalence (compared to baseline: 9%, instruction + self‑debugging: 27%) on neural modules, providing a highly reliable, scalable blueprint for cross‑framework migration. This has been validated across several state‑of‑the‑art models including SAM (segment anything), T5, Code Whisper amongst others showing high numerical equivalency. Code: https://github.com/AI‑Hypercomputer/accelerator‑agents/tree/main/MaxCode
Authors:Ji Feng, Zhouxing Shi
Abstract:
With large language models (LLMs) increasingly applied to mathematical reasoning, formal proof assistants such as Lean can be leveraged to verify reasoning outputs with machine‑checkable rigor, enabling use cases such as answer selection in test‑time scaling with K sampled candidate answers. However, employing Lean requires that LLM outputs, originally in natural language, first be formalized. Existing Lean‑based answer‑selection work uses an autoformalization model to generate a formal statement in Lean for each candidate answer independently, incurring a significant computational cost. We propose BASE, a base‑and‑edit pipeline that formalizes a single base candidate per problem and derives the remaining K‑1 statements by editing the answer expression in place. To facilitate this, we train a rewriter model LEANSCRIBE to localize the answer in the base formalization and generate a reusable edit function for the other K‑1 candidates. BASE simultaneously improves selection accuracy and reduces formalization cost ‑ a Pareto improvement that holds on all 12 (dataset, solver) configurations across four benchmarks and three solvers, cutting autoformalizer calls by about 5x at K=8, with the reduction expected to become larger as K grows. Code is available at https://github.com/ucr‑rai/base‑and‑edit.
Authors:Yuchao Wu, Junqin Li, XingCheng Liang, Yongjie Chen, Yinghao Liang, Linyuan Mo, Guanxian Li
Abstract:
Retrieval‑Augmented Generation (RAG) offers an effective approach for large language models to access external knowledge. However, existing methods rely on dense similarity retrieval and face inherent limitations in handling structured constraints and multi‑hop reasoning. Incorporating knowledge graphs partially alleviates these issues, but at the cost of semantic fragmentation, high maintenance overhead, and difficult incremental updates. This paper introduces SAG (SQLRetrieval Augmented Generation), a structured architecture for retrieval and agent systems. Instead of pre‑building a global static graph, SAG converts each chunk into one semantically complete event and a set of indexing entities, then uses SQL join queries to dynamically link events that share entities into local hyperedges,constructing, at query time, a dynamically instantiated local index structure. This design avoids the need for global graph rebuilding and ongoing maintenance; the system naturally supports incremental writes, concurrent processing, and continuous scaling through its reliance on standard database infrastructure. Across HotpotQA, 2WikiMultiHop, and MuSiQue, three standard multi‑hop benchmarks,SAG achieves the best results on 8 out of 9 Recall@K metrics, reaching 80.0% Recall@5 on MuSiQue, the benchmark with the highest multi‑hop reasoning demands.SAG has also been deployed at a production scale of hundreds of millions of data items, with online retrieval latency kept within seconds. Project site and code are available at https://github.com/Zleap‑AI/SAG‑Benchmark.
Authors:Adi Ahituv, Anat Ilivitzki, Moti Freiman
Abstract:
Anisotropic volumetric acquisitions are common in clinical MRI and volume electron microscopy (vEM), where sparse through‑plane sampling creates thick slices or sections that degrade orthogonal reformats and downstream analysis. We present CRIS, a cross‑plane self‑supervised framework for isotropic restoration without paired isotropic ground truth. CRIS casts 3D restoration as 2D stripe completion on orthogonal reformats of an isotropic grid: high‑resolution in‑plane slices are synthetically degraded and periodically masked for training, while at inference blank slices define the isotropic grid, two orthogonal reformats are restored, and predictions are fused by multi‑view averaging. We evaluate CRIS on two MRI cohorts and two microscopy benchmarks up to 8x anisotropy. On brain MRI, CRIS achieves 32.921 +/‑ 0.436 dB PSNR and 0.9631 +/‑ 0.0027 SSIM, outperforming interpolation, SMORE4, SIMPLE, SA‑INR, and ATME, and gives the best segmentation consistency (Dice 0.940 +/‑ 0.004, ASSD 0.245 +/‑ 0.014 mm, HD99 1.275 +/‑ 0.061 mm). On reference‑free abdominal MRI, CRIS reduces FID/KID to 48.714/0.023. On vEM, CRIS outperforms interpolation, NIIV, and vEMINR, reaching 29.133 dB/0.834 3D PSNR/SSIM at 4x, 27.123 dB/0.734 on EPFL at 8x, and 21.915 dB/0.699 on noisy hemibrain data. In a robustness experiment, one variable‑gap CRIS model evaluated across gap factors 3‑‑7 and coronal, axial, and sagittal degradations maintained higher PSNR/SSIM than interpolation (36.36‑‑31.14 dB and 0.977‑‑0.932 vs. 33.07‑‑27.85 dB and 0.951‑‑0.853). These results support CRIS as a modality‑flexible route to isotropic restoration without paired isotropic targets or configuration‑specific retraining. Code is available at https://github.com/adi‑hatav/CRIS.
Authors:Zhengyang Shen, Kai-Hung Chang, Erroll Wood, Deying Kong, Bo Peng, Timo Bolkart, Jinlong Yang, Bowen Zhao, Danhang Tang, Sasa Petrovic, Emre Aksan, Jérémy Riviere, Vassilis Choutas, Delio Vicini, Jay Busch, Shichen Liu, Zhe Cao, Hugh Liu, JingJing Shen, Jonathan Taylor, Mingsong Dou
Abstract:
Robust, high‑fidelity 3D hand capture, while fundamental to digital human creation, remains challenging with practical multi‑view systems that balance rich photometry with the geometric ambiguities of reconstruction arising from limited viewpoint density. This paper presents an end‑to‑end pipeline for dynamic hand performance capture and registration, specifically designed for view‑efficient setups (～20 views). We address key challenges with two primary innovations. First, to overcome reconstruction difficulties like limited view overlap and background clutter, our mask‑free neural method robustly extracts detailed hand geometry and appearance from unmasked images using scene parameterization and scenario‑specific density regularization. Second, addressing registration challenges such as accurately capturing non‑linear skin deformations and ensuring plausible results during severe self‑contact, we propose a physics‑inspired framework. It aligns reconstructions to a personalized hand model by optimizing intrinsic volumetric offsets within its canonical tetrahedral mesh, alongside pose parameters. This approach, supported by robust losses and optimization, captures fine surface deformations, ensures plausible results under severe articulation and self‑contact, and demonstrates strong tolerance to input noise. We demonstrate the scalability and robustness of our automated pipeline on an extensive dataset of over 12,000 sequences, from which we also derive a large‑scale, high‑quality synthetic 2D/3D hand dataset for training downstream tasks. This showcases its effectiveness for single hands, intricate two‑hand interactions, and natural hand‑object manipulations. Our method achieves state‑of‑the‑art reconstruction fidelity in view‑efficient, unmasked scenarios and highly accurate registration. Our project page are available at https://vephand.github.io/.
Authors:Gaston Besanson
Abstract:
Agentic AI systems act through tools and sub‑agents, yet the controls meant to bound their financial and environmental cost still sit on dashboards evaluated beside or after execution. Green SARC applies the SARC governance‑by‑architecture framework ‑‑ four enforcement sites in the agent loop ‑‑ to FinOps and GreenOps, contributing the theory of what to enforce and how to predict it. We report four policy‑independent results. (i) The unconstrained "State Snowball" is Θ(n^2) in loop depth; on 3,000 real multi‑step plans (SWE‑rebench) it holds on 100%, with median curvature \hatc_2=216 exceeding the linear‑accretion prediction p/2=134 ‑‑ real plans accrete faster than the model. (ii) On real residuals the Normal‑σ gate under‑covers (92% at nominal 95%); split‑conformal calibration holds (95.2%). (iii) A soft Lagrangian penalty tuned to the budget in expectation breaches it on 91.5% of seeds; the architectural gate breaches 0%. (iv) Under binding budgets the gate's over‑budget incidence is 0% on synthetic and real (BurstGPT) arrivals. End‑to‑end token/USD/carbon savings (47‑‑55%) are real but policy‑dependent in magnitude ‑‑ set by a scope‑cap knob, not by gate rejections. The library is open‑source, dependency‑free, and ships a regeneration script for every cited number.
Authors:Sasank Tumpati, Devansh Agarwal, Ayush Kedia, Arjun Neekhra, Murari Mandal, Krishna Garg, Yash Sinha, Suman Gupta, Dhruv Kumar
Abstract:
Existing financial‑NLP benchmarks mostly evaluate prepared artifacts such as filings, tables, or extracted values. Real accounting begins earlier: source documents must be reconciled into cited journal entries, aggregated into a balance sheet, and checked for contradictions. We introduce FinBalance, a multi‑document accounting reconciliation benchmark built from source‑document bundles across eight industries, three period types, and five difficulty levels. Human‑authored business scenarios, accounting policies, tax/FX treatments, document schemas, distractors, and inconsistency templates are composed by a deterministic generator whose ledger produces journal entries,balance sheets, and 23 inconsistency‑code labels. On a 710‑record evaluation split, six contemporary LLMs reach at most 46% exact final‑balance‑sheet accuracy. Four models show a 26‑41 pp gap between BS_exact, the model's reported balance sheet, and BS_recon, the balance sheet obtained by replaying its entries through our ledger. Models often recover numerically plausible entries but fail to bind them to supporting documents and aggregate them consistently. Citation‑pressure prompting barely changes document‑linking errors, while ledger‑feedback ablations substantially improve reported balance sheets and expose inconsistency‑detection trade‑offs. Expert finance reviewers validate the benchmark design and labels.
Authors:Zhangfeng Hu, Zefan Yang, Ge Wang, Tanveer Syeda-Mahmood, Anushree Burade, Mannudeep Kalra, Pingkun Yan
Abstract:
Chest X‑ray (CXR) interpretation often requires longitudinal comparison to assess disease progression. Existing approaches typically rely on temporal feature fusion or inter‑study discrepancy modeling, yet remain limited in capturing subtle progression semantics and overlook the inherently directional nature of disease trajectories. In this paper, we propose ProTrans, a novel vision‑language pretraining framework that formulates disease progression as a directional semantic transition between paired CXR studies. ProTrans leverages radiology reports to anchor individual CXR representations within interpretable disease states, and introduces a learnable progression feature map to explicitly encode semantic shifts between states, aligned with report‑derived progression descriptions. To enforce direction‑aware perception, ProTrans incorporates a reversed temporal modeling process and imposes bidirectional reconstruction consistency across states and transitions, thereby disentangling directional semantics and promoting coherent trajectory modeling. Extensive experiments on longitudinal downstream tasks, including disease progression classification and progression captioning, demonstrate that ProTrans consistently outperforms existing methods, establishing a unified pretraining framework for longitudinal CXR understanding. https://github.com/RPIDIAL/ProTrans
Authors:Jyothiraditya Lingam, Nikhileswara Rao Sulake, Sai Manikanta Eswar Machara
Abstract:
We present GOOSE‑M2F, a task‑specific adaptation of Mask2Former for the GOOSE 2D Fine‑Grained Semantic Segmentation (FGSS) Challenge at ICRA 2026. The GOOSE benchmark spans 64 fine‑grained classes across unstructured outdoor terrain with a severely long‑tailed distribution, where rare classes occupy fewer than 50 pixels per image. We extend the Swin‑Large Mask2Former baseline with three targeted contributions: (1) 200 object queries to eliminate representational saturation; (2) a Feature Refinement Module (FRM) combining ASPP‑lite and CBAM dual‑attention; and (3) an Auxiliary Supervision Head that delivers direct per‑pixel gradients for rare classes. A multi‑stage training strategy pairs Distribution‑Balanced loss, Rare‑Class Copy‑Paste augmentation, dynamic IoU‑aware re‑weighting, and EMA. At inference, a dense sliding‑window engine with 2D Gaussian kernel blending and 4‑scale TTA adds +10.57%. GOOSE‑M2F achieves 70.08% Official Composite mIoU (63.55% fine, 76.61% coarse), placing 3rd on the GOOSE 2D FGSS leaderboard. Code and trained models are publicly available at GitHub: https://github.com/Aditya‑Lingam‑9000/GOOSE‑M2F and Hugging Face: https://huggingface.co/XYZ9843/GOOSE‑M2F.
Authors:Xuanle Zhao, Qiushi Sun, Jingyu Xiao, Xuexin Liu, Haoyue Yang, Qiaosheng Chen, Xianzhen Luo, Jing Huang, Yufeng Zhong, Lei Chen, Shuai Fu, Zhenlin Wei, Jinhe Bi, Lei Jiang, Haibo Qiu, Siqi Yang, Peng Shi, Jian Hu, Zhixiong Zeng
Abstract:
While Large Language Models (LLMs) have substantially advanced text‑to‑code synthesis, many real programming tasks specify intent through visual artifacts such as screenshots, charts, vector drawings, videos, and interactive states. These tasks require models to connect visual perception to executable programs, because correctness depends not only on syntax but also on layout, data semantics, interaction behavior, and domain‑specific constraints that apply after execution. This survey examines Multimodal Code Intelligence, covering systems that generate, edit, refine, or reason with code under visually grounded inputs and outputs. We first formulate the field by the role that code plays in each task, distinguishing code as a rendered artifact, an editable symbolic structure, a scientific representation, an intermediate reasoning trace, or an executable policy or tool interface. We then organize benchmarks and methods into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. This taxonomy connects mature artifact‑generation problems to emerging agentic and unified settings and allows us to compare how different tasks treat evidence of correctness. Looking ahead, we argue that future research may benefit from four verification‑centered directions. Multi‑signal validation can combine complementary evidence of correctness, multi‑state verification can test behavior across execution trajectories, cross‑task transfer testing can probe reusable visual‑code skills, and verifiable agent traces can reveal whether agent actions are grounded in visual evidence. Together, these directions may move this field from single‑output imitation toward evidence‑grounded executable systems. An ongoing project and resources are available on \hrefhttps://github.com/xjywhu/Awesome‑Multimodal‑LLM‑for‑CodeGitHub.
Authors:Bo Peng, Xu Chen, Yi Gu, Hidenobu Matsuki, Mingsong Dou, Jingjing Shen, Deying Kong, Juyong Zhang, Zhengyang Shen
Abstract:
The growing demand for high‑fidelity 4D hand‑object interaction (HOI) data in embodied AI and spatial computing is currently bottlenecked by the reliance on pre‑scanned object templates and physical markers. While recent methods have demonstrated promising results in reconstructing 4D hand‑object interaction from videos, they are highly sensitive to initial estimates of hand and object poses. Yet, estimating these poses from images is challenging, in particular under severe occlusion which is inherent in hand‑object interaction scenarios. We propose a novel system for the robust and accurate reconstruction of hands and objects from synchronized and calibrated multi‑view videos without requiring any templates or markers. Our system consists of two main components with key innovations: (1) a multi‑view feed‑forward transformer model that aggregates cross‑view geometry and temporal cues to provide a reliable, metric‑consistent initialization for both poses and dense object geometry, and (2) a hand‑object physics‑aware Gaussian‑based optimization framework to refine the initial estimates, integrating tetrahedral constraints, collision refinement, and appearance decomposition to produce physically plausible and visually accurate reconstruction. Validated on public benchmarks and an extensive internal dataset, our pipeline achieves highly robust, artifact‑free reconstruction, providing an efficient foundation for automated 4D asset generation. Our project page are available at https://zyshen021.github.io/HOSTPG/.
Authors:Yilong Zuo, Xunkai Li, Jing Yuan, Qiangqiang Dai, Hongchao Qin, Ronghua Li
Abstract:
Long‑document multimodal question answering requires a system to locate sparse evidence in long PDFs and integrate clues from text, tables, images, charts, and complex layouts. Existing RAG methods mostly rely on fixed Top‑k retrieval over text chunks or pages. Text retrieval can compress the context but often loses visual and layout information; page‑level visual retrieval preserves the original page, yet it also sends large irrelevant regions to the reader, leading to a static trade‑off among evidence coverage, noise, and inference cost. This paper proposes MAGE‑RAG, a multigranular adaptive graph evidence framework for long‑document multimodal QA. MAGE‑RAG uses page retrieval as the entry point for query‑time evidence construction. Offline, it builds an evidence graph with page nodes and element nodes, encoding containment, reading order, layout adjacency, section hierarchy, and semantic‑neighbor relations. At query time, an online evidence controller iteratively activates, opens, searches, and prunes evidence under explicit budgets. The resulting evidence subgraph is then rendered into structured multimodal reader input, allowing the LVLM to consume compact and relevant evidence within a limited context. On LongDocURL and MMLongBench‑Doc, we establish a unified comparison and analysis protocol covering Direct MLLM, Text RAG, Page‑level Visual RAG, and Graph/Agentic RAG. Experiments show that MAGE‑RAG achieves 52.75 overall accuracy on LongDocURL, and 53.26 accuracy with 51.19 F1 on MMLongBench‑Doc. Fine‑grained breakdowns, budget‑performance curves, ablations, and trace‑based analysis further show that query‑time evidence subgraph construction can balance dispersed evidence coverage with context‑noise control. Our code is available at https://github.com/laonuo2004/MAGE‑RAG.git.
Authors:Dongxu Yang
Abstract:
Where an LLM sits in an agent memory pipeline ‑‑ between the recall plane that retrieves stored facts (extensively benchmarked) and the control plane that mutates them via supersede, release, purge (largely untested) ‑‑ shapes which forgetting failure modes the system recovers. Comparing thirteen system configurations on a 385‑case adversarial surface, we observe three placement regimes with partly complementary coverage: deterministic primitives suffice for lexical/temporal categories but fail canonicalization (5% on identifier‑obfuscation, 0% on cross‑lingual); inscribe‑time LLM recovers canonicalization (100%) but cannot help intent‑aware deletion (0% on prefix‑collision and compound‑fact); a mutation‑time hook recovers intent‑aware deletion (78‑85%) and brightens nearly all categories simultaneously (91.7‑93.2% overall, 0.17 per 385‑case run, 2.3s/case mutation latency vs. 64‑191ms/case deterministic, recall path unchanged). We expose the trade‑off via ForgetEval, a 1000‑case templated suite plus a 385‑case adversarial layer (132 hand‑crafted + 253 LLM‑drafted oracle‑validated) scored by deterministic substring match, paired with a six‑method Adapter Protocol with honest N/A scoring that lets heterogeneous memory stores enter in 130 lines. Admission is corroborated by 10‑annotator IAA (Fleiss' kappa = 0.958) and a 77‑case external‑authored subset (four blind contributors) that replicates the canonicalization asymmetry and amplifies the joint‑placement lift (+27.8 pt). Production failures are predominantly forgetting failures rather than recall failures, yet existing benchmarks measure only recall. ForgetEval and all adapters are released under MIT.
Authors:Kacper Wyrwal, İsmail İlkan Ceylan, Alexander Tong
Abstract:
Flow matching is a powerful generative modeling framework, valued for its simplicity and strong empirical performance. However, its standard formulation treats signals on structured spaces, such as fMRI data on brain graphs, as points in Euclidean space, overlooking the rich topological features of their domains. To address this, we introduce topological flow matching, a topology‑aware generalization of flow matching. We interpret flow matching as a framework for solving a degenerate Schrödinger bridge problem and inject topological information by augmenting the reference process with a Laplacian‑derived drift. This principled modification captures the structure of the underlying domain while preserving the desirable properties of flow matching: a stable, simulation‑free objective and deterministic sample paths. As a result, our framework serves as a drop‑in replacement for standard flow matching. We demonstrate its effectiveness on diverse structured datasets, including brain fMRIs, ocean currents, seismic events, and traffic flows.
Authors:Yanxin Xi, Xiang Su, Jie Feng, Yu Liu, Sasu Tarkoma, Pan Hui
Abstract:
Understanding urban wellbeing from multimodal data requires integrating heterogeneous spatial and temporal signals, posing significant challenges for current multimodal large language models (MLLMs). We introduce UrbanWell, a large‑scale benchmark designed to systematically evaluate the spatio‑temporal reasoning capabilities of MLLMs for urban wellbeing analytics through joint modeling of satellite and street view imagery. UrbanWell spans 38 cities across multiple years and includes diverse indicators covering (1) environmental conditions (CO_2, NO_2, PM2.5, and Normalized Difference Vegetation Index), (2) spatial accessibility (minimum distance to supermarkets and restaurants), (3) urban form (road length, road density, and land use), (4) urban vitality (population, economic activity diversity, and land use diversity), and (5) subjective perception attributes (e.g., safety, beauty, liveliness, wealth, and quietness). All indicators are aligned at grid level to enable standardized evaluation. Beyond static prediction, UrbanWell defines temporal reasoning tasks, including future value forecasting from historical observations and temporal trend classification. We benchmark 15 state‑of‑the‑art representative MLLMs in a zero‑shot setting, providing a comprehensive comparative evaluation across spatial and temporal dimensions. Experimental results indicate that while MLLMs capture salient spatial and perceptual cues, their performance varies substantially across heterogeneous urban indicators spanning environment and subjective perception. UrbanWell serves as a unified benchmark for evaluating multimodal spatial and temporal reasoning in urban wellbeing analytics, offering a standardized testbed for systematic assessment and future research on multimodal urban intelligence. Our codes and datasets are accessible via https://github.com/axin1301/UrbanWell‑Benchmark.
Authors:Jialong Mai, Jinxin Ji, Xiaofen Xing, Wencui Liu, Xiangmin Xu
Abstract:
Non‑verbal vocalizations (NVs), such as laughter, sighs, and coughs, are important acoustic cues for emotion and intent. Existing speech quality assessment methods typically focus on overall naturalness, while non‑verbal TTS evaluations mainly examine whether a target NV appears with the correct type and position. However, the perceptual quality of NV events themselves remains underexplored. To address this gap, we construct an NV‑MOS dataset containing outputs from multiple NV‑TTS systems and naturally occurring NV samples, with ratings collected from three acoustic experts on a perceptual quality scale. We further analyze audio‑capable multimodal large language models such as Gemini and find clear inconsistencies between their scores and expert ratings. These results suggest that general‑purpose multimodal models cannot reliably replace human judgments for NV quality assessment. We then propose NVMOS, to our knowledge the first model that can reliably predict the perceptual quality of NV events in speech. Experimental results show that, with a local NV‑event focusing module, NVMOS reaches expert‑level or stronger agreement with human MOS.
Authors:Kaiqing Lin, Zhiyuan Yan, Ruoxin Chen, Ke-Yue Zhang, Yue Zhou, Caiyong Piao, Bin Li, Taiping Yao, Bo Wang, Youchang Xiao, Shouhong Ding
Abstract:
Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding. As AI‑generated images become realistic, semantic‑level inconsistencies alone are often insufficient for reliable detection. This motivates a critical question: whether MLLMs can achieve full‑spectrum forensic signal perception, i.e., capturing low‑level generator artifacts without sacrificing pre‑trained semantic knowledge. We further perform a layer‑wise analysis of forensic signal perception in MLLMs, showing that semantic information is primarily formed in the early‑to‑middle layers, whereas direct fine‑tuning for artifact learning disrupts these semantic representations. Based on this insight, we propose Deep Visual Residual MLLM (Deep‑VRM) to preserve early semantic processing while injecting artifact‑specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal‑level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input, achieving robust and generalizable detection performance. Extensive experiments show that our method achieves state‑of‑the‑art across most benchmarks. The code and data are available at https://github.com/KQL11/Deep‑VRM.
Authors:Dimitar Nedanovski, Svetoslav Nenov, Dimitar Pilev
Abstract:
CMA‑ES is, per run, a local optimizer; multimodal search relies on restart strategies such as IPOP and BIPOP, which draw every restart uniformly and reuse no information from previous evaluations. Multi‑Start Clustering CMA‑ES (MSC‑CMA‑ES) makes restarts structure‑aware: in alternating cycles, a Sobol pre‑sample is partitioned into approximate basins of attraction by nearest‑better clustering, restarts are seeded basin by basin with locally scaled step sizes and population sizes, redundant basin visits are detected and excluded, and the remaining budget is spent on an unbounded local refinement of the best‑so‑far solution. We evaluate the method on four CEC suites (CEC2014, CEC2017, CEC2020, CEC2022) at their official budgets, across ten (suite, dimension) cells with dimensions 5‑30, 51 runs per function, against BIPOP‑CMA‑ES and five differential‑evolution algorithms (ARRDE, jSO, j2020, NL‑SHADE‑RSP, LSRTDE). Read per function class, MSC‑CMA‑ES leads on one class, is mixed on a second, and trails on the third. On composition functions, MSC‑CMA‑ES attains the best value on all four aggregate measures, with 2.7x the fixed‑budget target coverage of BIPOP‑CMA‑ES ‑ the highest composition coverage of any algorithm evaluated. On basic functions, it achieves the best (lowest) median error but exhibits a lower deep‑target coverage ‑ the measured price of spending budget on landscape discovery. On hybrid functions both CMA variants trail the leading DE algorithms; the deficit belongs to the CMA family, not to the restart mechanism. All results and scripts are publicly available.
Authors:Miso Choi, Seonga Choi, Mincheol Kwon, Woosung Joung, Jinkyu Kim, Jungbeom Lee
Abstract:
Recent advances in large language models (LLMs) have produced many specialized multimodal LLMs (MLLMs) that share common foundational LLMs, forming distinct model lineages. It remains unclear whether a fundamental behavioral link exists between the foundational LLMs and downstream variants. We investigate this question by quantifying head‑level context‑truthfulness scores. Across diverse LLM and MLLM lineages, including Vicuna‑, Qwen2.5‑, LLaMA2‑, and Mistral‑based models, we find that Truth Scores are strongly preserved within model families, even after instruction tuning or multimodal adaptation. We further show that this inheritance is consistent with attention‑head weight preservation, and that context‑truthful heads attend to query‑relevant evidence. Building on this finding, we propose TruthProbe, a soft‑gating strategy that amplifies context‑truthful heads while preserving other head contributions. TruthProbe improves contextual truthfulness on HaluEval and reduces multimodal hallucination on POPE and CHAIR, with base‑LLM Truth Scores transferring effectively to their fine‑tuned LLM and MLLM descendants. Code is available at https://github.com/miso‑choi/TruthProbe.
Authors:Siya Yang, Nanxiang Jiang, Zhaoxin Fan, Yunfeng Diao
Abstract:
The rapid progress of visual autoregressive (VAR) models has unlocked a transformative frontier for high‑fidelity text‑to‑image synthesis, while heightening concerns over the safety alignment of generated content. Naive application of existing erasure techniques to VAR models causes catastrophic semantic collapse and visual artifacts, since they are predominantly designed for the homogeneous denoising steps of diffusion models. To address this foundational challenge, we first propose the Semantic Singularity Axiom, which posits that any target semantic concept embedded within a prompt is definitively locked at Scale‑0. Then rigorously validate this axiom through our proposed Incremental Semantic Saliency Analysis (ISSA),which also enable the community to transparently inspect the coarse‑to‑fine semantic injection process. Guided by this insight, we introduce the first scale‑aware concept erasure framework (SACE) for VAR models. By strictly confining interventions to the first scale, our approach couples an Entropy‑Regularized Erasure Objective to prevent high‑entropy sampling degeneration, alongside a restorative preservation loss to safely anchor the integrity of entangled benign priors. Extensive experiments demonstrate that our method achieves surgical concept erasure performance across various domains with minimal training overhead, timely and elegently resolute the critical safety vulnerabilities inherent in emerging VAR architectures. Code is available at: https://github.com/limerenceysy/SACEhttps://github.com/limerenceysy/SACE.
Authors:Dabin Kim, Junwon Lee, Juhan Nam
Abstract:
This paper addresses timbral ambiguity in instrument timbre transfer under fine‑grained structural conditions. We argue this issue stems from instrument‑specific expressive details in these conditions, which conflict with the target timbral properties. For example, imposing a violin's pitch‑dominant vibrato contours onto a flute, which naturally exhibits loudness‑dominant vibrato, impairs timbral fidelity. We propose AdaTT, a target‑adaptive system that ensures high timbral fidelity across diverse timbre transfer scenarios within the ControlNet scheme. It selectively scales the frame‑wise influence of pitch and loudness controls via text prompts to match the target instrument's identity. We also present a semi‑automatic data construction pipeline to teach the model which expressive details to transform or preserve. Results show AdaTT achieves superior timbral fidelity and naturalness while retaining score‑level content. Audio samples are available at https://dabinkim0.github.io/adatt/.
Authors:Artyom Mazur, Nina Konovalova, Aibek Alanov
Abstract:
Mechanistic interpretability seeks to explain neural network behavior by decomposing model computations into interpretable features and circuits. While transcoder‑based circuit tracing has recently enabled detailed causal analyses of large language models, multimodal diffusion transformers for image generation remain comparatively opaque. We still lack tools for understanding how semantic information propagates across denoising steps and how text and image representations interact within double‑stream MM‑DiT architectures. Existing methods provide only partial insight: attention maps expose a limited view of token interactions, while sparse autoencoders can discover interpretable features but do not directly reveal how these features are transformed and composed through nonlinear MLP layers. In this work, we extend transcoder‑based circuit tracing to multimodal diffusion transformers. We train timestep‑conditioned transcoders that faithfully approximate the input‑output behavior of MLP sublayers in FLUX.1[schnell]. By replacing MLPs with transcoders and linearizing the remaining computation, we obtain exact feature‑to‑feature attribution and recover compact, interpretable circuits. Empirically, our transcoders match or slightly outperform sparse autoencoders on the sparsity‑faithfulness tradeoff. The resulting circuits reveal mechanisms underlying attribute binding and cross‑stream semantic propagation, and provide causal explanations for systematic generation errors. Moreover, circuit‑guided interventions are substantially more precise and effective than standard SAE‑based steering. Our results demonstrate that transcoder‑based circuit analysis is feasible for state‑of‑the‑art diffusion transformers and provides a powerful framework for understanding and controlling multimodal generative models. The code is available at https://github.com/Artalmaz31/DifFRACT
Authors:Nguyen Linh Dan Le
Abstract:
Alzheimer's disease (AD) progression is often described through the amyloid‑tau‑neurodegeneration, or AT(N), cascade. However, most longitudinal models represent this cascade either as a fixed sequence of biomarkers or as a black‑box forecasting task. This makes it difficult to determine when biologically guided biomarker relationships influence future regional pathology. In this study, we introduce Bayesian Networks with Latent Time Embedding (BN‑LTE), a Bayesian structural framework for stage‑aware modeling of AD progression. BN‑LTE estimates disease pseudotime from baseline biomarker profiles and constrains directed dependencies according to biologically plausible AT(N) ordering. Posterior spline‑varying structural equations are then used to link initial multimodal measurements with future annualized regional tau‑PET change. Across repeated subject‑disjoint evaluations using ADNI data, BN‑LTE shows strong spatial reconstruction of tau progression compared with the included forecasting baselines. Beyond spatial reconstruction, BN‑LTE recovers posterior stage‑varying AT(N)‑constrained effects and identifies a mid‑pseudotime window of amyloid sensitivity. This window is supported by model‑implied g‑formula contrasts, root‑adjusted AIPW, mechanism‑sensitive ablations, and robustness analyses across spline and prior specifications. Overall, these findings position BN‑LTE as a Bayesian structural framework for forecasting tau progression while examining stage‑dependent AT(N)‑cascade mechanisms in observational longitudinal neuroimaging data. Our code is available at https://github.com/danleneurocom/BN‑LTE.
Authors:Tai Tran Tan, An Dinh Thien
Abstract:
We present our approach to SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning. Our solution uses contrastive learning with fine‑tuned sentence transformers to capture narrative similarity across abstract themes, course of action, and outcomes. We develop two pipelines: (Track A) a single‑view method that encodes full narratives with smart layer freezing to reduce overfitting, and (Track B) a multi‑view method that models theme, plot, and outcome with view‑specific projection heads and self‑supervised alignment. Both pipelines build on sentence‑transformers models and are trained with contrastive loss on synthetic data. The code is available at the following GitHub repository: https://github.com/dinhthienan33/SemEval2026‑Task4‑ttda704.
Authors:Chakshu Gupta
Abstract:
The saturation number μ^(G) of a graph G is the minimum cardinality of a maximal matching, and H(G) is its harmonic index. TxGraffiti conjectured in 2023 that μ^(G) \le H(G) for every nontrivial connected graph G, and Bıyıkoğlu refuted this by showing that the ratio μ^(G)/H(G) can be made arbitrarily large. Restricting to trees bounds the ratio sharply. Every nontrivial tree T satisfies μ^(T) < \frac32 H(T), with the constant 3/2 best possible. A complementary bound H(G) < 4μ^(G) holds for every graph with an edge, so on a nontrivial tree the saturation number is pinned to \frac14 H(T) < μ^(T) < \frac32 H(T), both constants best possible. The friendship graph F_4 is a smallest counterexample to the conjecture, on nine vertices, and the smallest tree counterexample is the subdivided star on eleven vertices. For each positive integer m a family of graphs with m hubs has ratio approaching m+1, while the conjecture holds whenever all vertices have equal degree. Both invariants arise in applications, the harmonic index as a molecular descriptor and the saturation number as a measure of adsorption inefficiency, and the bounds estimate the latter, which is NP‑hard to compute, by the former, which is computable in linear time.
Authors:Marios Koulakis, Constantin Seibold
Abstract:
A significant gap exists between theory and practice in deep learning. Generalization and approximation error bounds are often derived for simplified models or are too loose to be informative. Many rely on the manifold hypothesis and on geometric regularity such as intrinsic dimension, curvature, and reach. Progress requires insight into data‑manifold geometry and suitable benchmarks, yet existing options are polarized: analytic manifolds with known geometry but limited applicability, or real‑world datasets where geometry is only coarsely estimable. We introduce a benchmarking framework for studying data geometry. We repurpose and extend dSprites and COIL‑20 with additional transformation dimensions and dense, axis‑aligned sampling, and pair them with finite‑difference estimators that recover curvature, reach, and volume at near‑ground‑truth accuracy in a regime where general‑purpose estimators are unreliable or difficult to deploy. The framework is intended as a controlled testbed, useful as a calibration environment for geometric estimators and a sandbox for probing theoretical assumptions. To illustrate its use, we present two application studies, namely assessing the scaling behavior of the bounds of Genovese et al. and Fefferman et al., and tracking the layer‑wise geometry of a β‑VAE, highlighting the behavior of current bounds and the value of controlled benchmarks for guiding and validating future theory. A reference implementation is available at https://github.com/koulakis/manifold‑microscope.
Authors:Hyebin Cho, Jaehyuk Jang, Changick Kim, Joon Son Chung
Abstract:
Audio‑Language Models (ALMs) have shown remarkable success in zero‑shot audio classification by aligning audio waveforms with text. Recent efforts to improve downstream performance focus on learning optimal text prompts. However, previous approaches focus on the text encoder, leaving the potential of learnable prompts within the audio encoder unexplored. In this paper, we propose a novel framework that introduces trainable prompts into the audio encoder to capture task‑specific acoustic features. We demonstrate that integrating audio‑side prompt learning with existing text‑side approaches enhances few‑shot adaptation. Through extensive experiments across 11 datasets show that integrating our method as a plug‑and‑play module alongside existing text prompt tuning generally leads to performance improvements. These findings suggest that explicitly modulating the audio representation space effectively complements text‑only prompting approaches. The code is available at https://github.com/hyebin‑c/aspl.
Authors:Hassan Ismkhan, Hamid Bouchahcia
Abstract:
This paper addresses the missing‑modality challenge in multi‑modal learning by introducing Unsupervised Learning for Missing Modalities in Multi‑Modal Learning (UL4M4), a flexible framework that imputes missing feature embeddings in a task‑independent manner before supervised prediction. We propose modality‑specific normalization and a novel partial‑modality distance metric to enable fair clustering of incomplete observations, capturing cross‑modal structures while preserving scale‑invariance across varying dimensionalities and modality counts. Cluster centers from this unsupervised stage guide an iterative greedy imputation process for any missing modalities during training or inference, supporting arbitrary numbers of modalities and arbitrary missing patterns per sample. The imputation module is lightweight, uses frozen encoders, and decouples from the downstream task, allowing easy integration with any fusion/prediction architecture. Extensive experiments under diverse and highly incomplete regimes demonstrate UL4M4's robustness, achieving, to the best of our knowledge, the first consistent F1‑Micro scores above 0.7 on challenging missing configurations even when more than 50% of modality slots are missing. Results are also stable across cluster sizes and significantly outperform state‑of‑the‑art baselines. Code is available here: https://github.com/h‑ismkhan/Multimodal‑Learning‑with‑Missing‑Modalities‑via‑Unsupervised‑Learning.
Authors:Shuaike Zhang, Shaokun Wang, Haoyu Tang, Jianlong Wu, Liqiang Nie
Abstract:
Embodied Continual Learning (ECL) aims to enable robots to continually acquire new manipulation tasks while retaining previously learned behaviors under closed‑loop control. Compared with conventional continual learning, ECL suffers from more severe catastrophic forgetting. Feature drift accumulated under closed‑loop control progressively propagates through sequential decision‑making, leading to degradation of previously learned behaviors. A key challenge in ECL lies in structured skill reuse across continually evolving tasks, since existing methods primarily focus on skill learning without explicitly organizing them for coherent task execution. To address this issue, we propose SCE, a Skill‑Compositional Experts framework for ECL. SCE builds a skill base via Compositional Skill Grounding (CSG), which decomposes task demonstrations into reusable skills. Based on this, Dual Execution‑and‑Transition Experts (DETE) enable new task learning through skill composition, where one branch ensures skill execution and the other supports transitions between skills for coherent behavior. Experiments on LIBERO benchmarks and real‑world manipulation tasks demonstrate that SCE consistently improves retention and overall task performance. Further feature drift analyses and ablation studies verify the effectiveness of our method. Project website: https://eqcy.github.io/sce/.
Authors:Yiran Wang, Zeyu Zhang, Yuanming Li, Ziming Wang, Yang Zhao
Abstract:
High‑quality 4D head avatars from one or a few source portraits are central to telepresence, AR/VR, and digital‑human interaction. 3D Gaussian Splatting (3DGS) has emerged as the dominant representation, with two complementary regimes (generalizable feed‑forward predictors and per‑subject refiners) maturing in parallel. However, existing feed‑forward predictors are trained on a single dataset family with a hard‑coded source count, inheriting the corresponding domain bias. Per‑subject refiners require 300K‑‑600K iterations and rely on adaptive densification that destroys upstream Gaussian layouts, preventing the two regimes from sharing a representation end‑to‑end. To bridge both regimes we propose SpatialAvatar‑0 on a shared FLAME‑mesh‑bound Gaussian representation: a feed‑forward generator with a parameter‑free K‑source mean‑pool and a monocular‑temporal to multi‑view‑spatial two‑phase schedule that anchors against identity‑prior collapse onto the smaller multi‑view set. We further introduce a 10K‑iter layout‑preserving per‑subject refinement loop that freezes the FLAME‑binding and Gaussian count and replaces densification with a three‑component anti‑spike regularization. On VFHQ/HDTF cross‑domain zero‑shot we surpass the in‑domain leader GAGAvatar by +1.5 dB PSNR despite never training on either test domain, and on the SplattingAvatar monocular benchmark we lead every reported metric, surpassing the 300K‑iter GeoAvatar by +1.3 dB PSNR at up to 60x shorter per‑subject schedule than common SOTA baselines. Website: https://spatialwalk.github.io/SpatialAvatar‑0.
Authors:Haochen Hu, Yanrui Bin, Zhengyan Zhang, Minchen Wei, Chih-yung Wen, Bing Wang
Abstract:
The underwater images are captured within diverse water‑medium conditions, leading to complex degradation, including color bias, low contrast, and blur effect. Recently, learning‑based methods have demonstrated their potential for underwater image enhancement (UIE). However, most of the previous work focus on the training strategy or network design to make the enhanced result aligned well with the labels in datasets, ignoring that the labels are selected from the enhanced results of previous UIE methods and these pseudo‑labels are noisy. Consequently, the performance of their models is not satisfactory to a certain extent. However, collecting the true labels of the underwater images is challenging. In this work, we propose a transfer learning‑based UIE that does not require underwater images to have paired noisy or true labels for learning. Instead, the UIE task is first divided into global color correction, haze removal, and background noise suppression following the underwater physics. Then multiple types of prior from other vision tasks are leveraged as cross‑domain supervision in each step. In this way, a novel UIE is available via transfer learning, and the physics‑aligned UIE decomposition provides theoretical soundness. Qualitative and quantitative experiments demonstrate that our proposal based on physics and priors fusion achieves SOTA performance in the UIE task and effectively boosts downstream vision tasks, significantly outperforming benchmark methods. Project repo: https://github.com/Haru2022/P2‑UIE.
Authors:Cheng Zhang, Qing Cai, Xingzheng Wu, Xun Yang, Xiaojun Chang, Bingkun Bao, Liqiang Nie, Xinwang Liu, Yi Yang
Abstract:
Foundation models have demonstrated impressive performance in enhancing healthcare efficiency across a wide range of medical applications. Nevertheless, their limited ability to perceive, understand, and interact with the physical world significantly constrains their effectiveness in real‑world clinical workflows, where safety‑critical decision‑making and physical execution are tightly coupled. Recently, embodied artificial intelligence (AI) has emerged as a promising physical‑interactive paradigm for intelligent healthcare, enabling agents to operate in complex medical environments. As research in this area rapidly expands, understanding how intelligent agents function as integrated, end‑to‑end systems in clinical environments becomes increasingly critical. However, existing surveys on medical embodied AI largely emphasize individual aspects or functional components, lacking a unified system‑level organization of the field. To support and consolidate recent advances, we systematically survey the core components of medical embodied AI, with a particular emphasis on the coordinated integration of perception, decision‑making, and action. We further review representative medical applications and relevant datasets, and we analyze the major challenges encountered in real‑world clinical practice. Finally, we discuss key directions for future research in this rapidly evolving field. The associated project can be found at https://github.com/VMVLab/Medical_Embodied_AI_Paper_List.
Authors:A Padmaprabhan, Amal Shaji, Prabhat Kumar
Abstract:
Soft robots leverage compliant materials to generate motion through controlled elastic deformation, making them ideal for delicate tasks such as underwater exploration and biomimetic marine systems. Although hydraulic/pneumatic actuation remains pivotal for such systems, the lack of systematic design frameworks has hindered the development of robots capable of complex 3D motion, such as fish‑like swimming. This work introduces a topology optimization method to automate the design of a hydraulic soft fish tail, explicitly addressing the design‑dependent coupling between fluidic actuation and structural deformation. We use a Darcy law‑based model augmented with a drainage term to simulate spatially varying hydraulic pressure loads, translating these into consistent nodal forces via finite element analysis. The employed robust multi‑criteria optimization formulation balances deformation efficiency, fluid‑structure interaction, geometric manufacturability, and required stiffness for optimizing a bioinspired soft fish tail for 3D swimming kinematics. The optimized tail topology is incorporated into a pneumatic network actuator and computationally validated under various hydraulic loads, achieving tunable undulatory amplitudes and multiaxis bending for depth adjustment. The optimized 2D tail outperforms its rectangular counterpart. By cascading optimized tail segments, we demonstrate programmable swimming patterns in soft robotic fish tails at different hydraulic loads. This work advances the systematic codesign of hydraulic actuators and soft structures, offering a pathway to automate underwater robots with optimized design and vertebrate‑like agility in confined aquatic environments. Our implementations and simulations are publicly available at 'https://github.com/PrabhatIn/TO‑SoFiT'.
Authors:Salman Hussain Ali, Umberto Cappellazzo, Mirco Ravanelli
Abstract:
Fine‑tuning Transformer‑based foundation models has become the dominant strategy for domain adaptation in audio and speech processing. To reduce the computational and memory costs of this process, parameter‑efficient transfer learning (PETL) methods have been widely explored. Meanwhile, Mamba, a recent state‑space model, has emerged as a promising alternative to Transformers for sequence modeling. In this work, we present MambAdapter, a parameter‑efficient transfer learning approach that integrates Mamba into low‑rank bottleneck adapters. Our design combines parameter sharing across adapters with the injection of a lightweight Mamba module, enabling more effective modeling of audio features. We demonstrate that MambAdapter matches or outperforms strong PETL baselines on four audio classification tasks and five speech recognition languages, even when operating under reduced parameter budgets.
Authors:Nils Matteson
Abstract:
Per‑token counterfactual credit estimation asks which token in a language‑model rollout caused the final answer to be right or wrong: cut the transcript at a pivot, substitute an alternative token, replay continuations, and compare outcomes. Published methods re‑feed the transcript prefix as a fresh prompt, assuming this reproduces the state the model passed through during generation. We measure what that assumption costs on a stock inference engine, with a three‑pass design: continuations resumed from the verified decode‑time KV state, an identical second exact pass (a replica noise floor), and a re‑feed pass. Across six configurations and three models (including a GRPO‑trained checkpoint), at low‑margin decision tokens, re‑feeding changes the credit estimate at rates 14‑28 percentage points above the replica floor (7‑21pp under a treatment‑independent conditioning; problem‑clustered t = 2.9‑6.4). Most changes are zero‑boundary crossings of the quantized estimator rather than polarity reversals, and the perturbation is consistent with mean‑zero, so averaged quantities are largely safe; but selection is not: a critical‑token set chosen by thresholding |\hatA_t| under re‑feed overlaps the exact‑resume selection at Jaccard 0.34‑0.90, versus a 0.63‑0.96 replica ceiling. A causal confirmation closes the loop: under vLLM's batch‑invariant kernels all three passes are identical on every measured channel, with both disagreement rates exactly zero. Replica passes themselves disagree on 9‑23% of eligible estimates: single‑sample credit measurements at decision tokens are unreliable under any replay. Settings were fixed in advance; exact‑pass cache hits in the second campaign are instrumented (100% hit rate, 3,434 pivots); total compute was under 10 USD. We recommend that counterfactual credit studies resume decoder state or use batch‑invariant kernels, and report a replica floor.
Authors:Hyunsoo Lee, Farrin Marouf Sofian, Kushagra Pandey, Stephan Mandt
Abstract:
Collaborative generation, which coordinates multiple diffusion trajectories to extend the capabilities of pretrained priors, has emerged as a powerful paradigm for extending the applicability of diffusion models. Among existing approaches, diffusion synchronization provides a scenario‑agnostic solution by introducing general guidance mechanisms. However, current synchronization approaches rely heavily on heuristics and still require task‑specific tailoring, which limits their generalizability and performance. In this work, we mathematically derive a synchronization framework based on optimal control, providing a principled explanation of diffusion synchronization. During sampling, we optimize control variables to guide multiple trajectories toward coherent solutions while remaining close to the underlying diffusion prior. Our method operates entirely at test‑time without additional training, thereby enabling broad applicability across diverse generation scenarios when combined with strong pretrained priors. We demonstrate consistent improvements over baselines on three representative collaborative generation tasks, covering a wide range of modalities and applications. Beyond performance gains, our work establishes a novel foundation for collaborative generation, opening a principled path toward extending pretrained generative models to new collaborative generation settings.
Authors:Fuyou Mao, Beining Wu, Yanfeng Jiang, Bohan Xu, Lixin Lin, Naye Ji, Hao Zhang, Yan Tang
Abstract:
Organ segmentation from PET/CT is critical for quantitative analysis and radiotherapy planning in oncology. To ease the high annotation cost of PET/CT segmentation, semi‑supervised learning (SSL) provides a practical and effective solution for developing deep models with limited labeled data. Recent developments in visual foundation models have demonstrated remarkable adaptability with improved efficiency. In this work, we propose a mutual distillation framework that seamlessly exploits both structural and functional foundation models, which act as modality‑specific generalists for distilling knowledge from structural CT and metabolic PET imaging. By bridging the gap between the task‑specific precision of student models and the segmentation priors of generalist foundation models, we propose MuDuo, a mutual distillation framework that synergistically leverages SAM‑Med3D for CT and SegAnyPET for PET to distill their knowledge into a lightweight student network. Our approach eliminates the need for manual prompts while maximizing the utility of unlabeled data for automatic segmentation, achieving state‑of‑the‑art performance on the AutoPET dataset with only 5 labeled cases. Our source code is available at https://github.com/Wu‑beining/MuDuo.
Authors:Zihan Wang, Guansong Pang, Zelin Liu, Wenjun Miao, Jin Zheng, Xiao Bai
Abstract:
Multimodal Large Language Models (MLLMs) are increasingly used as automated judges, e.g., for image quality and safety assessment. However, their adversarial robustness remains largely unexplored, threatening the fairness and reliability of automated judging. To bridge this gap, we introduce RobustMLLMJudge, the first general framework for evaluating the adversarial robustness of general‑purpose MLLMs when functioning as judges. It covers diverse attacks against popular judge approaches across quality and safety evaluation scenarios. Using RobustMLLMJudge, we reveal that i) different MLLM judges are highly vulnerable to score‑inflating adversarial attacks; and ii) although effective, these attack methods face a critical challenge due to unique constraints in the evaluation protocols of MLLM judges. We further propose MGSIA, namely Manifold‑Guided Semantic Induction Attack, a novel method that bypasses these constraints to enable more effective and transferable attacks on MLLM judges. The core idea of MGSIA is to combine affirmative semantic induction with high‑score manifold alignment: it maximizes the probability that judges yield affirmative responses (e.g., "Yes") to binary semantic queries, while regularizing adversarial representations toward high‑score centers estimated from proxy protocols. Together, these objectives yield transferable score‑inflating perturbations. Extensive experiments demonstrate the superiority and generalizability of MGSIA in deceiving advanced MLLM judges under different evaluation scenarios, highlighting the need for robust MLLM judges. Code and data will be made available at https://github.com/mala‑lab/RobustMLLMJudge.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Cardinality‑estimation (CE) research ranks estimators by q‑error, yet it is well known that q‑error is an imperfect proxy for query‑plan quality. We give a measurement‑driven account of when it is a good proxy and when it is not, and why. Modeling plan selection as an argmin over a piecewise‑linear cost landscape, we find that plan regret (the cost of the chosen plan relative to the optimal, under true cardinalities) is governed by plan‑cost geometry in a regime‑dependent way. (i) For small errors, a true‑point condition number kappa predicts regret and out‑predicts q‑error; its predictive power decays to zero as error grows, as a local linearization must. (ii) For large errors ‑‑ where deployed learned estimators operate ‑‑ an estimator‑independent average‑case sub‑optimality measure ACS‑infinity predicts which queries are regret‑prone (Spearman rho ~ 0.54 on STATS‑CEB), while q‑error is nearly uninformative at the query level (rho ~ 0.05). (iii) The worst case is Haritsa's maximum sub‑optimality (MSO). The three are one cost‑ratio spectrum under three weightings. We prove a limit law ACS‑infinity = sum_k r_k pi_k with cardinality‑independent combinatorial weights, and validate every claim on STATS‑CEB and JOB‑light with four released estimators under pre‑registered decision rules, and confirm on real PostgreSQL runtime that ACS‑infinity predicts regret where q‑error does not. The contribution is conceptual and empirical ‑‑ an average‑case companion to worst‑case robust query optimization, and a characterization of when an accuracy metric tracks plan quality ‑‑ rather than a new estimator. Code and the full pre‑registration are public.
Authors:Terry Tong, Yu Feng, Surbhi Goel, Dan Roth
Abstract:
For tool‑augmented language models, comparing natural‑language reasoning with code‑execution pipelines is difficult because the comparison changes both the intermediate representation and the execution mechanism. We separate these factors with an intermediate intervention: the model expresses its reasoning as executable code, and the language model simulates that code in context to produce an answer. On a 40‑task verifiable algorithmic benchmark, deterministic code execution outperforms natural‑language reasoning by +31.6pp. We observe that the intermediate intervention is not meaningfully different from natural‑language reasoning (+0.15pp). These results suggest that, in our evaluated setting, changing the intermediate representation alone does not explain the tool‑use advantage, providing evidence for the performance gains requiring reliable external execution. We formalize this intuition with a simple statistical decision‑theoretic model that characterizes when execution dominates end‑to‑end risk in our disentangled trace‑generation/execution regime. We validate our theory using a reconstruction intervention that leverages a proxy language model to infer natural‑language reasoning traces from code representations, recovering performance comparable to the original natural‑language reasoning pipeline. All experiments are at https://github.com/TerryTong‑Git/ToolProj.
Authors:Xiongjun Guan, Jianjiang Feng, Jie Zhou
Abstract:
Small‑area fingerprint sensing on mobile devices creates a fundamental mismatch between acquisition and recognition: each touch captures only a tiny, pose‑varying local patch, while reliable biometric matching ultimately requires a stable and sufficiently complete fingerprint representation. Existing pipelines largely cope with this mismatch by treating repeated touches as independent partial templates, which leads to repeated registration, repeated matching, and no guarantee of adequate global coverage. In this paper, we advocate a different formulation, namely \emphaccumulative fingerprint mapping and reconstruction for small‑area mobile sensing. Rather than matching every partial patch separately, the proposed perspective converts a sequence of local observations into a unified fingerprint state that is progressively refined as new touches arrive and can be matched only once after consolidation. As a concrete baseline, we present a classical pipeline that performs patch‑wise structural feature extraction, feature‑level registration and fusion, fingerprint map construction, and phase‑based ridge reconstruction. More importantly, we position this baseline within a broader mobile fingerprint framework that integrates structured token learning, two‑stage pose reasoning, and diffusion‑based generative reconstruction. This viewpoint reframes mobile fingerprint recognition from multi‑capture multi‑match processing to accumulative map building, state refinement, and one‑shot matching, offering a principled route toward efficient, pose‑robust, and deployment‑friendly biometrics for small‑area mobile platforms. The baseline implementation has been publicly released at https://github.com/XiongjunGuan/FpReconstruction.
Authors:Yiwei Ma, Ke Ye, Weihuang Lin, Jiayi Ji, Xiaoshuai Sun, Tat-Seng Chua, Rongrong Ji
Abstract:
In recent years, there have been notable advancements in the area of instruction‑based image editing (IIE), which focuses on the automatic alteration of input images using a model. Nevertheless, assessing the effectiveness of these editing models poses a considerable challenge due to the intricate nature of instructions and the wide variety of edits. To tackle this problem, one urgent task in this domain is the development of a robust evaluation framework that can precisely gauge the quality of editing outcomes and offer valuable benchmarks to guide future improvements. To address this challenge, we present a comprehensive evaluation benchmark named I2EBench2.0, designed for single‑round and multi‑round assessment of IIE models. I2EBench2.0 has four key features: 1) Evaluation Across Single and Multi‑rounds: I2EBench2.0 simultaneously evaluates both single‑round and multi‑round instruction‑based edits, assessing the precision and consistency of the edits. 2) Extensive Evaluation Criteria: I2EBench2.0 encompasses a broad range of criteria, evaluating both high‑level and low‑level aspects of each IIE model. Specifically, it incorporates 16 dimensions for single‑round evaluations and 7 for multi‑round evaluations. 3) Alignment with Human Judgment: To ensure our benchmark aligns with human evaluation, we conducted a comprehensive user study for each criterion. 4) Research‑driven Insights: By analyzing the strengths and weaknesses of current IIE models across all 16 single‑round and 7 multi‑round dimensions, we provide critical insights aimed at directing future research in this area. We tested eight recently developed IIE models using I2EBench2.0 and derived academic insights through meticulous comparison and analysis. The related code, dataset, and images generated by all IIE models are available on GitHub: https://github.com/cocoshe/I2EBench.
Authors:Eyup Engin Kucuk, Tarik Kelestemur, Ömer Dağlar Tanrikulu
Abstract:
Qualitative coding is central to social science, but expert annotation is difficult to scale. LLMs offer a possible extension, yet require careful validation when the target construct is interpretive, theoretically loaded, and only indirectly expressed. We study this problem in a difficult case: detecting whether authors treat Bayesian models as descriptions of mental and neural mechanisms (realism) or as useful mathematical tools (instrumentalism). Our method combines a theory‑driven codebook, expert‑coded reference annotations, a diagnostic‑gated prompt‑optimization search yielding a shared zero‑shot prompt for three frontier LLMs (GPT‑5.1, Claude Sonnet 4.6, Gemini 3 Pro Preview), and multi‑rater reliability analysis. The final prompt achieved a held‑out combined reliability score of 0.76 (harmonic mean of ICC = 0.79 and α = 0.74), with all diagnostics satisfied. Deployed on 6,858 quotes from 210 articles, the three LLMs reached substantial quote‑level agreement (ICC = 0.80; α = 0.76; combined = 0.78) and near‑perfect article‑level rank stability (r = 0.96‑0.97 across rater pairs). The corpus was predominantly weakly realist, but article‑level stances were rarely uniform: only 1.4% of articles used a single band, while 59.5% spanned four or more. Low‑level perception/motor articles scored 8.8 Realism points higher than high‑level cognition articles (p < .001, d = 0.60), quantifying a long‑held qualitative intuition. We present this as an expert‑led case study; the framework is intended to generalize to similar theoretically demanding tasks, not to all qualitative analysis.
Authors:Carlos R. B. Azevedo
Abstract:
AI systems increasingly delegate decisions to specialized models, evaluators, tools, and supervisory controllers. The central AI problem is no longer only model accuracy, but uncertainty‑aware governance: how much autonomy to grant, which evidence should calibrate trust, what performance ceiling a delegated AI system can sustain, and when human intervention becomes necessary. We propose the Minimum Sufficient Oversight Principle (MSO), a variational principle for principled autonomy delegation: minimize governance burden on the Fisher information manifold subject to a delivery constraint. The resulting Euler‑Lagrange solution yields a water‑filling allocation of governed delegation across the task space. Building on a revealed‑action governed delegation channel model, we prove a capacity theorem for stationary symbolwise review policies, derive a local first‑order approximation relating workflow complexity to quality degradation, and give a drift‑dominated autonomy‑time scaling law linking intervention timing to effective capacity, complexity, and drift. Within this framework, masking appears as a structural AI‑governance pathology: corrected performance can hide the competence signal needed to calibrate trust. Synthetic simulations and a semi‑real reconstructed workflow support design prescriptions including upstream‑first correction, sensitivity‑based intervention, and explicit feasibility checks before autonomy is expanded. The result is a computable framework for uncertainty, planning, and oversight in delegated AI systems. A companion Python package is available at https://github.com/crbazevedo/delegation‑lab.
Authors:Ruofei Bai, Jie Chen, Yuxin Cai, Jun Li, Wei-Yun Yau, Lihua Xie
Abstract:
The success of generative models in language and visual generation has inspired extensive applications to generative robot planning. However, most existing works either focus on single‑robot planning, or generate multi‑robot trajectories in a sequential manner with iterative post‑processing to resolve inter‑robot conflicts. In this work, we investigate whether coordinated multi‑robot trajectories, as a special spatiotemporal distribution, can be learned and generated with a generative model in a feed‑forward manner. We propose Robots as Tokens (Roken), a unified diffusion transformer that directly generates multi‑robot trajectories that satisfy both (individual) safety and (global) connectivity constraints. The core design of Roken is to represent each robot as a discrete token, allowing them to naturally interact with each other through self‑attention, and cross‑attend to map tokens for environment layouts. We further introduce several auxiliary tasks based on Bayes' theorem to provide multi‑scale spatial‑temporal supervision for efficient learning of the conditional distribution. In training, Roken absorbs diverse expert trajectories from different team sizes. During inference, Roken behaves as a versatile multi‑robot planner that can handle single‑robot planning, coordinated multi‑robot trajectory generation, and conditional trajectory generation by fixing some robot tokens as conditions. Experiments in diverse cluttered environments show that Roken can generate coordinated multi‑robot trajectories to perform connectivity‑constrained goal navigation tasks with high success rates, outperforming the baseline method used to generate the training dataset. Roken also demonstrates good scalability after training with mixed team sizes, and shows generalization to unseen or partially observed environments, verifying its potential to learn from diverse data and perform versatile tasks.
Authors:Feng Qiao, Zhaochong An, Zhexiao Xiong, Serge Belongie, Nathan Jacobs
Abstract:
Re‑rendering an existing video from a novel camera viewpoint requires the output to follow the prescribed camera trajectory while preserving the appearance and dynamics of the original scene across every frame. Existing methods rely on per‑frame pose embeddings, noisy point‑cloud renderings, or implicit learned correspondences, none of which provides an explicit, temporally continuous link between source and target pixels. We propose Track2View, which conditions a video diffusion transformer on paired 3D point tracks: sparse trajectories of scene points projected into both the source and target camera views. These tracks provide explicit spatiotemporal correspondences that are temporally continuous by construction, encoding what content should appear where and when. At the core of Track2View is a dual‑view track conditioner that transfers visual context from source to target view through parameter‑free geometric operations and learned temporal aggregation, ensuring generalization to arbitrary camera trajectories without memorizing specific motions. We further introduce a data curation pipeline that extracts one‑to‑one track correspondences by running a 3D point tracker on temporally concatenated multi‑camera view pairs. On a 400‑video benchmark spanning static and dynamic scenes, Track2View achieves state‑of‑the‑art results across visual quality, view synchronization, and camera accuracy, reducing rotation error by 30‑65% and translation error by 61‑72% relative to leading baselines. Project page is available at this https URL: https://qjizhi.github.io/track2view
Authors:Runyang Xu, Xueqi Cheng, Yushun Dong
Abstract:
Initial attack (IA) is the first wildfire suppression phase, when agencies must quickly decide which fires may escape early control. Existing IA failure prediction studies often use non‑public response records or regional settings, so it remains unclear how well public data available at fire discovery time can support IA failure prediction at national scale. We present WILDFIREIA, the first U.S. national‑scale benchmark for IA failure prediction from environmental and contextual data available at fire discovery time. WILDFIREIA aligns 38,128 naturally caused FPA‑FOD wildfire events with FIRMS/VIIRS thermal detections, gridMET weather and fire‑danger variables, LANDFIRE vegetation, fuel, and topography, OpenStreetMap access features, and WorldPop population density. To prevent data leakage, the benchmark fixes the event unit, size‑based label rule, chronological split, metrics, and forbidden‑feature list, and excludes final fire size, containment timestamps, and post‑discovery satellite detections from model inputs. We evaluate 16 representative models across tabular, temporal, spatial, and spatiotemporal families under the same protocol. Results show that public discovery‑time data provides useful but incomplete signal for IA failure prediction: XGBoost achieves the best AUPRC of 53.3%; FIRMS/VIIRS is the least redundant source; and fuel is the strongest static predictor when dynamic observations are unavailable. We release preprocessing outputs and model‑ready caches to support reproducible research on early wildfire risk assessment: https://github.com/LabRAI/WildfireIA#.
Authors:Ryo Tamura, Naruki Yoshikawa, Koji Tsuda, Shoichi Matsuda
Abstract:
Self‑driving laboratories (SDLs), where artificial intelligence proposes subsequent experiments and robotic systems execute them, are rapidly becoming the vanguard of materials discovery. A critical bottleneck, however, lies in seamlessly bridging diverse AI algorithms tailored for specific exploration goals with the heterogeneous robotic hardware found across different laboratories. Here, we present NIMO, an open‑source software platform designed to dissolve this barrier through three core paradigms: a modular AI‑robot decoupling mediated via simple CSV file exchange, a discrete candidate‑pool architecture that seamlessly absorbs domain knowledge, and a unified Python interface pre‑loaded with twelve distinct AI algorithms. In this Perspective, we review the operational principles of each algorithm alongside six diverse SDL implementations driven by NIMO, covering electrolyte discovery, organic synthesis, thin‑film exploration, fuel‑cell process informatics, coffee‑ring phase exploration, and legacy liquid‑handling automation. One of these also demonstrates NIMO's seamless interoperability with the IvoryOS orchestration framework. To democratize autonomous science, we also introduce a no‑code desktop application that enables intuitive, human‑in‑the‑loop exploration for non‑programmers. NIMO is freely available at https://github.com/NIMS‑DA/nimo, offering a versatile, plug‑and‑play foundation to accelerate autonomous materials exploration across diverse experimental landscapes.
Authors:Hassan Ismkhan, Hamid Bouchahcia
Abstract:
Robotic systems perceive the world through multiple input modalities ‑‑ including visual camera streams and natural language instructions ‑‑ and must select appropriate actions based on these signals. However, assuming the permanent availability of all input devices is unrealistic, as sensors may fail, become occluded, or drop out entirely during deployment. Robust handling of such missing‑modality scenarios is therefore essential for real‑world robot operation. This paper introduces RL4IL, a reinforcement learning guided method for imitation learning that selects the most suitable action for a given observation by identifying the most relevant expert demonstrations from a training library. A reinforcement learning policy, trained via Proximal Policy Optimisation over Breadth‑First Search candidate sets, ranks candidate demonstrations and a soft cross‑attention fusion head aggregates their action signals to produce the final prediction. When a modality is missing at inference time, a dedicated per‑modality RL retrieval policy identifies donor demonstrations from the training library, and a soft imputation head reconstructs the missing embedding via cross‑attention over the top‑ranked donors ‑‑ without requiring any retraining of the system. Experiments on three LIBERO benchmark suites demonstrate that RL4IL substantially outperforms state‑of‑the‑art imitation learning methods under sensor dropout conditions, while requiring no policy network training. The code can be found at https://github.com/h‑ismkhan/Reinforcement‑Learning‑via‑kNN‑for‑Robotic‑Learning‑with‑Missing‑Camera
Authors:Umut Baser, Ismail Emir Yuksel, F. Nisa Bostanci, Konstantinos Sgouras, Ataberk Olgun, Emre Hakan Demirli, Zhiheng Yue, Harsh Songara, Oguz Ergin, Onur Mutlu
Abstract:
We experimentally demonstrate that it is possible to generate unique, repeatable, and device‑specific signatures suitable for use as Physical Unclonable Function (PUF) responses in commercial off‑the‑shelf (COTS) DRAM chips by leveraging simultaneous multiple‑row activation (SiMRA). Based on a rigorous experimental characterization of 112 modern DDR4 DRAM chips (from 10 modules), we introduce SiMRA‑PUF, the first DRAM‑based PUF that uses SiMRA‑generated signatures as PUF responses. We analyze SiMRA‑PUF in terms of reliability, uniqueness, and evaluation latency for varying numbers of simultaneously activated DRAM rows (i.e., 2, 4, 8, 16, and 32), DRAM chip density & die revision, and evaluate how temperature affects the similarity of SiMRA‑generated responses. Among our 8 key experimental observations, we highlight two major results. First, SiMRA‑PUF provides average intra‑Jaccard indices of 89.02%, 89.81%, 93.03%, 94.06%, and 94.86%, and average inter‑Jaccard indices of 3.98%, 2.37%, 3.44%, 2.92%, and 3.24% for 2‑, 4‑, 8‑, 16‑, and 32‑row activations, respectively, showing that SiMRA‑generated signatures are both repeatable within a device and unique across devices. Second, 2‑row activation‑based SiMRA‑PUF provides 5.75% lower evaluation latency than the state‑of‑the‑art DRAM‑based PUF. We open‑source our infrastructure and datasets at https://github.com/CMU‑SAFARI/SiMRA‑PUF.
Authors:Shao-Yi Yu, Jen-Wei Wang, Maya Horii, Masayoshi Tomizuka, Vikas Garg
Abstract:
Robotic systems deployed in uncertain and dynamically changing environments often face variations in contact conditions, aerodynamic effects, and external disturbances that challenge reliable control. To remain effective under model‑based control, these systems require dynamics models that can adapt to such changes, especially when direct access to complete environmental information is limited. To enable adaptability and facilitate integration with model predictive control, we propose a context‑aware dynamics model based on neural ordinary differential equations, which infers environmental factors from state‑action histories using a two‑phase training procedure. We validate the approach across diverse robotic platforms, including a quadrotor in simulation, as well as a Sphero BOLT robot and a Fanuc manipulator in real‑world experiments. The results demonstrate that our method effectively adapts to temporally and spatially varying environmental changes across different tasks. Videos are available at https://youtu.be/PY0sNyF2rqE , and the source code is available at https://github.com/syyu410‑yu/context‑aware‑neural‑ode‑control.git .
Authors:Sivaperuman Muniyasamy, Surendar Devasundaram
Abstract:
Vision‑based perception is fundamental to Space Situational Awareness and autonomous on‑orbit operations such as rendezvous, docking, servicing, and navigation. However, progress in this area is limited by the scarcity of annotated space imagery and by challenging visual‑domain characteristics including severe illumination changes, low signal‑to‑noise ratio, and high contrast. We address Stream 1 of the SPARK 2026 Challenge, which requires a single model for spacecraft classification, detection, and fine‑grained component segmentation across multiple target types. We propose a compact architecture that integrates a MobileNetV3 encoder with a U‑Net‑style decoder, combining computational efficiency with accurate dense prediction. Detection is derived analytically from the union of predicted component masks, avoiding a separate bounding‑box regression head in the single‑spacecraft setting. Our method achieved an overall leaderboard score of 0.9482, with task‑specific scores of 1.0000 in classification, 0.9788 in detection, and 0.8917 in segmentation. The proposed approach ranked second overall in the SPARK 2026 Challenge, demonstrating that lightweight encoder‑decoder architectures can deliver strong multi‑task performance for practical onboard space vision systems.
Authors:Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang
Abstract:
Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilingual settings, they lack adaptation to Chinese‑specific regulatory policies, cultural context and linguistic nuances, failing to support fine‑grained risk classification for diverse deployment needs. In this paper, we introduce a 5‑macro, 31‑micro category fine‑grained risk taxonomy for Chinese scenarios, and build CHILLGuard: a dedicated Chinese LLM content safety guardrail. To address the critical scarcity of high‑quality annotated Chinese safety data, we propose a scalable multi‑stage data construction pipeline: we expand multi‑source corpus via retrieval‑augmented generation, generate implicit harmful samples through prompt engineering rewriting, and refine high‑quality data via multi‑model voting‑based label calibration. Based on this, we build CHILLGuardTrain, a large‑scale training set with 405,007 samples, and CHILLGuardTest, a rigorously curated annotated test set with 51,745 samples. We then train CHILLGuard on CHILLGuardTrain under a generator‑classifier collaborative framework via Model‑aware Direct Preference Optimization. Extensive experiments under multiple settings demonstrate the state‑of‑the‑art performance of CHILLGuard, e.g., a 15.92% improvement of F1 score over Qwen3Guard‑8B‑Strict on our benchmark. We will release our resources at https://github.com/cswbyu/CHILLGuard.
Authors:Yixuan Wang, Yiyang Zhou, Yiming Liang, Congyu Zhang, Fuxiao Liu, Jiawei Zhou, Huaxiu Yao
Abstract:
LLM agents can improve without weight updates by accumulating natural‑language skills from experience, but current systems entrust every decision about which skills to keep and how to apply them to LLM judgment alone. We argue that this conflates two distinct roles: generating a skill from experience is a creative act that judgment handles well, while deciding whether that skill actually helps requires empirical evidence across many tasks. Measuring per‑skill causal contributions via randomized masking, we find that skill libraries exhibit pervasive causal heterogeneity: individual skills routinely help on some task types while hurting on others, yet their opposing effects cancel in aggregate, making them invisible to global curation methods. We propose ASSAY, a framework that separates generation from curation: it computes a per‑skill causal attribution on a small development set, restructures the library offline, and suppresses skills with negative predicted effect for each test task. Across seven base models spanning four providers and two benchmarks (AppWorld and tau‑bench), ASSAY consistently improves over prior skill‑curation approaches. On AppWorld's hardest split, DeepSeek‑V3 achieves 69.3% task‑goal completion (47.4% relative improvement), a new state of the art among all published methods including weight‑tuned approaches. On tau‑bench retail, GPT‑4.1 improves by 8.7% relative, advancing past o4‑mini, o1, and GPT‑4.5 on the public leaderboard without any weight modification. Ablation traces the dominant gain to per‑task masking, confirming that the bottleneck is matching skills to tasks at inference time, not removing bad skills globally. Code is available at https://github.com/aiming‑lab/assay.
Authors:Ömer Veysel Çağatan, Xuandong Zhao
Abstract:
Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety. Yet most known instances have been discovered post hoc in frontier systems where controlled study is impractical. We adapt the AI Safety Gridworlds framework into a text‑based evaluation suite that reformulates classic reinforcement learning safety tasks for language‑based agents. Across frontier and mid‑scale models, we find that specification gaming emerges zero‑shot: models systematically achieve high observed reward while underperforming on hidden safety objectives, and even apparently safe behaviors can reflect misunderstanding rather than principled safety. Reinforcement learning does not correct these failures: direct reward optimization widens the gap between observed and hidden reward, as the model's initial competence causes it to lock into locally rewarding strategies before discovering safer alternatives. This pattern persists across model scales (1.5B‑‑14B) and is not resolved by finer credit assignment, exploration prompts, or entropy regularization. Our results show that reward hacking arises naturally when optimizing proxy objectives with capable language model agents and resists standard mitigations, suggesting that proxy‑reward failures in agentic settings may require approaches beyond standard exploration and credit‑assignment fixes. To facilitate reproducibility, the code for this work is available at \hrefhttps://github.com/asparius/verl‑agent‑safetyour public repository.
Authors:Joy Bose, Om Thomas
Abstract:
Standardized examinations are typically treated as uniform syllabus coverage problems. We argue they are better understood as adversarial systems with stable latent cognitive structures diverging systematically from official syllabi. We introduce LearnOpt, which recovers this structure from historical question papers and generates personalized, time‑bounded study plans. Applied to nine years of NEET questions (2016‑2024, n=1,496), LearnOpt builds an exam knowledge graph from LLM‑tagged questions, extracts a five‑category latent skill distribution, and formulates study planning as a knapsack‑variant optimization over prerequisite‑aware subgraphs with Bayesian Knowledge Tracing. Central finding: NEET's latent skill distribution is stable within a syllabus regime (consecutive‑year KL divergence 0.004‑0.032 for 2016‑2021, non‑significant under permutation testing) but shifts significantly with NCERT's 2023 syllabus rationalization: pooling 2016‑2021 (n=1,072) vs 2023‑2024 (n=392) gives KL=0.040 (p=0.0005), with Elimination/Negation questions rising from ~20‑29% to ~31‑35%. Latent structure, while not permanently stationary, is piecewise stable, with shifts detectable and attributable to curricular events. Within either regime, subject predicts skill profile more strongly than year. An optimization evaluation, using one real and two synthetic mastery profiles, shows the skill‑weighted objective produces a modest but real reordering of recommended topics over a mastery‑conditioned frequency baseline. Applying the pipeline to JEE Advanced reveals a profile dominated by Multi‑concept Integration (80.9% vs. 33.3% for NEET), with a JEE‑vs‑NEET divergence (KL=0.505) exceeding NEET's largest cross‑subject divergence: exam tier shapes latent cognitive structure more than subject, which shapes it more than time within a regime. Code, knowledge graph, and annotated dataset are released publicly.
Authors:Mengshi Qi, Changsheng Lv, Zijian Fu, Xianlin Zhang, Huadong Ma
Abstract:
In this paper, we propose SGFormer++, a novel Semantic Graph Transformer for 3D scene graph generation (SGG), which aims to parse point cloud scenes into semantic structural graphs, where nodes denote detected object instances and edges encode their pairwise relationships, with the core challenge lying in modeling complex global scene structure. While existing graph convolutional network (GCN)‑based methods suffer from over‑smoothing and limited receptive fields, SGFormer++ leverages Transformer layers as its backbone to enable global message passing. Specifically, we introduce two key components tailored for 3D SGG: (1) a Graph Embedding Layer++ that efficiently integrates edge‑aware global context with linear computational complexity, and (2) a Semantic Injection Layer++ that enriches visual features with linguistic priors from large language models (LLMs) and vision‑language models (VLMs), boosting semantic representation without introducing extra trainable parameters. To further address the practical challenge of incremental SGG (I‑SGG), where new relationship categories arrive sequentially, we equip SGFormer++ with a novel Spatial‑guided Feature Adapter, which calibrates predicate features using subject‑object spatial geometry to counter scale variation, and a Cascaded Binary Prediction Head that mitigates catastrophic forgetting via task‑incremental classifier expansion and logit distillation. Extensive experiments on the 3DSSG benchmark demonstrate that SGFormer++ achieves state‑of‑the‑art performance in both standard and incremental settings: it yields a significant 4.49% absolute improvement in Predicate A@1 under the incremental setting. Code and data are available at: https://github.com/Andy20178/SGFormer.
Authors:Liming Wang, Cody Karjadi, Rhoda Au, James Glass
Abstract:
A key challenge of speaker de‑identification is the balance between privacy and utility. Many utility variables, such as the cognitive health status of the speaker, are correlated with the privacy variable, such as the speaker identity, violating the independence assumption held by the disentanglement‑based approaches, causing leakage of private information and the loss of useful information for downstream tasks. To tackle this challenge, we propose a general framework, DDPO‑VC, for speaker de‑identification through reinforcement learning‑based post‑training with diffusion models. Learning from reward signals combining knowledge from privacy‑focused and utility‑focused teachers, our method outperforms various strong \deid/ methods in both privacy preservation and cognitive utility on two commonly used dementia speech benchmarks. Please check out our code\footnote\hrefhttps://github.com/cactuswiththoughts/DDPO‑VChttps://github.com/cactuswiththoughts/DDPO‑VC and demo\footnote\hrefhttps://cactuswiththoughts.github.io/SpeakerDeID‑Demo/https://cactuswiththoughts.github.io/SpeakerDeID‑Demo/.
Authors:Yuxin Zhang, Ju Fan, Meihao Fan, Shaolei Zhang, Xiaoyong Du
Abstract:
Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real‑world development. Such environments typically involve both complex code and large‑scale data (i.e., file system). However, existing benchmarks usually evaluate code‑centric or data‑centric capabilities in isolation, leaving a clear gap with real development scenarios. In this paper, we bridge this gap by introducing CODA‑BENCH, the first benchmark to jointly evaluate code and data intelligence in a data‑intensive environment. We construct a data‑intensive Linux sandbox based on the Kaggle ecosystem (containing hundreds of datasets), where agents must actively explore complex file hierarchies to identify relevant resources and generate code for data‑driven analytical tasks. CODA‑BENCH comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files, simulating realistic data scale and noise. Evaluations of advanced agents reveal that even top‑performing systems struggle to effectively integrate data discovery with code execution, achieving a success rate of only 61.1%. These results highlight a substantial gap in current agentic capabilities for data‑intensive tasks and point to promising directions for future research.
Authors:Chenyang He, Xinyi Shao, Shun Huang, Bosong Huang, Daoqiang Zhang, Ming Jing, Cheng Ding
Abstract:
Photoplethysmography (PPG) plays a central role in wearable health monitoring and clinical decision support. Yet existing approaches to universal PPG representation learning largely focus on signal‑level objectives and often overlook patient‑level health context, which limits generalization to complex clinical tasks and heterogeneous cohorts. To address this gap, we construct a large‑scale paired PPG‑EHR multimodal dataset by distilling fragmented medical histories and clinical records into cohesive, patient‑level electronic health records (EHR). Building on this resource, we propose Clinical Anchored Pretraining for PPG (CAP). During pretraining, CAP performs cross‑modal contrastive alignment that anchors PPG representations to patient‑level clinical semantics, guiding the encoder beyond waveform fitting toward modeling consistency in a patient's overall physiological state. During downstream adaptation, the pretrained PPG encoder provides clinically grounded representations that strengthen inductive bias and improve robustness and transferability. Experiments demonstrate that CAP consistently outperforms strong baselines on four diverse downstream tasks. CAP achieves a particularly large gain on respiratory rate prediction (up to +87.6% relative improvement over the state‑of‑the‑art baseline) and delivers an average relative +26.7% across all tasks. We further enhance the interpretability of our approach through comprehensive analyses, including ablations and multiple complementary visualizations of the learned representations. The code for our experiments is available at: https://github.com/gody123gody/CAP .
Authors:Mustafa Bora Çelik
Abstract:
Fine‑grained action recognition demands temporal reasoning that general‑purpose architectures address through different cost‑accuracy tradeoffs: 3D dense operators couple computation to the input volume, while difference‑based methods approximate motion through rigid, hand‑crafted subtraction of uncontextualized features ‑ each reflecting a deliberate design choice with corresponding limitations in expressiveness or flexibility. We present MamBOA, a backbone‑agnostic temporal framework built upon a novel interleaved scan structure that recasts the selective state‑space recurrence (S6) as a native motion synthesizer. By interleaving consecutive feature representations extracted from a pretrained backbone into a single alternating sequence, the proposed scan structurally drives the recurrence to encode both temporal observations of each position within a shared hidden state, separated by only a single decay step ‑ rendering the inter‑frame transition an intrinsic component of the state dynamics rather than an externally computed quantity. A cascade of dedicated alignment and decoding operations then distills this joint encoding into an explicit motion representation, which a dual‑path pooling mechanism adaptively aggregates by balancing attention‑driven selection with uniform temporal coverage. The framework interfaces seamlessly with CNN, Transformer, and Mamba backbone families, adding only ~2.1 GFLOPs per feature pair. On Diving48, MamBOA achieves 85.02% Top‑1 accuracy with an image‑pretrained backbone and 86.24% with a video‑pretrained backbone processing the entire video in a single forward pass ‑ demonstrating that structurally induced state‑space dynamics constitute a principled and general foundation for motion modeling.
Authors:Qiheng Sun, Junxu Liu, Xiaokai Mao, Haocheng Xia, Jinfei Liu, Kui Ren, Haibo Hu
Abstract:
Shapley value‑based feature attribution methods face challenges in scenarios involving complex feature interactions and causal relationships, even when a causal structure is provided. Existing methods typically adopt a node‑centric view, attributing importance solely to individual features. Consequently, they often fail to simultaneously capture the externality and exogenous influence of features, leading to unreasonable interpretations. To overcome these limitations, we propose a novel feature attribution method called DAG‑SHAP, which is based on edge intervention. DAG‑SHAP treats each feature edge as an individual attribution object, ensuring that both externality and exogenous contributions of features are appropriately captured. Additionally, we introduce an approximation method for efficiently computing DAG‑SHAP. Extensive experiments on both real and synthetic datasets validate the effectiveness of DAG‑SHAP. Our code is available at https://github.com/ZJU‑DIVER/DAG‑SHAP.
Authors:Zhenwei Mou, Liping Chen, Yajun Hu, Zhen-Hua Ling, Xin Fang, Jianqing Gao
Abstract:
Personalized text‑to‑speech (TTS) aims to clone the target speaker in the synthesized speech, imitating both the voice and speaking style. Current large language model (LLM)‑based TTS methods ignore the style‑specific prosodic patterns in generated speech, resulting in deficient style learning and thus limiting speaker similarity in synthesized speech. To this end, we investigate the prosody learning conditioned on the synthesized speech, and propose to predict the prosody of the current syllable based on previously predicted speech. Experimental results obtained on three datasets demonstrated the efficacy of the proposed dynamic prosody prediction method in enhancing the prosody learning capability, thereby improving the speaker similarity of the generated speech. Audio samples are available at https://muzw.github.io/dynapros/.
Authors:Zhenwei Mou, Weili Jiang, Liping Chen, Zhen-Hua Ling, Kong Aik Lee, Kai Gao, Boyu Zhao
Abstract:
Large language model (LLM)‑based text‑to‑speech (TTS) models have achieved remarkable voice cloning capabilities, raising concerns about potential deepfake misuse. Speech watermarking mitigates this by embedding traceable information into generated speech. Mainstream watermarking methods operate at the signal level (waveform or spectrogram), rendering the watermark vulnerable to generative attacks (e.g., neural codec and vocoder). To address this, we propose DuraMark, a robust information‑level watermarking framework. It utilizes syllable duration editing to achieve watermark embedding. Specifically, DuraMark integrates a duration‑controllable LLM‑based TTS model to edit syllable durations during synthesis, coupled with a duration extractor to extract these durations for detection. Experiments demonstrate DuraMark's superior robustness against generative attacks, significantly outperforming signal‑level baselines. Audio samples are available at https://muzw.github.io/duramark_demo/.
Authors:Jierui Zhang, Siyuan Tan, Xinhang Li, Longzhuangzhi Lin, Dailin Li, Chengfeng Gu, Xinping Li, Yaxian Hao, Shengjia Liang, Yuxiang Ren, Wenhao Liu
Abstract:
Large language models (LLMs) are increasingly capable of mathematical problem solving and can even assist with research‑level proofs, yet we still lack a scalable and reproducible way to measure step‑level reasoning in long proofs across diverse sources. This evaluation gap limits trustworthy AI assistance in proof‑certified scientific progress. Existing evaluations often emphasize final answers or rely on costly expert grading, while end‑to‑end proof generation remains open‑ended and hard to verify automatically. We introduce Mask‑Proof, a pipeline that turns real proofs into automatically checkable masked‑step tasks. It masks key formula steps, provides the necessary surrounding context, and evaluates model reconstructions with an LLM‑based equivalence judge using repeated votes for stability. The resulting Mask‑ProofBench contains 292 curated problems across diverse research areas. Experiments with 17 models show that reasoning‑enhanced models outperform standard models by 12% to 27%. Our evaluator achieves 96.8% agreement with expert annotators, enabling faithful, reproducible, and comparable measurement of step‑level mathematical reasoning. Benchmark, annotations, and code are available at https://github.com/weating/Mask‑Proof.
Authors:Thanh Nguyen Canh, Thang Tran Viet, Phuc Van Dinh, Xiem HoangVan, Nak Young Chong
Abstract:
Coordinating heterogeneous multi‑robot systems (MRS) for complex, long‑horizon tasks requires both flexible high‑level reasoning and efficient low‑level scheduling. Existing LLM‑based approaches address the reasoning side but introduce two critical bottlenecks: (1) repeated LLM inference during execution, which inflates latency with agent count, and (2) offline, pre‑committed scheduling, which forces robots to idle while waiting for sequentially ordered predecessors even when independent work is available. This paper presents OSDAG, a novel framework that integrates LLM‑based task reasoning with Directed Acyclic Graph (DAG) representation and constraint‑aware online scheduling. The LLM is invoked once to decompose a natural‑language instruction into a dependency‑annotated task graph, and a lightweight online scheduler then allocates ready tasks to idle agents in real time. The DAG representation encodes both precedence and resource constraints, ensuring correctness while exposing all available parallelism. Experiments across five benchmark scenarios demonstrate that OSDAG achieves 5‑15x faster reasoning time compared to dialogue‑based methods, reduces makespan by up to 38% over sequential baselines, and maintains competitive success rates. Both simulation and real‑world experiments on dual‑arm manipulation tasks validate the effectiveness and practicality of the proposed approach for efficient multi‑robot coordination. The website and resources are available at http://thanhnguyencanh.github.io/LLM_DAG4MultiRobot
Authors:Kun Ma, Qilong Han, Chengjing Song, Jingzheng Yao, Xiao Han, Yuee Zhou, Changmao Wu
Abstract:
Modern trajectory predictors increasingly condition on external spatial context, such as map geometry, signed distance fields (SDFs), and nearby moving agents. While this context improves prediction quality, constructing it for every training anchor has become a hidden systems bottleneck. In a representative maritime AIS pipeline, spatial context construction requires roughly 17 CPU‑days for a 5.48M‑anchor corpus, dominating the cost of the downstream predictor. We present M‑CTX, an exact and scalable spatial context‑retrieval framework for trajectory analytics. M‑CTX recasts context construction as an ingest‑once, query‑many spatial database workload and replaces three brute‑force stages ‑‑ OSM range retrieval, SDF computation, and moving‑vessel neighbour lookup ‑‑ with composable, index‑backed operators. Its learned range‑index backend, BR‑LZ, provides recall‑complete MBR‑overlap range retrieval and reduces candidate amplification by 1.1x‑‑2.7x relative to global‑expansion one‑curve baselines. Across four maritime regions, eight baseline systems, synthetic workloads with up to 40M spatial features, and 10^7‑record AIS streams, M‑CTX reproduces the reference context exactly. On the 5.48M‑anchor corpus, it reduces context construction from about 17 CPU‑days to 1.8 hours, a measured 226x end‑to‑end speed‑up. An optional storage mode further compresses SDF context by 64x with only a 0.04 m ADE change. These results establish exact spatial context retrieval as a first‑class database problem in modern trajectory analytics. Code and datasets are publicly available at https://github.com/mark000071/M‑CTX‑Traj.
Authors:Yi Xie, Jiawei Du, Yu Cheng, Jiuan Zhou, Zhaoxia Yin
Abstract:
Skills are becoming the capability layer through which LLM agents turn plans into actions, but their use introduces security risks such as data leakage, unauthorized operations, and tool misuse. Existing vetting usually evaluates each skill in isolation, while real agent tasks often invoke multiple skills in a shared execution context. This creates Skill Composition Risk (SCR): a skill that appears benign alone can become harmful when its outputs, trust signals, authorization cues, or side effects influence later invocations along an activated path. We introduce SCR‑Bench to evaluate this risk in controlled, sandboxed skill environments. Rather than relying only on textual intent or surface behavior, SCR‑Bench records downstream state changes and path‑level outcomes across composed skill executions. It contains three sub‑benchmarks: SCR‑CapFlow for capability‑flow composition, SCR‑TrustLift for trust‑transfer composition, and SCR‑AuthBlur for authorization‑confusion composition. Across SCR‑Bench, composed paths expose risks that are largely absent under isolated evaluation. In SCR‑CapFlow, attack success rate reaches 33.6 percent under composition, compared with near‑zero isolated baselines. In SCR‑TrustLift, attack success rate exceeds 96.5 percent on four of five backends. In SCR‑AuthBlur, the risky‑approval rate increases by 71.8 percent relative to the L0 isolated baseline under the L1 context setting. These results show that agent skill security should be assessed at the level of activated paths rather than isolated artifacts. SCR and SCR‑Bench provide a foundation for path‑aware risk evaluation and defense in LLM agent skill ecosystems. Benchmark: https://github.com/saint‑viperx/SCR_Bench.
Authors:Weichen Fan, Haiwen Diao, Penghao Wu, Ziwei Liu
Abstract:
Pixel‑space diffusion models are trained on full‑bandwidth noisy images, yet the useful signal available to the denoiser is strongly frequency dependent. Under rectified‑flow diffusion and natural‑image power‑law spectra, the per‑band data‑to‑noise contour k^(t) = (1‑t)^‑2/α separates a signal‑bearing low‑frequency region from a noise‑dominated high‑frequency region at each time t. We show that this implicit coarse‑to‑fine structure is not merely descriptive: it induces a capacity‑allocation problem. A standard pixel‑space denoiser must discover the moving bandwidth boundary internally and can spend computation on frequency‑time regions where the optimal prediction collapses to deterministic baselines rather than data‑distribution modeling. To make this boundary explicit, we introduce Spectral Forcing, a parameter‑free, time‑conditional 2D‑DCT low‑pass operator applied to the noisy input before the patch embedder. Its cutoff expands monotonically with the diffusion time and becomes the identity at the data endpoint. Through controlled synthetic experiments, we identify the regime in which the operator is beneficial: coarse patch tokenization and data whose high‑frequency content is predominantly noise rather than essential signal. On ImageNet‑256 with JiT‑700M/32, Spectral Forcing consistently improves both FID and Inception Score across different training epochs, demonstrating robust gains throughout training; at finer tokenization, the spectral forcing is still competitive. We further insert the unchanged operator into SenseNova‑U1, a unified text‑to‑image model, where it improves DPG‑Bench and GenEval, showing that the input‑side spectral prior transfers beyond class‑conditional generation. These results suggest a route to capacity‑efficient pixel‑space diffusion by showing the signal and hiding the noise.
Authors:Zhengbo Zhang, Changtao Miao, Jinbo Su, Zhaowen Zhou, Chunxia Zhang, Xukai Wang, Ruiqi Liu, Kaiyuan Zheng, Jiansheng Cai, Bo Zhang, Zhe Li, Shiming Xiang, Ying Yan
Abstract:
Multimodal large language models (MLLMs) have demonstrated impressive capabilities in many visual tasks, but they often struggle with factual grounding when confronted with complex, open‑world scenarios. While recent multimodal deep search agents attempt to address this issue by utilizing external tools, the visual‑native search paradigm remains underexplored. Existing methods primarily rely on simple images with explicit semantics and text‑only evidence trajectories, limiting the agent's ability to perform multi‑hop, cross‑modal reasoning and search. To address these limitations, we propose Visual‑Seeker, a visual‑native multimodal deep search agent via active visual reasoning. Rather than treating vision as a static input, our agent actively attends to fine‑grained visual details, dynamically harvests visual evidence throughout the search process. To unlock its visual‑native potential, we design an active visual reasoning data pipeline and synthesize 5K high‑quality multimodal trajectories for model training. Extensive experiments demonstrate the state‑of‑the‑art performance across five challenging multimodal search benchmarks, even surpassing several proprietary models, validating robust visual‑native reasoning and search in real‑world web environments. The code and data can be accessed at: https://github.com/ZhengboZhang/Visual‑Seeker.
Authors:Peihao Li
Abstract:
Targeted advertising systems can pair audiences selected by advertisers with ad units that expose visible user actions. When an interaction remains linked to the campaign that elicited it, the advertiser may receive an observation tied to a user rather than only an aggregate report. We model that channel as a noisy oracle for attribute inference. The model separates targeting predicates, exposure, interaction, and disclosure. These boundaries capture the gap between eligibility and delivery, and the gap between interaction and advertiser visibility. We build a reproducible benchmark using synthetic populations calibrated with public data, each with known sensitive labels. A generated campaign semantics layer provides topic variants and response priors. The simulator generates the ground truth, event traces, disclosed observations, and metrics. The evaluation compares Bayesian, supervised, positive and unlabeled, and adaptive attacks under common campaign and disclosure definitions. The final evaluation uses four topic variants, seven simulator seeds, and two interaction settings. Repeated campaigns with identity exposure produce measurable but bounded inference signal. At 160 campaigns, Bayesian and supervised attacks reach about 0.64 AUC in the main setting and about 0.65 AUC in the higher interaction setting. Disclosure policy is the strongest control. Aggregate reporting removes the evaluated oracle input tied to users. Type filtering and randomized disclosure reduce the released signal. The result is a model, artifact, and defense evaluation method for privacy in interactive targeted advertising. The code is available at https://github.com/P‑HOW/Interactive‑Ad‑Oracle.
Authors:Yun Wang, Junbin Xiao, Han Lyu, Yifan Wang, Jing Zuo, Zhanjie Zhang, Hong Huang, Dapeng Wu, Angela Yao
Abstract:
We introduce UCS‑Bench, a dataset spanning 170+ hours of egocentric visual observations with 8.1K+ timestamped questions for diagnosing User‑Centric Continual Spatial intelligence in egocentric video streams. UCS‑Bench targets a new problem that emphasizes dynamic spatial reasoning, long‑term memory, and their alignment with users' real‑time locations. We propose DirectMe, a framework that incrementally constructs and maintains a structured spatial memory from streaming egocentric observations. DirectMe enables robust tracking and recall of object locations, all relative to the user's movement over time. By tightly coupling visual perception with memory updates and spatial reasoning, our approach supports long‑horizon queries that require recalling interactions, resolving viewpoint‑induced ambiguities, and adapting to dynamic scenes. Our experiments show that DirectMe significantly improves the spatial reasoning of leading multimodal LLMs; it also surpasses many spatially aware and long‑form streaming video models. We hope our benchmark and solution will advance spatial intelligence research for egocentric AI assistants. Data and code are available at https://github.com/cocowy1/UCS‑Bench.
Authors:Jeahun Sung, Dahyeon Kye, Soo Ye Kim, Jihyong Oh
Abstract:
Reference‑guided generation (e.g., object compositing, customization) has progressed rapidly, yet current pipelines share a fundamental limitation: the object‑centric high‑resolution reference image (HRRI) provided by users is downsampled to a fixed low‑resolution (LR) before being fed into the model, so the fine‑grained details are discarded before the output is even produced. In addition, the generation step then introduces its own artifacts (e.g., identity distortion) on top of this loss. Existing reference‑guided generated content refinement (RefGCR) methods can correct some of these artifacts but still operate in the LR domain; reference‑guided super‑resolution (RefSR) methods recover resolution but assume natural‑image degradations and ignore the artifact distribution of generative pipelines. To address both gaps in a single formulation, we introduce a new task: reference‑guided generated content super‑resolution‑refinement (RefGC‑SR^2), where the original HRRI is reused at the post‑processing stage to recover lost details, refine generative artifacts, and upscale the output simultaneously. We construct the first real‑world triplet data generation pipeline for this RefGC‑SR^2 task, training a diptych‑conditioned generator to synthesize paired low‑quality anchors that public pretrained models cannot provide. We further present a frequency‑aware diffusion transformer model for RefGC‑SR^2 that selectively injects fine details from the HRRI while removing generative artifacts. Extensive experiments demonstrate that our RefGC‑SR^2 model successfully (i) refines the object identity faithfully with respect to the reference, and (ii) recovers high‑resolution details, so that the final result is significantly higher quality and practically more usable compared to existing RefGCR and RefSR baselines.
Authors:Bharawee Phoompho, Gokul Puthumanaillam, Yan Miao, Ruben Hernandez, Tim Bretl, Sayan Mitra, Melkior Ornik
Abstract:
Reliable trajectory planning under partial observability depends not only on computing a feasible geometric path, but also on whether the robot receives informative observations while executing that trajectory. Existing approaches usually keep the environment fixed and adapt the robot through belief‑space planning, active localization, or added sensing, often incurring costly uncertainty propagation and brittle behavior in observation‑poor regions. We flip this perspective and address the largely open problem of \emphtask‑aware environment augmentation: given a mapped environment, a planned task trajectory, and a small budget of visual fiducial markers, where should the environment be augmented so that the planned trajectory can be executed reliably under uncertainty? Our key observation is that useful marker layouts are defined by the localization support they provide along the task trajectory: a small number of well‑timed observations can be sufficient to prevent uncertainty from accumulating in regions where state‑estimation error would otherwise compromise control. Building on this observation, we present \tbpSCoDA, Shielded Conditional Diffusion for Environment Augmentation. \tbpSCoDA learns a conditional distribution over high‑performing fiducial layouts from data, using the environment, planned trajectory, disturbance context, and desired execution profile as conditioning. Its shielded sampler reasons over where along the planned execution pose corrections should occur, and steers this distribution toward task‑relevant, finite‑budget augmentations. Across simulated benchmarks and hardware deployments, we show that \tbpSCoDA improves trajectory execution reliability and completion time over strong baselines. Code, models and dataset available at: \hyperlinkscoda‑diffusion.github.iohttps://scoda‑diffusion.github.io/
Authors:Shijun Wan, Xuehai Wu, Jiwen Zhang, Siyuan Wang, Zhongyu Wei
Abstract:
Social interaction depends on both language and visible social signals, such as facial expressions, posture, gaze, and emotional shifts. Yet existing social‑agent benchmarks are largely text‑based and rarely test whether multimodal agents can use visual cues to guide interaction. We introduce \textsc\benchmarkname, a benchmark evaluating visual social intelligence in multimodal social simulation. It contains 240 scenarios, 585 role instances, and 2,340 role‑task instances, combining aligned textual‑visual evidence, structured role profiles, and four role‑level tasks: expression task, characteristic task, interaction regulation task, and interaction outcome task. Evaluating seven recent MLLMs under verbalized‑vision and direct‑vision reveals a clear gap between local role enactment and interaction management: role‑specific expression and conflict handling are near saturation, whereas interaction regulation and visually grounded outcome achievement remain substantially more difficult. The code is released at https://github.com/JunsWan/AgentViSS, and the dataset is available at https://huggingface.co/datasets/JunsWan/AgentViSS.
Authors:Nonghai Zhang, Siyu Zhai, Yanjun Li, Zeyu Zhang, Zhihan Yin, Yandong Guo, Boxin Shi, Hao Tang
Abstract:
Generating realistic humanoid motion from scene images and text involves both low‑frequency pose semantics and high‑frequency physical dynamics. However, many existing methods tokenize motion with a single shared codebook, forcing heterogeneous motion signals into the same quantization space. Our frequency‑domain analysis of human motion data reveals a clear mismatch between single‑codebook quantization and motion statistics: five DCT coefficients capture 93% of joint‑position energy but only 37% of joint‑velocity energy, which can bias quantization toward pose statistics and under‑represent high‑frequency velocity components. A second challenge lies in adapting a standard autoregressive model to effectively model high‑frequency physical signals in motion sequences. Therefore, we propose DSFT, a dual‑stream frequency tokenizer that separates motion into Base and physical streams and compresses them independently with DCT truncation and BPE. Furthermore, we present MotionVLA, a Qwen3.5‑based model that arranges Base and physical tokens in a unified sequence, where Phys tokens are predicted after Base tokens. Experiments on HumanML3D and MBench show that, despite using a lightweight 2B backbone, MotionVLA reduces the Diversity gap to real data by over 50% on HumanML3D and improves Motion‑Condition Consistency by 3.8% on MBench, supporting frequency‑aware dual‑stream decoupling as an effective formulation for autoregressive motion generation. Code: https://github.com/AIGeeksGroup/MotionVLA. Website: https://aigeeksgroup.github.io/MotionVLA.
Authors:Tianshan Zhang, Yijia Duan, Yanjun Li, Zeyu Zhang, Hao Tang
Abstract:
Dexterous interaction with articulated objects is important for household, assistive, and humanoid manipulation, where multi‑finger hands can provide compliant contact patterns beyond parallel‑jaw grasping. However, articulated‑object manipulation differs from static‑object manipulation: the target part cannot be directly actuated, and its motion must emerge through sustained physical hand‑‑handle contact. This makes the transition from object‑centric articulated generation to hand‑driven dexterous hand‑‑object interaction non‑trivial, since geometric trajectory replay or open‑loop execution does not model the contact dynamics required to move the articulated part. Moreover, policies trained only for task completion under fixed dynamics can overfit nominal contact loads, especially without tactile or force feedback, and may degrade when the contact load changes. To address these challenges, we present DragMesh‑2, a contact‑driven framework for dexterous interaction with articulated objects that extends articulated interaction from object‑centric generation to hand‑driven dexterous hand‑‑object interaction, where articulated motion must arise through physical contact. We further propose PICA, a physically informed contact‑aware training mechanism that injects physical signals into policy learning without tactile or force feedback, improving robustness and task success under changing contact loads. Finally, we conduct systematic evaluation across multiple damping conditions and articulated‑object categories to study robustness under contact‑load variation, and provide a pure‑geometry dexterous interaction resource to support future loco‑manipulation and humanoid hand‑‑object interaction research. Across seven GAPartNet objects, DragMesh‑2 achieves stronger robustness under contact‑load variation than the compared methods while maintaining high task success across damping conditions.
Authors:Weilong Guo, Yuhan Sun, Shengyang Li
Abstract:
Weak objects are common in images and videos of space applications. However, it is hard to learn proper representations from their limited appearance information. Inspired by multi‑view learning, we develop simple multi‑view attentions, treating their outputs as multi‑view features. We also propose a multi‑view feature high‑order fusion method (MHF) to aggregate more accurate and richer features of weak objects. Our MHF extends the commonly used low‑order feature fusion method to higher orders. It enhances the model's capacity to capture relevant and complementary information about weak objects. This is achieved by introducing high‑order multi‑view features perception and a recursive task‑contribution gated selection of multi‑view features. The new operation is highly flexible and customizable. It is compatible with various variants of multi‑view feature representations. We conduct extensive experiments on two newly constructed space science datasets and an open, large‑scale satellite video dataset. Our MHF serves as a plug‑and‑play module and significantly improves various vision transformers and convolution‑based detection and segmentation models. We achieve all state‑of‑the‑art accuracies on both tasks across three datasets. Our MHF can be a new basic module for visual modeling that effectively represents weak objects in terms of multi‑view learning. The code will be available at https://github.com/Kingdroper/MHF.
Authors:Yiyi Zhu, Yaolin Wen, Xiang Xia, Xin An, Hanyi Si, Xiang Shu, Yangde Fu, Liang Dou, Hong Qian
Abstract:
Multi‑objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives. In many practical scenarios, function evaluations are unavailable or prohibitively expensive, necessitating optimization solely based on a fixed offline dataset. In this setting, known as offline MOO, the goal is to find out the Pareto set without access to the true objective functions. This setting suffers from the out‑of‑distribution (OOD) issue, where the surrogate model is not accurate for unseen designs. Due to the OOD issue, surrogate errors may cause the optimizer to select solutions that do not lie on the true Pareto front and are biased toward its extremes. To address this, this paper proposes Diversity‑driven Offline Multi‑Objective Optimization (DOMOO), which aims to find out a diverse and high‑quality set of solutions. First, DOMOO incorporates an accumulative risk control module that estimates the potential risk of candidate solutions and alleviates the OOD issue between the training data and the generated solutions. In addition, a nested Pareto set learning (PSL) strategy is proposed to jointly learn preference and PSL parameters, then optimize them, enabling adaptation to diverse Pareto front geometries. To further enhance solution quality, we design a diversity‑driven selection strategy that extracts a representative and well‑distributed set of final solutions. To achieve this diversity‑driven selection strategy, we propose \textIGD_\textoffline, a tailored indicator for the offline setting that considers both diversity and convergence, and avoids the bias of hypervolume indicator. Extensive experiments on synthetic and real‑world benchmarks show that DOMOO achieves the best average rank across tasks in both convergence and diversity among the compared methods.
Authors:Lingtong Zhang, Wenlei Li, Mu He, Li Xiao, Yang Ji
Abstract:
Zero‑Shot Self‑Supervised Learning (ZS‑SSL) has emerged as a promising paradigm for accelerated Magnetic Resonance Imaging (MRI) reconstruction, eliminating the reliance on fully‑sampled external datasets. However, learning solely from a single under‑sampled scan suffers from supervision scarcity and optimization instability, often leading to overfitting or artifacts. To address these challenges, we propose a robust physics‑driven ZS‑SSL framework that synergizes physical consistency with image‑domain non‑local priors. Our method introduces three core innovations: (1) a Coil Sensitivity Map (CSM)‑Guided Dynamic Repository, which stabilizes the training trajectory by filtering physically inconsistent artifacts based on coil sensitivity constraints; (2) a SPIRiT‑based regularization, which enforces k‑space self‑consistency via a learned correlation kernel and stochastic masking; (3) a Non‑Local Self‑Similarity (NSS) Pixel Bank, which leverages the high‑fidelity reference established by the former modules to explicitly mine non‑local anatomical similarities, thereby augmenting supervision in the image domain. Extensive experiments on the FastMRI dataset demonstrate that our approach achieves state‑of‑the‑art performance, particularly under high acceleration factors, effectively bridging the gap between zero‑shot learning and supervised methods. The code is available at https://github.com/Zolento/NS‑SSL.
Authors:Sanhorn Chen, Xiaoyang Chen, Boyu Liu, Roy Zhao
Abstract:
Time series data in real‑world deployments is overwhelmingly irregular. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows. However, existing Time Series Question Answering (TSQA) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how large language models (LLMs) and AI agents perform under irregular conditions. To bridge this gap, we introduce IRTS‑ToolBench, a benchmark of 1,700 questions spanning 10 task types across 13 domains. IRTS‑ToolBench is designed to be used independently by any researcher working on LLM‑based irregular time series analysis, providing standardized inputs and a reproducible evaluation protocol. Code can be found in https://github.com/SanhornC/IRTS‑ToolBench.
Authors:Huan Kang, Hui Li, Tianyang Xu, Tao Zhou, Xiao-Jun Wu, Josef Kittler
Abstract:
Infrared and visible image fusion aims to integrate complementary modalities, while existing Euclidean methods impose rigid distance metrics that distort multi‑modal interactions and parent‑to‑child semantic hierarchies. To overcome these limitations, we introduce a text‑driven fusion framework empowered by hyperbolic manifold learning. During training, BLIP‑extracted text prompts serve as topological anchors within the hyperbolic space, guiding vision‑attribute alignment through hyperbolic embeddings that naturally accommodate varying semantic granularities. By exploiting the exponential volume growth dictated by the Poincaré ball's negative curvature, this approach seamlessly embeds hierarchical trees to encode coarse‑to‑fine semantics without metric saturation, while the vast peripheral space prevents texture distortion during cross‑modal fusion. At inference, the fusion process autonomously adapts to input content using the learned text‑attribute priors, completely eliminating the need for textual input. Experimental results show our method outperforms state‑of‑the‑art approaches on benchmark datasets, with code available at https://github.com/Shaoyun2023/TEDFusion.
Authors:Qianyu Wang, Xiaoman Wang, Yuanyuan Liang, Xinyuan Li, Yunshi Lan
Abstract:
Grammatical error correction (GEC) systems are usually trained and evaluated on GEC benchmarks, but their performance often drops sharply once the surrounding context is slightly perturbed or extended. This indicates that the existing GEC models usually fail to understand the error patterns in the varying contexts. In this paper, we thoroughly investigate the counterfactuals for GEC tasks, where the subtle changes to the contexts could lead to the label flipping issue. We propose CoCoGEC, a counterfactual generation framework that creates copies of training instances with error‑irrelevant contexts altered. Our framework systematically generates counterfactuals by (1) generating intra‑ and inter‑sentence counterfactuals that maintain the error patterns as well as syntax of the original instances by altering the word‑level and sentence‑level contexts; (2) revising the generated counterfactuals by selecting the instances with flipped labels and high GEC Mutual Information (MI) coefficient. Extensive experiments show that our method substantially improves the stability of GEC models, outperforming a set of data augmentation baselines. Particularly, it could achieve absolute F0.5 gains of +9.9, +11.3, and +20.8 points on the perturbed BEA‑19,CoNLL‑14, and TEM‑8 data set.Our code is released at https://github.com/Quinnok/CoCoGEC
Authors:Aarav Bedi
Abstract:
Manipulation demonstrations have temporal phase structure, and a natural hypothesis is that demonstration‑curation metrics should be applied within phases rather than globally. The idea is to segment each trajectory into phases, score each phase with the metric that is locally most informative, and then aggregate. This follows directly from prior work showing that a single global metric can be the best detector of a defect and yet the worst curator of the resulting policy. We test the per‑phase hypothesis on three contact‑rich LIBERO pick‑and‑place tasks with a controlled early‑release structural defect, comparing phase‑gated curation against the same metrics applied uniformly and against a strong single global metric. Across all three tasks and five random seeds per condition, phase‑gated curation is never the best curation strategy, and it is the worst of the three on two of the three tasks (Task 1: 86.0 vs. 92.0 for global; Task 3: 22.7 vs. 48.0 for uniform). We trace the failure to a concrete mechanism. When the defect signal is concentrated in a single phase, rank‑aggregating across phases dilutes that signal with uninformative scores from defect‑free phases, selecting a worse demonstration subset than simply applying the defect‑informative metric everywhere. We further show that the per‑phase metric selection does not transfer across tasks, since no phase shares a winning metric between any two tasks, so the selection cannot be reused and must be re‑derived per task from a noisy sweep. These results bound a plausible and previously untested method, and they argue that practitioners should prefer identifying a single defect‑informative metric over decomposing curation by phase. We release the full pipeline, all metric implementations, and per‑seed results.
Authors:Xinhang Ma, Taoran Li, Chaowei Xiao, Zhiyuan Yu, Ning Zhang, Yevgeniy Vorobeychik
Abstract:
Indirect prompt injection (IPI) is a major security threat to LLM‑powered agents. Thus, a growing body of work have proposed a variety of defensive approaches against IPI. These can be grouped into three broad categories: 1) prompt‑based (using prompting as a way to prevent agents from following malicious instructions), 2) detection‑based (identifying and filtering malicious instructions), and 3) system‑level (using systems insights, such as control and data isolation, for defense). However, commonly used benchmarks for evaluating defense, such as AgentDojo, are \emphinherently static, generating a fixed distribution of IPI attacks. Consequently, static benchmarks do not usefully evaluate defense robustness to adaptive threats. We address this issue by developing AutoDojo, an adaptive extension of AgentDojo that optimizes IPI against a given defense. Using AutoDojo against state‑of‑the‑art IPI defenses across three task suites and five target models, we make two key observations. First, many defenses offer only limited protection: a cheap, black‑box adaptive attack using a frontier LLM to iteratively optimize the injection raises attack success rate (ASR) well above the level achieved by static injections against nearly all evaluated defenses. Against a filter that reduces static ASR to 0%, AutoDojo recovers 28% overall and 64% on action‑open tasks. Second, for prompt‑level and filter‑based defenses, ASR is substantially higher on \emphaction‑open tasks ‑‑ where the user's request delegates the action itself to attacker‑controlled content ‑‑ than on precisely specified tasks. This is a structural limit: on such tasks the injection can pose as ordinary data rather than an explicit instruction, bypassing defenses that rely on detecting instruction‑like text. AutoDojo is publicly available at https://github.com/xhOwenMa/AutoDojo.
Authors:Jingyi Yan, Ke Sun, Zhenglin Li, Hongying Jia
Abstract:
Moving Target Defense (MTD) is a proactive security strategy designed to enhance cyber‑resilience by dynamically altering system parameters, thereby preventing adversaries from acquiring the critical information needed to execute stealth attacks. In this paper, we consider the case in which the operator modifies the admittance of branches to enable MTD, and focus on the problem of effectively protecting the system with fewer number of branch admittance modifications and shorter computational time. Specifically, we identify the ineffectual branches whose admittance modification do not contribute to the improvement of MTD effectiveness via theoretical analysis. Building on these insights, we propose the Bus Traversal‑based MTD (BT‑MTD), which is a bus‑oriented algorithm that traverses over the buses of the network according to analytically derived guidelines. The performance of the BT‑MTD is evaluated and compared with four existing strategies on standard IEEE test systems, demonstrating its robustness and superior performance in effectiveness, efficiency, and computational cost. The code of BT‑MTD is available at: https://github.com/YJY101/BT‑MTD.
Authors:Jason Chan, Jonathan C. Kao
Abstract:
We present Token Steering (TS), a method for dynamically steering trajectories generated by an autoregressive vision‑language‑action (VLA) model through direct intervention in the action‑token space. TS injects low‑dimensional user inputs into the model's native action‑token representation, allowing users to influence trajectory generation without modifying the underlying vision‑language model (VLM) architecture. Because TS operates entirely at inference time, it requires no additional training or finetuning. User inputs guide rather than override the pretrained policy, allowing users to influence robot actions while preserving the dexterity, smoothness, and task priors learned by the VLA. We evaluate TS on two household manipulation tasks ‑‑ drawer closing after object placement and state‑aware object swapping ‑‑ and improve success rates from 10.0% to 72.5% and from 16.7% to 93.8%, respectively. By enabling lightweight, intuitive steering over robot foundation models, our interface has the potential to improve human‑robot interaction in consumer environments and broaden accessibility for individuals with limited physical control. Project website: https://jasontchan.github.io/token‑steering/ .
Authors:Jea Kwon, Dong-Kyum Kim, Jiwon Kim, Yonghyun Kim, Woong Kook, Meeyoung Cha
Abstract:
Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces analogous to biological memory units remains an open question. This work introduces a geometric framework to identify such "AI engrams" by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed‑form estimator that isolates individual memory traces from globally entangled parameters, and show that this biologically‑derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning and offer geometric insight into how deep networks simultaneously support functional specificity within distributed storage.
Authors:Yilin Wu, Zilin Si, Zeynep Temel, Oliver Kroemer, Andrea Bajcsy
Abstract:
Inference‑time steering adapts pre‑trained generative robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this verification only with visual observations, vision alone is often insufficient for contact‑rich manipulation, where success depends on both global task progress and subtle local interactions such as contact force. We introduce ViTaL, a visuo‑tactile inference‑time steering framework that formulates multimodal guidance as a bi‑level optimization problem. At the high level, visual sampling‑and‑verification performs long‑horizon mode selection, deciding what behavior the robot should execute. At the low level, tactile‑guided diffusion editing refines the selected action sequence over a shorter horizon to satisfy local contact requirements. To support outcome‑based steering, ViTaL learns a visuo‑tactile latent world model and employs semantically aligned visual and tactile verifiers, including a novel text‑conditioned tactile reward that scores predicted tactile futures directly in latent space. Across three real‑world contact‑rich manipulation tasks, ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive multimodal fusion by at least 20%. Website: https://yilin‑wu98.github.io/vital_website.
Authors:Haoru Tan, Sitong Wu, Yanfeng Chen, Jun Xia, Ruobing Xie, Bin Xia, Xingwu Sun, Xiaojuan Qi
Abstract:
While large and diverse datasets have driven recent advances in large models, identifying the optimal data mixture for pre‑training and post‑training remains a significant open problem. We address this challenge with FASTMIX, a novel framework that automates data mixture discovery while training only a single proxy model. Instead of relying on predefined heuristics or resource‑intensive simulations, FASTMIX jointly optimizes mixture coefficients and model parameters, substantially improving efficiency and scalability over prior approaches. At the core of FASTMIX is a reformulation of mixture selection as a bilevel optimization problem. Under this reformulation, we show that optimizing mixture ratios is mathematically equivalent to assigning per‑source loss weights under uniform source sampling. This embeds the mixture coefficients directly into the differentiable iterative optimization objective, enabling efficient, gradient‑based optimization of both mixture and model. To solve the optimization problem, FASTMIX implements an approximate iterative optimization procedure, alternating between (i) updating model parameters on data sampled according to current mixture ratios (inner loop) and (ii) updating mixture ratios based on validation feedback (outer loop). Across pre‑ and post‑training, FASTMIX outperforms baselines while drastically reducing search cost. Code (https://github.com/hrtan/fastmix)
Authors:Adnan El Assadi, Roman Solomatin, Isaac Chung, Chenghao Xiao, Deep Shah, Manan Dey, Shriya Sudhakar, Zacharie Bugaud, Wissam Siblini, Ayush Sunil Munot, Yashwanth Devavarapu, Rakshitha Ireddi, Michelle Yang, Márton Kardos, Niklas Muennighoff, Kenneth Enevoldsen
Abstract:
We introduce the Massive Video Embedding Benchmark (MVEB), a 23‑task benchmark for video embeddings spanning classification, zero‑shot classification, clustering, pair classification, retrieval, and video‑centric question answering. We evaluate 33 models and find that no single model dominates: MLLM‑based embeddings lead on classification, clustering, pair classification, and QA; multimodal binding leads on retrieval and zero‑shot classification; generative MLLMs without contrastive adaptation collapse on cross‑modal tasks. Paired video‑only vs. audio+video evaluations show that audio's contribution depends on dataset annotation provenance: audio helps when labels were produced from both modalities and hurts when they were produced from visuals alone, a six‑point gap consistent across model families. MVEB is derived from MVEB+, a 184‑task pool, and is designed to maintain task diversity while reducing evaluation cost. It integrates into the MTEB ecosystem for unified evaluation across text, image, audio, and video. We release MVEB and all 184 tasks along with code and a leaderboard at https://github.com/embeddings‑benchmark/mteb.
Authors:Zhengtao Gui, Sheng Yang, Zhouxing Shi
Abstract:
Proof autoformalization aims to translate a mathematical informal proof written in natural language into a formal proof in a formal language such as Lean~4. Several works have developed LLM‑based models for proof autoformalization. However, existing evaluations have typically focused on translating well‑formed informal proofs from curated datasets. We argue that a robust proof autoformalizer must remain faithful even for informal proofs that diverge from these idealized ones, and we present the first study on the robustness of proof autoformalization models. We formulate two categories of perturbations and evaluate robustness under each: a global perturbation paraphrases the informal proof in a different style, under which the formalization should remain consistent; a local perturbation alters a value, symbol, or proof step, possibly in a counterfactual way, and a robust formalization should faithfully reflect the perturbation rather than reverting to the original one or inferring a different one on its own. We build a benchmark with both perturbations on miniF2F and MATH‑500, and automatically measure how stable a proof autoformalization's correctness is under global perturbations and how faithfully its output reflects local perturbations. We evaluate seven recent models, all of which are sensitive to global perturbations and mostly fail to remain faithful under local perturbations. Code and data are available via https://github.com/ucr‑rai/robust‑proof‑autoformalization.
Authors:Igor Itkin
Abstract:
A content‑moderation system can score well on every standard accuracy metric and still cause real harm, if its mistakes fall on the few users who connect otherwise separate communities. We show this in an agent‑based model where N=240 learning agents on a community‑structured network each post harmless, productive, or dangerous content, and a regulator removes or penalizes whatever a noisy classifier flags. Overall usefulness barely moves as the noise changes (one‑way ANOVA, p=0.96): by aggregate measures, nothing looks wrong. The damage instead concentrates on these bridge users, whose useful posts are wrongly suppressed and whose dangerous posts are wrongly spared. A governance loss (L_gov) that prices these two mistakes separately from the cost of enforcement more than doubles under false‑positive‑heavy noise. Aggregate accuracy hides who is harmed, and the cheap quantity to audit is how many connections a user has (degree), a near‑perfect proxy for the betweenness that defines a bridge (r=0.96).
Authors:Yizhao Huang, Haoyang Chen, Shiqin Wang, Pohsun Huang, Jiayuan Li, Haoyuan Du, Yandong Shi, Zheng Wang, Zhixiang Wang
Abstract:
This paper argues that a systemic lack of Agency constrains the implicit reasoning capabilities of current Vision‑Language Models (VLMs). Implicit reasoning refers to the ability to autonomously discover and utilize hidden visual evidence to bridge information gaps, rather than merely relying on explicitly specified targets. This capacity underlies human visual understanding and everyday reasoning. We argue that this limitation arises from a tendency to approach visual reasoning primarily as passive semantic retrieval, rather than as active, situated reasoning that depends on autonomous visual exploration. As a result, most existing benchmarks primarily assess Passive Capacity, leaving this aspect of reasoning largely unmeasured. To address this gap, we introduce the Visual Implicit Reasoning Diagnosing Benchmark (V‑IRD), which targets this missing quadrant by requiring models to derive answers strictly through autonomous visual analysis. Our results show that, despite strong retrieval abilities, prominent VLMs struggle to utilize reference objects and to attend to visual evidence that requires self‑directed inquiry. Simply put, strong semantic recognition does not equate to active visual exploration, revealing a critical gap in current VLMs. More information can be found at https://haoychen.github.io/Implicit‑Reasoning/
Authors:Fengrui Liu, Ruiyang Huang, Qijian Zheng, Yuanfang Wang, Feng Liu
Abstract:
Self‑supervised learning advances audio representation for multimedia analysis. However, prevailing data‑centric approaches rely on massive real‑world corpora, increasing training costs, curation burdens, and privacy barriers. To address this, we present AudioPG, a procedural synthesis framework eliminating real audio recordings during pre‑training. AudioPG trains a Transformer‑based masked autoencoder on waveforms generated on‑the‑fly from basic acoustic primitives and composition rules. The encoder transfers effectively to real audio benchmarks, achieving 90.60% accuracy on ESC‑50, 0.546 mAP on FSD50K, 88.17% on UrbanSound8K, and 97.03% on Speech Commands V2. Notably, pre‑training completes in under 20 minutes on a single GPU. Latent space analysis reveals physical factors, including fundamental frequency and relative intensity, emerge in orthogonal subspaces, making representations linearly decodable. These results establish procedural synthesis as an efficient, interpretable pre‑training signal when large‑scale corpora are unavailable. Our code is available at: https://github.com/Freyliu0516/audioPG.
Authors:Tianhao Chen, Yuheng Wu, Kelu Yao, Xiaogang Xu, Xiaobin Hu, Dongman Lee
Abstract:
Multimodal Large Language Models (MLLMs) achieve strong vision‑language reasoning, but long visual contexts enlarge the KV cache and increase decoding latency. Existing compression methods rely on observation window attention for stable token‑importance estimation, yet this aggregation can dilute sparse visual evidence and discard answer‑critical tokens under aggressive compression. Therefore, we identify last‑query attention as a complementary source for recovering such evidence, but its answer‑irrelevant signals can mislead retention. We propose BACON, a plug‑and‑play method that calibrates observation window attention with last‑query evidence and suppresses isolated noise via intra‑layer coherence and inter‑layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%. Our project page is available at https://ryu1ion.github.io/official_BACON/
Authors:Daniel Torres, Julia Navarro, Catalina Sbert, Joan Duran
Abstract:
Underwater imaging plays a crucial role in ocean engineering, although captured data often suffer from poor visibility and color distortion. To address these challenges, we propose a model‑based deep unfolding network for underwater image enhancement that integrates variational modeling into a learnable architecture. The framework is guided by a variational formulation based on a dehazing decomposition, incorporating a multiplicative residual component to absorb remaining artifacts and a nonlocal gradient‑type constraint to preserve structural details and enhance edge sharpness. We provide a theoretical analysis establishing the existence of solution for the associated minimization problem. The proposed unfolding method incorporates Mamba layers to efficiently capture self‑similarities in the scene. In addition, we introduce a proximal trajectory loss that enforces consistency between the unfolding stages and the iterations of an ideal restoration regularizer. Experimental results demonstrate that the proposed unfolding approach achieves improved visual quality and competitive quantitative performance compared with recent state‑of‑the‑art methods. The source code will be available at https://github.com/MIA‑UIB/Variational‑Unfolding‑Mamba‑Underwater‑Enhancement .
Authors:Jinwen Wen
Abstract:
We present Double‑Helix Vision (DH), a geometry‑based visual sampler that compresses 2D images into compact 1D signals using paired golden‑ratio‑inspired spiral trajectories. Rather than processing every pixel uniformly, DH employs two phase‑shifted helices (Alpha and Beta, offset by 180 degrees) to sample the image with biologically‑inspired foveation: high density at the center, sparse coverage at the periphery. At 4K resolution, DH achieves a 1,433x compression ratio (99.93% reduction) while preserving the geometric structure of the scene. The full perception pipeline ‑‑ including spatial mapping, temporal collision detection, and intra‑frame structural disparity estimation ‑‑ runs in 0.52 ms at 1080p on CPU‑only hardware, with no neural network dependencies. On CIFAR‑10 at extreme sampling budgets (K=128 points per helix), DH achieves a +6.03% accuracy gain over uniform random sampling. A JSON‑serializable Robotics API is provided, delivering sub‑millisecond spatial perception reports in 2.7 KB packets. Code and benchmarks are available under the MIT License.
Authors:Emirhan Bilgiç, Baptiste Caramiaux, Zhi Yan, Gianni Franchi
Abstract:
As Vision‑Language Models are increasingly deployed in safety‑critical applications, the trustworthiness of their explanations becomes crucial. Explainable AI (XAI) methods for Vision‑Language Models often suffer from semantic hallucination, where attribution maps highlight prominent image regions even when prompted with incorrect text descriptions (e.g., highlighting a dog when prompted ``cat''). Although this problem is widespread, a formal mathematical analysis of XAI methods and CLIP embeddings is largely missing in the literature. We demonstrate that this phenomenon is not specific to a single architecture but is a fundamental consequence of Linear Semantic Leakage in high‑dimensional embedding spaces. We propose a unified theoretical framework, Linear Semantic Attribution (LSA), which generalizes across discriminative methods. We introduce OSP, a geometric intervention that utilizes the residual property of OMP to disentangle unique semantic signals from shared concepts. We prove theoretically and demonstrate empirically that OSP minimizes hallucination by orthogonalizing the query vector against distractor concepts, rendering the attribution model blind to shared features while preserving fidelity for correct prompts. Our code is available at: https://github.com/emirhanbilgic/Orthogonal‑Semantic‑Projection
Authors:Nadav Orenstein, Aviad Cohen Zada, Shai Avidan, Gal Oren
Abstract:
Texture segmentation stresses foundation segmentation because meaningful regions are defined by material or repeated appearance rather than object identity. Segment Anything Models (SAMs) often fail by default on such texture‑defined partitions, but this failure is ambiguous: the texture evidence may be absent, missing from the proposal bank, or present but selected or assembled incorrectly by an object‑centric readout. We ask what texture‑relevant evidence is already preserved in frozen SAM before adaptation. We study two frozen evidence spaces: multiscale features, probed with a minimal clustering readout, and the automatic proposal bank, treated as evidence for a supervised consolidation readout. SAM is frozen throughout; we do not fine‑tune the backbone or retrain the proposal generator. Across RWTD, STLD, an ADE20K‑selected refined‑crop complement, and a ControlNet‑stitched PTD bridge archive, frozen SAM is not a texture segmenter by default, but its failures are not simple texture blindness. Coarse frozen features preserve texture organization, and proposal banks often contain texture‑aligned masks or fragments. Natural scenes more often require assembly and commitment over fragments, while cleaner synthetic cases more often reduce to selecting an already coherent proposal. Default mask failure should therefore be decomposed into representation evidence, proposal‑bank support, readout mismatch, and commitment failure.
Authors:Aviad Cohen Zada, Nadav Orenstein, Shai Avidan, Gal Oren
Abstract:
Images can be segmented based on visual cues (i.e., texture segmentation) or into objects (i.e., semantic segmentation). We propose a new category of sub‑semantic image segmentation that blurs the line between the two. In sub‑semantic image segmentation, language is not used to name whole objects. Instead, it is used to partition an image into stable appearance patterns that can be described by language. To do that, we couple a general‑purpose vision‑language model to SAM 3, a promptable segmentation backbone whose native text pathway can ground rich descriptions into masks. Simple coupling fails for a number of reasons that we identify in the paper, and we overcome them by introducing DETECTURE that resolves three concrete failure modes ‑‑ language leakage between texture regions, prompt competition inside the segmentation backbone, and semantic distortion at the language‑to‑mask interface. Since there is no dataset of sub‑semantic image segmentation, we introduce one, termed TextureADE. The new dataset is derived from the ADE20K dataset using a system we designed. We compare DETECTURE to a number of baselines and find that it achieves the strongest performance on several datasets using different metrics. Code is available at https://github.com/Scientific‑Computing‑Lab/TextureDetecture.
Authors:Xirui Kang, Yanpei Shi, Lucy Liang, Roy Gan, Dongxiu Liu, Pushi Zhang, Danpeng Chen, Xiaoyi Qin, Yinan Zheng, Jinliang Zheng, Hao Wang, Xianyuan Zhan, Hang Su
Abstract:
Modern Vision‑Language‑Action (VLA) models must bridge pretrained vision‑language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing codes that preserve motion geometry but provide only weak semantic supervision to the backbone. We therefore formulate action tokenization not as mere compression, but as semantic interface learning between multimodal reasoning and executable control. To this end, we introduce X‑Tokenizer, a lightweight encoder‑Semantic Residual Quantization (SRQ)‑decoder architecture that provides a shared action interface across diverse robotic arm embodiments. Its key component, SRQ, imposes an asymmetric structure on residual vector quantization: the first level is trained with Masked Action Modeling (MAM) to form a discrete action language that captures coarse motion intent, while deeper levels remain reconstruction‑oriented residuals that preserve fine‑grained details. To further align action tokens with multimodal semantics, X‑Tokenizer is pretrained with contrastive alignment to the representation space of a pretrained foundation model and with next‑frame vision‑language feature prediction. Pretrained on 2.4M trajectories (2.0B action frames), a single frozen X‑Tokenizer plugs into a mixed discrete‑continuous VLA as a representation‑shaping supervision signal. X‑Tokenizer achieves top real‑world aggregate and strong RoboTwin 2.0 simulation results. Outperforming FAST in multimodal grounding (+13.5%) and long‑horizon tasks (+8.25), it shows that action tokenizers serve as semantic interfaces for VLA pretraining beyond mere action compression.
Authors:Shiwen Zhang, Haoyuan Wang, Xianghao Zang, Haibin Huang, Chi Zhang, Xuelong Li
Abstract:
Content‑Preserving Style transfer, given content and style references, remains challenging for Diffusion Transformers (DiTs) due to entangled content and style features. With a reverse triplet synthesis pipeline to build a million‑scale training set and a dual‑branch Style‑Content DiT (SC‑DiT) that decouples style and content via separate ROPE embeddings and causal masking, we observe that such a one‑stage training paradigm on mixed style categories causes semantic styles to dominate, hindering texture style learning, and harming content preservation. To address these issues, we propose Style‑CCL, a Multi‑Stage Curriculum Continual Learning framework that trains SC‑DiT from semantic (easy) to texture (hard) styles, and from clean to synthetic data, with Random Memory Rehearsal across stages to avoid catastrophic forgetting. Extensive experiments demonstrate that our Style‑CCL achieves state‑of‑the‑art performance in three core metrics: style similarity, content consistency, and aesthetic quality.
Authors:Romiyal George, Sathiyamohan Nishankar, Selvarajah Thuseethan, Roshan G. Ragel
Abstract:
Vision Transformers (ViTs) have demonstrated strong representation capability in image classification. However, their quadratic self‑attention complexity and large parameter counts limit deployment on resource‑constrained mobile and edge devices. This paper introduces UtVAA, an ultra‑tiny Vision Transformer architecture designed for efficient visual recognition under strict computational budgets. It incorporates a novel Affix Attention block that combines depthwise‑pointwise local feature extraction, linear self‑attention, coordinate attention for spatial dependency modelling, and a lightweight ternary fusion strategy to integrate local and global representations. In addition, Dilated Bottleneck blocks expand the receptive field using dilated depthwise separable convolutions while maintaining low FLOPs and stable optimisation through residual connections. UtVAA is implemented in scalable Tiny, Medium, and Large variants, with the smallest model containing 204.67K parameters and 53.95M FLOPs. Experimental results on CIFAR‑10, CIFAR‑100, PlantVillage‑Tomato and SLIF‑Tomato datasets show that UtVAA achieves competitive accuracy within a sub‑million‑parameter regime. Overall, the results demonstrate that transformer‑based vision models can be redesigned into ultra‑tiny architectures without significant loss in discriminative performance, making UtVAA suitable for mobile and edge deployment. Code is available at https://github.com/romiyal/UtVAA
Authors:Matiur Rahman Minar, Seunghun Oh, GangHyeon Jeong, Unsang Park
Abstract:
Autoregressive video diffusion models enable streaming generation but often degrade over long rollouts: static scene layouts drift, while mechanisms that improve spatial stability tend to suppress motion, causing natural flows such as water, fire, or smoke to stagnate. We study this stability‑motion trade‑off in fixed‑camera long‑horizon nature video generation, where the two failure modes can be more clearly separated than in moving‑camera settings. We propose Steady‑Forcing, a memory and training framework combining a persistent visual anchor (V‑Sink), an exponential moving‑average motion memory (EMA‑Sink), block‑relative temporal encoding, periodic cache purification, and distillation from a Wan2.1‑14B teacher with motion‑rewarded priors under task‑focused configurations. Together, these components are designed to preserve background identity while sustaining visually plausible fluid dynamics over multi‑minute autoregressive rollouts. Evaluations across seven baselines show that Steady‑Forcing improves long horizon background consistency and imaging quality, while a blind user study indicates stronger perceived stability and motion continuity. The benchmark evaluation further suggest that generic VBench aggregate scores under‑penalize fixed‑camera artifacts as well as rewarding drift‑induced optical flow as Dynamic Degree while not directly penalizing texture hardening or flow stagnation ‑ motivating future task‑specific benchmarks for static‑camera nature‑flow evaluation. Project page: https://minar09.github.io/steadyforcing/
Authors:Mostafa Dahshan, Quazi Mamun, Tanmoy Debnath
Abstract:
Large language models (LLMs) provide strong performance across a wide range of tasks but are typically hosted on centralised cloud infrastructure, incurring significant bandwidth, latency, and privacy costs. In contrast, small language models (SLMs) can run on edge devices but have limited capability and robustness. This paper introduces SWARM‑LLM, a routing and collaboration layer that coordinates a small swarm of edge‑hosted SLMs with an optional cloud foundation model (FM). SWARM‑LLM decides, for each query, whether to answer locally, collaborate with peer SLMs, or "summon" a cloud FM, using lightweight uncertainty estimates and safety signals. We implement a working prototype on commodity hardware with three heterogeneous SLMs and a 70B‑parameter cloud FM accessed via API, and evaluate it on a controlled study workload of easy, hard, and safety‑oriented queries. Our results show that SWARM‑LLM substantially improves performance on hard questions compared to an edge‑only deployment, while limiting cloud usage to roughly one quarter of queries, illustrating a practical trade‑off between accuracy, latency, and cost for privacy‑conscious edge deployments. The implementation code is available at the GitHub repository https://github.com/mdahshan/swarm_llm.
Authors:Xinyue Cai, Chaoyou Fu, Yi-Fan Zhang, Ran He, Caifeng Shan
Abstract:
Current automated pipelines for audio‑visual Question Answering (QA) generally adopt a ``video‑caption‑QA'' paradigm. However, these methods typically segment videos into short clips and generate separate descriptions for audio and visual modalities. This decoupled processing severs inherent associations between sounds and their visual sources, while independent clip processing often causes inconsistent descriptions of the same entity across segments. Furthermore, coupling long‑text comprehension and QA synthesis into a single step often restricts models to localized events, yielding questions lacking long‑term temporal connections and deep cross‑modal reasoning. To address these issues, we propose an automated data engine featuring two mechanisms: (1) Entity‑Anchored Video Scripting transforms videos into structured scripts, comprising summaries, main entity lists, and segment‑wise audio‑visual descriptions. The entity list serves as a global prior to ensure cross‑segment referential consistency and reconstruct audio‑visual associations. (2) Clue‑Guided QA Generation prompts models to first mine cross‑segment, multimodal clues from the script, and subsequently generate QA pairs based on these high‑value clues. Leveraging this pipeline, we construct the instruction‑tuning dataset OmniVideo‑100K and a human‑verified test set, OmniVideo‑Test. Fine‑tuning VITA‑1.5, Qwen2.5‑Omni‑7B and Qwen3‑Omni‑30B on OmniVideo‑100K yields performance gains of up to 20.59% on OmniVideo‑Test, demonstrating strong generalization (up to 12.64% improvements) across established benchmarks like Daily‑Omni and JointAVBench.
Authors:Sicheng Yang, Hangjie Yuan, Wenjun Zhang, Jinwang Wang, Yichen Qian, Weihua Chen, Fan Wang, Lei Zhu
Abstract:
Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical hallucination benchmarks mainly focus on data collection, but often ignore where hallucinations originate within the reasoning process. We find that hallucination sources vary across samples: errors may arise from visual misrecognition, incorrect medical knowledge recall, or flawed reasoning integration. To enable source‑level hallucination diagnosis, we introduce ClinHallu, a benchmark for stage‑wise hallucination diagnosis in medical MLLM reasoning. ClinHallu contains 7,031 validated instances, where each instance is augmented with a structured reasoning trace decomposed into Visual Recognition, Knowledge Recall, and Reasoning Integration. We also use stage‑replacement interventions to measure how correcting specific stages affects the final answer. Beyond evaluation, we show that trace‑supervised fine‑tuning reduces stage‑wise hallucinations. ClinHallu provides a fine‑grained hallucination testbed for diagnosing and mitigating reasoning failures in medical MLLMs. The benchmark is publicly available at https://github.com/alibaba‑damo‑academy/ClinHallu.
Authors:Jinsu Kim, Jihoon Tack, Noah Lee, Jongheon Jeong
Abstract:
Language Models (LMs) have shown remarkable potential as role‑playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona. However, applying these capabilities to real‑world applications (e.g., ecosystems with numerous NPCs interacting simultaneously) exposes a critical inefficiency due to the excessive computational cost. In this paper, we question the necessity of dedicating a full, generalist model to a single persona, hypothesizing that a specific character identity relies on only a fraction of the model's total capacity. We observe that naively pruning LMs often severely degrades the role‑playing performance for a specific persona; it does not distinguish between redundant knowledge and essential character traits. We propose Persona‑Pruner, a framework that sculpts a lightweight role‑playing model by isolating persona‑specific sub‑networks from a single description. Our experiments consistently show that Persona‑Pruner preserves role‑playing performance substantially more effectively than existing state‑of‑the‑art LLM pruning techniques, reducing the performance drop from the dense model by up to 93.8% over the strongest baseline on RoleBench in LLM‑as‑a‑judge score, while still maintaining general LLM capabilities. Code is available at https://github.com/jsu‑kim/Persona‑Pruner.
Authors:Xuan Wei, Longbin Ji, Guan Wang, Xiangrui Liu, Zhenyu Zhang, Shuohuan Wang, Yu Sun, Qingqi Hong
Abstract:
Long‑form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next‑shot continuations without verifying whether the historical memory preserves identity‑critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject‑reconstruction‑guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next‑shot generation with memory‑based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long‑range subject evidence from short‑range cues, Memento introduces a dual‑query memory mechanism, where one query retrieves identity‑relevant memory and the other selects short‑context keyframes for coherent continuation. Additionally, a subject‑aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun‑free subject descriptions. Experiments demonstrate that Memento achieves state‑of‑the‑art performance in long‑term subject consistency, cross‑shot coherence, and visual quality.
Authors:Yue Xu, Mingtao Nie, Tianle Li, Hong Li, Yibo Luo, Siyuan Huang, Yong-Lu Li
Abstract:
Robot learning from real‑world demonstrations is currently constrained by data scaling. Universal Manipulation Interface (UMI) provides an efficient robot‑free data collection interface, yet current UMI‑style pipelines often collect redundant demonstrations and lack global scene context. To improve data efficiency, we present EgoGuide, a collection interface that records synchronized wrist and head/egocentric observations and couples them with online visual‑geometric data quality guidance. We also introduce a Gated Egocentric Residual Policy for robust learning from a viewpoint‑varying egocentric camera, allowing head/egocentric context to correct ambiguous local observations while preserving stable wrist‑view control. Real‑world experiments show that EgoGuide reduces the required number of data episodes and improves data efficiency. The residual policy further improves robustness under visual occlusion. Project Page: https://silicx.github.io/EgoGuide
Authors:Yijun Liu, Jie Huang, Zeyue Xue, Yuming Li, Ruizhe He, Haoran Li, Shijia Ge, Siming Fu
Abstract:
Reward models guide text‑to‑image (T2I) systems toward outputs aligned with human preferences. However, typical reward models such as HPSv3 are trained on pre‑annotated data from earlier T2I models, without accounting for quality discriminative shifts arising from evolving model capabilities and reinforcement learning (RL) iterations, limiting their broader applicability. In this work, we propose HPSv3++, a reward model framework that elevates the HPSv3 model for varying T2I model capabilities and their RL iteration changes across the full capability‑iteration spectrum. Specifically, we first introduce HPDv3++, a 212K dual‑dimension preference dataset annotated for text fidelity and aesthetic quality using a recent high‑capability (Qwen‑Image) model with human supervision. We then propose a two‑stage training framework. Stage 1 employs data‑aware orthogonal gradient projection to incorporate diverse aesthetic perception from HPDv3++ while preserving the original effective human preference knowledge in HPSv3. Stage 2 further leverages unlabeled data from T2I models spanning different capability levels and RL iterations, and introduces a joint capability‑iterations conditioned signal for the reward model together with a standard deviation‑driven unsupervised guidance mechanism, strengthening reward model across the capability‑iteration spectrum. HPSv3++ achieves state‑of‑the‑art preference prediction, outperforming HPSv3 9.8% on HPDv3, 5.5% on GenAI‑Bench, while achieving 79.1%/88.1% on our proposed HPDv3++. When used for T2I RL training, it consistently improves GenEval scores across diverse T2I models, demonstrating its wide‑range capabilities. The code is available at https://github.com/PlantPotatoOnMoon/HPSv3‑PlusPlus.
Authors:Imane Meddour, Andréa Macario Barros, Cédric Gouy-Pailler
Abstract:
In this work, we propose StereoGeo, an end‑to‑end network‑based approach for stereo camera calibration. Our method estimates the focal lengths and gravity directions of the left and right cameras, as well as the relative extrinsic transformation relating them. Existing methods often rely on calibration patterns in structured environments or address only a single camera configuration, being limited to either intrinsic or extrinsic estimation, and depending on a multi‑view setups. StereoGeo extends the GeoCalib algorithm, integrating deep neural network feature extraction with a differentiable optimizer. Extensive experiments on real‑world benchmarks demonstrate that StereoGeo achieves competitive performance for intrinsic calibration and provides accurate stereo extrinsic estimation, outperforming existing methods that are limited to monocular settings. The dataset used in this work is partially publicly available at https://github.com/meddourimane/StereoGeo‑dataset.
Authors:Armand de Villeroché, Sibo Cheng, Vincent Le Guen, Marc Bocquet, Rem-Sophia Mouradi, Patrick Armand, Alban Farchi, Patrick Massin
Abstract:
Transformer‑based neural operators have shown remarkable performance for approximating solution operators of partial differential equations on complex geometries. However, existing approaches implicitly assume a fixed domain size, which limits their ability to generalize at inference. In this work, we investigate domain extension, namely zero‑shot inference on spatial domains that are significantly larger than those encountered during training. We argue that this setting fundamentally requires spatial locality and translation equivariance. We propose to implement this locality via a decomposable bias in the attention logits computation, enabling finely controllable locality while remaining fully decomposable into query‑key inner products and directly compatible with optimized attention kernels. Combined with rotary positional embeddings, it enables expressive embeddings with controllable spatial support without altering the transformer architecture. We empirically show that our approach substantially improves zero‑shot generalization to larger domains across two PDE benchmarks and a 3D industrial atmospheric flow application. Our code and datasets are available at https://github.com/cerea‑daml/domain‑extension.
Authors:Wei Wu
Abstract:
LLM agent systems increasingly run as long‑lived autonomous runtimes: scheduling jobs, calling tools, maintaining memory, and pushing results to humans. We present a longitudinal study of silent failures in one such system: a personal‑assistant agent runtime in continuous production since March 2026, with roughly 40 scheduled jobs, 8 LLM providers, a tool‑governance proxy, and a knowledge‑base memory plane, defended by 4,286 unit tests and 827 governance checks. Over eight weeks we documented 22 incidents with full root‑cause postmortems, in which one meta‑pattern ‑‑ a failure whose error signal never reaches a human in actionable form ‑‑ manifested at least 28 times. We derive a five‑class, mechanism‑oriented taxonomy: (A) environment and platform quirks, (B) design‑assumption mismatches, (C) error swallowing and dilution, (D) chained hallucination and fabrication, (E) operational omission and forensic blind spots. Class D is unique to LLM systems and the most dangerous: the system does not merely fail to report an error ‑‑ the LLM transforms it into fluent, plausible narrative delivered to the user. We term this fail‑plausible: gray failure's differential observability escalated ‑‑ the observer is not just blind, it is convincingly lied to by the failure itself. Three findings: about 70% of silent failures were caught by human user‑view observation, not tests or audits; a retrospective audit of 15 incidents found 0% ex‑ante prevention but 87% regression blocking ‑‑ audits are regression engines, not prediction engines; incident latency (13 hours to 60 days) tracks failure mechanism, not code complexity ‑‑ the longest‑lived failures lived in the seams between components, where no test runs. We describe the resulting defense framework and distill design principles for agent systems whose failures are loud, attributable, and boring. All postmortems and artifacts are public.
Authors:Guanming Liu, Yuqi Ren, Hansu Gu, Peng Zhang, Weihang Wang, Jiahao Liu, Ning Gu, Tun Lu
Abstract:
A central role of personal‑agent memory is to turn stored information and prior interactions into future‑oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two‑step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow‑up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow‑up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow‑up behavior, even when evidence is stored or feedback is incorporated locally. StreamMemBench is publicly available at https://github.com/landian60/StreamMemBench.
Authors:F. Nisa Bostanci, Haocong Luo, Ataberk Olgun, Maria Makeenkova, Geraldo F. Oliveira, A. Giray Yaglikci, Onur Mutlu
Abstract:
Cycle‑level DRAM simulators provide accurate and flexible models for DRAM and memory controller operations and enable research on current and future memory systems. Therefore, they are critical for improving the performance, efficiency, and robustness of DRAM‑based memory systems. Ramulator 2.0 (successor of Ramulator) is a highly modular and extensible cycle‑accurate DRAM simulator that enables rapid exploration of new ideas in DRAM‑based memory systems. A MICRO 2024 best paper runner‑up publication, A Mess of Memory System Benchmarking, Simulation and Application Profiling, which we refer to as "the Mess paper," with all three artifact badges awarded (including "Reproducible"), proposes a new benchmark to evaluate real and simulated memory system performance. While doing so, it makes strong negative claims about Ramulator 2.0 and shows unexpected results. In this talk and the associated extended abstract, we demonstrate that these results and claims in the Mess paper are incorrect and are due to configuration and simulator usage errors made in the Mess paper. We describe four best practices to aid users and developers of simulation tools to avoid such issues in the future. We emphasize the importance of contacting simulator authors and developers when unexpected results are observed (especially and importantly before publishing such results), to ensure these simulators are used with correct configurations and as intended. Our investigation also aims to stimulate discussion on artifact evaluation practices and on mechanisms for correcting results and artifacts after publication. To aid future works and reproduction of all our results, we open source all our code and scripts at https://github.com/CMU‑SAFARI/Cleaning‑up‑the‑Mess. We refer the reader to our full ISPASS 2026 paper and its artifact for the complete analysis, detailed methodology, and extended results.
Authors:Zihao Li, Ranpeng Qiu, Yincong Chen, Guoqiang Ren, Weiming Zhi
Abstract:
Robots under autonomous operation may require decisions based on evidence that is no longer visible. We study delayed‑evidence tasks, where an early cue disappears before a later decision point, so visually similar observations can require different actions. In these settings, the current observation is not a sufficient state for control. We introduce TRAjectory‑routed Causal Evidence (TRACE), a memory framework for visuomotor imitation policies. TRACE stores task‑relevant visual and robot‑state evidence, such as object identity, target choice, or route‑dependent state, in a fixed‑size latent memory that remains bounded over long episodes. Instead of indexing memory by raw time or manually provided task labels, TRACE uses path signatures: compact, order‑sensitive features of the executed robot‑state trajectory. These signatures do not store the visual cue itself; rather, they provide trajectory‑conditioned keys for writing and retrieving the evidence stored when the cue was visible. When the robot later reaches an ambiguous observation, the policy conditions on TRACE memory to recover the missing context and choose the correct branch. TRACE attaches through lightweight adapters to policies, without changing the policy backbone, action head, or imitation objective. Across real‑world long‑horizon manipulation tasks with visually ambiguous branch points, TRACE improves branch selection and task success over alternative baselines, including short‑history and recurrent memory. Project page: https://jeong‑zju.github.io/trace
Authors:Carlo Di Cicco
Abstract:
Large language models encode rich information in their hidden states. This work asks whether code correctness is legible in the hidden states of Qwen3‑4B‑Instruct‑2507, before it generates and as it repairs a failed attempt, studied on 444 LiveCodeBench tasks. It reports two findings connected by a single confound‑control tool: residualization. First, the correctness of the model's first‑attempt code is linearly decodable from the prompt‑final hidden state, with a leakage‑free held‑out AUC of 0.931 +/‑ 0.008 across 50 outer splits. After the linear effect of prompt length is removed from each hidden state dimension, the probe still reaches 0.911 +/‑ 0.010, well above a prompt‑length baseline of 0.754 +/‑ 0.014. Second, on 236 cleaned cases where the model attempts to repair a failed first attempt, the hidden state shift from the failing attempt to its repair carries a statistically detectable contrastive direction, significant on both a magnitude and a split‑half test against label‑shuffled nulls. This direction does not survive a conditional residualization against repair‑context covariates that differ between successful and failed repairs, marking it as a correlate of repair success driven by the repair context rather than an isolated repair‑comprehension feature. The probe layer is selected by nested cross‑validation, and the same residualization approach that upholds the pre‑generation correctness result overturns the repair‑direction interpretation. The contribution is as much methodological as empirical: a diagnostic honest enough to report a negative result alongside a positive one.
Authors:Qingkai Fang, Shoutao Guo, Yang Feng
Abstract:
Real‑time, full‑duplex speech interaction is a key feature of next‑generation spoken chatbots, allowing the model to listen and speak at the same time and to handle natural phenomena such as overlap, hesitation, and barge‑in. Existing speech language models (SpeechLMs) such as LLaMA‑Omni and GLM‑4‑Voice are still turn‑based and rely on an external Voice Activity Detection (VAD) module to mark the end of the user's turn, which fundamentally limits their interactive ability. In this paper, we introduce BayLing‑Duplex, a native full‑duplex SpeechLM where a single autoregressive LLM decides when to listen, when to speak, and when to stop, with no auxiliary turn‑taking module. The design adds only a few special tokens to the standard vocabulary, so it transfers across LLMs and reuses existing training and serving stacks with no architectural adaptation. Starting from the public GLM‑4‑Voice checkpoint and using only 400K full‑duplex samples for fine‑tuning followed by a lightweight DPO stage, BayLing‑Duplex reaches 92% turn‑taking success and 100% interruption success on InstructS2S‑Eval, while improving the speech‑response score from 2.17 to 3.39 over Moshi. BayLing‑Duplex also matches or surpasses its turn‑based counterpart on Llama Questions, Web Questions, and Alpaca‑Eval, showing that simultaneous listen‑and‑speak modeling does not sacrifice response quality.
Authors:Liou Tang, James Joshi, Ashish Kundu
Abstract:
The removal of learned data from Machine Learning models through Machine Unlearning (MU) has been widely studied; however, there has yet to be an agreed‑upon scheme for auditing MU. Existing work has shown that a dishonest model owner can falsify evidence to avoid executing MU, while curious auditors (and adversaries) can infer the privacy‑sensitive properties of the model and its training data even with limited access. Yet auditing of MU under mutual distrust between the model owner and the auditor remains unexplored. We provide an information‑theoretic proof for this scenario: for convex ML models, a generic audit scheme that relies solely on querying the model for behavioral signals cannot identify insufficiently unlearned models without revealing membership information of the retained set. Therefore, auditing MU under the assumption of a dishonest model owner and an honest‑but‑curious auditor faces an inherent privacy‑audit tradeoff. Our empirical results on convex models strongly supports this result, while further experiments demonstrate that this privacy‑audit tension persists in non‑convex models. Our results call for a more careful consideration of the privacy‑audit tension under a realistic auditor threat model, and serve as a foundation for more scrutiny of designs of privacy‑preserving audit schemes for the MU pipeline. We also release our code implementation at https://github.com/LiouTang/Behavioral‑Unlearn‑Audit.
Authors:Xihang Shan, Ye Luo
Abstract:
We present a recipe‑controlled decoder audit (RCDA) for structural transductive knowledge‑graph completion (KGC). The audit asks a simple reporting question: before attributing gains to an encoder or training recipe, what changes when the decoder is swapped under the same recipe? Using ComplEx and DistMult as the primary controlled pair, with targeted RotatE/TransE spot‑checks, we evaluate seven benchmarks. On five standard KGs, ComplEx‑vs‑DistMult differences are modest but consistent under our recipe (+0.005 to +0.012 MRR), whereas CompGCN‑style encoder effects vary more by dataset. On small KGs, decoder effects become the main diagnostic: Kinship shows a stable ComplEx advantage of +0.143 MRR (6 seeds), while UMLS favours ComplEx by +0.022 MRR in a clean 6‑seed server rerun but reverses in an earlier provenance variant. We therefore treat small‑KG decoder choice as recipe‑ and provenance‑sensitive rather than as a fixed dataset winner. We further show that decoder choice interacts with encoder depth on WN18RR, and that under our recipe L=0 ComplEx on YAGO3‑10 reaches 0.6971 +/‑ 0.0048 MRR at d=128. The result is a compact audit protocol: report matched decoder rows, log small‑KG provenance, and sweep decoder x depth before making encoder‑level claims.
Authors:Piotr Kitłowski, Dominik Wiącek, Mateusz Modrzejewski
Abstract:
This paper investigates the fragility of post‑hoc explanation methods in audio deepfake detection. While previous work on explanation manipulation focused on images using standard L_p metrics, we introduce a psychoacoustic framework that optimizes inaudible perturbations to decouple model attributions from final classifications. We evaluate this vulnerability across state‑of‑the‑art architectures under strict prediction‑preserving constraints. By evaluating the manipulation cost through domain‑specific perceptual audio quality metrics alongside explanation alignment criteria, our framework demonstrates that an adversary can systematically distort automated explanation heatmaps while preserving the predicted deepfake label. Full code available at: https://github.com/cncPomper/Audio‑XAI
Authors:Ayoub Elkhouzari, Youssef Iraqi, Loubna Mekouar
Abstract:
We present MaskedFOP, a system for closed‑set polyglot speaker identification under two simultaneous challenges: the face modality is entirely absent at test time, and speech comes from Urdu, a language unseen during face‑supervised training. The system integrates three complementary mechanisms. First, a modality‑dropout dual‑head network built on the Fusion and Orthogonal Projection (FOP) backbone forces the audio branch to develop independent discriminative power via per‑sample face masking, ensuring that the audio encoder remains capable when face is absent. Second, two MaskedFOP instances trained on Emphasized Channel Attention, Propagation, and Aggregation in Time Delay Neural Network (ECAPA‑TDNN) features with different random seeds produce complementary audio embeddings whose element‑wise average yields a more robust 512‑dimensional representation than any single model. Third, a two‑stage cascaded inference procedure first refines multimodal labels through a fused Graph Label Propagation (GLP) pass (Stage 1), then assigns audio‑only labels by cosine nearest‑centroid (Stage 2), replacing the 70 sparse training prototypes with ~1,500 in‑domain test‑set centroids from Stage 1. Submitted to the POLY‑SIM 2026 Grand Challenge, the system achieves a mean P‑accuracy of 0.9989, placing first among all submissions evaluated on the challenge server. An ablation identifies cascaded seeding as the single largest gain (>8 pp on P4/P6). The code is available at https://github.com/Ayoub‑Elkhouzari/POLY‑SIM2026.
Authors:Sihan Zhuang, Xinyuan Chen, Tianfan Xue, Yaohui Wang
Abstract:
Recent advances in diffusion‑based video generation have significantly improved visual quality and short‑term temporal coherence. However, existing methods still struggle to produce videos with physically consistent and causally plausible dynamics, especially in scenarios involving long‑horizon interactions. This limitation arises from the fact that video diffusion models primarily learn physical consistency implicitly, while vision‑language models can directly model physical laws. Based on this idea, in this work, we propose CausalMotion, a training‑free framework that injects explicit physical reasoning into video generation through structured intermediate representations. Our key idea is to decouple reasoning from generation by leveraging a vision‑language model to decompose a text prompt into a sequence of causally consistent keyframes and object‑centric motion trajectories. These representations are then aligned and integrated as soft constraints to guide a pretrained video diffusion model during inference. This design enables explicit modeling of object dynamics and causal transitions without requiring additional training or supervision. Extensive experiments show that our method consistently improves physical plausibility and temporal coherence, particularly in dynamics‑intensive scenarios, while maintaining high perceptual video quality.
Authors:Victor Barberteguy, Ahmet Iscen, Mathilde Caron, Alireza Fathi, Gül Varol, Cordelia Schmid
Abstract:
Recent advances in 3D feedforward reconstruction neural networks have achieved remarkable success in dense reconstruction from images without any camera parameters. Yet, equipping these models with robust semantic understanding remains an open problem. Here we introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework. We build on existing 3D reconstruction models and augment them with a set‑based mask decoder. The approach is jointly trained with a geometric and semantic loss, which are shown to be mutually beneficial. More precisely, the features are initialized from the geometric information and then finetuned to capture jointly geometry and semantics. We demonstrate the generality of our approach by successfully applying our framework both to online and all‑to‑all attention reconstruction backbones. Our method achieves state‑of‑the‑art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets. Ablation studies show that such joint training of a unified model equips 3D feedforward reconstruction neural networks with panoptic segmentation and yields mutually beneficial improvements.
Authors:Jiaen Lv, Leran Qi, Shaowei Wang
Abstract:
Time series prototype learning is fundamentally challenged by observational ambiguity. Discrete architectures fail to resolve this, as they lack the capacity to decouple stochastic noise from continuous dynamics. Furthermore, rigid closed‑set assumptions fail to capture unseen diversity. To address these limitations, we propose a hierarchical ordinary differential equation clustering network, which utilizes neural ordinary differential equation to model latent state evolution as a continuous integral curve. This formulation enforces temporal continuity to effectively disentangle smooth feature trends from stochastic noise, while our adaptive hierarchical mechanism autonomously determines the appropriate number of prototypes without rigid prior constraints. Validated on the early link failure detection task with irregularly sampled time series, the proposed method effectively extracts underlying physical prototypes, thereby enabling robust failure detection. Our code is available at https://github.com/NJ‑LNN/Hierarchical‑ODE.
Authors:Haidong Hou, Zhangguo Yu, Tao Han, Hengbo Qi, Khaleel Ghazal, Yu Zhang, Yidong Du, Xuechao Chen, Fei Meng
Abstract:
Fall recovery is critical for autonomous legged locomotion. Existing methods have demonstrated that some legged robots, such as humanoids and quadrupeds, are capable of fall recovery from diverse postures by utilizing arms or coordinating multi‑legs to generate support forces. Without arms or other legs to provide supportive assistance, a bipedal‑wheeled robot must rely solely on the actuation of its legs, making recovery particularly difficult. To address this, we introduce FTSR (Force‑guided Teacher‑student framework with Stage‑wise Rewards). The force‑guided method constructs an external auxiliary force during simulation training that correlates directly with the robot's real‑time height, explicitly formulating this force as an optimizable constraint. Through constrained reinforcement learning, the policy is guided toward reducing force dependency gradually and increasing the body height, developing internal recovery strategies despite having no arms for support. Height‑progressive stage‑Wise rewards progressively structure posture stabilization during recovery and transition to sustained locomotion, integrated with teacher‑student architecture distilling privileged knowledge of force effects and recovery dynamics. After simulation training, the policy is deployed on a physical armless bipedal‑wheeled robot and extensively evaluated. Experiments confirm robust and reliable fall recovery under diverse challenging conditions, demonstrating strong environmental adaptability and motion robustness, while maintaining full post‑recovery motion capability. The framework also generalizes effectively to a high‑DOF humanoid, confirming its practical generalizability. The project page is available at https://2350575870.github.io/force‑guided.github.io/
Authors:Yanzhao Guo, Wenkai Chen, Jianwei Zhang
Abstract:
Diffusion‑based Vision‑Language‑Action (VLA) policies have demonstrated strong capability in modeling expressive and multimodal action distributions. However, their reliance on iterative sampling introduces substantial inference latency, which limits their applicability to reactive closed‑loop robot manipulation. To address this limitation, we propose \textttReactVLA, a lightweight and low‑latency VLA framework for real‑time robotic manipulation. \textttReactVLA combines two complementary designs: (1) an improved Mean Flow (iMF) action generator that reduces expensive multi‑step diffusion sampling to one‑to‑few‑step action generation, and (2) Attention Residuals (AttnRes), a dynamic depth‑wise feature routing mechanism that replaces uniform residual accumulation to better preserve task‑relevant multimodal representations. We evaluate \textttReactVLA on large‑scale simulation benchmarks, including LIBERO and RoboIMI, as well as real‑world robotic manipulation tasks. Experimental results show that \textttReactVLA consistently outperforms similarly sized VLA baselines, including SmolVLA and π_0. On challenging precision manipulation tasks, \textttReactVLA achieves up to a 1.65× improvement in task performance while providing more than a 4× increase in inference speed compared with leading VLA models. Finally, it reduces real‑world policy latency to below 38.6 ms, enabling fast reactive control on physical robot platforms. Please check out our project website at: https://game‑loader.github.io/ReactVLA/.
Authors:Hyejin Oh, Woo-Shik Kim, Sangyoon Lee, YungKyung Park, Je-Won Kang
Abstract:
Multispectral (MS) imaging extends beyond conventional RGB imaging by capturing more spectral bands, thereby improving illuminant spectrum estimation (ISE). However, existing methods often fail to fully exploit spectral information, resulting in suboptimal performance under diverse lighting conditions and across different sensor domains. Hence, we propose a deep learning framework with a spatio‑spectral feature extraction block, which incorporates spectral attention mechanisms to enhance spectral correlation and preserve illuminant‑relevant spatial features. Through the inclusion of an illuminant prior (IP), our approach prioritizes specific channels that provide more meaningful information in an MS image. We also propose a spectral‑domain transform across different MS sensor spaces. The results demonstrate that illuminant spectra learned in high‑dimensional sensor spaces can be effectively transformed to various lower‑dimensional camera sensor spaces without any additional training. To facilitate evaluation, we introduce a real‑world MS dataset containing high‑dimensional ground‑truth illumination spectra captured under diverse lighting conditions. Through extensive experiments, we demonstrate that our method achieves superior accuracy compared to existing models, thus providing a practical solution for real‑world ISE. The code and dataset are available at https://github.com/hyejin5/Spectrum‑Aware‑Illumination‑Estimation‑Using‑Multispectral‑Image.
Authors:Yifan Jiang, Meige Yang, Zitong Li, Jay Pujara
Abstract:
Affordance reasoning, the inference of an object's action possibilities from its physical properties (e.g., shape and material), is fundamental to human physical understanding and increasingly critical for Large Language Models (LLMs). However, existing affordance benchmarks largely expose explicit object identities in the evaluation setup, allowing models to rely on memorized object‑affordance mappings rather than reasoning over physical properties. To address this gap, we introduce Affordance20Q, a novel affordance reasoning benchmark formulated as a 20‑Questions game without exposing the object's identity. In each game, the model identifies a hidden object's affordance from a candidate set by asking yes/no questions about its physical properties. Affordance20Q comprises 1,009 games over 454 objects and 59 affordances, all manually filtered, refined, and annotated. We conduct comprehensive experiments with 15 state‑of‑the‑art LLMs and find a substantial gap (~20 points) compared to human performance. A KL‑based information‑gain (IG) analysis further shows that models fail to ask discriminating questions as the game progresses. To close the gap, we develop KB‑Anchored Rule Induction (KARI), a pipeline based on LLMs that generates affordance rules grounded in evidence from knowledge bases (KBs). KARI improves open‑source LLMs by up to 15.2 points, while the limited coverage of KBs hinders further gains. We release all our code and data at https://github.com/1171‑jpg/Affordance20Q.git
Authors:Xilin Dai, Yiding Liu, Hongjie Xia, Yifan Hu, Zewei Dong, Jiang-Ming Yang, Qiang Xu
Abstract:
The rapid evolution of Time Series Foundation Models (TSFMs) has advanced zero‑shot forecasting across diverse domains. Inspired by the current form of Large Language Models, future TSFMs may be offered as commercialized, closed‑source API services. However, many existing online adaptation methods still rely on white‑box access for parameter fine‑tuning or gradient backpropagation. This paradigm mismatch raises a question: In black‑box online adaptation for TSFMs, what should we learn? We answer this with an insight: the predictive errors of the base model are conditioned on both the input and output of the base model (i.e., the context of errors). To validate this insight, we propose ORCA (Online Residual Contextual Adaptation). We conduct extensive experiments across 5 state‑of‑the‑art TSFMs and 8 datasets to demonstrate the effectiveness of our approach. Furthermore, through ablation studies, we quantitatively analyze the impact of different adapter learning hypotheses on the final adaptation performance in black‑box online adaptation. Code available at https://github.com/Fifthky/ORCA.
Authors:Zixuan Gu, Xiaojun Ye, Yang Liu
Abstract:
Large language models (LLMs) are increasingly deployed in privacy‑sensitive domains, where users must balance the risk of data exposure through external APIs against the high computational cost of local deployment. Split learning has therefore emerged as a promising paradigm for LLM fine‑tuning and inference under limited local resources. However, it introduces new privacy risks. Prior work primarily studies leakage of private input prompts, typically via inversion attacks on intermediate representations, while the potential for sensitive information leakage through generative response outputs remains largely unexplored. In this work, we unveil novel vulnerabilities of Split‑LLM by presenting Patched Model Inversion with Dual‑Sided Initialization (PIDI), a two‑stage attack that simultaneously targets both private input prompts and output responses in Split‑LLM settings. It combines dual‑sided initialization with a patched inversion strategy to tackle long sequences, substantially outperforming prior inversion methods. To counter threats from both sides, we further propose the Adapter‑based DualGuard with Mutual Information Defense (ADMI), which integrates an adapter‑based local warmup strategy and mutual information regularization to provide a strong empirical privacy protection with minimal impact on task performance. Extensive experiments across diverse tasks and models demonstrate that ADMI effectively defends against PIDI and other state‑of‑the‑art inversion attacks. Our code is publicly available at https://github.com/FLAIR‑THU/VFLAIR‑LLM.
Authors:Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap
Abstract:
Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness‑driven post‑training pulls them toward a homogeneous, overly agreeable assistant register, creating a behavioral Sim2Real gap. We present OdysSim, the largest open systematic investigation of behavioral foundation models, i.e., models trained to simulate human behavior at scale. We propose SOUL, a taxonomy of five capability axes (CONV, SS, COG, ROLE, EVAL) that unifies 62 datasets and 23 benchmark tasks under one framework. Specifically, we curate the OdysSim corpus (21.4M interactions, 10B tokens, retrofitted with back‑generated social contexts), construct the SOUL‑Index benchmark, and develop an end‑to‑end training recipe combining midtraining, task‑specific RL, and expert distillation. The resulting open 8B OSim model ranks first or tied‑first on 8 of 23 tasks, outperforming any individual frontier model by this count, with the strongest gains on conversational and social tasks. Its outputs are also more human‑like in length, formatting, and word choice, and it transfers zero‑shot to out‑of‑distribution user simulation on τ‑bench, nearly matching real users on reaction alignment (93.2 vs. 93.5). We further show that LLM‑as‑judge RL induces reward‑hacking patterns, and that our detectors can mitigate them during post‑training. Together, our findings suggest that behavioral foundation models require rethinking the LLM training paradigm. We release all artifacts to support future research.
Authors:Kaiwen Chen, Shuhai Zhang, Zimo Liu, Linxiao Li, Ying Sun, Yuchen Li, Yifan Zhang, Bo Han, Mingkui Tan, Qiuwu Chen
Abstract:
Large‑scale neural network training increasingly relies on matrix‑aware optimizers that exploit the structure of weight parameters beyond element‑wise adaptation. However, existing matrix‑aware methods such as Muon have an underappreciated vulnerability: their core operation, Newton‑Schulz iteration, depends critically on input conditioning, yet the raw momentum matrices exhibit severe coordinate‑wise scale heterogeneity. In this paper, we first verify this scale heterogeneity through a chi‑square uniformity test, showing that intra‑matrix scale imbalance is prevalent across Transformer layers and that coordinate whitening effectively corrects it. Motivated by this finding, we propose Zeta, a dual whitening optimizer that applies coordinate whitening and spectral whitening in a strictly ordered pipeline. The ordering is not a tunable choice but follows from a mathematical dependency: coordinate whitening establishes the statistical isotropy that spectral whitening requires to function reliably. We further prove that this dual pipeline strictly reduces orthogonalization error relative to pure spectral methods by improving the condition number of the input. Empirically, Zeta matches or surpasses strong baselines across language modeling (0.6B to 8B parameters), mixture‑of‑experts architectures, and vision tasks, demonstrating that resolving scale imbalance before orthogonalization leads to faster convergence and better generalization. Code is available at https://github.com/AIGCodeOS/aigcode_zeta_optimizer.
Authors:Yufeng Xu, Taiming Lu, Kunjun Li, Jiachen Zhu, Mingjie Sun, Zhuang Liu
Abstract:
Pruning promises a shortcut to strong small language models. In this work, we examine this promise by pruning Llama‑3.1‑8B at pruning ratios of 0.5‑‑0.8 with six methods spanning depth, width, and sparse granularities, under two controlled token‑matched settings. (1) With the same training token budget, pruned initialization consistently outperforms random initialization. This shows that the parent model provides a strong starting point, although the advantage narrows as the training token budget grows and as the pruning ratio rises, nearly vanishing at the highest pruning ratio we study. (2) When training from scratch is instead given the full token budget consumed by the whole pipeline, pruning at finer granularities still retains an advantage, while coarser structured pruning can be matched or surpassed. This suggests that the parent model transfers knowledge that additional training tokens alone cannot fully recover, but only at fine granularity. Taken together, our results yield a clear recommendation: with a large pretrained model in hand and a limited training token budget, pruning is better than training from scratch; when the training budget is not limited, training from scratch can be competitive for coarser pruning, so a large pretrained parent is not always necessary.
Authors:Zheyuan Zhan, Hongchen Li, Can Wang, Yinfei Ma, Mingzhen Huang, Ruoshi Bai, Jiawei Chen, Siwei Lyu, Defang Chen
Abstract:
Inversion‑based image editing offers flexible and training‑free control but still struggles with inversion accuracy and the trade‑off between editing fidelity and background preservation. While recent methods improve inversion formulations or attention interactions, the role of textual conditioning in shaping diffusion dynamics and editing behavior remains underexplored. We show both empirically and theoretically that the precision of textual conditioning influences inversion stability by modulating the geometry of the diffusion velocity field, while also affecting the consistency of cross‑branch attention during editing. These effects directly impact background preservation and semantic fidelity. Building on this analysis, we propose SimEdit, a conditioning‑aware framework with two complementary components: (a) conditioning refinement, which constructs conditioning signals with improved semantic precision and structural alignment to facilitate stable inversion and consistent attention manipulation, and (b) token‑wise cross‑branch attention control, which separates edit‑relevant and structure‑preserving components and modulates them asymmetrically during attention manipulation. Extensive experiments on PIE‑Bench demonstrate that SimEdit consistently improves both inversion reconstruction quality and editing performance over previous attention‑manipulation approaches. Our code is available at https://github.com/zju‑pi/SimEdit.
Authors:Xiaoran Yan, Cheng Tang, Atsushi Shimada
Abstract:
Deployed knowledge‑tracing models are typically frozen after training, yet systematic per‑item logit bias arises, from limited per‑item expressivity in backbone architectures and from post‑deployment shifts in item properties, degrading prediction quality. Global post‑hoc calibrators such as Platt scaling, temperature scaling, and isotonic regression improve probability estimates but leave discriminative ability, as measured by AUC, unchanged. This AUC invariance is a structural consequence of monotone score‑only transforms; recovering the stranded discrimination requires conditioning on item identity. We propose SLC (State‑space Logit Correction), which converts binary observations to Gaussian pseudo‑observations via Laplace/IRLS, applies empirical‑Bayes shrinkage through a Kalman smoother, and fits an offset‑Platt link. The state‑space formulation also yields a detectability bound that characterizes the Bernoulli information floor, explaining why temporal tracking provides no benefit at current data densities. Across four datasets, five backbones, and three seeds, SLC improves AUC on all four datasets and NLL on three, with the advantage concentrating on sparse items. Cross‑domain controls suggest that the same phenomenon can arise beyond education when the deployed backbone leaves entity‑level bias.
Authors:Ziwei Wang, Xingyi He, Tianwang Jia, Hongbin Wang, Dongrui Wu
Abstract:
Auditory attention decoding (AAD) aims to infer the attended speaker from neural responses in multi‑speaker acoustic environments and is a key problem for neuro‑steered hearing systems. Although recent studies have achieved encouraging progress, existing AAD models still do not fully exploit frequency domain electroencephalography (EEG) information. In particular, most approaches introduce multi‑band information through handcrafted feature extraction or direct cross‑band feature concatenation, which mainly exploit frequency information at a shallow level and may overlook band‑specific patterns and cross‑band interactions. To address these limitations, this paper proposes FAConformer, a frequency‑aware CNN‑Transformer framework for AAD that explicitly integrates band‑specific encoding and adaptive cross‑band interaction. Specifically, FAConformer first decomposes EEG signals into multiple frequency bands and assigns each band to an independent CNN‑Transformer encoder for band‑specific modeling. The resulting band‑wise features are then adaptively fused by a carefully designed frequency‑aware attention (FAA) module that models cross‑band dependencies by treating band‑wise features as tokens. Further, band‑wise auxiliary supervision (BAS) is introduced to prevent weakly contributing branches from being under‑optimized during joint training. In this way, FAConformer performs frequency‑aware modeling that more effectively exploits frequency domain information. Extensive experiments on two public AAD datasets with three decision‑window lengths demonstrated that FAConformer consistently outperformed 12 competitive baselines, surpassing the current state‑of‑the‑art model by 4.9%. Further analyses of band importance, ablation, and parameter sensitivity verify the effectiveness, robustness, and interpretability of the proposed framework. Code is available at https://github.com/wzwvv/FAConformer.
Authors:Yanbin Hao, Pengyu Liu, Xing Wei, Xun Yang, Dan Guo, Meng Wang
Abstract:
Micro‑actions are short‑duration, low‑amplitude subtle body movements at the whole‑body level that can reveal latent intentions, involuntary reactions, and fine‑grained affective changes. Our previous MA‑52 benchmark has provided an important foundation for micro‑action recognition, but it remains limited in scale, scene diversity, task coverage, and evaluation protocols. To advance micro‑action analysis toward more realistic and comprehensive settings, we introduce MMA‑82, a large‑scale multi‑domain extension of MA‑52. MMA‑82 expands the label space from 52 to 82 fine‑grained micro‑action categories and covers four distinct domains, including laboratory interviews, street interviews, psychiatric patient interviews, and emotion‑rich television videos, resulting in 77,856 annotated instances from 454 subjects. Built upon MMA‑82, we establish two core tasks: Micro‑Action Recognition and Multi‑label Micro‑Action Detection. For recognition, we further define in‑domain and cross‑domain protocols, including few‑shot and zero‑shot settings, to evaluate model robustness, transferability, and generalization. Extensive experiments show that current methods still struggle with realistic micro‑action understanding, especially under domain shift, long‑tailed category distributions, and complex temporal localization. Beyond benchmarking, we investigate the relationship between micro‑actions and emotion, showing that micro‑actions are strongly associated with emotional states and provide complementary cues to facial micro‑expressions for improved emotion recognition. These results demonstrate that MMA‑82 serves as a comprehensive and challenging benchmark for realistic micro‑action analysis and a valuable resource for human‑centered AI. MMA‑82 is available at https://lpynow.github.io/MMA‑82‑AIM/.
Authors:Shiao Wang, Xiao Wang, Chao Wang, Yitao Li, Menghao Liu, Bo Jiang, Yaowei Wang, Yonghong Tian, Jin Tang
Abstract:
Conventional RGB cameras have been widely used in multi‑object tracking due to their ability to capture rich appearance and semantic information. However, their performance is often degraded under complex real‑world challenges, such as motion blur, low illumination, and overexposure. Bio‑inspired event cameras offer high temporal resolution and high dynamic range, providing complementary cues under extreme scenarios. Nevertheless, RGB‑event multi‑object tracking remains underexplored due to the lack of large‑scale and well‑annotated datasets. To address this issue, we propose FEMOT, a large‑scale RGB‑event multi‑object tracking dataset that covers diverse real‑world scenarios and 14 challenging attributes. With both RGB and event data as well as high‑quality annotations, FEMOT provides a reliable platform for systematically evaluating RGB‑event multi‑object tracking methods. Based on FEMOT, we retrain and evaluate over ten strong trackers, thereby establishing a comprehensive benchmark for future research. Furthermore, we propose FEMOTR, a multimodal tracking framework that decouples RGB and event features and fuses them in the frequency domain, thereby effectively exploiting their complementary characteristics for robust object localization and identity association. Extensive experiments on FEMOT and DSEC‑MOT datasets demonstrate the effectiveness of the proposed method. The source code and benchmark dataset have been released on https://github.com/Event‑AHU/FEMOT.
Authors:Yao-Ting Huang, Jake Honma, Omar Hernandez, Logan Li, Kaitlin Calimbahin, Bryce Hackel, Michael C. Yip
Abstract:
Humans rely on directional fingertip forces to probe and regulate contact during manipulation, yet most wearable haptic gloves render only vibration or single‑axis force, leaving force direction ambiguous. Without directional cues, users must infer contact force from vision alone, often leading to over‑pressing, inconsistent control, and reduced precision in robotic teleoperation. We present the N2D Haptic Glove, a multi‑finger wearable device that renders planar flexion‑extension fingertip forces using capstan‑drive transmissions for high‑transparency force feedback. Through benchtop validations and a user study involving haptic teleoperation of a robotic arm and hand, we demonstrate that compared to visual‑only and single‑axis haptic baselines, planar fingertip feedback significantly reduces contact force error during precise manipulation, improves trial‑to‑trial consistency, and enhances overall user experience in axial probing tasks. These findings establish the N2D Haptic Glove and directional finger‑based haptics devices as a promising modality for contact‑rich teleoperation, immersive virtual reality simulations, and robot learning from demonstrations. N2D Haptic Glove's hardware and software system will be fully open‑sourced at \hrefhttps://ucsdarclab.github.io/n2d‑glove/this https URL.
Authors:Guangzong Si, Dong Wang, Zhenhao Li, Yifan Yu, Panwang Pan, Wentao Zhu
Abstract:
Existing studies on gender bias in LLMs have largely focused on stereotypes, occupational associations, or explicit harmful outputs. In this work, we ask whether LLMs apply consistent response standards to the same negative behavior under matched male‑actor and female‑actor conditions. We introduce GAMA‑Bench, a gender‑mirrored benchmark of 1,298 scenarios covering intimate relationship and public social conflicts. It constructs gender‑neutral misconduct templates through controlled grids and cross‑model review, then compiles them into paired first‑person prompts with matched actor‑gender and role‑reference variations. We further design a structured response‑framing protocol to measure how models allocate punishment, empathy, escalation, instruction, and blame. Experiments on 10 representative LLMs reveal a consistent male‑disadvantaging asymmetry: male actors receive more punitive, escalatory, and blame‑centered framing, whereas female actors receive more therapeutic and empathy‑oriented framing for the same misconduct. Further analyses show that this pattern persists across model families, scenario tracks, model scale, and explicit thinking‑style reasoning. The official code is available at https://github.com/xufeiqiong/GAMA‑Bench.
Authors:Shaoqiu Zhang, Maoquan Wang, Yuling Shi, Yuhang Wang, Xiaodong Gu, Yongqiang Yao, Tori Gong, Sheng Chen, Rao Fu, Anisha Agarwal, Spandan Grag, Gabriel Ryan, Colin Merkel, Yufan Huang, Shengyu Fu
Abstract:
Large Language Model (LLM) coding agents have achieved strong results on software engineering tasks, yet repository exploration remains a major bottleneck: locating relevant code consumes substantial token budget and pollutes the agent's context with irrelevant snippets. In most agents, the same model explores the repository and solves the task, leaving exploratory reads and searches in the solver's history. We present FastContext, a dedicated exploration subagent that separates repository exploration from solving. Invoked on demand, FastContext issues parallel tool calls and returns concise file paths and line ranges as focused context. FastContext is powered by specialized exploration models spanning 4B‑‑30B parameters. We bootstrap them from strong reference‑model trajectories and refine them with task‑grounded rewards for broad first‑turn search, multi‑turn evidence gathering, and precise citation generation. Across SWE‑bench Multilingual, SWE‑bench Pro, and SWE‑QA, integrating FastContext into Mini‑SWE‑Agent improves end‑to‑end resolution rates up to 5.5% while reducing coding‑agent token consumption up to 60%, with marginal overhead. These results show that repository exploration can be separated from solving and handled effectively by specialized models. Code and data: https://github.com/microsoft/fastcontext
Authors:Dongjian Ma, Silin Chen, Yufei Yang, Yuling Shi, Yanfu Yan, Xiaodong Gu
Abstract:
Coding agents powered by large language models have demonstrated strong performance on software engineering tasks. Yet most agents consume repositories almost entirely as text, which differs from how human developers use visual structure such as folder hierarchies and dependency relationships to orient themselves in large codebases. With multimodal large language models (MLLMs), it is an open question whether agents can effectively benefit from visual representations of repositories. This paper presents the first systematic empirical study of visual repository representations for LLM‑based agents on repository‑level issue resolution. We evaluate four recent multimodal models. Our results show that a strictly vision‑only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries. In contrast, integrating visual graphs of repository structure as a supplementary modality alongside standard text interfaces helps agents understand structure more efficiently: input token consumption decreases by up to 26% while issue‑resolution accuracy is maintained or improved. Visualization is most useful during fault localization and when the agent autonomously controls exploration depth. These findings point to a practical hybrid text‑and‑vision design for next‑generation coding agents.
Authors:Shiyao Wang, Xijuan Zeng, Hui Wang, Shiwan Zhao, Feng Deng, Chen Zhang, Yong Qin
Abstract:
We present FoleyGenEx, a unified video‑to‑audio (VTA) framework integrating multi‑modal control, frame‑level temporal alignment, and fine‑grained semantics, enabling synchronized, versatile audio synthesis for diverse tasks. Existing VTA methods either have multi‑modal control but weak temporal alignment or strong alignment but lack reference audio conditioning and semantic precision. FoleyGenEx fills this gap via three core innovations: a conditional injection mechanism for audio‑controlled VTA and Foley extension, a multi‑modal dynamic masking strategy preserving training synchronization, and an adverb‑based data augmentation algorithm leveraging signal processing and large language models to enhance textual supervision with nuanced semantics. Experiments on AudioCaps, VGGSound, and Greatest Hits demonstrate its competitive controllable VTA performance against existing methods. Demo samples are available at https://foleygenex.github.io/FoleyGenEx.
Authors:Minghan Li, Jeremy Moebel, Mengyu Wang
Abstract:
One‑step image editing is important for making text‑guided editing fast, practical, and easy to deploy, but its underlying mechanism is still not fully understood. We revisit ChordEdit through reproduction, ablation, and simplification. Our analysis shows that a) the chord window δ largely acts as an effective timestep shift from t to t ‑ δ; b) chord transport acts on high‑noise images and mainly performs low‑frequency semantic editing; and c) proximal alignment acts on low‑noise images and complements it by adding high‑frequency target details. In this view, ChordEdit naturally decomposes editing into a coarse low‑frequency transport stage and a fine high‑frequency alignment stage. These findings suggest a path toward prompt‑conditioned dynamic timestep selection for adaptive image editing. All code and results can be found at \hrefhttps://github.com/Harvard‑AI‑and‑Robotics‑Lab/ChordEdit‑Reproductionlink.
Authors:Zixin Wang
Abstract:
The modern software supply chain, taking Node Package Manager (npm) dependency network for example, relies heavily on shared open‑source dependencies. While this promotes rapid development, it introduces systemic vulnerabilities as well. Concerning this potential risk, we analyze the npm dependency network by modeling 53,481 packages and 78,520 dependency edges, and classify the network as a scale‑free topology. Thus, we demonstrate its inherent vulnerability to targeted attacks on high‑degree hubs. To mitigate this, we propose and evaluate a dual‑pronged defense strategy consisting of Centrality‑Based Node‑Hardening and Dependency Weight Warning system. Moreover, by simulating the network under various attack scenarios, we prove that applying strict security protocols to just the top 1% of nodes, combined with pruning 30% of structurally trivial edges, prevents catastrophic network collapse and neutralizes cascading malware infections. The source code can be found at https://github.com/5tarWhee1/Centrality‑Based‑Protection‑Strategy‑for‑Supply‑Chain‑Security‑in‑npm‑Dependency‑Network.
Authors:Dinh-Khoi Vo, Nhut-Thanh Le-Hinh, Viet-Tham Huynh, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le
Abstract:
Zero‑shot text‑guided diffusion has significantly advanced image editing; however, its practical usability remains constrained by three persistent challenges: prompt brittleness that requires meticulous prompt engineering, spillover edits that unintentionally affect non‑target regions, and failures on small or cluttered objects caused by limited fine‑grained supervision in training data. We propose FocusDiff (Target‑Aware Refocusing for Tuning‑Free Diffusion Editing), a tuning‑free framework for precise and region‑specific image manipulation based on refocusing cross‑attention. Given a target region obtained through automated segmentation or manual selection, FocusDiff applies selective blurring to non‑edit areas to guide attention toward the masked region while accurately transferring the object's identity, structure, and appearance to the edited output. Integrated context‑preserving modules further ensure background fidelity and global coherence, enabling accurate edits from simple text prompts in a single pass. We also extend FocusDiff to 360‑degree indoor panorama editing and demonstrate its effectiveness within virtual reality environments. Extensive experiments on our localized editing benchmark LIMB, comprising 30 multi‑object images and 100 annotated examples including challenging small‑object cases, show that FocusDiff outperforms existing zero‑shot editors in text‑image alignment and background preservation, achieving superior precision, photorealism, and usability. The project page is available at https://vdkhoi20.github.io/FocusDiff.
Authors:Yuewen Mei, Tong Nie, Jie Sun, Haotian Shi, Wei Ma, Jian Sun
Abstract:
Closed‑loop adversarial training improves autonomous driving safety by exposing policies to rare safety‑critical scenarios. Standard pipelines first generate adversarial scenarios and then sample them for policy optimization. However, most existing frameworks remain attack‑oriented: collision‑driven generators often synthesize unsolvable extreme situations, which can degrade learning, while heuristic samplers ignore the evolving capability of the driving policy, causing sample inefficiency and delayed convergence. We propose AlignADV, a learnability‑guided closed‑loop adversarial training framework that converts adversarial scenarios into resolvable and capability‑aligned curricula. First, we reformulate adversarial scenario generation as a preference alignment problem and employ direct preference optimization to guide the generator toward critical yet resolvable scenarios. Second, we introduce behavioral fingerprints to capture the intrinsic characteristics of the evolving policy and construct a multi‑modal capability prediction model that estimates policy performance without expensive closed‑loop simulations. By combining resolvability‑aligned scenarios with capability predictions, AlignADV develops a dynamic curriculum sampling mechanism that prioritizes scenarios targeting the current policy's vulnerabilities. Experiments on the Waymo Open Motion Dataset demonstrate that AlignADV improves convergence efficiency and final performance, reducing training steps by up to 40.6 percent compared with baseline methods while lowering collision rate and improving route completion under both normal and adversarial traffic conditions. These results highlight a shift from attack‑oriented scenario generation to learnability‑guided policy improvement, offering a principled direction for safer and more efficient autonomous driving training. Project page: https://meiyuewen.github.io/AlignADV/.
Authors:Xilong Wang, Xiaoxing Chen, Patrick Li, Dawn Song, Neil Gong
Abstract:
Agentic browsers integrate autonomous AI agents into web browsers, enabling users to accomplish web tasks through natural‑language instructions. The same‑origin policy (SOP) is a fundamental browser security mechanism that prevents unauthorized automated cross‑origin data flows induced by scripts. However, whether SOP remains effective in agentic browsers is an open question that has not been systematically studied. In this work, we bridge this gap. We first observe that an agentic browser can itself serve as an automated channel for cross‑origin data flows, potentially leading to SOP violations. To investigate this phenomenon, we construct SOPBench, a benchmark for evaluating SOP violations in agentic browsers. Our evaluation shows that existing agentic browsers frequently violate SOP, both in benign settings and under attacks. To address this problem, we propose SOPGuard, an SOP enforcement mechanism tailored to agentic browsers. We implement SOPGuard in BrowserOS, an open‑source agentic browser. Extensive evaluations demonstrate that SOPGuard effectively enforces SOP while preserving utility and incurring only a small runtime overhead. Our code and data are available at https://github.com/wxl‑lxw/BrowserOS‑SOPGuard.
Authors:Duong-Duy-Khang Bui, Minh-Tan Pham, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le
Abstract:
Fashion sketching is a cornerstone of design workflows, allowing rapid visualization of creative concepts prior to physical prototyping. Yet, progress in sketch‑based fashion image synthesis has been hindered by the absence of large‑scale, high‑quality paired resources. To bridge this gap, we present GarmentSketch, a novel dataset comprising 26,249 fashion sketches across 21 garment categories, each paired with detailed textual descriptions. Captions were produced through a multi‑stage pipeline that integrates multiple multimodal large language models (MLLMs) with human‑in‑the‑loop refinement, ensuring both semantic accuracy and descriptive richness. We benchmark GarmentSketch on state‑of‑the‑art generative models, providing baseline performance for sketch‑guided text‑to‑image generation. Our experiments reveal both the promise and the current limitations of existing methods. By offering a comprehensive and richly annotated resource, GarmentSketch establishes a foundation for advancing sketch understanding, fine‑grained fashion image generation, and creative human‑AI collaboration in design. The dataset will be available at: https://khangbdd.github.io/garmentsketch.
Authors:Krispin Wandel, Jingchuan Wang, Hesheng Wang
Abstract:
Vision Transformers (ViTs) have become a dominant architecture for visual representation learning, providing exceptionally strong and broadly reusable backbone features. However, ViTs are commonly operated on relatively small patch‑token grids due to the quadratic cost of global self‑attention, which creates a persistent bottleneck for dense prediction tasks such as semantic segmentation and depth estimation. This has motivated the development of task‑agnostic feature upsamplers. While recent state‑of‑the‑art methods produce visually sharp dense representations, their reliance on shallow image encoders for guided upsampling can introduce feature leakage, fragmentation, and blur. We introduce ViT‑Up, an implicit feature upsampling framework that replaces external image guidance with layer‑wise query construction from intermediate ViT hidden states. This enables feature prediction at arbitrary continuous image coordinates while preserving alignment with the backbone feature space. Experiments demonstrate that ViT‑Up consistently outperforms state‑of‑the‑art image‑guided upsamplers across dense prediction and semantic correspondence. On DINOv3‑S+, ViT‑Up improves over prior methods by up to +2.07 mIoU on Cityscapes and +4.17 PCK@0.10 on SPair‑71k. With the larger DINOv3‑B backbone, these gains increase to +3.36 mIoU and +8.09 PCK@0.10, demonstrating that ViT‑Up scales favorably with backbone capacity.
Authors:Ibrahim Denis Fofanah
Abstract:
Sierra Leone's agriculture operates with almost no data‑driven decision support, and no published machine learning study has examined the country's crop yields. We ask whether rice yield can be forecast from data Sierra Leone currently has. Using 25 years of FAOSTAT production data (2000‑2024) for nine major crops, we train XGBoost, Gradient Boosting, and Random Forest under a strict anti‑leakage protocol with expanding‑window walk‑forward evaluation across seven held‑out years, benchmarked against naive persistence. No model trained on crop statistics alone outperforms persistence. Augmenting with free satellite climate data (CHIRPS rainfall, NASA POWER temperature) reverses this result: a climate‑only XGBoost reduces forecast error by one third (RMSE 284 vs 428 kg/ha), a gain that holds for a linear model and is robust to excluding the anomalous 2018 season. Early‑season (May‑June) rainfall is the dominant predictor, implying seasonal yield risk is observable months before harvest. No model anticipated the 2018 collapse, whose origins were institutional rather than climatic. We translate the findings into policy recommendations for Sierra Leone's Feed Salone Strategy, with a fully open‑source pipeline.
Authors:Hexuan Yu, Chaoyu Zhang, Heng Jin, Shanghao Shi, Ning Zhang, Y. Thomas Hou, Wenjing Lou
Abstract:
Modern LLM‑powered autonomous agents increasingly rely on rich user interface (UI) state observations to achieve reliable action grounding in complex digital environments. However, many deployments transmit the full UI state to remote inference servers even when most elements are irrelevant to the current task, which can leak sensitive but unnecessary context such as authentication codes, private notifications, and background application states. We propose MINIM, a trusted local broker that performs privacy‑aware minimization on the client side before any observation leaves the device. Grounded in Contextual Integrity (CI), MINIM learns a dual‑score representation for each UI element by predicting an inherent sensitivity score (s) and a task‑conditioned necessity score (n). These scores drive a ternary disclosure policy that keeps essential elements, abstracts sensitive attributes when needed, and removes task‑irrelevant content. We optimize a CI‑aware objective that penalizes necessity errors more strongly on high‑risk content, enabling aggressive pruning while preserving task‑critical information. Experiments on real‑world UI observations derived from WebArena show that MINIM substantially reduces task‑irrelevant sensitive leakage while preserving task‑critical semantic context and the interactive affordances required for reliable agent actions.
Authors:Ziqing Wang, Weihao Li, Shijie Chen, Yuan Luo, Kaize Ding
Abstract:
Automated International Classification of Diseases (ICD) coding is a core medical‑coding task for billing, epidemiology, and clinical decision support. Generative large language models (LLMs) are often reported as weak medical coders, but this finding mainly comes from inference‑time settings such as prompting, retrieval, reranking, or tool use, leaving the role of task‑specific post‑training underexplored. We present a controlled empirical study of post‑training for generative ICD coding, comparing discriminative baselines with LLM coders across prompting, supervised fine‑tuning, and reinforcement learning under a common protocol and metric set. To our knowledge, this is the first study to evaluate RL‑based post‑training for generative LLM coders in ICD coding. We further introduce PHI, a diagnostic curriculum that extends GRPO to refine missed‑code cases. Our results show that prompting‑only evaluation substantially underestimates the potential of LLMs for ICD coding. SFT provides the main capability jump, GRPO further improves code‑set prediction beyond SFT, and PHI provides targeted gains on macro‑level performance. These findings suggest that the main bottleneck is not the generative formulation alone, but how the model is adapted and optimized for full‑taxonomy recall. We release our code, data splits, and checkpoints at https://github.com/AlexandreWANG915/LLM4ICD.
Authors:Li Zhang, Yuzhen Shi, Yiran Hu, Jingwen Zhang, Wenbo Lv, Yubo Ma, Wei Wang, Rongyao Shi, Yuanyang Qiu, Xinran Xu, Yuemeng Qi, Linlin Miao, Jaromir Savelka, Yun Liu, Kevin Ashley, Bing Zhao, Hu Wei, Lin Qu
Abstract:
Lawyer‑client consultation is a critical starting point for legal services. Effective legal assistance hinges on eliciting sufficient and truthful information from clients in order to devise strategies that best protect their interests. This task requires Large Language Models (LLMs) not only to perform robust legal reasoning, but also to strategically elicit material facts through multi‑turn interactions and effectively guide clients with diverse personalities. Yet existing legal benchmarks overlook this interactive capability. To fill this gap, we introduce DLawBench, a diagnostic benchmark for real‑world legal consultation. Drawing on realistic client behavior, we characterize lawyer‑client interactions into four types: Cooperative, Dependent, Withdrawn, and Adversarial. Using dialogues grounded in real cases, DLawBench evaluates whether LLMs can effectively conduct legal consultation under realistic conditions. DLawBench comprises 461 cases from Chinese and U.S. law, 5,532 paired fact entries, 3,411 inquiry rubrics, and 3,348 issue‑resolution rubrics, and evaluates 26 representative LLMs. Systematic experiments show substantial headroom: the best‑performing model, GPT‑5.5, achieves only 0.562 on consultation‑grounded legal reasoning. More importantly, DLawBench exposes both sycophancy in legal consultation and a paradox: models perform worse when clients need guidance most.
Authors:Yijun Liang, Hengguang Zhou, Ming Li, Lichen Li, Cho-Jui Hsieh, Tianyi Zhou
Abstract:
Vision‑language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non‑trivial, visual‑centric and grounded questions remains underexplored. Existing visual questioners' performance is bottlenecked by the availability of high‑quality training data or the cost of curating them. We show that a VLM can continuously improve itself as a visual questioner without any external supervision. We propose a self‑evolving framework that uses a VLM itself as both a proposer and a filter to produce harder, more informative, and visual‑centric questions, while maintaining their exploration diversity to avoid training collapse. These questions are then used to train the VLM in both questioner and answerer modes. To evaluate the questioner, we introduce an agentic protocol that assesses questions along perception, reasoning, and diversity dimensions. Experiments across various backbone VLMs show that our method substantially enhances the quality and substantially expands the difficulty boundary of autonomous question generation. Under the same budget, our self‑supervision is more effective than training on the static source data. Moreover, the self‑evolving questioner remains a competitive or even better answerer.
Authors:David Montero, César Aybar, Miguel D. Mahecha, Luis Gómez-Chova
Abstract:
Earth observation (EO) is essential to understanding the Earth system, enabling the transformation of planetary properties into measurable variables that can be analysed, compared, and modelled. In recent decades, EO capabilities have grown rapidly, accompanied by an even faster expansion in the number and variety of available EO instruments. Today, EO includes instruments deployed on satellites, airborne platforms, and terrestrial or in‑situ systems. However, despite this proliferation of instruments, users often lack a single, reliable source describing their existence and key characteristics. Although existing data catalogues have substantially improved dataset discovery, they primarily describe data products rather than providing persistent, curated metadata about the instruments that produced them. Here we present Awesome Earth Observation Instruments, an open, standardized, and community‑oriented registry providing machine‑readable metadata for EO instruments. The catalogue is hosted on GitHub and allows contributors to submit instrument metadata following a common schema. The schema combines a lightweight core with modular extensions covering spectral, geometric, and data access‑related metadata, enabling both standardization and flexibility across diverse EO systems. All submissions undergo automated schema validation and human review. Because the schema is open, versioned, and extensible, the catalogue can continuously evolve as new instruments and metadata requirements emerge. This facilitates the discovery, interpretation, and analysis of EO data in light of instrument characteristics. To support programmatic access and interoperability, we further envisage an API for integration within common EO analysis environments. The catalogue is openly available at https://github.com/awesome‑spectral‑indices/awesome‑earth‑observation‑instruments.
Authors:Isai Daniel Chacón, Zhongqi Miao, Bruno Demuro, Caleb Robinson, Rahul Dodhia, Lasha Otarashvili, Jason Holmberg, Kirk Larsen, Howard Frederick, Nathan J. Pamperin, Pablo Arbeláez, Juan M. Lavista Ferres
Abstract:
Automated aerial wildlife surveys increasingly rely on deep learning, yet standard object detectors require bounding‑box annotations, reported to be up to seven times slower and three times more expensive to produce than point‑level labels. To address this bottleneck, we introduce the Overhead Wildlife Locator (OWL), a weakly supervised density‑estimation framework with three variants: OWL‑C, a fully convolutional model for high‑throughput screening; OWL‑T, a Swin‑augmented hybrid for heterogeneous, cluttered scenes; and OWL‑D, built on a frozen DINOv3 ViT‑H+/16 encoder with a DPT‑style fusion decoder. We benchmark all three against POLO, YOLOv11n, and YOLOv11l across five public aerial datasets, from sparse fixed‑wing savanna surveys to dense UAV paddock imagery, and against the published HerdNet baseline on its native Delplanque split. OWL‑D sets a new state of the art on Delplanque (0.934 AP vs. HerdNet's 0.840) and records the highest AP on four of the five datasets. Performance is regime‑dependent: on the extreme‑density SheepCounter UAV dataset the hybrid OWL‑T leads (0.978 AP) and the convolutional variants attain the lowest counting error, whereas the foundation‑based OWL‑D degrades, indicating which variant suits which survey type. We further validate operational readiness on the Alaska Department of Fish and Game's 2022 Central Arctic Caribou census: under cross‑herd and cross‑temporal transfer, OWL‑C fine‑tuned on the 2017 Porcupine Caribou Herd split attains F1 = 0.965 on a held‑out patch test set, with a signed count error of +3.1% aggregated across the released test patches. We release the OWL code, model weights, and the annotated Porcupine Caribou Herd 2017 (PCH) and Central Arctic Herd 2022 (CAH) patches, the first open patch‑level datasets for large‑scale caribou aerial surveys, at https://github.com/microsoft/MegaDetector‑Overhead.
Authors:Stella Katharina Wermuth, Qazi Arbab Ahmed, Klaus Neumann, Thorsten Jungeblut
Abstract:
Autonomous staff‑free public transport requires reliable in‑vehicle passenger monitoring. However, perception inside moving vehicles is challenged by confined spaces, variable illumination, motion‑induced background variation, occlusion, and limited viewpoints. To mitigate these spatial constraints, ceiling‑mounted fisheye cameras provide full‑scene coverage from a single viewpoint. Yet existing public overhead fisheye datasets are recorded in static environments and do not capture the domain shift introduced by vehicle motion. To fill this gap, we introduce PMOF, Passenger Monitoring using Overhead Fisheye cameras, the first public dataset of top‑view fisheye imagery captured inside a moving vehicle, comprising over 19k manually annotated frames. PMOF provides rotated bounding boxes, tracking identifiers, and action labels, supporting object detection, tracking, and action recognition. We benchmark PMOF using YOLO26m‑obb models fine‑tuned under multiple dataset configurations that combine PMOF with existing overhead fisheye datasets. Cross‑domain fine‑tuning with custom rotation‑aware augmentation achieves 94.8% AP50 on PMOF and 96.5% AP50 on an unseen overhead fisheye dataset from a different domain. Our results highlight the domain gap between static and moving environments and show that incorporating PMOF improves detection performance and advances generalization beyond passenger monitoring to broader fisheye‑based person detection tasks. The dataset and code are available at https://swermuth.github.io/pmof/.
Authors:Jafar Bakhshaliyev, Niels Landwehr
Abstract:
Spiking Neural Networks (SNNs) have emerged as an energy‑efficient alternative to conventional neural networks, demonstrating strong performance in computer vision and robotics. More recently, SNNs have been applied to time series forecasting (TSF), with methods exploring spiking temporal backbones, spike‑compatible positional encodings, Fourier‑domain processing, and redesigned neuron dynamics. However, existing SNN forecasting approaches process variables independently, lacking explicit mechanisms for modeling inter‑variable dependencies. This is a critical limitation in multivariate settings, where cross‑variable correlations carry substantial predictive information. We propose Spiking Fourier Graph Operators (SpikF‑GO), which addresses this gap by combining a hypervariate graph formulation in which every scalar observation becomes a graph node with spike‑driven spectral processing. SpikF‑GO introduces a Hard Concrete frequency gate for learnable sparse frequency selection and a Complex LIF gate that applies independent spiking neurons to real and imaginary Fourier components, preserving binary, event‑driven computation throughout the spectral domain. We further present a variant incorporating Central Pattern Generator‑based positional encodings for stronger long‑range temporal modeling. Evaluated on eight benchmarks under a unified experimental protocol, SpikF‑GO achieves the best average rank among all SNN methods and outperforms its ANN counterpart, FourierGNN, at reduced energy cost. SpikF‑GO maintains competitive accuracy even at substantially smaller embedding dimensions, thereby achieving significant energy reductions. To our knowledge, this is among the first works to bring graph‑based multivariate modeling into the spiking domain for TSF and the first to provide a unified comparison across SNN forecasting architectures under a common experimental protocol.
Authors:Nadav Benedek, Tomer Koren, Ohad Fried
Abstract:
AdamW is a default optimizer for modern deep learning, but its first and second moment states add roughly two parameter‑sized buffers to training memory. We propose Gefen, a memory‑efficient optimizer that automatically shares second‑moment estimates across parameter blocks and quantizes the first moment using a learned codebook, thereby reducing AdamW's memory footprint by ~8x while maintaining the same performance, corresponding to a reduction of 6.5 GiB per billion parameters. The method is motivated by a theoretical result showing that large mixed Hessian entries constrain the ratio of squared gradients toward one, suggesting that Hessian‑aligned parameters are natural candidates for sharing second‑moment statistics. Since computing Hessians is impractical at scale, Gefen infers block structure from the initial squared gradients, requiring no architecture‑specific metadata or hyperparameters beyond AdamW defaults. Gefen learns an exact histogram‑based dynamic‑programming quantization codebook and reuses the same blocks for first‑moment scaling. Across diverse experiments, Gefen achieves the lowest peak optimizer memory among the compared AdamW‑like methods while maintaining AdamW‑level performance. In FSDP and DDP training, the reduced memory footprint enables larger microbatches and improves throughput significantly over AdamW, providing a practical drop‑in replacement with lower memory usage that can increase throughput and enable training larger models or using larger batch sizes. We provide the complete Python implementation, including fused CUDA kernels at https://github.com/ndvbd/Gefen
Authors:Lev V. Utkin, Andrei V. Konstantinov, Stanislav K. Kogan, Natalya M. Verbova, Maksim I. Goriunov
Abstract:
Estimating the probability that a treatment outperforms a control for an individual patient, called the Individual Probability of Treatment Benefit (IPTB), offers a clinically intuitive alternative to population‑average metrics. However, existing methods for IPTB estimation are largely confined to binary treatment settings, despite the prevalence of dose‑varying interventions in clinical practice. We propose a general framework for IPTB estimation with ordinal outcomes under discrete dose assignments, called Dose‑AIPTB (Dose Attention‑based IPTB). Our approach recasts the problem as binary classification over the unobserved sign of the individual treatment effect, constructing pseudo‑labels from covariate‑similar pairwise comparisons and aggregating them via attention mechanisms or Nadaraya‑Watson kernel regression. This formulation naturally accommodates multiple discrete dose levels, extending beyond the binary treatment paradigm. Through numerical experiments on real‑world and synthetic data under covariate shift, varying sample sizes, and heterogeneous outcomes, we demonstrate that attention‑based aggregation consistently outperforms kernel alternatives. The framework provides a foundation for personalized dose selection grounded in individual‑level benefit probabilities. Codes implementing the model are publicly available at https://github.com/NTAILab/AIPTBDose.
Authors:Tejas Agrawal, Vu Le, Sumit Gulwani, Gust Verbruggen
Abstract:
Predictive code completion greatly accelerates how quickly developers work. In spreadsheets, despite being much more common, such auto‑completion features are virtually non‑existent. To address this gap, we introduce a benchmark for systems that observe a sequence of user actions in a spreadsheet and predict future actions. Two challenges are (1) the absence of edit histories in public spreadsheet corpora and (2) the complex space of spreadsheet actions (spatial, temporal, composite). To address (1), we manually curate 52 sequences of 12K actions that recreate spreadsheets from public corpora, seeded by parametrized heuristics and LLM refinement. To address (2), we propose an online evaluation that expects a prediction after each user action, accepts or rejects that prediction, updates the future actions upon acceptance, and repeats this until the target spreadsheet is obtained. We use multiple baseline predictors (including zero‑shot LLMs, fine‑tuned SLMs, and classical models) and analyze different properties that our benchmark teaches us, including but not limited to: properties of saved actions and false positives, efficiency, effect of user profiles, effect of triggers, and effect of context.
Authors:Jorge Miguel Silva
Abstract:
Finding the shortest program that generates a sequence is uncomputable, and for six decades that fact has been mistaken for a wall around finding any generating program. It is not a wall but a price, and this paper measures it. For every algorithm that learns about a candidate program only through its score, a class spanning Levin search, evolutionary methods, simulated annealing, and the cross‑entropy method, we define the coupling width of a search problem and prove an unconditional worst‑case lower bound, exponential in that width with base one less than the domain size. From it follows a conservation law: structural knowledge injected into a search trades one for one against the search it removes, and their sum can never fall below the length of the program sought. Levin's 1973 upper bound and the lower bound proved here are the two ends of one conserved quantity, closing on each other as the instruction set grows. The only escape is to read a candidate's structure rather than its score, and its price, which we prove for generic targets, is incompleteness. A deterministic engine built on this theory recovers a generating program, certified by compressing its data and predicting an unseen continuation, for 2,383 of 3,914 sequences across four independent populations, including 244 of the 256 elementary cellular automata, with measured discovery cost rising along program length more than an order of magnitude inside the score‑oracle worst case.
Authors:Haozhe Jiang, Haiwen Feng, Pieter Abbeel, Jiantao Jiao, Angjoo Kanazawa, Nika Haghtalab
Abstract:
RL post‑training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy‑gradient methods are often unstable and cannot achieve reliable policy improvement. We identify the cause as the double‑drift phenomenon: optimizing a variational surrogate can let the ELBO separate from the true log‑likelihood, which then makes the resulting proxy policy gradient misaligned with the true policy gradient of expected return. We propose DiPOD, a diffusion policy optimization framework that maintains tight‑bound behavior throughout training by interleaving self‑distillation with policy‑improving gradient updates. This leads to a simple and practical algorithm: augmenting each diffusion policy‑gradient update with an on‑policy ELBO regularizer. Across diffusion language model post‑training and continuous‑control diffusion policies, DiPOD substantially stabilizes training and reaches higher rewards than previous methods.
Authors:Sharath Girish, Tsai-Shien Chen, Zhikang Dong, Mukesh Singhal, Hao Chen, Sergey Tulyakov, Aliaksandr Siarohin
Abstract:
Cinematic video depicts multiple subjects acting or interacting at specific moments, captured with deliberate camera movement, and stitched together by shot transitions. Together, these elements demand a level of fine‑grained control beyond current text‑to‑video models. Existing work addresses each axis in isolation: multi‑subject personalization, temporal control, multi‑shot synthesis, or camera control; no prior framework jointly integrates all four. We present CineOrchestra, a unified video diffusion model that controls subjects, events, cameras, and shot transitions simultaneously. Our key insight is that these heterogeneous cinematic elements share a fundamental structure: each is an entity acting over a specific temporal interval, which can therefore all be expressed through one shared structure of entity‑centric conditioning primitives, augmented with reference images for visual entities. This formulation reduces the architectural challenge to a single positional encoding problem, which we solve with two parameter‑free coordinated rotary embeddings: (a) an interval‑sampled temporal RoPE that yields consistent attention behavior across events of dramatically varying duration, and (b) a 2D entity‑temporal cross‑attention RoPE that disambiguates per‑entity conditions and routes each to its corresponding spatiotemporal region. On two new benchmarks, CineOrchestra outperforms six per‑axis specialists on dense caption following and shot‑transition timing, with consistent gains in a pairwise user study and component ablations. Project page: https://snap‑research.github.io/CineOrchestra
Authors:Elijah Cadenhead, Cristian McGee, Xin Li, El Houcine Bergou, Aritra Dutta
Abstract:
Low‑rank adaptation (LoRA) and its variants provide a memory‑ and compute‑efficient alternative to full fine‑tuning of pre‑trained models. However, questions remain about the comparative generalizability of these approaches and how the structural restrictions on low‑rank updates preserve effective adaptation performance. We present a historical framing, covering the past (full fine‑tuning and original LoRA), the present (different variants of LoRA), and propose simpler, cheaper, parameter‑efficient extensions by inducing sparsity within existing LoRA variants: Cheap LoRA (cLA), training a single low‑rank factor with the other fixed (deterministically or, in its randomized variant, stochastically), and the chained circulant variant, c^3LA. We frame cLA as a structured instance of asymmetric LoRA, serving as a controlled column‑subspace restriction of full fine‑tuning. We derive information‑theoretic generalization error bounds for these variants, marking one of the first endeavors in this area. Empirically, we evaluate 11 fine‑tuning methods across 10 pre‑trained models and 14 datasets, analyzing the fine‑tuned models' performance and generalization using tools such as loss landscapes and spectral analysis. Despite the sensitivity of fine‑tuned models to the pre‑trained model, datasets, and other factors, our study suggests that restricting LoRA‑based PEFT methods' adaptation to a sparse, structured column space remains competitive across tasks with their parameter‑matched baselines while reducing up to 10% training time and peak GPU memory up to 15%, even with a naïve, non‑optimized, sparse implementation. Our theoretical and empirical generalization measures provide a more consistent and principled approach to their cost‑effective adaptation than commonly used analytical tools. Overview and code are available at: https://elicaden.github.io/Beyond_LoRA/.
Authors:Jisoo Jang Wen-Syan Li
Abstract:
Business intelligence (BI) increasingly combines dashboard interaction with LLM‑based assistance, but these two modes often fall out of sync during multi‑step analysis. As users switch between direct dashboard manipulation and natural‑language queries, it becomes difficult to preserve a consistent analytical state across filters, hierarchies, metrics, and chart context. We present TwinBI, an agentic digital‑twin framework that couples an LLM‑based agent system with an executable BI dashboard state. TwinBI unifies conversational interaction, dashboard manipulation, semantic grounding, and provenance tracking through a shared analytical state reconstructed from a unified interaction log. It also exposes artifacts such as schema views, SQL, logs, and an /insights command for state‑grounded analytical summaries. We evaluate TwinBI in two complementary ways. In a controlled A/B benchmark with the same backbone agent, TwinBI improves exact‑match accuracy from 43.3% to 63.3%, partial‑credit accuracy from 48.3% to 70.8%, and substantially reduces timeout rate from 40.0% to 10.0% relative to Dashboard alone. In a usability study, participants benefited from the integrated dashboard‑and‑chat workflow, with high task accuracy, moderate workload, and favorable ratings for state‑aware interaction mechanisms. These results suggest that TwinBI improves both agent‑level analytical reliability and user‑facing analytical support by turning visible dashboard state into richer actionable context. Our dataset and source code are available at: https://github.com/simonjisu/TwinBI
Authors:Carlos Cueto Zumaya, Iacopo Catalano, Jorge Peña-Queralta, Wallace Moreira Bessa
Abstract:
Hierarchical 3D scene graphs (3DSGs) for indoor robots organize geometric and semantic information across spatial scales, with a room layer that connects object‑level perception to room‑scale reasoning. Existing systems construct this layer from different spatial substrates (\eg place clusters, wall planes, or segmentation outputs), and as a result, room nodes are not evaluated on a common geometric criterion. We present an occupancy‑grounded 3DSG pipeline in which room nodes are anchored to tracked free‑space regions derived from occupancy decomposition, giving each room an explicit polygonal footprint. We evaluate the pipeline on 12 Matterport3D scenes by matching predicted room polygons to annotated room instances and compare against Hydra, a representative state‑of‑the‑art place‑connectivity baseline. The results show that occupancy‑grounded anchoring recovers substantially more room instances than place‑connectivity construction, at the cost of lower precision, and that wall‑accurate room boundaries remain an open problem for both methods. Code is available at https://github.com/crcz25/OccuSG.
Authors:Maohua Nie, Jiang Zhu, Jingqun Zhang, Zhichen Zeng, Jiayi Wang, Sibo Zhang, Jialin Wang, C. -J. Richard Shi
Abstract:
We present HierSVA, an integrated suite that combines a pipeline, dataset, and benchmark for LLM‑driven hierarchical hardware formal verification. HierSVA‑SP pairs an RTL preprocessing toolchain with an LLM‑in‑the‑loop formal verification flow to produce reference SystemVerilog Assertions (SVA) on hierarchical RTL. Applying it to BaseJump STL yields HierSVA‑DS, a dataset of 342 modules, with hierarchy metadata and depths 0‑‑9, accompanied by a deep subset of 28 module‑bug pairs with natural‑language specifications and bug variants. HierSVA‑B decomposes assertion quality into six metric axes: syntax correctness, assertion proof success rate, vacuity, specification faithfulness, mutation coverage, and formal core coverage. Applying HierSVA‑B to twelve recent LLMs reveals three findings. First, the module‑level compile rate is 67.1%; among generated assertions in evaluable runs, 82.1% prove non‑vacuously, but the corresponding assertion sets detect only 70.2% of eligible injected faults and cover 36.2% of the formal core. Second, on 211 evaluable model‑‑module entries in the deep subset, assertion sets flag buggy RTL with 0.87 recall, but 40% of predicted‑buggy outcomes are false positives on correct RTL, limiting precision to 0.60. Third, agentic mode improves S1‑style provability and strength metrics, but gains plateau and oscillate. Codes and artifacts are available at \hrefhttps://github.com/HierSVAAnon/HierSVACodeAndArtifactshttps://github.com/HierSVAAnon/HierSVACodeAndArtifacts. Dataset is available at \hrefhttps://huggingface.co/datasets/AnonymousHierSVA/HierSVAhttps://huggingface.co/datasets/AnonymousHierSVA/HierSVA.
Authors:Phuc Nguyen H
Abstract:
Human pose estimation (HPE) utilizing wireless WiFi signals has emerged as a promising technology owing to its device‑free nature, privacy preservation, and robustness against occlusion and poor lighting. However, existing methods often overlook the physical complex phase information of WiFi signals and fail to generalize across diverse environments due to severe domain shifts. In this paper, we present C‑MambaPose, a physics‑informed complex‑valued Mamba‑GraFormer hybrid framework for robust cross‑environment WiFi‑based 3D HPE. Our framework first sanitizes raw WiFi Channel State Information (CSI) phase errors and constructs a phase‑preserving complex‑valued representation. We then employ a Spatiotemporal Complex Mamba encoder with a dynamic selective receptive field to capture fine‑grained phase dynamics. A cross‑attention joint‑query mapper maps the unstructured sequence tokens to human joints, which are decoded by a Graph Convolutional Network (GCN) to predict anatomically coherent 3D coordinates. Extensive evaluations on the MM‑Fi dataset show that C‑MambaPose achieves competitive or superior performance to state‑of‑the‑art baselines across all settings, setting a new state‑of‑the‑art specifically on the challenging cross‑environment split, requiring only 3.78 M parameters‑an 83.1% reduction compared to GraphPose‑Fi~\citechen2026graph and an 85.7% reduction compared to MetaFi++~\citezhou2023metafi++, while maintaining a comparable size to DT‑Pose~\citechen2025towards (which is only 18% smaller) but achieving significantly superior performance without requiring any pretraining. Our code is publicly available at https://github.com/phucngvinuni/cmampose.git.
Authors:Dian Zheng, Harry Lee, Manyuan Zhang, Kaituo Feng, Zoey Guo, Ray Zhang, Hongsheng Li
Abstract:
Recent image generators have demonstrated impressive photorealism and instruction‑following capabilities in single‑image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text‑image sequence), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open‑source Unified Multimodal Models (UMMs) exhibit limited performance in this regard. In this paper, we introduce InterleaveThinker, the first multi‑agent pipeline designed to endow any existing image generator with interleaved generation capabilities. Specifically, we employ a planner agent to organize the image‑text input sequence, instructing the image generator on the required execution at each step. Subsequently, we introduce a critic agent to evaluate the generator's outputs, identify samples that deviate from the planned instructions, and refine the instructions for regeneration. To implement this pipeline, we construct the Interleave‑Planner‑SFT‑80k and Interleave‑Critic‑SFT‑112k to perform a format cold‑start. Then we develop Interleave‑Critic‑RL‑13k to reinforce the step‑wise instruction correction capability within a generation trajectory using GRPO. Since a single interleaved generation trajectory may involve over 25 generator calls, optimizing the entire trajectory is computationally impractical. Therefore, we propose accuracy reward and step‑wise reward, allowing single‑step RL to effectively guide the entire generation trajectory. The results show that InterleaveThinker improves performance across various image generators. On interleaved generation benchmarks, it achieves performance comparable to Nano Banana and GPT‑5. Surprisingly, it also significantly enhances the base model on reasoning‑based benchmarks; for example, on 4‑step FLUX.2‑klein, we observe substantial gains on WISE and RISE.
Authors:Zhao-Heng Yin, Guanya Shi, Pieter Abbeel, C. Karen Liu
Abstract:
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact‑rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim‑to‑real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse‑to‑fine pipeline that transforms procedurally‑generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero‑shot sim‑to‑real transfer for both grasping and in‑hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.
Authors:Junke Wang, Qihang Zhang, Shuai Yang, Yiming Luo, Yujun Shen, Zuxuan Wu, Yu-Gang Jiang, Yinghao Xu
Abstract:
This work presents RepWAM, a representation‑centric world action model (WAM) built on representation visual‑action tokenizers. Existing WAMs typically inherit reconstruction‑oriented video tokenizers from pretrained video generation models. Although these tokenizers preserve visual fidelity, pixel reconstruction alone provides limited guidance for learning instruction‑following dynamics that connect future prediction with robot control. To address this, we explore a semantic visual‑action latent space for representation‑centric world action modeling. Specifically, we train a representation visual‑action tokenizer that maps visual inputs into aligned visual and latent action tokens. We then pretrain our WAM to jointly model future visual states and the latent actions that connect them under language instructions, followed by adaptation to real robot trajectories for closed‑loop manipulation. Experiments on real‑world manipulation tasks and simulation benchmarks show that RepWAM delivers strong performance across diverse manipulation settings, while ablations highlight the value of semantic visual‑action tokenization over reconstruction‑oriented alternatives. These results establish representation visual‑action tokenization as a promising foundation for world action models and a step toward generalist robot policies. Code and weights will be available at https://github.com/wdrink/RepWAM.
Authors:Arnav Kumar Jain, Yilin Wu, Jesse Farebrother, Gokul Swamy, Andrea Bajcsy
Abstract:
The potential impacts of world models (WMs, i.e., learned simulators) on robotics are far‑reaching ‑‑ policy evaluation, policy improvement, and test‑time planning ‑‑ all with limited real‑world interaction. To unlock these downstream capabilities, a WM needs to jointly satisfy three desiderata: (i) fidelity (i.e., producing simulated trajectories that correlate with reality), (ii) consistency (i.e., producing simulated trajectories that are coherent over long horizons), and (iii) efficiency (i.e., producing simulated trajectories quickly). We propose \textttWEAVER (World Estimation Across Views for Embodied Reasoning): a WM architecture that simultaneously achieves all three desiderata, providing state‑of‑the‑art results on robotic manipulation tasks. \textttWEAVER is a multi‑view WM trained to predict future latents and reward values via a flow‑matching loss. We distill the key design decisions across model architecture, memory, and prediction objectives required to unlock the kinds of long‑horizon dynamic manipulation tasks that have confounded prior world modeling approaches. We apply \textttWEAVER in robotic hardware, demonstrating its effectiveness at policy evaluation (ρ=0.870 correlation with real‑world success rate), policy improvement (real‑world success rate improvement of 38% on top of the π_0.5 robot foundation model), and test‑time planning (real‑world success rate improvement of 14% with a 5‑10× speedup over prior WMs). \textttWEAVER also demonstrates better performance than prior WMs when evaluated on out‑of‑distribution scenarios. Code, models, and videos at: https://arnavkj1995.github.io/WEAVER/ .
Authors:Guo Yu, Wenlin Liu, Yulan Hu, Hao-Xuan Ma, Jun-Peng Jiang, Han-Jia Ye
Abstract:
On‑policy distillation (\textscOPD) has recently become a prominent post‑training recipe as it combines two desirable ingredients: on‑policy student trajectories and dense teacher supervision, yet how this hybrid changes a model's parameters remains unclear. Across several language and vision‑language model pairs and use cases, our analysis yields two main findings. On sparsity, \textscOPD‑style updates are small and coordinate‑sparse. They are distributed across layers and are usually FFN‑heavy. This sparse structure is operationally useful: training only the discovered subnetwork recovers nearly the same performance as full \textscOPD. However, the sparsity‑inducing SGD optimizer underperforms AdamW in our optimizer ablation, likely because dense teacher supervision preserves heterogeneous coordinate‑wise gradient scales where AdamW's adaptive scaling remains useful. On geometry, the updates are numerically full‑rank but spectrally concentrated; they lie mostly away from the principal singular subspaces of the source weights and fall disproportionately on coordinates where the source weights are close to zero. These findings suggest that dense teacher supervision does not turn \textscOPD into ordinary dense parameter rewriting; instead, \textscOPD retains important geometric signatures of on‑policy post‑training.
Authors:Hao Zhang, Mohamed El Banani, Jen-Hao Cheng, Paul Zhang, Yi Hua, Ben Mildenhall, Christoph Lassner, Narendra Ahuja, Gengshan Yang
Abstract:
Image‑to‑3D methods often trade off faithfulness and completeness: depth estimators are anchored to input pixels but stop at the visible surface, while image‑to‑3D models generate complete shapes that are often misaligned with the input. We introduce World Tracing, a generative pixel‑aligned geometry representation that predicts 3D points aligned with observed pixels while completing geometry beyond the visible surface. For each input pixel, World Tracing predicts an ordered stack of camera‑space 3D points, where the first layer represents the visible surface and subsequent layers represent front‑to‑back intersections with occluded surfaces. We instantiate this representation with a world‑tracing diffusion transformer, WT‑DiT, which treats multiple geometry layers as separate denoising tokens coupled through factorized and global attention. WT‑DiT is trained with pixel‑space flow matching and a mixed noise schedule that balances visible‑surface reconstruction with occluded‑geometry generation. World Tracing achieves strong performance on visible‑surface reconstruction and complete geometry generation across object, scene, and dynamic benchmarks, outperforming both depth predictors and image‑to‑3D generators. It also preserves 2D‑to‑3D correspondence, enabling text‑driven 3D scene editing, geometry‑conditioned novel‑view video synthesis, and training‑free integration with textured‑mesh generators.
Authors:Antoine Guédon, Shu Nakamura, Nicolas Dufour, Jiahui Lei, Ko Nishino, Angjoo Kanazawa
Abstract:
Geometry is invariant to viewpoint, which makes any collection of images a redundant encoding of a single 3D state. Existing feed‑forward reconstruction models fail to exploit this: per‑view methods emit overlapping, unaligned pointmaps that grow linearly with input count, while global‑latent methods commit to a fixed, low‑resolution output. We introduce Surflo, which compresses a variable number of unposed RGB views into K latent tokens‑one global state‑and decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching. This frees the output from any fixed grid or token budget: the same latent yields from a few thousand to a million points in a single forward pass. To suppress the local inconsistencies inherent to independent per‑point decoding, an inference‑time guidance term correlates nearby points by injecting a photometric gradient during ODE integration. Surflo matches or surpasses feed‑forward baselines on surface metrics, runs an order of magnitude faster than optimization‑based methods that require hundreds of views, and is the only feed‑forward approach to combine a global latent with arbitrary‑resolution decoding.
Authors:Vinícius Orrú, Bruno H. Foggiatto, Gabriel E. Lima, David Menotti, Rayson Laroca
Abstract:
Vehicle color recognition is an important cue for vehicle identification in surveillance systems, especially when license plates are illegible due to low resolution, occlusion, motion blur, or poor illumination. However, real‑world vehicle color distributions are highly imbalanced, making overall accuracy insufficient to assess performance on rare but operationally relevant colors. This paper presents a comprehensive study of vehicle color recognition under severe class imbalance using UFPR‑VeSV, a challenging real‑world surveillance dataset. We investigate synthetic minority‑class augmentation through two off‑the‑shelf generative strategies: text‑conditioned image generation with RunDiffusion/JuggernautXL and image‑conditioned color editing with Gemini 2.0 Flash. The curated synthetic data are combined with modern visual representations, loss reweighting, learning‑rate scheduling, color‑safe augmentation, foreground‑aware preprocessing, and ensemble fusion. The bestperforming approach achieves 94.6% micro accuracy and 79.7% macro accuracy, improving macro accuracy by 8.2 percentage points over recent literature. A manual error analysis further shows that many remaining failures are visually ambiguous even for human annotators, highlighting the practical limits of color‑based vehicle identification in unconstrained surveillance imagery. The generated images and source code are publicly available at https://github.com/viniciusorru/vcr‑synthetic
Authors:Achraf Hsain, Sultan Almuhammadi
Abstract:
Shielded reinforcement learning is typically presented as a runtime safety mechanism that compiles temporal‑logic specifications into automata restricting an agent's actions. We argue this is the wrong product. The same automata‑theoretic machinery ‑‑ specification compilation, product game construction, attractor computation, and winning‑region extraction ‑‑ is better read as a design‑time analytical instrument whose outputs are structural insights about a system rather than runtime constraints on a deployed agent. We instantiate this through a constrained two‑player safety game for network defense. The two specifications are enforced asymmetrically: the defender specification defines the unsafe region of the game, whereas the attacker specification restricts the adversary's legal actions during attractor computation. Solving the game yields a defensibility verdict ‑‑ a formal certificate that a topology‑specification pair is or is not defensible ‑‑ with the associated winning region and shield. Beyond the binary verdict, we derive topology‑level metrics from the attractor structure and combine them with post‑convergence behavior from shield‑constrained adversarial multi‑agent reinforcement learning. Together these form a defensibility fingerprint capturing both a network's formal safety properties and its operational behavior under adaptive play. A what‑if analysis shows that formal defensibility and operational effectiveness capture distinct aspects of security: small architectural changes can produce large shifts in operational outcomes while leaving formal safety margins nearly unchanged. Shield synthesis is thus most valuable not as a deployment mechanism for safe agents, but as a framework for answering architectural questions about whether, where, and how a system can be defended. The defensibility verdict is the output, not the safe policy.
Authors:Minghao Luo, Liang Chen
Abstract:
Search‑augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: generative recommenders may consume polluted web content, such as fake reviews and promotional pages crafted to mislead recommendations. We ask: to what extent do search‑augmented LLMs become unwitting promoters of fake products when consuming polluted retrieval results? To answer this, we introduce FORGE (Fake Online Recommendations in Generative Environments), a benchmark for measuring fake‑product promotion under controlled web‑content pollution. Given an upstream search result, FORGE locally rewrites real products in retrieved web pages into fake ones to simulate web‑content pollution, and measures how often the LLM recommends the fake product. FORGE covers 225 real‑world products across 15 categories and 5 consumer scenarios. Across 12 commercial and open‑weights LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top‑3 replacement raises this to 73.8%. Vulnerability varies substantially across categories, increasing when models lack stable prior knowledge of the relevant products. Reasoning does not mitigate this vulnerability; instead, it often generates spurious social proof to justify false recommendations. We evaluate three defenses: skepticism prompting and consensus filtering (over model priors or cross‑document evidence). Skepticism can exacerbate vulnerability, much like reasoning, while filtering risks suppressing legitimate products. We release FORGE at https://github.com/leoluolol/forge‑benchmark.
Authors:King Yeung Tsang, Zihao Zhao, Vishal Venkataramani, Haizhou Shi, Zixuan Ke, Semih Yavuz, Shafiq Joty, Hao Wang
Abstract:
Multi‑Agent Systems (MAS) built on Large Language Models (LLMs) require effective orchestration to coordinate specialized agents, yet training such orchestrators is hindered by limited supervision and high computational cost. We propose Orchestration Reward Modeling (OrchRM), a self‑supervised framework for evaluating orchestration quality without human annotations. OrchRM leverages intermediate artifacts from multi‑agent executions to construct win‑lose pairs for Bradley‑Terry reward model training. Unlike existing MAS test‑time scaling and orchestrator training frameworks that rely on costly sub‑agent rollouts, OrchRM operates directly at the orchestration level, enabling efficient and high‑performing reward‑guided orchestrator training and MAS test‑time scaling. OrchRM improves training efficiency by up to 10x in token usage while improving MAS test‑time scaling performance by up to 8% in accuracy. These gains consistently transfer across multiple domains, including mathematical reasoning, web‑based question answering, and multi‑hop reasoning, demonstrating orchestration‑level reward modeling as a scalable direction for robust multi‑agent orchestration. Code will be available at https://github.com/Wang‑ML‑Lab/OrchRM.
Authors:Dachun Kai, Jiayao Lu, Yueyi Zhang, Xiaoyan Sun
Abstract:
Event‑based vision has drawn increasing attention owing to its distinctive properties, including ultra‑high temporal resolution and extreme dynamic range. Recent works have introduced it to video super‑resolution (VSR) to enhance flow estimation and temporal alignment. In contrast, this paper shifts the focus of event signals from motion refinement to texture enhancement in VSR. We propose EvTexture++, the first event‑driven framework dedicated to texture enhancement in VSR. It leverages high‑frequency spatiotemporal details from events to improve texture recovery. EvTexture++ incorporates a customized texture enhancement branch, along with an iterative texture enhancement module that progressively exploits high‑temporal‑resolution event information for texture restoration. This enables gradual refinement of texture regions across iterations, yielding more accurate and detailed high‑resolution outputs. Besides intra‑frame texture recovery, large motions could degrade inter‑frame temporal consistency, particularly in texture regions, leading to texture flickering. To mitigate this, we further exploit the continuous‑time motion cues of events to enhance temporal consistency, introducing a temporal texture alignment module that estimates event‑guided texture‑aware flow for precise inter‑frame texture alignment. Moreover, EvTexture++ is designed as a plug‑and‑play tool to flexibly boost the performance of existing VSR models. Experiments on five datasets demonstrate that EvTexture++ achieves state‑of‑the‑art performance. When integrated into recent VSR models, it yields significant improvements, with gains of up to 1.55 dB in PSNR on the texture‑rich Vid4 dataset. Code: https://github.com/DachunKai/EvTexture.
Authors:Baochang Ren, Xinjie Liu, Xi Chen, Yanshuo Liu, Chenxi Li, Daqi Gao, Zeqin Su, Jintao Xing, Zirui Xue, Rui Li, Xiangyu Zhao, Shuofei Qiao, Minting Pan, Wangmeng Zuo, Lei Bai, Dongzhan Zhou, Ningyu Zhang, Huajun Chen
Abstract:
Scientific laboratories increasingly rely on AI systems to reason about experiments, but the physical act of doing science remains largely outside their reach. AI can help read literature, generate hypotheses, and plan protocols, yet the execution of those protocols at the bench still requires a human operator. Vision‑Language‑Action (VLA) models provide one possible interface between written protocols and robot execution, but existing policies are trained mostly on household and tabletop demonstrations and rarely encounter the instruments, transparent liquids, or fixed protocol workflows found in scientific laboratories. Closing this gap requires both laboratory‑specific supervision and a unified learning framework that can accommodate the diverse robot embodiments used to execute experimental protocols. We therefore identify data and embodiment as central bottlenecks alongside model design. To address the data side, we build RoboGenesis, a simulation‑based workflow and data engine that composes configured laboratory workflows from atomic skills, validates and filters rollouts, and exports structured demonstrations across supported robot profiles. On the policy side, we present LabVLA, trained with a two‑stage recipe: FAST action token pretraining first makes the Qwen3‑VL‑4B‑Instruct backbone action aware before any continuous control is learned, and flow matching posttraining then attaches a DiT action expert under knowledge insulation. On the LabUtopia benchmark, LabVLA achieves the highest average success rate among all evaluated baselines under both in‑distribution and out‑of‑distribution settings.
Authors:Yifan Hu, Hongzhou Chen, Peiyuan Liu, Yiding Liu, Zewei Dong, Jiang-Ming Yang
Abstract:
Real‑world time series are often highly incomplete and irregular due to sensor dormancy, transmission delays, and event‑driven sampling, making reliable forecasting fundamentally challenging. Existing methods have evolved from impute‑then‑forecast pipelines to continuous‑time models such as Neural ODEs and continuous‑time graph networks. While these approaches improve the modeling of historical irregularity, they still rely on an implicit oracle assumption at inference time: the timestamps of future valid observations are presumed to be known in advance. This assumption limits practical relevance, since in many real systems the more fundamental question is not only what the future value will be, but also whether a valid observation will occur at all. In this paper, we propose Timeflies, a unified framework that reformulates forecasting as a joint problem of future observability inference and value estimation. To explicitly model the interaction between observation dynamics and state evolution, Timeflies adopts an observation stream and a value stream, coupled through three dedicated modules for reliability‑aware embedding, observation‑guided dependency modeling, and joint prediction. We further construct Shadow, a benchmark that combines natural missingness from public datasets with real‑world industrial data, and introduce the Observation‑Value Joint Entropy (OVJE) metric to comprehensively evaluate this coupled predictability. Extensive experiments show that Timeflies consistently outperforms existing methods, highlighting the importance of explicitly modeling future observability in time series forecasting with missing values. Code and dataset are available in https://github.com/ant‑intl/Timeflies.
Authors:Mingkun Lei, Tong Zhao, Liangyu Yuan, Chi Zhang
Abstract:
Step‑level caching accelerates diffusion models by exploiting temporal redundancy across denoising steps. Existing methods make per‑step cache decisions using threshold‑based heuristics, without directly optimizing for final output quality. As a result, their inference latency varies across inputs and is difficult to control at deployment. In this work, we propose BudCache, which inverts this formulation: rather than letting per‑step error thresholds dictate the runtime cost, we fix the compute budget in advance and search for the cache policy that best preserves the final output. To tackle the combinatorial complexity of step selection, we combine Simulated Annealing with deterministic Hill Climbing. This offline search identifies high‑quality cache policies within minutes and introduces no online search or thresholding overhead during inference. When the compute budget is very tight, we further introduce cache‑aware schedule alignment, which adapts the time discretization to the selected cache policy to reduce cache‑induced trajectory mismatch. Experiments on FLUX.1‑dev and Wan2.1 show that BudCache achieves better generation quality than heuristic caching baselines under the same inference budgets. Code is available at https://github.com/Westlake‑AGI‑Lab/BudCache
Authors:Daichi Azuma, Taiki Miyanishi, Koya Sakamoto, Shuhei Kurita, Yaonan Zhu, Petr Khrapchenkov, Motoaki Kawanabe, Yusuke Iwasawa, Yutaka Matsuo
Abstract:
Goal‑conditioned visual navigation requires a robot to act under partial observability by anticipating how its motion will change the future egocentric view and whether that change brings it closer to the goal. Navigation world models provide such visual foresight, but they remain prediction modules that require an external planner to convert predicted futures into closed‑loop control. We propose Navigation World Action Model (NavWAM), a diffusion‑transformer policy that turns navigation world‑model prediction into executable action by representing future observations, goal‑progress values, and action chunks in a shared latent sequence. By learning future prediction jointly with the action and value targets that determine closed‑loop behavior, NavWAM makes visual foresight directly usable for robot control. We build NavWAM through simulation pretraining and real‑robot adaptation, and evaluate it on image‑goal navigation against planning‑based world models and a representative direct navigation policy. Across offline benchmarks and closed‑loop real‑robot deployment, NavWAM improves over planning‑based world‑model baselines in our evaluations while using the default policy mode without CEM‑style action search. Project page: https://dachii‑azm.github.io/navwam/
Authors:William Smits
Abstract:
Anomaly detection in multivariate time series is challenged by four structurally distinct anomaly types ‑‑ point (isolated spikes), distributional (level shifts), temporal (rhythm changes), and collective (inter‑sensor correlation breakdowns) ‑‑ each requiring different feature representations. Most unsupervised methods target only one or two types and provide limited interpretability. We present CRAFTIIF (Cross‑Resolution Analytic Four‑Type Interpretable Isolation Forest), a fully unsupervised framework targeting all four types without dataset‑specific tuning. CRAFTIIF generates K=500 random analytic wavelet feature draws across four families (Morlet, DOG, Haar, Coiflet), each targeting a specific anomaly type, feeding five structured Isolation Forests ‑‑ one per type plus a meta‑IF for compound anomalies. An adaptive Otsu/MAD threshold calibrates detection automatically across anomaly rates from 0.1% to 69.2%. Because each IF is trained exclusively on type‑specific features, branch firing provides direct anomaly‑type attribution by construction, without post‑hoc explanation. Evaluated on all 19 datasets of the mTSBench benchmark (Zhou et al., TMLR 2026), CRAFTIIF achieves mean F1=0.228 (all 19 datasets) and F1=0.322 (13 detectable datasets), ranking first among all 25 evaluated methods on VUS‑PR (0.463 vs. previous best 0.329, +40.7%). A diagnostic framework ‑‑ oracle F1, detectability limits, and branch separation ratios ‑‑ identifies 6 of 19 datasets as fundamentally undetectable by any unsupervised method. Ablation over 11 conditions confirms adaptive thresholding (+38% F1), four‑branch structure (+20%), and meta‑IF (+23%) are each essential. Code: https://github.com/smitswil/craftiif
Authors:Tianyi Ma, Yijun Ma, Zehong Wang, Weixiang Sun, Ziming Li, Connor R. Schmidt, Chuxu Zhang, Matthew J. Webber, Yanfang Ye
Abstract:
Supramolecular chemistry, which includes the study of non‑covalent host‑guest assemblies, has advanced various applications. However, designing host‑guest systems remains time‑consuming, requiring days of dry‑lab verification per candidate pair. Although LLMs have emerged as a fast alternative with strong performance on molecular binding tasks, no benchmark currently systematically evaluates LLMs for host‑guest reasoning across fundamental supramolecular chemistry tasks, e.g., binding affinity prediction. To this end, we collaborate with domain experts to release the first Supramolecular Benchmark, called SupraBench, to evaluate LLMs in chemistry reasoning. Specifically, we design four fundamental tasks, i.e., binding affinity prediction, top‑binder selection, solvent identification, and host‑guest description, plus an auxiliary vision‑based task for molecular identification. We also release SupraPMC, a curated 16M‑token corpus of Supramolecular chemistry articles distilled from Europe PMC, to support the adaptation to the supramolecular domain. We benchmark a broad range of open and proprietary LLMs and find that LLMs leave substantial headroom across all tasks. Domain adaptation pretraining over SupraPMC transfers cleanly to in‑distribution regression but trades off against strict letter‑format output. Moreover, the difficulty profile differs sharply across task families, revealing distinct failure modes that indicate specific gaps in current supramolecular chemistry reasoning. Our source codes and benchmark datasets are available at https://github.com/Tianyi‑Billy‑Ma/SupraBench.
Authors:Pengfei Liu, Gen Li, Junqiao Fan, Boyu Ma, Jindou Jia, Yang Xiao, Jianfei Yang
Abstract:
Human communication is inherently multimodal, where language is often accompanied by non‑verbal cues such as gestures to convey intentions. However, current Vision‑Language‑Action (VLA) models treat robotic manipulation as a pure text‑driven task, overlooking the important role of gestures in Human‑Robot Interaction (HRI). This often leads to inaccurate intent grounding and unreliable manipulation when language instructions are ambiguous or underspecified. To address this challenge, we propose GIVE (Gesture Intent via Visual‑Semantic Enhancement), an effective approach that enhances pre‑trained VLA models with human gesture understanding without architectural modifications. Specifically, GIVE incorporates gesture information through two complementary pathways: a visual pathway that overlays hand skeletons and fingertip rays onto robot observations for explicit object grounding, and a semantic pathway that generates high‑level descriptions of human gestures and task instructions for robust intent grounding. By jointly leveraging visual and semantic guidance, GIVE enables VLA policies to better associate gestures with manipulation behaviors and adapt to dynamic interaction intents. In real‑world HRI experiments, GIVE substantially outperforms the baseline, improving target object recognition accuracy by 40% and overall task success rate by 80%, while demonstrating strong robustness and generalization to unseen spatial layouts and diverse participants.
Authors:Jiwen Liu, Shujuan Li, Zhixue Fang, Xiaohan Li, Yan Zhou, Zijie Meng, Zhimin Zhang, Yawen Luo, Guoxin Zhang, Yu-Shen Liu, Pengfei Wan
Abstract:
Cloning camera motion from reference videos is an important task in video generation, as videos provide intuitive and precise control. Existing methods either directly use parametric representations that fail to handle multi‑shot generation or synthesize cross‑paired data, which suffer from data scarcity, resulting in poor performance in complicated camera motion cloning. To address these issues, we introduce a general camera motion representation that encodes cameras as grid motion videos. This camera grid represents the camera parameters visually and supports the integration of diverse trajectories for multi‑shot video generation. Building upon this, we propose OmniDirector, a unified framework trained on a million‑scale camera grid‑video pairs that coordinates characters, actions, and cameras to provide director‑level control for multimodal diffusion transformers. Furthermore, we design a novel hierarchical prompt expansion agent that harmoniously integrates different control signals by systematically describing camera motion and visual content through understanding signal relationships. Extensive experiments demonstrate the superior performance and outstanding controllability of our framework. Project page: https://ymlinfeng.github.io/OmniDirector.github.io/
Authors:Hoang-Nguyen Cao, Le-Hoang Bui, Dinh-Khoi Vo, Minh-Triet Tran, Trung-Nghia Le
Abstract:
Cultural garments pose a unique challenge for visual retrieval systems, as their identity often depends on subtle structural and symbolic details that are poorly captured by standard AI models. We introduce VietFashion, a new benchmark for sketch‑text composed image retrieval centered on the Ao Dai, a traditional Vietnamese garment. VietFashion enables designers and researchers to retrieve culturally meaningful outfits using a combination of hand‑drawn sketches, which convey garment structure, and textual descriptions, which encode cultural semantics. The dataset is initialized with 650 sketches and expanded using generative models to produce over 21,000 photorealistic images with aligned captions. Textual prompts that describe detailed outfit attributes, which are extracted from fashion magazines to ensure authenticity and diversity. To better reflect the inherent ambiguity of design intent, VietFashion adopts a multi‑target retrieval setting, where a single query may correspond to multiple valid results. We establish standardized evaluation protocols and benchmark state‑of‑the‑art composed image retrieval methods. Experimental results reveal significant performance gaps in modeling fine‑grained cultural semantics and multi‑modal composition, positioning VietFashion as a challenging benchmark for fine‑grained fashion retrieval. The dataset is publicly available at: https://hng0303.github.io/VietFashion.
Authors:Jianming Ma, Qiyue Yang, Yang Zhang, Liyun Yan, Zhanxiang Cao, Yazhou Zhang, Yue Gao
Abstract:
While flow‑based generative models have demonstrated strong performance across a wide range of domains, deploying them in safety‑critical physical systems remains challenging due to strict constraint requirements. Existing approaches typically enforce safety through post‑hoc corrections, which incur substantial computational overhead and may distort the learned distribution. We propose PolyFlow, a polytope‑constrained flow matching framework that embeds constraints directly into the model and flow dynamics. PolyFlow introduces a discrete‑time flow formulation and a projection‑free architecture, which eliminate the discretization error and guarantee strict satisfaction of arbitrary polyhedral constraints, without the need for expensive iterative solvers. Experimental results show that PolyFlow achieves zero constraint violation while maintaining high distributional fidelity across a range of planning and control tasks. Compared to state‑of‑the‑art constrained generation baselines, PolyFlow significantly reduces inference latency and demonstrates a favorable trade‑off between safety, efficiency, and generative quality. Code is available on https://github.com/MJianM/PolyFlow.
Authors:Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu, Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, Pengyu Zhao
Abstract:
Ultra‑long‑context capability is becoming indispensable for frontier LLMs: agentic workflows, repository‑scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale. We introduce MiniMax Sparse Attention (MSA), a blockwise sparse attention built upon Grouped Query Attention (GQA). A lightweight Index Branch scores key‑value blocks and independently selects a Top‑k subset for each GQA group, enabling group‑specific sparse retrieval while maintaining efficient block‑level execution; the Main Branch then performs exact block‑sparse attention over only the selected blocks. Designed around a principle of simplicity and scalability, MSA is deliberately streamlined, making it straightforward to deploy efficiently across a broad range of GPUs. To translate sparsity into practical speedups, we co‑design MSA with a GPU execution path that uses exp‑free Top‑k selection and KV‑outer sparse attention to improve tensor‑core utilization under block‑granular access. On a 109B‑parameter model with native multimodal training, MSA performs on par with GQA while reducing per‑token attention compute by 28.4x at 1M context. Paired with our co‑designed kernel, MSA achieves 14.2x prefill and 7.6x decoding wall‑clock speedups on H800. Our inference kernel is available at: https://github.com/MiniMax‑AI/MSA. A production‑grade natively multimodal model powered by MSA has been publicly released at: https://huggingface.co/MiniMaxAI/MiniMax‑M3.
Authors:Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
Abstract:
Web agents driven by large language models (LLMs) are increasingly deployed in real‑world environments, where they operate over untrusted web content and execute actions with direct consequences. This makes them vulnerable to prompt‑injection attacks, in which seemingly benign content embeds adversarial instructions that manipulate agent behaviour. Existing security benchmarks adopt an attack‑centric perspective, focusing on the technical feasibility of injections while overlooking the nuanced distribution of resulting harms. In practice, however, prompt‑injection risk is victim‑dependent: a single exploit can produce asymmetric consequences for different stakeholders, and the same attack pattern may exhibit substantially different effectiveness depending on whom it targets. To capture these properties, we introduce \sysname, a stakeholder‑centric benchmark to systematically categorize and attribute harm in real‑world web agent systems. It distinguishes between affected entities (e.g., user, seller, platform), decomposes the attacks into concrete objectives, and evaluates each case with complementary outcome‑ and process‑level metrics. Our results reveal substantial and heterogeneous vulnerabilities: not a single attack objective is reliably resisted by current agents, and failures distribute across qualitatively distinct modes ranging from \emphstealthy parasitism (attack succeeds without disrupting the user's delegated task) to \emphmisaligned disruption (task disrupted without attack success) and \emphcompounded failure (both adversarial objective and task integrity simultaneously violated). These patterns are missed by conventional evaluation, highlighting the need for stakeholder‑aware assessment of LLM‑based agents in real‑world deployments. Benchmark is available at https://github.com/StakeBench/SBC.
Authors:Xinnan Zhu, Ruijie Xu, Jiayu Ying, Daoguo Dong, Jiachen Xu, Yuan Xie, Xin Tan
Abstract:
Existing 3D scene editing methods typically rely on per‑scene optimization over explicit 3D representations or cascaded edit‑and‑reconstruct pipelines, resulting in high test‑time cost, limited 3D awareness, and structural inconsistencies. To couple appearance synthesis and geometry prediction during editing, we build on a unified RGB‑geometry reconstruction‑generation latent space and adapt it to feed‑forward 3D scene editing. The resulting framework, JointEdit3D, performs asymmetric latent inpainting by observing only a single edited RGB reference latent and generating the remaining RGB views and edited geometry latent under source‑scene anchoring. JointEdit3D introduces a dedicated SceneAnchor Branch to inject source‑scene structure without forcing direct copying, and adopts edit/background‑aware losses to balance edited‑region fidelity with unedited‑content preservation. To address the lack of paired resources for standardized 3D scene editing evaluation, we introduce SceneEdit3D‑15K, a dataset with 15K paired editing samples and renderer‑provided 3D annotations, together with SceneEdit3D‑Bench, a curated 100‑sample benchmark. Experiments show that JointEdit3D improves edited‑region quality and 3D structural completeness over prior baselines while maintaining competitive background preservation.
Authors:Nikhil Garg
Abstract:
This paper presents EconCSLib, a Lean 4 library and workflow for formalizing research papers in Economics and Computation with language‑model assistance. The central design principle is a human‑AI‑Lean workflow: an LLM writes Lean code, Lean checks formal statements and proofs, and humans (assisted by an LLM) verify the translation boundary from paper claims to formal statements. EconCSLib is organized around research papers, preserving their formal statements and following their proof structure to the extent possible; reusable mathematical statements are elevated into shared EconCS infrastructure. The workflow is designed to be author‑facing: researchers can formalize their own papers, inspect the Lean code's translations of paper‑facing statements, and contribute reusable components back to the library; this is supported by post‑formalization validation reports, paper result dependency graphs, and a review dashboard. The current public repository contains 11 formalized papers and 3 partially formalized papers, along with initial libraries for probability, auctions, matching markets, and graph tools. The library and workflow are available at https://github.com/nikhgarg/EconCSLib, with corresponding project webpage at https://gargnikhil.com/EconCSLib/. To our knowledge, we are also among the first applied math researchers to systematically pursue Lean formalization of one's own publications in the process of building such a community library. We welcome users and contributors to the project.
Authors:Wei Li, Zhen Huang, Xinmei Tian
Abstract:
Contrastively trained vision‑language models like CLIP, have made remarkable progress in learning joint image‑text representations, but still face challenges in compositional understanding. They often exhibit a "bag‑of‑words" behavior‑‑struggling to capture the object relations, attribute‑object bindings, and word order dependencies. This limitation arises not only from the reliance on global, single‑vector representations for optimization, but also from the insufficient exploitation and modeling of the rich compositional information inherently present in paired image text data. In this work, we propose MACCO (MAsked Compositional Concept MOdeling), a framework that masks compositional concepts in one modality and reconstructs them conditioned on the full contextual information from the other, enabling the model to capture and align cross‑modal compositional structures more effectively. To facilitate this process, we introduce two auxiliary objectives that jointly align and regularize masked features both inter‑modally and intra‑modally. Extensive experiments on five compositional benchmarks, along with in‑depth analyses, demonstrate that our approach not only significantly enhances compositionality in VLMs but also improves their ability to capture syntactic structure and linguistic information. Additionally, the improved compositionality also benefits text‑to‑image generation and multimodal large language model. Code is available at https://github.com/hiker‑lw/MACCO.
Authors:Aitor Sánchez-Ferrera, Elisabeth Wetzer, Kristoffer Wickstrøm, Michael Kampffmeyer, Robert Jenssen
Abstract:
Recent advances in time series anomaly detection (TSAD) have highlighted the effectiveness of self‑supervised classification‑based approaches. These methods apply transformations to normal training samples, training a classifier to recognize transformation‑specific patterns that help identify anomalies through increased classification errors. Despite their strong performance, a significant challenge is their lack of explainability, as they provide limited insight into the characteristics of flagged anomalies. To address this limitation, we propose ProtoX‑AD, a prototype‑based self‑explainable framework for self‑supervised TSAD. ProtoX‑AD learns transformation‑aware latent representations alongside interpretable prototypes, enabling both accurate anomaly detection and the identification of distinct anomalous profiles through prototype‑based explanations. Additionally, it allows for systematic analysis of how transformation design impacts detection performance and explainability. Experimental results on synthetic and real‑world datasets demonstrate that ProtoX‑AD achieves detection performance comparable to its black‑box counterparts while offering more consistent and semantically meaningful explanations than existing explainable baselines. Our code is publicly available at https://github.com/Aitorzan3/ProtoX‑AD.
Authors:Kirato Yoshihara
Abstract:
Weight‑space geometry plays a central role in neural network optimization, yet manifold constraints are often applied uniformly across all weight matrices. In this work, we ask whether different transformer modules prefer different manifold geometries. We study Manifold Muon for GPT‑2 pretraining and compare layer‑wise assignments of Stiefel and DGram constraints across attention and MLP blocks. Our results show a clear asymmetry: constraining attention layers with Stiefel geometry while assigning DGram geometry to MLP layers gives the best performance among the tested configurations, whereas the inverted assignment and all‑DGram configuration become unstable under the shared hyperparameter setting. We trace this failure to singular value growth in DGram‑constrained attention weights, which can amplify attention logits and induce softmax saturation. These findings suggest that symmetry‑aware and geometry‑aware optimization for transformers should be module‑specific rather than uniform.
Authors:Anugrah Aidin Yotolembah, Novanto Yudistira, Gembong Edhi Setyawan
Abstract:
This paper presents Custom ZeroCLIP, a retrieval‑augmented vision‑language framework for zero‑shot captioning of Indonesian traditional garments. The dataset contains 3,800 expert‑annotated images from all 38 Indonesian provinces. Using a province‑level inductive zero‑shot protocol, the model is trained on 24 seen provinces, validated on 6 seen provinces, and evaluated on 8 unseen provinces. The framework combines a frozen CLIP ViT‑B/32 image encoder, a CLIP text encoder, a BERT text encoder, and an LSTM caption decoder. During inference, unseen‑province labels and captions are unavailable, and retrieval uses only captions from training provinces. No unseen‑province image, label, or caption is used during training, validation, or retrieval‑bank construction. Custom ZeroCLIP achieves a CLIPScore of 0.8536, BLEU‑4 of 0.3342, and METEOR of 0.4859, outperforming existing baselines. Ablation results show that retrieval improves cultural vocabulary recovery with a 19.3% METEOR gain, while human evaluation confirms stronger cultural accuracy and fluency. The results demonstrate the effectiveness of retrieval‑augmented domain adaptation for culturally grounded caption generation in low‑resource heritage settings. The dataset is publicly available at https://github.com/AnugrahAidinYotolembah/Traditional‑Indonesian‑Clothing‑Captioning‑Dataset.
Authors:Minlin Zeng, Zhipeng Zhou, Yang Qiu, Martin J. McKeown, Zhiqi Shen
Abstract:
Gait‑based Parkinson's disease assessment increasingly relies on heterogeneous sensors, but clinical systems rarely collect all modalities simultaneously. New sensors may arrive through device upgrades, protocol changes, or multi‑center deployment, while historical patient data are often unavailable because of privacy and storage constraints. This modality‑incremental setting faces three challenges: unreliable cross‑modal distillation, modality‑specific statistical shifts, and reduced plasticity after preservation. We propose MOSAIC, a compact continual learning framework. First, we identify the Toxic Teacher phenomenon and introduce Modality‑Specific Warm‑Up to stabilize newly learned modality representations before distillation. Second, we propose a statistics‑decoupled MSBN architecture that isolates sensor statistics while maintaining a shared semantic backbone. Third, we design a curriculum‑guided repulsive objective for Plasticity Recovery, preserving legacy knowledge while recovering modality‑specific capacity. Experiments on three multimodal Parkinson's gait datasets show that MOSAIC improves final performance and mitigates forgetting. Project code is available at: https://github.com/minlinzeng/MOSAIC_Modality‑Specific‑Adaptation‑for‑Incremental‑Continual‑Learning‑in‑PD‑Gait‑Assessment.git
Authors:Sihwa Lee, Janghwan Lee, Donghoon Yoo, Jae Gon Kim, Hanyul Ryu, Soojung Ryu, Jungwook Choi
Abstract:
Large reasoning models (LRMs) improve complex problem‑solving by generating long intermediate reasoning traces, but this substantially increases inference costs. NVFP4 inference offers a promising approach to reduce both computational and memory costs through hardware‑supported low‑precision execution. However, directly applying NVFP4 to LRMs introduces two practical limitations: reasoning accuracy degrades under quantization, and existing NVFP4 kernels do not fully realize latency benefits in small‑batch autoregressive decoding. In this work, we analyze the effect of NVFP4 quantization on token‑level uncertainty during reasoning. We show that quantization increases incorrect sampling at low‑entropy symbolic tokens, while causing over‑concentration on a small set of tokens in high‑uncertainty reasoning steps. Based on this observation, we propose ReSET, a reasoning‑step entropy‑based temperature‑scaling method that estimates step‑level uncertainty online and adapts the decoding temperature using both token‑level and step‑level entropy signals. To address the latency gap, we further design a CUDA‑core small‑M NVFP4 kernel for latency‑critical autoregressive decoding. Across reasoning benchmarks and model scales, ReSET improves NVFP4 reasoning accuracy by up to ～\!2 points over the NVFP4 baseline. Our CUDA‑core small‑M kernel further improves latency‑critical decoding, delivering up to 2.5\!× kernel‑level speedup over NVFP4 vLLM and approximately 2\!× end‑to‑end decoding speedup over BF16. Code is available at https://github.com/aiha‑lab/ReSET.
Authors:Jaehwi Jang, Zhaoyuan Gu, Alfred Cueva, Zimeng Chai, Junjie Sheng, Thong Nguyen, Himank Galundia, Yifan Wu, Huishu Xue, Isaac Legene, Ojas Mediratta, Davin Doan, Andrew Collins, Sarah Sadegh, KyoungMok Kim, Rishita Dhalbisoi, Zun Chen, Ye Zhao
Abstract:
Whole‑body humanoid manipulation of bulky, deformable, and shared‑load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly. On the other hand, different demonstration sources provide complementary modalities with inherent trade‑offs: human demonstrations capture natural contact forces but not robot‑executable actions, while teleoperation directly records robot actions but with less natural force regulation. This paper presents WT‑UMI, a wearable whole‑body tactile interface worn by human operators or mounted on humanoids, providing accurate observations of tactile images, contact forces, and end‑effector poses across both human demonstration and humanoid teleoperation modes. We introduce a force‑conditioned target‑pose correction module that converts measured human poses into contact‑aware robot targets by learning corrections from teleoperation data. To leverage the natural force interaction in human data, we propose a force‑supervised planner that predicts end‑effector pose chunks and contact‑force trajectories. The predicted contact force serves as the reference for a tactile‑based admittance controller. Across five contact‑rich tasks spanning deformable objects, bulky rigid objects, and human‑‑humanoid collaboration, WT‑UMI improves success rate and reduces contact‑position tracking error over four policy baselines. Our project page is available at https://wt‑umi.github.io/WTUMI/.
Authors:Yurun Chen, Tianyuan Gao, Yizhong Ge, Shikun Ban, Yizhou Wang, Hongkai Xiong, Wenjun Zeng, Wentao Zhu
Abstract:
Distinguishing self from others is a prerequisite for social intelligence, yet humanoid robots that increasingly share workspaces with humans still lack this ability. Here we show that a humanoid robot can learn self‑other distinction from proprioceptive‑visual correspondence, without any identity labels or kinematic models. Once established, this distinction bootstraps a predictive self‑model that maps joint configurations to three‑dimensional body occupancy, capturing how the robot's body changes with action. In multi‑agent scenes involving humans or morphologically identical robots, the system reliably identifies itself, learns a 3D self‑model, and supports downstream tasks including target reaching, collision‑aware motion planning, and human‑to‑robot motion retargeting. Together, these results outline a route toward bodily self‑representation in robots that act and coordinate alongside others in shared physical environments. Project page: https://euron‑zc.github.io/humanoid‑self‑model/.
Authors:Bora Kargi, David Salinas
Abstract:
Evaluating new large language models typically requires costly human annotation campaigns at scale. LLM‑as‑a‑judge offers a cheaper alternative, but judge scores carry systematic errors ‑ such as position bias, self‑preference, or intransitivity ‑ that can strongly miscalibrate the resulting rankings. We quantify the resulting judge‑human disagreement at two complementary levels. At the local level, we estimate per‑battle uncertainty from the judge's own score differences by propagating calibrated win probabilities rather than hard labels into the Bradley‑Terry procedure. This alone provides a drastic improvement to Elo estimation accuracy, bringing LLM‑derived ratings within 17.9 Elo MAE of human‑derived ones when averaged over 55 held‑out models on LMArena. At the global level, we apply split conformal prediction to the residual gap between LLM‑derived and human‑derived Elo ratings across held‑out models, producing prediction intervals with distribution‑free marginal coverage guarantees that account for irreducible LLM‑human disagreement. Together, these two layers yield a low‑cost evaluation tool that provides developers with calibrated Elo estimates and honest uncertainty bounds, without access to large‑scale human annotations.To facilitate reproducibility, we release our code at https://github.com/kargibora/SoftElo .
Authors:Eshaan Tanwar, Pepa Atanasova
Abstract:
Reward models are a key component of reinforcement learning from human feedback (RLHF), aligning language models toward both helpful and harmless behaviour. However, the internal mechanisms underlying these objectives and their conflicts remain poorly understood. We study alignment tension in reward models trained under helpfulness‑only, harmlessness‑only, and mixed‑objective settings. We find that mixed‑objective models often underperform single‑objective models, indicating interference between objectives. Using activation‑based methods, we identify neurons associated with each objective and study their functional roles via targeted ablations. We find that these neurons causally support their corresponding objectives while often negatively affecting the opposing one. We find that a substantial proportion of neurons are shared between helpfulness and harmlessness, and that these shared neurons exert a disproportionate influence on model behaviour, contributing to alignment tension. Additionally, our results provide insights and mechanistic interpretation into how alignment objectives are represented in reward models and why multi‑objective alignment remains challenging, motivating future work on disentangled and controllable alignment methods.
Authors:Yujun Zhou, Kehan Guo, Haomin Zhuang, Xiangqi Wang, Yue Huang, Zhenwen Liang, Pin-Yu Chen, Tian Gao, Nuno Moniz, Nitesh V. Chawla, Xiangliang Zhang
Abstract:
Interactive LLM agents are becoming part of daily work, but they do not reliably become easier to work with over time: a correction remembered in one session may still be violated in the next. We study this gap between preference access and preference compliance. In tasks derived from anonymized real‑user friction cases, Mem0 memory still leaves 57.5% of applicable preference checks violated. We introduce Test‑time Rule Acquisition and Compiled Enforcement (TRACE), a drop‑in skill‑layer pipeline for coding‑agent runtimes that mines user corrections, rewrites them as atomic rules, and compiles them into runtime checks that must pass before an agent completes future tasks. Unlike runtime checks written ahead of time by developers, TRACE skills come from the user's own chat corrections. We evaluate TRACE with simulated user‑in‑the‑loop experiments on ClawArena coding‑agent tasks and MemoryArena‑derived memory‑intensive tasks. On ClawArena, TRACE reduces held‑out preference violation from 100.0% to 37.6% on in‑distribution tasks and from 100.0% to 2.0% on out‑of‑distribution tasks. On MemoryArena‑derived tasks, TRACE reduces in‑distribution violation from 100.0% to 60.5% while matching or exceeding the strongest memory baseline on task pass. These results suggest that compiling corrections into runtime enforcement can address a repeated‑friction failure mode that memory alone does not reliably solve, reducing the need for users to restate the same correction across future sessions. Experiment code is available at https://github.com/YujunZhou/TRACE_exp, and the deployable skill is available at https://github.com/YujunZhou/tellonce.
Authors:Lilan Peng, Yandi Liu, Qingren Yao, Chongshou Li, Tianrui Li
Abstract:
Spatio‑Temporal forecasting is crucial in diverse fields, such as transportation, climate, and energy. Urban spatio‑temporal data exhibits temporal mirage: similar short‑window inputs have divergent future trends, and vice versa. Existing spatio‑temporal graph neural networks (STGNNs) cannot effectively identify such mirages. We argue that the core reason lies in the short‑window inputs that have incomplete period observation, heterogeneous global spatial correlation, and cross‑period superposition causality. To bridge this gap, we develop a novel Multi‑ Period Pattern Pre‑training (MP3), a plug‑and‑play pre‑training plugin for distinguishing temporal mirages. MP3 presents two core innovations: (1) The multi‑period pattern learning is designed to learn multi‑period patterns from long time series. Specifically, multi‑period temporal modeling leverages edge convolution to identify different multi‑period patterns. Multi‑period spatial modeling uses a bottleneck project and a global memory bank to capture heterogeneous global spatial relations efficiently. Cross‑period pattern interaction employs a causality‑enhanced Transformer to capture dependencies across different period patterns. (2) This plugin can seamlessly integrate into existing STGNN backbones to strengthen their forecasting performance. The experiment on five STGNN baselines across five real‑world datasets (including a large‑scale dataset CA) verify the effectiveness, superior scalability and strong adaptability of MP3, which brings consistent and robust performance improvements across all evaluated baselines. On average, MP3 reduces the MAE 4.7% and the RMSE 5.0%. The code can be available at https://github.com/YAN‑outlook/MP3.
Authors:Aryan Khurana, Aravind Ramana RN, Dhruv Kumar
Abstract:
Large language models are increasingly deployed in citation‑augmented settings, yet the effect of citation presence on model behavior independent of factual content remains poorly understood. We introduce AuthorityBench, a 220,564‑prompt multi‑domain benchmark that isolates how citation‑based authority signals influence epistemic behavior in LLMs. The benchmark uses a fully balanced 2x2 factorial design crossing claim veracity with citation veracity, the first to do so, across four domains (general knowledge, science, law, and medicine), with controlled variation over 40 prompt templates, four venue prestige tiers, and a country‑coded author name dataset. Evaluating seven models on 12 structured research questions, we find that citation presence, whether real or fabricated, consistently increases hallucination rates relative to a no‑citation baseline. The effect is strongest when fabricated citations accompany true claims, raising hallucination rates by 3 to 22 percentage points and reaching 35 to 77% in the general knowledge domain, while legal claims are comparatively robust and venue prestige and author demographics show negligible impact. All datasets and evaluation code are available at: https://github.com/floating‑reeds/AuthorityBench
Authors:Hongbo Wang
Abstract:
Scale buys interpolation; structure buys a certified horizon. A world model's average error says nothing about whether a particular prediction can be trusted, or for how long. For equivariant latent world models we give a computable, multi‑step certificate of the predictable horizon: T‑step rollout error is provably constant over each symmetry orbit (Theorem A) and stratified channel‑by‑channel by the predictor's Lyapunov spectrum, T_j(ε)～\log(1/ε)/λ_j. The horizon is two‑sided ‑‑ a matching lower bound makes approximate equivariance provably horizon‑limited ‑‑ and the certificate is exclusive to structure: orbit‑constant error characterizes equivariance, so no non‑equivariant model has it at any scale. Empirically, on 40‑D Lorenz‑96 only a \mathbbZ_N‑equivariant network recovers the full Lyapunov spectrum (R^2=0.98); dense and recurrent baselines fail. Because the spectrum is faithful, the certificate acts, a priori: under a fixed sensing budget a c×‑inflated certificate provably needs c× the budget, and the equivariant certificate meets a budget its inflated dense counterpart cannot ‑‑ with zero calibration data. The same read‑out, unchanged, audits public pretrained world models training‑free: TD‑MPC2 checkpoints land on the certificate's own scope taxonomy ‑‑ calibrated where strongly expansive (ratio 0.94‑1.02), optimistic where weakly expansive, correctly abstaining where contracting ‑‑ a map a deployed monitor replicates cell‑by‑cell, out‑of‑sample. Across the official 1M‑317M multitask ladder, calibration does not improve with parameters. On V‑JEPA 2‑AC (1B, real robot data) the measured cross‑check correctly overrides an over‑promising tangent spectrum ‑‑ the cross‑validated audit, not the raw number, is the deployable object. Scale buys interpolation, not a calibrated horizon.
Authors:Zhixiong Zhao, Zukang Xu, Zhixuan Chen, Xing Hu, Zhe Jiang, Dawei Yang
Abstract:
Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity. However, existing methods struggle with heavy‑tailed activation distributions and therefore keep activations in high precision, fundamentally limiting end‑to‑end inference acceleration. To overcome this limitation, we propose TWLA, a post‑training quantization (PTQ) framework that achieves 1.58‑bit weight compression and 4‑bit activation quantization while maintaining high accuracy. TWLA comprises three components: (1) Euclidean‑to‑Manifold Asymmetric Ternary Quantizer (E2M‑ATQ) minimizes layer‑output error under weight ternarization via a two‑stage optimization from Euclidean initialization to manifold relocation; (2) Kronecker Orthogonal Tri‑Modal Shaping (KOTMS) applies a Kronecker‑structured orthogonal rotation to reshape weights into ternary‑friendly tri‑modal distributions, while the shared rotation statistically suppresses activation outliers; and (3) Inter‑Layer Aware Activation Mixed Precision (ILA‑AMP) explicitly introduces adjacent‑layer second‑order interaction costs in bit allocation and jointly optimizes for the layer‑wise disparity of activation quantization gains induced by the shared orthogonal transform, preventing cascades triggered by a few weak layers. Extensive experiments demonstrate that TWLA maintains high accuracy under W1.58A4, while delivering significant inference acceleration. The code is available at <https://github.com/Kishon‑zzx/TWLA>.
Authors:Fabien Maury, Solène Grosdidier, Maud de Dieuleveult, Adrien Coulet
Abstract:
Despite advances in information extraction driven by deep learning and large language models, performance gaps remain in highly specialized biomedical fields, where domainspecific complexity poses challenges for generalist models. In this work, we focus on the domain of autoimmunity, where the main entities of interest are autoimmune diseases, autoantibodies (i.e., molecules that may mark or cause these diseases), their molecular targets, their location in the body, and their associated clinical signs. Herein, we present AAbAAC (AutoAntibodies and Autoimmunity Annotated Corpus), a corpus of 115 abstracts selected from PubMed, where we manually annotated entities and their relationships. First, AAbAAC was used to evaluate several methods on the task of named entity recognition (NER), and secondly, to fine‑tune NER models. Our study demonstrates the utility of AAbAAC for information extraction in the domain of autoimmunity, showing expected improvement in NER performance after finetuning. This illustrates the value of small‑scale annotation efforts for specialized domains and contributes to the computational study of autoimmunity. The AAbAAC corpus is available at https://github.com/f‑maury/AAbAAC.
Authors:Luyao Zhang, Ke Li, Yuan Ding, Xulong Zhao, Guo Yu, Chengwei Yan, Fuyu Dong, Jiawei Hu, Di Wang, Nan Luo, Gang Liu, Quan Wang
Abstract:
Quadruped robots are capable of traversing a wide range of complex terrains with high flexibility. As highly mobile ground‑based intelligent platforms, they can be equipped with modules for navigation control, environmental perception, and intelligent interaction, thereby serving as real‑world mobile deployment platforms for various algorithms. In this paper, we introduce Y‑BotFrame, an extensible embodied platform that turns a robot into an intelligent ground assistant. Y‑BotFrame integrates multimodal perception capabilities, including speech, vision, and LiDAR, and employs a large language model as the cognitive core for environmental understanding, contextual reasoning, and task planning. The system maps user natural‑language instructions into executable embodied task units that can be carried out by the robot. Y‑BotFrame supports natural interaction through voice commands and visual feedback, removing the need for a remote controller and enabling efficient human‑robot collaboration. With a highly extensible framework, Y‑BotFrame supports plug‑and‑play integration of new functional modules as well as modular upgrades and iterative development, offering a reference implementation for the real‑world deployment of general‑purpose, instruction‑driven embodied agents.The supplementary video is available at https://xdei‑group.github.io/Y‑BotFrame/.
Authors:Dayu Xia, Yue Shi, Yao Mu, Huiting Ji, Chaofan Ma, Yingjie Zhou, Hua Chen, Yang Liu, Jiezhang Cao, Guangtao Zhai
Abstract:
Vision‑language models (VLMs) are increasingly explored as visual critics, reward generators, and failure detectors in robotic manipulation. These roles implicitly require models to judge not only final task success, but also how a manipulation execution is physically and temporally progressing. However, existing evaluations fail to test whether VLMs possess fine‑grained process understanding. To address this gap, we present RoboProcessBench, a benchmark for process‑aware understanding in vision‑language robotic manipulation. RoboProcessBench decomposes such capability into two complementary dimensions, \emphstatic monitoring and \emphdynamic reasoning, instantiated as 12 diagnostic question families covering phase, contact, motion, coordination, primitive‑local progress, temporal order, outcome, and primitive‑level transitions. Built from physically grounded execution traces, the curated benchmark corpus ProcessData contains \textasciitilde 58k question‑answer pairs across 260 manipulation tasks, which is further split into ProcessData‑SFT and ProcessData‑Eval for post‑training and evaluation purposes. Extensive evaluation of various VLMs on ProcessData‑Eval reveals broad limitations across 12 diagnostic task families, suggesting current models still lack robust process‑aware understanding of manipulation executions. But with ProcessData‑SFT, the post‑trained Qwen2.5‑VL‑7B and InternVL‑3‑8B exhibit consistent gains on local state, motion, progress, and primitive‑aware cues. These results demonstrate that RoboProcessBench serves as both an evaluation benchmark and a learnable supervision source for developing VLMs capable of monitoring and evaluating robotic manipulation processes. Project webpage: \hrefhttps://processbench‑2026.github.io/RoboProcessBench‑Web/https://processbench‑2026.github.io.
Authors:Haowei Qian
Abstract:
As LLM agents proliferate in prediction markets and collective decision‑making, they risk a cognitive monoculture: agents built on shared foundation models produce correlated forecasts, and recent measurement finds frontier‑model errors correlated at r ~ 0.77. We ask whether human cognitive diversity can be recovered from behavior and transferred to LLM agents. Nous extracts a structured eight‑dimension behavioral profile from real Polymarket trading activity and injects it into agents through prompts. Our central finding is a dissociation between the two halves of that pipeline. Extraction works, partially: across 100 wallets, 8 of 14 parameters are temporally stable (split‑half ICC >= 0.5, bootstrap CI lower bound > 0.3; contrarian score reaches ICC ~ 0.9); wallets are identifiable from their profiles well above chance (top‑1 retrieval 17‑22% vs. 1% chance); and two of four pre‑specified dimensions rank‑correlate with future realized profit out‑of‑sample, though the correlations do not survive behavioral‑confound controls. Prompt‑level injection does not measurably transmit it: on a semantic embedding metric, structured injection shows no significant advantage over a length‑matched control on any model, and the diversity it induces neither reduces ensemble error correlation nor improves Brier score ‑‑ a null that persists across exploratory checks on sampling temperature, profile diversity, and question difficulty. Measuring the prompts themselves locates the compression before the model: the structure‑to‑narrative translator emits near‑uniform prompts whose spread does not track profile spread. We position Nous as measuring the cognitive‑monoculture problem and the limits of a prompt‑level remedy, motivating deeper, below‑the‑prompt injection (fine‑tuning, activation steering). Code, frozen profiles, prompts, and model outputs: https://github.com/WillChienT/nous‑paper
Authors:Yifan Zhao, Lang Qin, Jintai Chen
Abstract:
Cyclic peptides represent a promising class of therapeutic compounds in modern drug discovery, often offering improved stability and binding affinity. However, the de novo design of cyclic peptides remains challenging because methods must identify pocket‑adaptive cyclization patterns and linkage sites while simultaneously controlling drug‑relevant properties. This challenge is particularly pronounced for recent generative models trained predominantly on linear peptide data, which may fail to capture cyclization‑specific constraints. To address the limitation, we introduce APCyc, a target‑aware de novo cyclic peptide generation framework that explicitly models cyclization and jointly optimizes multiple essential physicochemical properties. By using an expanded residue vocabulary and explicitly encoding cyclization‑site and linkage‑type information, APCyc learns cyclization‑aware representations and leverages Bayesian posterior guidance to steer sampling toward cyclic peptides satisfying multiple property objectives. Experimental results demonstrate that our model learns target‑dependent cyclization preferences, and enables effective and controllable multi‑property optimization for cyclic peptide design. The source code of this paper is available at https://github.com/HKUSTGZ‑ML4Health‑Lab/APCyc.
Authors:Sathira Silva, Abrham Kahsay Gebreselasie, Muhammad Umer Sheikh, Kartik Kuckreja, Daniel Harari, Muhammad Haris Khan
Abstract:
Learning grounded word meaning from natural experience requires resolving two ambiguities in infant‑view recordings: when the named referent appears and where it is in a cluttered frame. In SAYCam‑style data, caregiver speech is sparse and weakly synchronized with egocentric video, so single‑frame contrastive pairing yields noisy positives in which the intended object is absent or entangled with distractors. We propose BabyMind, an object‑first bias for child‑view contrastive learning under sparse, noisy supervision. BabyMind extracts candidate object embeddings using an offline mask‑based region interface, links candidates across a short utterance‑centered window into lightweight object files via tracking, and aligns utterances to bags of object files with a prototype‑space multiple‑instance contrastive objective. Track‑coherence and global‑object agreement regularizers stabilize learning and transfer object‑file structure into the global frame embedding used at evaluation. On SAYCam‑S, BabyMind improves Labeled‑S 15 forced‑choice accuracy by +2.6 points over CVCL and yields consistent gains on in‑vocabulary out‑of‑distribution benchmarks. Code is available at https://github.com/sathiiii/BabyMind.
Authors:Shihefeng Wang, Kangchen Lv, Mingrui Yu, Xiang Li
Abstract:
Scalable robot imitation learning relies on large‑scale heterogeneous data from diverse robots or body‑free data, making Cartesian end‑effector actions a key interface for embodiment‑agnostic policy learning. However, end‑effector‑only abstraction leaves Cartesian policies unaware of the deployed robot body, making them brittle under robot‑specific constraints such as whole‑body collision avoidance. To overcome this limitation, we present EmbodiSteer, a training‑free framework that steers embodiment‑agnostic visuomotor policies toward zero‑shot, embodiment‑aware deployment. EmbodiSteer keeps policy learning in Cartesian space while efficiently lifting inference‑time diffusion sampling into the target robot's joint space via forward kinematics and Jacobian‑based updates. With whole‑body collision‑aware guidance over joint trajectories after each denoising step, the arm can be steered away from collisions while preserving learned end‑effector behavior. Compared with Cartesian‑only execution, EmbodiSteer reduces collision rate by 46.1% and improves task success rate by 28.5% across 9 simulated robots, and further achieves 90.0% collision rate reduction and 36.7% success rate increase on two physical robots in highly constrained scenarios. Our project page is at https://frankwang67.github.io/EmbodiSteer‑Page.
Authors:Ching-Yu Tsai, Chia-Min Lin, Chih-Hsiang Yang, Yung-Che Wang, Jen-Shiun Chiang
Abstract:
Crack detection plays an important role in infrastructure inspection and Structural Health Monitoring (SHM). However, cracks typically appear as thin, low‑contrast structures and are easily affected by background noise, posing challenges for existing object detection models. This study proposes an improved YOLO‑based architecture with integrated attention mechanisms, termed YOLO‑AMC (YOLO with Attention Mechanisms for Crack Detection), to enhance automated crack detection performance. Based on YOLOv11, the original C2PSA module is removed, and multiple attention mechanisms, including Global Attention Mechanism (GAM), Residual Convolutional Block Attention Module (Res‑CBAM), and Shuffle Attention (SA), are introduced into the multi‑scale feature fusion layers of the Neck to strengthen cross‑scale feature integration. Experimental results demonstrate that YOLO‑AMC consistently outperforms baseline models YOLOv11n and YOLOv8n across multiple evaluation metrics. Among the evaluated attention modules, GAM achieves the best detection performance, obtaining mAP@0.5 = 0.9917 and mAP@0.5:0.95 = 0.9506 on the test dataset, which are higher than those of YOLOv11 (0.9833 / 0.9112) and YOLOv8 (0.9707 / 0.8921). Furthermore, while maintaining a computational complexity of 7.6 GFLOPs, the proposed model achieves 110.95 FPS on an NVIDIA RTX 4090 platform and approximately 5 FPS on a Raspberry Pi 5 edge device, demonstrating a favorable trade‑off between accuracy and deployment efficiency. The implementation code for this study is available on GitHub at https://github.com/CY‑Tsai24/YOLO‑AMC.
Authors:Xiang Li, Yixuan Zhou, Jingran Xie, Zhiyong Wu, Hui Wang
Abstract:
Neural speech codecs based on Vector‑Quantized VAEs (VQ‑VAEs) are core audio tokenizers for speech LLMs, yet their reconstruction fidelity is bottlenecked by quantization error. Modifying the quantizer or increasing model capacity are common fixes, but they complicate downstream language modeling. Our core idea is to align the decoder's internal feature manifolds when processing both the quantized tokens and their original continuous embeddings, using a lightweight feature‑mapping loss. This requires minimal training overhead and no inference‑time changes. Applied to XCodec2, self‑guidance improves all reconstruction metrics, achieving state‑of‑the‑art low‑bitrate performance. Notably, it enables a 4x codebook reduction without fidelity loss, which downstream TTS experiments show significantly improves LLM‑based synthesis by simplifying the token modeling space. Multiple statistical observations and visualizations corroborate the enhanced internal manifold alignment in the decoder. Extensive experiments confirm its generality across various inductive biases. Self‑guidance thus establishes an efficient, broadly applicable method for high‑fidelity neural audio coding.
Authors:Inseok Kong, Geunyoung Jung, Jiyoung Jung
Abstract:
3D point cloud models suffer significant performance degradation under distribution shifts caused by sensor noise, occlusions, and environmental changes. Test‑time adaptation (TTA) has emerged as a practical paradigm for mitigating this issue during inference. Recently, leveraging multi‑view augmentation has shown promise in improving 3D TTA performance. However, existing multi‑view approaches are often constrained by sequential optimization that treats each view independently. This sequential optimization leads to substantial inference latency due to repetitive optimization steps, making real‑time adaptation impractical. To address this, we propose Masked Multi‑View Test‑Time Adaptation (MAMVI), which replaces sequential optimization with a unified single‑step adaptation. Specifically, MAMVI utilizes a hybrid masking strategy that combines fixed ratios for stability with Beta‑distributed sampling for diversity. By aggregating losses across multiple views, MAMVI performs adaptation through a single backward pass based on multi‑view consensus. Additionally, a confidence‑based adaptive learning rate is used to dynamically adjust the adaptation intensity for each sample. Extensive experiments on ModelNet‑40C, ShapeNet‑C, and ScanObjectNN‑C demonstrate that MAMVI achieves state‑of‑the‑art accuracy on ShapeNet‑C and ScanObjectNN‑C. Moreover, it remains competitive on ModelNet‑40C while delivering 4.9‑8.9 times faster inference, making it highly suitable for real‑time applications. Our code is available at https://github.com/Inseok‑kong/MAMVI
Authors:Zehong Wang, Yijun Ma, Connor R. Schmidt, Tianyi Ma, Weixiang Sun, Ziming Li, Xiaoguang Guo, Chuxu Zhang, Matthew J. Webber, Yanfang Ye
Abstract:
Molecular dynamics (MD) is the canonical in‑silico method for atomistic molecular science, simulating molecular behavior from first‑principle physics. Designing an MD pipeline for a new system requires substantial expert knowledge: running it on even one molecule is expensive, ruling out trial‑and‑error. We automate this expert pipeline‑design process with an LLM agent. Unlike existing MD agents that orchestrate a predefined tool set, we treat pipeline design as open‑ended code generation in which the agent's behavior is reshaped online by verbal reward. Specifically, we build MDForge, an LLM agent whose in‑context update rule densifies the sparse reward via a multi‑agent debate among physics experts. On three SAMPL host‑guest binding free‑energy benchmarks, MDForge automatically designs MD pipelines competitive with human experts. Deployed on a library of unseen candidate guests, its CB[7] pipeline discovers a novel binder that wet‑lab competition NMR confirms is a high‑affinity, picomolar CB[7] binder. Our data and code are available at https://github.com/Zehong‑Wang/MDForge.
Authors:Allison Andreyev, Landon Eum, Nestor Tiglao, Romel Gomez
Abstract:
For robotics to be effectively integrated into household or industrial environments, machines must adapt to natural‑language prompts in real time. Although Vision‑Language Models (VLMs) have enabled zero‑shot generalization in robot task and motion planning (TAMP), current state‑of‑the‑art approaches often remain computationally "heavyweight" or require extensive training on thousands of demonstrations. We present GRASP (Grounded Reasoning and Symbolic Planning), a framework designed as a step toward open‑vocabulary tabletop manipulation. Our approach leverages a pretrained VLM to translate natural‑language queries into neuro‑symbolic goal states, grounded in the physical world via a bounding‑box detection pipeline. Unlike methods that rely on fixed color lists or hard‑coded coordinates, GRASP enables robots to interpret abstract spatial concepts such as "top shelf" and execute tasks without additional fine‑tuning. We achieve 73.3% overall success across 90 real‑robot trials at three difficulty levels, requiring no task‑specific training.
Authors:Wen Zhang, Xiaocui Yang, Zhuoyue Gao, Shi Feng, Daling Wang, Yifei Zhang
Abstract:
Empathetic spoken dialogue systems require not only semantically appropriate responses but also emotionally aligned prosodic expression. However, cascade pipelines often discard acoustic cues during speech‑to‑text conversion, while end‑to‑end speech models lack interpretable control over emotion and knowledge integration. To address these challenges, we propose PRISM, a multi‑agent framework for empathetic spoken dialogue that decouples speech perception, response generation, and speech synthesis into coordinated components. PRISM introduces a prosody‑to‑language translation mechanism to stabilize large language model reasoning and enables on‑demand invocation of external knowledge tools for empathetic dialogue generation. Experimental results demonstrate that PRISM achieves consistent improvements in empathy, prosodic appropriateness, and text response generation quality across objective and subjective metrics. Our code is available at: https://github.com/Bxzfrm/PRISM.
Authors:Jiahao Yang, Shuhai Zhang, Hailong Kang, Feng Liu, Qi Chen, Mingkui Tan
Abstract:
Large language models (LLMs) often hallucinate by generating factually incorrect or unfaithful content, posing significant risks to their safe use. Detecting such hallucinations is particularly challenging under the zero‑source constraint, where no model internals or external references are available, and detection must rely solely on the textual query‑answer pair. In this paper, we propose Human‑like Criteria Probing for Hallucination Detection (HCPD), a paradigm that emulates the multi‑faceted reasoning of human evaluators. Its core is a Human‑like Criteria Probing (HCP) mechanism, in which a LLM agent adaptively decomposes its judgment into a weighted set of interpretable criteria and aggregates criterion‑specific scores into a final truthfulness measure. To achieve this adaptive capability, we introduce a reward‑based alignment scheme using only weak supervision from semantic consistency. At inference, we employ a multi‑sampling aggregation strategy to ensure robust decisions while preserving full interpretability. We further provide theoretical analysis supporting the reliability of our approach. Extensive experiments show that HCPD consistently outperforms state‑of‑the‑art baselines, offering an effective and explainable solution for zero‑source hallucination detection. Code is available at https://github.com/TRISKEL10N/HCPD.
Authors:Xinrui He, Qiyu Kang, Xuhao Li, Zheng-Jun Zha
Abstract:
Spiking Neural Networks (SNNs) are well‑regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typically rely on first‑order Ordinary Differential Equations (ODEs) to govern neuronal state transitions. This first‑order assumption imposes a "memoryless" bottleneck, limiting the model's capacity to capture the complex, long‑range dependencies inherent in long‑sequence tasks. In this work, we propose LongSpike, a novel SNN framework that integrates fractional‑order State‑Space Modeling, or f‑SSM, from control theory into the spiking domain. By extending traditional integer‑order SSMs to the fractional‑calculus regime, LongSpike enables the hierarchical integration of neuronal dynamics with long‑memory kernels. To mitigate the computational overhead and parallelization challenges typically associated with fractional operators, we leverage a state‑space formulation that supports efficient, parallel training. Empirical evaluations on challenging benchmarks, including Long Range Arena (LRA), large‑scale WikiText‑103, and Speech Commands, demonstrate that LongSpike outperforms state‑of‑the‑art SNNs in accuracy while preserving sparse synaptic computation. The code is available at https://github.com/xinruihe389‑commits/LongSpike.
Authors:Jingxuan Han, Wei Liu, Mingyang Zhu, Youpeng Wang, Ziwen Wang, Lin Qiu, Xuezhi Cao, Xunliang Cai, Zheren Fu, Licheng Zhang, Zhendong Mao
Abstract:
Search Agents (SAs) typically leverage large language models (LLMs) to support complex information‑seeking tasks by autonomously exploring web sources and synthesizing information into comprehensive responses. For SAs evaluation, prior benchmarks mainly focus on specialized tasks that are unlikely to arise in real‑world user scenarios. Moreover, their reliance on coarse task‑level rubrics often limits evaluation interpretability. To bridge this gap, we introduce DailyReport, an open‑ended benchmark to evaluate SA capabilities on daily search tasks. It contains 150 open‑ended tasks with 3,546 associated rubrics, capturing widely discussed and timely information demands of real‑world users. Each task is decomposed into subtasks and evaluated with cascade rubrics across disentangled dimensions. Through cascade performance attribution and user‑centric aggregation, we derive highly interpretable scores for each dimension, along with a user preference score. Our results on 17 agentic systems show that current systems still fall short of users' expectations. To facilitate future research, our dataset and code are made publicly available at https://github.com/AGI‑Eval‑Official/DailyReport.
Authors:Xu-Jing Ye, Yuan-Gen Wang, Ruping Wang
Abstract:
The Abstraction and Reasoning Corpus (ARC) is viewed as a critical avenue to Artificial General Intelligence (AGI), as it enables models to learn abstract transformation rules from few‑shot examples and then generalize to new tasks. However, prevalent ARC methodology is either pure language or vision‑only (i.e., VARC). The former depends heavily on LLMs, consuming billions of parameters. The latter often struggles to capture high‑level semantics, leading to overfitting on pixel‑level patterns. To bridge this gap, we propose L‑VARC, a novel framework that enhances visual reasoning via a language‑guided Learning Using Privileged Information (LUPI) branch. Specifically, we design a Semantic Compression Module by feeding a unified, task‑agnostic prompt into DeepSeek‑V3. In this way, the raw LARC (a crowd‑sourced language description dataset) can be substantially refined and structured, fitting with the context length constraint of standard text encoders (e.g., CLIP). Moreover, we design a Cross‑Attention Projector to align visual features with semantic embeddings, aiming to guide the training of the ARC model. Notably, the LUPI branch is taken in the training process and will be discarded during inference, thereby yielding a lightweight model with a mere 18 million parameters. Extensive experiments demonstrate that our L‑VARC effectively leverages linguistic priors to boost visual reasoning and outperforms state‑of‑the‑art. Ablation studies further confirm the contribution of the two new designs towards the L‑VARC framework. The code is available at https://github.com/GZHU‑DVL/L‑VARC.
Authors:John Fields, K M Sajjadul Islam, Ruchitha Thota, Victor Chen, Praveen Madiraju
Abstract:
This study explores privacy‑preserving machine learning (PPML) techniques using the PySyft platform to enable collaborative prediction of student retention between institutions. We developed a remote data science (RDS) framework with a semi‑air‑gapped architecture consisting of high‑side and low‑side servers, allowing researchers from three universities to build predictive models on sensitive student data without direct data access. Using historical data from a small private university (N=720), we evaluated three synthetic data generation approaches and validated the framework through inter‑institutional collaboration. The results demonstrate consistent classification performance across institutions (Macro F1: 0.690‑‑0.695) while maintaining strict Family Educational Rights and Privacy Act (FERPA) compliance. We also propose Data‑Type‑Aware Templates, a novel synthetic data method that prioritizes privacy over distributional fidelity. Our findings confirm that RDS‑based PPML is technically feasible for educational settings and offers a practical alternative to federated learning for small‑scale inter‑institutional collaborations. The code is available at https://github.com/jtfields/NAIRR240195‑Privacy‑Preserving‑Machine‑Learning.
Authors:Rasul Khanbayov, Hasan Kurban
Abstract:
Do research topics in artificial intelligence grow gradually, or do they advance through abrupt, detectable jumps? Analyzing 80,814 accepted main‑track papers from five premier AI conferences (ACL, CVPR, ICLR, ICML, NeurIPS) spanning 2017 to 2025, we show major AI topics advance through topical phase transitions: remaining marginal for years, then surging across venues within one to three years. Large language models became the dominant cross‑venue topic by 2025, diffusion models rose with comparable abruptness, and language‑model methods crossed into computer vision via vision‑language models, whereas reinforcement learning compounded smoothly, distinguishing genuine phase transitions from ordinary growth. This structure is our primary contribution: a large‑scale, cross‑venue characterization of how AI research reorganizes. We then ask whether a transition leaves a detectable footprint before it peaks. We define an early‑warning signature, four publication‑dynamics criteria frozen on 2017‑2021 data, and evaluate it out of sample on 2023‑2025 transitions, obtaining a precision of 27% and recall of 63% against a 13.5% base rate. Applied to 2025 data, the signature flags reasoning and test‑time compute, agentic AI, multimodal LLMs, retrieval‑augmented generation, and world models as topics to monitor over 2026‑2028. The source code is also publicly available on GitHub at https://github.com/KurbanIntelligenceLab/ai‑phase‑transitions.
Authors:Xiao Ren, Yuhui Yang, Zongbiao Weng, Zhijie Liu, He Kong
Abstract:
Recent reinforcement learning approaches have shown great promise in improving humanoid motion tracking performance and achieving fall recovery under disturbances. However, most existing works treat motion tracking and fall recovery as different tasks and require multi‑stage training with specialized recovery rewards and/or separate recovery policies. Moreover, existing reinforcement learning‑based methods often terminate training episodes immediately after severe tracking failures, limiting recovery‑oriented exploration in unstable or fallen states. To address the above issues, we propose Stubborn, a streamlined and unified reinforcement learning framework to achieve robust humanoid motion tracking and fall recovery. Specifically, Stubborn uses an asymmetric Actor‑Critic architecture and consists of three major components. First, a yaw‑aligned tracking representation is adopted to reduce sensitivity to global drift and heading disturbances while preserving gravity‑related balance information. Second, we introduce a Bernoulli‑based probabilistic termination mechanism that enables the policy to encourage exploration of fall‑recovery behaviors under varying failure modes. Third, we propose a probabilistic termination and tracking‑error‑driven strategy that dynamically reshapes the sampling distribution based on tracking performance, increasing the training efficiency for difficult motion segments and unstable states. Extensive comparisons with SOTA methods and ablation studies show that Stubborn achieved competitive performance, and the proposed probabilistic termination mechanism and adaptive sampling strategy contributed to the performance and robustness gains. For real‑world demonstrations, please refer to https://aislab‑sustech.github.io/Stubborn/.
Authors:He Li, Haoang Chi, Qizhou Wang, Yunxin Mao, Zhiheng Zhang, Jie Tan, Tongliang Liu, Wenjing Yang, Bo Han
Abstract:
Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large‑scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open‑sourced in https://github.com/lihe‑maxsize/Lifelong_Unlearning_main.
Authors:Yijun Ma, Zehong Wang, Yiyang Li, Ziming Li, Xiaoguang Guo, Weixiang Sun, Chuxu Zhang, Yanfang Ye
Abstract:
Self‑evolving agents are expected to improve through interaction without external supervision, but this remains difficult in partially observable environments where agents must explore actively, learn from limited feedback, and decide when to trust prior experience. Existing LLM‑agent methods often rely on memory or planning modules, yet they rarely close the loop between them to continually refine an internal understanding of environment dynamics. We introduce ProPlay, a procedural world model that supports procedure‑level preplay, where agents can rehearse future procedural paths using the learned world knowledge. Rather than representing experience as isolated rules or low‑level action constraints, ProPlay abstracts successful trajectories into procedures and organizes them in a procedure graph that captures causal transitions among task stages. Each transition is associated with a reliability record embedding to estimate its task‑specific contribution from past outcomes. Before each episode, ProPlay simulates future procedural trajectories over known graph structures as structured soft guidance; after execution, it refines the graph using environment feedback. Experiments on public benchmarks show that ProPlay consistently improves environment understanding and self‑evolution capability over strong baselines. Our code has been released in https://github.com/antman9914/proplay.
Authors:Tianyu Liu, Allen Xin Wang, Antonia Panescu, Lisa Xinyi Chen, Wenxin Long, Xinyu Wei, Yueqian Jing, Ziyao Zeng, Jihang Chen, Sihan Jiang, Ziqing Wang, Siyi Gu, Siyu Chen, Xinyang Hu, Haoran Shao, Leqi Xu, Wangjie Zheng, Zhiyuan Cao, Ada Fang, Botao Yu, Kunyang Sun, Rex Ying, Arman Cohan, Qingyu Chen, Lingzhou Xue, Kaize Ding, Yuanqi Du, Wengong Jin, Zhuoran Yang, Marinka Zitnik, James Zou, Hua Xu, Hongyu Zhao
Abstract:
AI agents are increasingly being developed to accelerate scientific discovery, yet their practical capabilities in real research settings remain poorly understood. Existing benchmarks for AI agents rarely capture the complexity, heterogeneity, and extended reasoning required by scientific work, whereas benchmarks for scientific tasks often reduce research to static, direct problems and provide limited support for interactive evaluation. Here, we introduce SciAgentArena, a systematic benchmark for evaluating AI agents in real‑world scientific research scenarios drawn from emerging needs across multiple domains. SciAgentArena comprises approximately 200 tasks with stepwise verification and an interactive, agent‑agnostic environment for assessing diverse AI agents. Using this benchmark, we find that current agents can contribute effectively to well‑specified data‑analysis workflows, particularly when the task structure and evaluation criteria are clear. However, their performance remains uneven across scientific contexts: agents struggle to generate genuinely novel insights, sustain self‑directed exploration, and formulate robust solutions for open‑ended research questions. We further characterize common failure modes across agents and identify opportunities for improving their reliability, autonomy, and scientific reasoning. Together, SciAgentArena provides a practical framework for measuring progress in AI agents for science and for guiding the design of future agents capable of addressing complex scientific challenges. Full codes, tasks, and datasets can be accessed via this link: https://sciagentarena.github.io/.
Authors:Weijie Chen, Alan B. McMillan
Abstract:
Federated learning (FL) enables collaborative model training without sharing raw patient data, but standard approaches such as FedAvg treat each client as a black box and provide no mechanism for isolating an adversarial contributor, auditing per‑client influence, or honoring a departed participant's right to be forgotten. We present Fed‑FBD (Federated Functional Block Diversification), a modular federated architecture that decomposes a ResNet backbone into six functional blocks (the stem, four residual groups, and the classification head) and maintains a warehouse of N color variants, each assembled from independently tracked and contributor‑stamped blocks. Fed‑FBD provides three capabilities absent in FedAvg: (i) architecturally guaranteed block‑level isolation, so that an adversarial or mislabelled client cannot contaminate the clean colous; (ii) privacy‑by‑design, where membership inference advantage is already indistinguishable from chance before any privacy mechanism is applied; and (iii) surgical machine unlearning of a departed participant's contribution at sub‑second cost and without retraining. Experiments on six MedMNIST‑2D datasets, PathMNIST at 224x224, and CIFAR‑10 show that Fed‑FBD trades a modest 0.3%‑3.1% IID accuracy gap on the adequately sized datasets for these guarantees, remains within 0.8%‑4.0% of FedAvg at Dirichlet alpha=1.0 on three of four datasets, and confines all six adversarial attacks we study to the poisoned client's own blocks with at most +/‑0.01 AUC drift on the clean colors.
Authors:Kushal Raj Bhandari, Ling Yue, Ching-Yun Ko, Dhaval Patel, Shaowu Pan, Pin-Yu Chen, Jianxi Gao
Abstract:
Compact language models (LMs) reduce cost, latency, and deployment risk for tool agents. Yet MCP‑style tool use requires more than isolated function calling: an agent must discover tools from live catalogs, satisfy schemas, preserve dependencies across intermediate outputs, and ground final responses in executed evidence. Small planners often generate plausible workflow graphs that fail under tool resolution, parameter validation, dependency tracking, or execution. We argue that this failure mode is poorly handled by small‑corpus distillation. A few hundred teacher traces can teach workflow format, but rarely cover the recovery behavior needed to repair failed plans over changing tool catalogs. We introduce Evoflux, an inference‑time evolutionary search method that treats compact tool use as the repair of executable tool workflows. It evolves typed workflow graphs through structured edits, execution feedback, adaptive intensity, meta‑guided redesign, and diversity pruning. On held‑out MCP‑Bench tasks spanning live MCP servers and 250 tools, Evoflux raises execution feasibility from roughly 3% to 17‑24% across small planners. In contrast, SFT and SFT+DPO on the same search‑mined data match, underperform, or collapse below zero‑shot performance; ReAct reaches higher peaks, but with higher variance and token cost. These results show that execution‑grounded search is more reliable under scarce teacher‑trace budgets.
Authors:Jiangtao Kong, Peijun Zhao, Chun-Fu Chen, Youngwook Do, Shaohan Hu, Tianyi Zhou, Huajie Shao
Abstract:
Incremental Learning (IL) for Open‑ended Image‑to‑Text Generation (OpenITG) enables models to continuously generate accurate, contextually relevant text for new images while preserving previously acquired knowledge. Unlike prior studies, this paper addresses a more practical scenario in which the predominant category of visual data shifts over time as environments evolve. In this context, we introduce a new notion of continual alignment, which incrementally adapts the alignment module within pre‑trained VLMs to preserve high‑quality cross‑modal representations. Based on this idea, we propose Efficient Continual Alignment (ECA), a novel exemplar‑free IL approach for OpenITG. The key challenge is enabling the model to acquire new, task‑specific features while minimizing interference with the established alignment without accessing raw data from previous tasks. To address this, ECA employs three core mechanisms: a Mixture of Query (MoQ) module that adapts task‑specific query tokens, a Fisher Dynamic Expansion (FeDEx) that dynamically expands model structure based on a Fisher Information Matrix (FIM)‑based metric, and an embedding dictionary with Dictionary Replay (DR) to retain past knowledge. To evaluate ECA's performance, we construct four new IL OpenITG benchmarks that better reflect real‑world scenarios. Experimental results demonstrate that ECA significantly mitigates catastrophic forgetting and improves IL performance compared to baseline methods. Code and benchmarks are available at https://github.com/Snowball0823/ECA.
Authors:Binay Kumar Singh, Niels Da Vitoria Lobo
Abstract:
Object detection in autonomous driving requires precise localization and an inherent understanding of the relational context between co‑occurring objects. In extremely complex heterogeneous environments rare classes, small‑scale objects, and frequently appearing objects are difficult for standard object detection frameworks to handle. In this paper, we propose a novel framework called Context‑Centric Feature Fusion (CCFF), which utilizes two attention‑based modules, Local Context Fusion Module (LCFM) uses the RoI‑to‑RoI self‑attention mechanism to resolve spatial interactions, mainly considering small and partially obscured objects, while Global Context Attention Module (GCAM) converts the co‑occurrence of objects priors by pooling top‑K RoI features into a global context attention token, avoiding the computational overhead of pixel‑level global pooling. This fusion of local and object‑centric global features yields contextualized embeddings that enhance classification results and co‑occurring objects detection. Our method is evaluated on two datasets, Cityscapes and BDD100K which demonstrate significant improvement on relational consistency, achieving a Category‑level Consistency Strategy (CCS) of 0.973 and 0.969, respectively. Furthermore, our approach produces substantial gains in small object detection (AP_S: 14.1%) and successfully recovers rare classes such as "Train" that are typically lost in large distributions. Our efficiency report shows that the framework processes images in real time with a 0.2 FPS overhead. The code is available at https://github.com/BinayKSingh/CCFF.
Authors:Arthur Bigot, Harmon Bhasin, Core Francisco Park, Eugene Shakhnovich, Dianzhuo Wang
Abstract:
Protein language models are trained on highly imbalanced datasets, raising the question of how they represent underrepresented biological sequences. Using viral proteins as a case study across ESM model families, we identify a dominant nativeness axis in embedding space, aligned with masked reconstruction perplexity, that orders sequences from well‑modeled cellular proteins through viral proteins to shuffled and random sequences. Scaling contracts this axis unevenly across viral families. Despite this, protein language model embeddings retain viral‑specific signal: viral proteins remain linearly separable beyond zero‑shot perplexity and shallow sequence features. Together, these results suggest that pLM representations are structured by a general notion of nativeness while preserving information specific to distinct biological groups.
Authors:Tanmay Bishnoi, Riddhiman Laha, Tobias Löw, Jose Alex Chandy, Luis F. C. Figueredo, Sami Haddadin
Abstract:
Reactive motion generation in unstructured environments remains an open challenge in robotics. Due to the computational complexity of collision‑free motion generation, existing methods either generate global trajectories for static scenarios, or employ models that make conservative assumptions about the environment. This paper identifies the primary bottleneck as the runtime performance demand of planning on high‑fidelity environments, and the temporal integration between the perception and planning modules. Therefore, we propose a framework that does not compromise on runtime performance and world representations for perception and planning by accelerating world modeling and vector‑field based planning using the GPU. This allows us to achieve faster parallel state exploration for quasi‑global trajectory planning, and tighter coupling of the perception‑action loop in real‑time for dynamic cluttered environments with off‑the‑shelf depth sensors. We quantitatively evaluate the computation‑time and success rate differences for the CPU and GPU versions of our planner, and perform qualitative evaluations of our coupled framework using real‑world experiments on a 7‑DoF Franka Emika robot. Experimental results demonstrate that our GPU‑based framework achieves up to a 5x speedup over the CPU version and successfully avoids collisions across both trivial and challenging physical world scenarios.
Authors:Alireza Heidari, Amirhossein Alimohammadi, Wallace Michel Pinto Lira, Adi Bar-Lev, Ali Mahdavi-Amiri
Abstract:
Transferring hairstyles between images is an important but challenging task in computer graphics, computer vision, and visual effects. It enables users to explore new looks without physically altering their hair, with applications in virtual try‑on systems, augmented reality, and entertainment. Most prior works operate best under small pose gaps, and they fall short under large viewpoint and scale differences, where missing hair content must be synthesized rather than transferred. We propose HairPort, a 3D‑aware hairstyle transfer framework that attempts to solve these issues by explicitly separating hair removal from transfer and enforcing geometric consistency before synthesis. We introduce a Bald Converter, which produces realistic bald versions of faces through LoRA‑based in‑context adaptation of FLUX.1 Kontext. To train our Bald Converter, we introduce a new dataset, Baldy, containing 6,000 paired bald and original images across diverse identities and conditions. We also use a 3D‑Aware Transfer Pipeline that reconstructs and re‑renders the reference hairstyle from the target viewpoint before compositing it onto the source image. Being 3D aware, our method supports large pose and scale discrepancies between the source and target. Finally, a conditional flow‑matching generator synthesizes the transferred result from the bald source and geometry‑aligned reference guidance. Together, our method enables accurate, pose‑consistent, and identity‑preserving hairstyle transfer, outperforming existing methods both qualitatively and quantitatively.
Authors:Zeyue Tian, Lei Ke, Zhaoyang Liu, Ruibin Yuan, Liumeng Xue, Yujiu Yang, Weijia Chen, Xu Tan, Qifeng Chen, Wei Xue, Yike Guo
Abstract:
Audio and music generation based on flexible multimodal control signals is a widely applicable topic, with the following key challenges: 1) a unified multimodal modeling framework, 2) large‑scale, high‑quality training data, and 3) the prohibitive inference cost of multi‑step diffusion sampling. As such, we propose AudioX‑Turbo, a unified and efficient framework for anything‑to‑audio generation that integrates varied multimodal conditions (i.e., text, video, and audio signals) in this work. AudioX‑Turbo follows a teacher‑student paradigm. The teacher AudioX‑Base is built on a Multimodal Diffusion Transformer with a Multimodal Adaptive Fusion module that aligns diverse multimodal inputs for high‑fidelity synthesis, and is then distilled into the few‑step student AudioX‑Turbo via Distribution Matching Distillation adapted to flow matching, complemented by a diffusion‑based discriminator for high‑quality few‑step generation. To support the training of AudioX‑Turbo, we construct a large‑scale, high‑quality dataset, IF‑caps‑Pro, comprising approximately 9.2M samples curated through a two‑stage data collection and annotation pipeline. We benchmark AudioX‑Turbo across a wide range of tasks, finding that our model achieves superior performance, especially on text‑to‑audio and text‑to‑music generation, while operating at only 4 sampling steps and requiring approximately 25x fewer function evaluations (NFE) than multi‑step baselines. These results demonstrate that our method is capable of audio generation under flexible multimodal control, showing efficient and powerful instruction‑following capabilities. The code and datasets will be available at https://zeyuet.github.io/AudioX‑Turbo/.
Authors:Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky, Aydar Bulatov, Artem Pshenitsyn, Yuri Kuratov, Alexey Skrynnik, Aleksandr I. Panov, Alexey K. Kovalev
Abstract:
Vision‑language‑action (VLA) models predict chunks of future actions from the current observation, an assumption that fails under partial observability, where decisions depend on information no longer visible. Existing memory‑augmented VLAs simultaneously introduce recurrence, retrieval, compression modules, auxiliary objectives, hierarchical memory, or task‑specific architectural changes, so the contribution of recurrence itself remains entangled with surrounding machinery. We present a controlled isolation study of recurrence in a strong pretrained VLA backbone. Our formulation augments the transformer with a small set of learnable memory tokens carried across timesteps and updated through self‑attention, trained end to end with truncated backpropagation through time, with no auxiliary losses and no architectural changes. We instantiate this as μVLA, a family of OpenVLA‑OFT variants parameterized by memory width m, TBPTT length K, and the memory update rule (cross‑step gradients or a detached EMA), so that recurrence is the only varying factor. On MIKASA‑Robo, μVLA improves average success rate on five training tasks from 0.42 to 0.84 at the strongest setting and reaches 0.23 on held‑out tasks with the same memory structure versus 0.07 for the memoryless baseline. On tasks requiring different memory structure, performance remains near baseline. On LIBERO, the strongest recurrent variant achieves 96.2% average success, indicating no regression under full observability. We interpret these results as a calibration of the capability envelope of minimal in‑backbone recurrence, identifying the regime in which it is sufficient and the regime where additional memory structure is required. Demos and videos can be found in https://avanturist322.github.io/mu‑vla/.
Authors:Peng Jia, Li Dai, Jia Li, Zhenzhen Hu, Ye Zhao, Richang Hong
Abstract:
Accurate and robust multimodal speaker identification is essential for multimedia understanding and biometric authentication. However, real‑world polyglot scenarios pose two key challenges: speaker‑discriminative representations should generalize across languages, and the model should remain reliable when face information is unavailable. To address these challenges, we propose MRAF, a Missing‑Token Prompted Reliability‑Aware Fusion framework for polyglot speaker identification across complete‑modality, missing‑face, and cross‑lingual scenarios. MRAF represents unavailable face inputs with a learnable missing token instead of fixed zero‑valued features, providing a trainable representation of the missing visual state. This design reduces the distribution gap caused by missing inputs and allows subsequent reliability estimation and cross‑modal fusion to operate within a unified token space. To adaptively integrate modalities with different reliability, MRAF further introduces a reliability‑aware cross‑attention fusion module, which estimates face and audio reliability scores, normalizes them into modality weights, and applies these weights to token representations before bidirectional cross‑attention. In this way, the model can emphasize reliable modality cues while suppressing unreliable ones. During training, MRAF jointly optimizes multi‑branch classification losses, audio‑only knowledge distillation, and center loss to improve speaker discrimination and missing‑modality robustness. Experiments on the official POLY‑SIM 2026 test set demonstrate the effectiveness of the proposed framework. In the final evaluation, MRAF achieves 100% accuracy on P3 and P5, and obtains competitive results on the more challenging missing‑face settings P4 and P6. The source code will be released at https://github.com/MSA‑LMC/MRAF.
Authors:Guangyu Wang, Zhaonan Wang
Abstract:
Reducing real‑world trial and error has long been a central goal of decision making, and generative simulators advance this goal by modeling the evolution of future states. An even more challenging yet meaningful task is simulating how disturbance events (e.g., accidents) propagate their impacts across real‑world networks. The existing approaches fall short of modeling both structured attributes and unstructured semantics of events, and capturing topological structures in simulating network event evolution. Therefore, we are motivated to propose Net‑Ev^2 (\underlineNetwork \underlineEvent \underlineEvolution), a novel generative simulator that jointly leverages event cues while preserving network topology in simulations. Specifically, the framework consists of two stages, namely structure‑guided masked pre‑training and topology‑aware diffusion process, which is achieved by U‑Net‑like graph downsampling and upsampling during denoising. At inference time, Net‑Ev^2 can generate simulations using natural‑language event input only, with greater flexibility for practical usage. Furthermore, we introduce Net‑Ev^2‑6.5M, a multimodal benchmark of aligned event and network traffic data across four large‑scale road networks, as well as a new topology‑aware metric, namely JL‑MMD, to evaluate topological fidelity in generated network dynamics. Extensive experiments demonstrate the state‑of‑the‑art performance and strong generalization ability of Net‑Ev^2. Code is made available at https://github.com/Guangyu4/Net‑Ev‑2.
Authors:Niccolò Biondi, Federico Pernici, Simone Ricci, Alberto Del Bimbo
Abstract:
Learning compatible representations aims to learn feature representations that can be used interchangeably over time whenever a model undergoes updates. In this paper, we demonstrate that stationary representations learned by d‑Simplex fixed classifiers imply compatibility as in its formal definition. This result establishes a foundation for future works and can be directly exploited in practical learning scenarios. We address the challenge of learning compatibility using d‑Simplex fixed classifiers when the model is sequentially fine‑tuned. Learning according to a d‑Simplex fixed classifier with the cross‑entropy loss aligns feature distributions at the first‑order statistics. Consequently, it may not fully capture higher‑order dependencies in the representation between model updates. To address this issue, we demonstrate that training the model using a d‑Simplex fixed classifier through a convex combination of the cross‑entropy loss and a contrastive loss not only captures higher‑order dependencies, but is also equivalent to learning with the cross‑entropy under the compatibility constraints. We confirm our findings with extensive experiments also considering a new scenario where a pre‑trained model is sequentially fine‑tuned and occasionally replaced with an improved model. We show that stationary representations enable uninterrupted retrieval services (without reprocessing gallery images) while improving performance during model updates and replacements, achieving state‑of‑the‑art. Code at https://github.com/miccunifi/iamcl2r.
Authors:Longkun Hao, Hongyu Lin, Hao Li, Zhichao Yang, Haojie Hao, Dongshuo Huang, Haitao Yang, Hongyu Ge, Ming jie Xie, Yanjun Wu, Zi Hao Yin, Yan Bai, Yihang Lou
Abstract:
Training interactive web agents through imitation learning from expert trajectories has emerged as a highly effective approach. However, determining the optimal timing for expert intervention presents a critical challenge in this context. Delayed intervention often leads to the accumulation of early‑stage errors, pushing the page state into an irrecoverable regime. Conversely, premature or excessive intervention causes the agent to become overly reliant on expert policies, trapping the model in local optima characterized by a single, rigid trajectory. We propose Speculative Rollback Correction (SRC), a branch‑level imitation framework for resettable agent environments. Instead of requesting teacher labels at every visited state or correcting only after a completed trajectory, SRC uses fixed‑horizon branch review: the student executes a short speculative segment before teacher review, and the teacher localizes the first harmful deviation only when local progress breaks. Rollback preserves useful prefixes, while successful rollouts are filtered by a hard verifier and retained in a lightweight quality‑diversity archive. The resulting data supports next‑action supervised fine‑tuning on both localized corrections and verifier‑passing trajectories. On WebArena‑Infinity, SRC collects 977 verifier‑passing trajectories and 9,183 next‑action examples; fixed‑horizon review improves the recovery‑versus‑query tradeoff over step‑level review while retaining verifier‑passing solution variants. Code is available at https://github.com/LongkunHao/SRC_gui_agent.
Authors:Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber, Sahil Bansal
Abstract:
Large language models deployed as agents over large tool catalogs face a critical tool‑retrieval bottleneck. As embedding‑based retrieval approaches rely on compact encoders that may under‑capture specialized tool semantics, parametric tool retrieval addresses this by encoding each tool as a virtual token appended to the LLM vocabulary, fine‑tuned in two stages (memorization then retrieval SFT) to use the LLM as a retriever, achieving strong performance on standard ToolBench retrieval benchmarks. Yet these benchmarks use verbose, fully‑specified queries, and their evaluation applies constrained decoding that restricts outputs to valid token paths, neither reveals whether the model actually understands its tools. We introduce ToolSense, an open‑source LLM‑powered diagnostic framework that takes any tool catalog as input and automatically generates three benchmarks: a Realistic Retrieval Benchmark (RRB) with queries at three ambiguity tiers, an MCQ probing benchmark, and a QA probing benchmark. Applying ToolSense to ToolBench (~47k tools) and evaluating five parametric model training configurations reveals a knowledge‑retrieval dissociation: on RRB queries, several configurations collapse by ~50‑64 percentage points compared to fully‑specified ToolBench benchmarks, falling below the embedding‑model baseline. Additionally, despite strong retrieval performance, some models score near‑random on factual probes, suggesting a knowledge‑retrieval dissociation. We open‑source the ToolSense framework and the ToolBench diagnostic benchmarks at https://github.com/SAP/toolsense.
Authors:Cheng-Yu Yang, Shao-Yuan Lo, Yu-Lun Liu
Abstract:
Vision‑language models (VLMs) project images into hundreds to thousands of visual tokens, making decoder inference expensive in both attention computation and KV‑cache memory. Existing visual‑token reduction methods largely follow a rank‑and‑remove paradigm: they score visual tokens, keep a compact subset, and permanently discard the rest. We show that this irreversible action is fragile because visual‑token importance changes across decoder depth; tokens ranked low at one stage may become relevant in later layers, especially for grounding‑sensitive queries. We propose Reroute, a training‑free plug‑in that replaces removal with recoverable routing. At each routing stage, selected vision tokens pass through decoder blocks, while deferred tokens bypass the stage and re‑enter the candidate pool at the next routing decision. Reroute reuses existing attention‑score ranking rules and stage‑wise schedules, preserving the theoretical TFLOPs and KV‑cache budget class of the pruning method it augments. Across FastV, PDrop, and Nüwa variants on LLaVA‑1.5 and Qwen backbones, reroute improves grounding under aggressive token reduction while maintaining general VQA performance. These results suggest that VLM token reduction should not be viewed only as irreversible pruning, but also as recoverable routing. The code can be found here: https://github.com/elmma/mllm‑reroute/
Authors:Jin Yao, Dhruva Dixith Kurra, Tom Lampo, Zezhou Cheng, Danhua Guo, Burhan Yaman
Abstract:
Vision‑language‑action (VLA) models can describe scenes and reason about them in language, yet still struggle to ground their actions in the dense 3D world around them. Existing approaches either inject features from a frozen 3D foundation model without an objective that ensures the policy uses them, or constrain geometry with sparse box and map losses that provide no dense spatial signal. We introduce VLGA, the first vision‑language‑action model supervised to reconstruct the dense 3D world it drives through. VLGA introduces geometry as a fourth modality alongside vision, language, and action through a dedicated expert supervised by a per‑pixel pointmap regression loss against LiDAR. Extensive experiments conducted on challenging nuScenes and Bench2Drive datasets for open‑loop and closed‑loop evaluations, respectively, show the superiority of VLGA over counterpart VLA methods. In particular, on open‑loop nuScenes, VLGA sets a new state of the art among VLA methods without ego status, with the lowest L2 (0.50\,m average) and 3‑second collision rate (0.18%). On closed‑loop Bench2Drive, VLGA attains the state‑of‑the‑art driving score of 79.08, +0.71 over the strongest prior VLA, at comparable efficiency and comfort.
Authors:Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang
Abstract:
Human‑in‑the‑loop reinforcement learning (HiL‑RL) has emerged as an effective paradigm for real‑world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL‑RL frameworks remain intervention‑intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real‑world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high‑value states, taking over the bulk of interventions from human operators. Specifically, UniIntervene first performs future‑conditioned action‑value estimation, predicting the latent consequence of the current action and evaluating its induced value, which provides a more stable progress signal. Building on this, a temporal value‑risk critic aggregates recent value dynamics and triggers intervention when the estimated value exhibits sustained stagnation or degradation. When intervention is required, UniIntervene retrieves a high‑value recovery target from a memory of past intervention episodes and produces executable corrective actions through a goal‑conditioned recovery policy. In this way, UniIntervene turns intervention from passive human correction into a value‑aware recovery process for efficient real‑world RL. Extensive experiments on diverse real‑world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state‑of‑the‑art HiL‑RL baselines.
Authors:Zhen Zhao, Gang Zhang, Xiaolin Hu, Liang Tang
Abstract:
Object detection and instance segmentation tasks are closely related. Existing top‑down instance segmentation methods usually follow a detect‑then‑segment paradigm, where an initial detector is used to recognize and localize objects with bounding boxes, followed by the segmentation of an instance mask within each bounding box. In such methods, the detection accuracy directly influences the subsequent segmentation performance. However, previous research has seldom explored the impact of the instance segmentation task on object detection. In this paper, we present a turbo‑inference strategy for the top‑down methods that leverages the complementary information between detection and segmentation tasks iteratively. Specifically we design two modules: turbo‑detection head and turbo‑segmentation head, which facilitate communication between the tasks. The two modules form a closed loop that interlaces the detection and segmentation results without retraining the model. Comprehensive experiments on the COCO, iFLYTEK, and Cityscapes datasets demonstrate that our method substantially enhances both detection and segmentation accuracies with a certain increase in computational cost. The proposed method represents a tradeoff between prediction accuracy and inference speed. Codes are available at https://github.com/zhaozhen2333/Turbo‑Learning.git.
Authors:Kechun Xu, Zhenjie Zhu, Anzhe Chen, Rong Xiong, Yue Wang
Abstract:
Vision‑Language‑Action (VLA) models that couple pretrained Vision‑Language Models (VLMs) with continuous action experts have achieved strong manipulation performance, yet generalization to out‑of‑distribution (OOD) language instructions remains poor. A known challenge is the structural imbalance in VLA data, where language is far less diverse than visual and action content, making policies prone to visual shortcuts. While discrete‑action methods mitigate this through vision‑language co‑training, continuous action experts lack such protection: they start from random initialization and learn entirely from imbalanced data, producing noisy gradients that corrupt the VLM and fail to exploit its language capability. We address this from a Bayesian perspective, factorizing the policy into a language‑agnostic Vision‑Action (VA) prior and a language‑conditioned VLA likelihood, and propose APT, a two‑stage training method emphasizing Action expert PreTraining. In Stage 1, the action expert is pretrained as a VA prior on vision‑action pairs from a frozen VLM, bypassing the language imbalance. In Stage 2, language tokens are injected through a gated fusion mechanism that integrates VLM features while preserving the learned visuomotor prior. APT applies to mainstream VLA architectures, including the π and GR00T‑style architectures. Comprehensive experiments validate that APT achieves consistent gains on unseen instructions and compositional tasks. Project Page: https://xukechun.github.io/papers/APT/
Authors:Mengyu Zheng, Kai Han, Boxun Li, Haiyang Xu, Yuchuan Tian, Wei He, Hang Zhou, Jianyuan Guo, Hailin Hu, Lin Ma, Chao Xu, Guohao Dai, Lixue Xia, Yunchao Wei, Yunhe Wang, Yu Wang
Abstract:
General‑purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE‑bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and prediction contract required for scoring. We introduce Claw‑SWE‑Bench, a multilingual SWE‑bench‑style benchmark and adapter protocol that makes heterogeneous agent harnesses, or claws, comparable under fair settings including a fixed prompt, runtime budget, workspace contract, patch extraction procedure, and evaluator. The full benchmark contains 350 GitHub issue‑resolution instances across 8 languages and 43 repositories, drawn from SWE‑bench‑Multilingual and SWE‑bench‑Verified‑Mini after future‑commit cleanup. We also release Claw‑SWE‑Bench Lite for faster validation, which is an 80‑instance subset selected by a cost‑aware, rank‑aware procedure over 17 calibration columns. On the full benchmark, OpenClaw with a minimal direct‑diff adapter scores only 19.1% Pass@1, whereas the full adapter reaches 73.4% with the same GLM 5.1 backbone, showing that adapter design is essential for enabling OpenClaw‑style harnesses to perform coding tasks effectively. Across an OpenClaw × nine‑model sweep and a five‑claw × two‑model sweep, model choice changes Pass@1 by 29.4 pp and harness choice by 27.4 pp under fixed models; systems with similar accuracy can differ substantially in total API cost. Claw‑SWE‑Bench therefore treats harness and cost accounting as first‑class axes of SWE‑style coding‑agent evaluation, providing both a full benchmark and a low‑cost reference set for reproducible comparison. The data is available at https://github.com/opensquilla/claw‑swe‑bench and https://huggingface.co/datasets/TokenRhythm/Claw‑SWE‑Bench.
Authors:Gege Gao, Bernhard Schölkopf, Andreas Geiger
Abstract:
Memory is not merely the storage of data; it is the scaffolding of reality. When biological memory fades, the world does not simply turn black; it regresses into an unrecognizable chaos. Echoes of the Prior is an interactive installation that attempts to visualize this subjective phenomenology of forgetting. By inducing controlled synaptic decay within a Feed‑Forward 3D Reconstruction model, we create an artistic analogy for the erosion of the brain's predictive priors. We position the Neural Network not as a tool for engineering, but as a cognitive proxy ‑ a silicon brain whose structural degeneration evokes the disorienting, poetic, and terrifying experience of losing one's grip on the world. Ultimately, we offer this framework as a catalyst, inviting the wider community to explore the uncharted potential of neuromorphic aesthetics in visualizing the fragility of intelligence. Interactive demo see https://decart‑4d.github.io/.
Authors:Jiang Wang, Runwu Shi, Yaozhong Kang, Benjamin Yen, Takeshi Ashizawa, Kazuhiro Nakadai
Abstract:
Sound source distance estimation (SDE) is a critical capability in human‑robot interaction. An inappropriate interaction distance not only reduces the reliability of speech acquisition and understanding, but also compromises the naturalness and comfort of the interaction. Most existing SDE methods rely on microphone arrays, however, multi‑microphone systems typically require careful hardware synchronization, geometric calibration, and additional space and computational resources, which limits applicability to size‑constrained and computability‑limited embodied platforms. To alleviate these issues, we propose Fast‑SDE, a lightweight single‑microphone SDE framework that is suited for deployment on robot platforms with limited computational resources and strict size constraints. Specifically, Fast‑SDE employs a subband‑based backbone that decomposes the frequency axis into multiple subbands, rather than processing the entire spectrum with a wide full‑band backbone. A shared subband encoder then maps each subband to a compact latent representation and learns the relationship between acoustic structure and time‑frequency patterns. Finally, a lightweight regression head converts the fused subband representations into the estimated distance. Extensive simulation and real‑world experiments demonstrate the merits of the proposed method. To benefit the broader research community, we have open‑sourced our code at https://github.com/JiangWAV/FAST‑SDE.
Authors:Balázs Gyenes, Emiliyan Gospodinov, Jan Frieling, Enrico Krohmer, Nicolas Schreiber, Xiaogang Jia, Niklas Freymuth, Gerhard Neumann
Abstract:
High‑precision robotic manipulation requires fine‑grained spatial reasoning that is often difficult to achieve with RGB‑only policies due to depth ambiguity and perspective scale issues. Policies that leverage 3D information directly, such as those based on point clouds, offer a stronger geometric prior over purely image‑based ones, yet their performance remains highly task‑dependent. We hypothesize that this discrepancy may be due to the spectral bias of neural networks towards learning low frequency functions, which especially affects architectures conditioned on slow‑moving Cartesian features. We thus propose to map point clouds from Cartesian space into high‑dimensional Fourier space, effectively equipping the point cloud encoder with direct access to high‑frequency features. We experimentally validate the use of Fourier features on challenging manipulation tasks from the RoboCasa and ManiSkill3 benchmarks and on a real robot setup. Despite their simplicity, we find that Fourier features provide significant benefits across diverse encoder architectures and benchmarks and are robust across hyperparameters. Our results indicate that Fourier features let policies leverage geometric details more effectively than Cartesian features, showing their potential as a general‑purpose tool for point cloud‑based imitation learning. We provide source code and videos on our project page: https://fourier‑il.github.io/fourier‑il
Authors:Ripon Chandra Malo, Tong Qiu
Abstract:
AI coding assistants now support a growing share of software work, from quick scripts to production applications. Yet these agents remain largely stateless: each new session re‑reads project files, re‑derives prior decisions, and ‑ most costly ‑ may repeat debugging attempts that already failed. Reconstructing this context can consume an estimated 5,000‑20,000 tokens per session; the bottleneck is often not model capability but missing project memory. We present projectmem, an open‑source, local‑first memory and judgment layer for AI coding agents. projectmem records development as an append‑only, plain‑text event log of typed events ‑ issues, attempts, fixes, decisions, and notes ‑ and deterministically projects that log into compact, AI‑readable summaries served through the Model Context Protocol (MCP). Beyond storage, projectmem adds a deterministic pre‑action gate that warns an agent before it repeats a previously failed fix or edits a known‑fragile file. We frame this as Memory‑as‑Governance: memory that does not merely answer the agent but acts on its next action. The system runs fully offline with no telemetry; its immutable log also serves as a provenance trail for reproducible, auditable AI‑assisted development. projectmem ships as a three‑dependency Python package (14 MCP tools, 19 CLI commands, 37 automated tests) and is evaluated through a two‑month self‑study across 10 projects comprising 207 logged events. Source code: https://github.com/riponcm/projectmem.
Authors:Yuchen Xian, Yunqiu Xu, Yang He, Yi Yang
Abstract:
Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image‑level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non‑local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi‑metric improvements in both global coherence and local fidelity. Project page: https://zju‑xyc.github.io/1D‑Fusion‑Project‑Page/
Authors:Sukmin Seo, Geewook Kim
Abstract:
Temporal grounding‑‑returning the interval [t_s, t_e] for a natural‑language query over a video‑‑is the language interface to long‑form video, yet has been studied on short videos; the dynamics of hour‑scale natural‑language grounding remain underexplored. We take the position that at hour‑scale, the binding constraint is search, not recognition: Video‑LLMs are bottlenecked not by localizing a nearby event, but‑‑given a natural‑language query‑‑by searching for the relevant region of a long video. To test this, we release ExtremeWhenBench, the first open hour‑scale grounding benchmark (2,273 queries over 194 videos, mean 75.7 min, max 9 hr) with an open‑form query distribution. Every open Video‑LLM collapses while a frame‑level retrieval baseline outperforms them; a failure taxonomy attributes 85% of failures to search; and a retrieve‑then‑ground hybrid recovers 6.7x over the monolithic Video‑LLM‑‑mirroring retrieve‑then‑read in open‑domain QA.
Authors:Alexander Martin, Dengjia Zhang, Joel Brogan, Francis Ferraro, Jeremy Gwinnup, Reno Kriz, Teng Long, Kenton Murray, Andrew Yates, Xiang Xiang
Abstract:
This overview paper presents the results of the shared task for the second workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR). In this shared task participants submitted systems focused on either (i) video retrieval or (ii) grounded generation of articles given retrieved videos. Teams could submit to either task. For the retrieval task, we had 2 participating teams that submitted a total of 17 systems ‑‑ all of which beat a baseline derived from the winner of last year's shared task. On the generation side, we had 4 teams submit 16 systems. All teams had at least one generated report that was labeled the best by a human annotator.
Authors:Benjamin Eckhardt, Dmytro Fishman, Stuart Fawke, Andrew Curtis, Bo Fussing, Constantin Pape
Abstract:
Counting living cells is an important step in many biological research workflows. Our collaborators at the Wellcome Sanger Institute study vital genes in humans via large scale saturation genome editing screening, which requires repeatedly counting cells a great number of times. Computer Vision based automation is crucial for high throughput and resource efficiency. In this work, we develop a regression‑based deep learning computer vision algorithm to detect and count cells in phase‑contrast microscopy images. To reduce annotation effort, which in practice often becomes a bottleneck, we focus on counting cells only using sparse point annotations, which are fast and easy to acquire. By comparison to state‑of‑the‑art 0‑shot methods, we show that regression‑based counting is a promising alternative in low data regimes. Through developing methods to automatically count living cells in microscopy images, we contribute to valuable research on the human genome. The code is available at https://github.com/beijn/cellnet.
Authors:Dmitrii Gavrilev
Abstract:
Expressive performance rendering (EPR) aims to generate realistic performances constrained on sequences of notes. However, flow matching audio editing models manipulate only synchronized music samples of the same duration, limiting their understanding of expressive timing. We introduce PianoKontext, a flow matching rendering model for classical piano music that generates variable‑length performances in the latent space of a pretrained Music2Latent model. We synthesize MIDI scores into deadpan audio and employ Dynamic Time Warping (DTW) in the latent space to construct paired data for training. The aligned embeddings are concatenated in DiT blocks, allowing for a simple and effective learning of the dependencies between the score and performances. Audio samples are available at our demo page: https://realfolkcode.github.io/pianokontext_demo/.
Authors:Jinyuan Zu, Xiaowei Lv, Yongcai Wang, Deying Li, Yunjun Han, Wenping Chen, Fengyi Zhang, Naiqi Wu
Abstract:
In Decentralized Training and Decentralized Execution (DTDE) for cooperative Multi‑Agent Reinforcement Learning (MARL), action‑advising‑based knowledge sharing promotes interpretable and scalable cooperation among agents. However, current action advising approaches often adhere too much to the teacher's guidance without evaluating teacher‑student compatibility, which causes excessive advising, suboptimal stability, and degraded performance. To overcome these challenges, this paper presents a Consensus‑based Communication and Knowledge Sharing (CCKS) framework, which allows agents to adopt recommendations based on consensus‑derived constraints and to follow the teacher's instructions more smartly. This mechanism enables agents to balance exploration and learning from experienced teachers, improving overall performance. The key is the consensus model construction, for which we propose to employ contrastive learning to construct consensus models based on local observations in the agents' training phase. In action selection, agents score and choose actions based on consensus and shared knowledge. Designed as a plug‑and‑play solution, CCKS integrates seamlessly with existing DTDE algorithms. Experiments conducted in the Google Research Football environment and the complex StarCraft II Multi‑Agent Challenge demonstrate that the integration with CCKS significantly improves cooperation efficiency, learning speed, and overall performance compared with current DTDE baselines. The code is available at https://github.com/yuanxpy/CCKS.
Authors:Yuchen Xian, Yang He, Yunqiu Xu, Yi Yang
Abstract:
Speculative decoding (SD) addresses the high inference costs of LLMs by having lightweight drafters generate candidates for large verifiers to validate in parallel. Existing draft‑verify methods use binary decisions: accept or fully recompute. Yet we find that many rejected tokens can be verified correctly by a slim submodel derived from the full verifier via intra‑model routing, instead of the full verifier. This motivates our slim‑verifier to handle tokens requiring moderate verification resources, reducing expensive large‑model calls. We propose Verification via Intra‑Model Routing for Speculative Decoding (VIA‑SD), a multi‑tier framework using a routed slim‑verifier. Draft tokens are processed hierarchically: direct acceptance for high‑confidence cases, slim‑verifier regeneration for medium‑confidence cases, and full‑model verification for uncertain cases. Across four representative tasks and multiple model families, VIA‑SD reduces rejection rates by 0.10‑0.22 and delivers 10‑20% speedups over strong SD baselines, while achieving 2.5‑3x acceleration over non‑drafting decoding. Moreover, VIA‑SD is compatible with existing SD frameworks without modifying their training procedures. Our results suggest multi‑tier SD as a general paradigm for scalable and efficient LLM inference. Project page: https://zju‑xyc.github.io/VIA‑SD‑Project‑Page/
Authors:Ali M Karaoglu, Shreyank N Gowda
Abstract:
Can financial news reliably predict short‑term stock movements? Despite advances in large language models, this question remains unresolved. We revisit this problem using a zero‑shot natural language processing framework, investigating whether models can extract actionable signals from financial news without domain‑specific training. We design a structured pipeline that combines zero‑shot natural language inference with temporal aggregation, explicitly modelling recency and event‑dependent impact horizons when integrating information across articles. To address the need for transparency in high‑stakes settings, we introduce a multi‑layered explainability framework that links predictions to token‑level, article‑level, and aggregate evidence, and produces grounded natural language rationales. Across multiple models and prediction horizons, we find that zero‑shot approaches consistently fail to outperform simple baselines, with particularly weak performance on negative movements, suggesting deeper structural limitations in mapping news sentiment to short‑term price dynamics. However, explainability signals reliably distinguish between trustworthy and unreliable predictions, offering practical value even when accuracy is limited. These findings highlight the limits of zero‑shot financial NLP and motivate a shift toward decision‑support systems that prioritise transparency and uncertainty awareness. Code: https://github.com/alimert05/zero‑shot‑stock‑xai
Authors:Changyue Wang, Weihang Su, Qingyao Ai, Yichen Tang, Runzhong Qiao, Xuancheng Li, Min Zhang, Yiqun Liu
Abstract:
Large language models (LLMs) are widely used to tackle complex tasks with autonomous workflows. Recently, reusable natural language skills have emerged as a popular paradigm to inject procedural knowledge into LLM applications. Since popular skills are often invoked repeatedly, placing their full text in every context significantly increases prefill cost and latency. While text compression techniques have the potential to solve this problem, most existing methods are designed to compress factual knowledge in documents instead of procedural knowledge, making them insufficient for skill compression. In this paper, we argue that an effective skill compression method should: 1) preserve logical dependencies among workflows and tool protocols, 2) enable lightweight, offline compression for frequently updated community skills, and 3) be adaptable to varying complexities across skills. To address this, we present SKIM (SKIll coMpression), an adaptive multi‑resolution soft token compression framework for procedural skills. Depending on the complexity of each skill, SKIM creates different numbers of soft tokens that not only improve the efficiency of LLM inference, but also preserve the effectiveness of skill usage. Experiments indicate that SKIM compresses skills to 30 to 60 percent of their original token length while preserving task performance better than existing compression methods.We have released our code at https://github.com/bebr2/SKIM .
Authors:Weirong Chen, Keisuke Tateno, Hidenobu Matsuki, Michael Niemeyer, Daniel Cremers, Federico Tombari
Abstract:
We address 4D reconstruction from partial point cloud sequences, where depth‑sensor observations are incomplete, unordered, and lack explicit temporal correspondences. This geometry‑only setting is challenging due to missing observations and ambiguous dynamics. While recent progress has largely relied on image‑based methods, existing point‑based approaches typically focus on single objects, assume relatively complete inputs, or require explicit correspondences. To address these limitations, we propose DynaTok, a point‑based framework for correspondence‑free 4D reconstruction from partial point cloud sequences without images. DynaTok encodes frames into compact latent tokens, aggregates incomplete observations over time with a Transformer‑based spatiotemporal encoder, and decouples geometry and motion through residual tokens in a unified model. A flow‑matching decoder then reconstructs complete, temporally consistent 4D point‑cloud sequences conditioned on the latent tokens. Experiments on object‑ and scene‑level benchmarks demonstrate improved reconstruction quality and temporal coherence from partial point cloud observations. Project page: https://wrchen530.github.io/dynatok/.
Authors:Jiawei Niu, Jian Chen, Di Zhang, Junbo Lu, Zhangcheng Liao, Xuhao Liu, Honglin Zhong, Mireia Crispin-Ortuzar, Chen Li, Zeyu Gao, Yi Cai
Abstract:
Existing computational pathology methods predominantly operate within whole‑slide image (WSI)‑level multiple instance learning (MIL) paradigms, while patient‑level modeling remains underexplored. In routine pathological practice, however, pathologists derive diagnostic and prognostic conclusions by integrating evidence across multiple WSIs rather than relying on any single slide. This discrepancy creates a fundamental misalignment when patient‑level supervision is directly imposed on conventional MIL frameworks, often leading to unstable optimization and degraded predictive reliability. To address this issue, we propose Anchor‑Guided Evidence MIL (AGE‑MIL), a weakly supervised framework for patient‑level prediction. AGE‑MIL constructs a patient‑level anchor from slide representations to capture global pathological context and guide the retrieval and integration of diagnostically relevant local patches, enabling robust patient‑level modeling. Patient‑level risk is further modeled as an evidence accumulation process, promoting stable optimization under weak supervision. AGE‑MIL is evaluated on six clinically relevant patient‑level prediction tasks from two independent cohorts. Experimental results show that the proposed framework consistently outperforms eight state‑of‑the‑art MIL methods. Code is available at https://github.com/wodeniua/AGE‑MIL.
Authors:Pankhuri Vanjani, Zhuoyue Li, Jakub Suliga, Moritz Reuss, Gianluca Geraci, Xinkai Jiang, Rudolf Lioutikov
Abstract:
Vision‑language‑action (VLA) models inherit a shared synchronous clock from vision‑language pretraining, processing every input at one rate. This is misaligned with physical interaction, where a high‑frequency modality changes at hundreds of hertz, vision evolves more slowly, and language stays constant across an episode. A synchronous VLA oversamples slow modalities, undersamples fast ones, and caps action generation at the lowest effective frequency. We hypothesize that decoupling temporal processing per modality, letting each update and retain information at its own sensor rate, yields stronger representations and more robust control. We present DAM‑VLA, which maintains per‑modality latent buffers refreshed at sensor rates and read continuously by the action head, integrating new high‑frequency modalities through gated cross‑attention that leaves the pretrained backbone intact. Across seven contact‑rich real‑world manipulation tasks, DAM‑VLA more than doubles the average success rate of the strongest synchronous baseline (95.2% vs.\ 40.95%) while sustaining smooth, reactive 100\,Hz control. Project website: \hrefhttps://intuitive‑robots.github.io/DAM‑VLA/intuitive‑robots.github.io/DAM‑VLA/
Authors:Jia Deng, Yimeng Chen, Xiaoqing Xiang, Ziyang Zeng, Shuo Tang, Wayne Xin Zhao, Feng Chang, Chuan Hao, Yuan Wei, Ran Tao, Bryan Dai, Ji-Rong Wen
Abstract:
Training deep search agents requires verifiable questions whose answers remain unavailable until sufficient evidence has been acquired through search. Existing synthesis methods often increase apparent difficulty by enriching graph structures, but structural complexity alone does not guarantee realized search difficulty: the intended search process can collapse through a cheaper identifying route. We formalize this gap with a shortcut‑aware difficulty framework and identify four actionable shortcut risks: evidence co‑coverage, single‑clue selectivity, exposed constants, and prior‑knowledge binding. To diagnose their realized effects, we use trajectory signatures including solving cost, answer hit time, and prior‑shortcut rate. Guided by this framework, we introduce FORT, a Framework of Shortcut‑Resistant Training‑Data Synthesis. FORT constructs shortcut‑resistant training data by controlling shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. Experiments show that FORT induces longer pre‑answer search and fewer shortcut patterns than existing open‑source deep search datasets. Using the resulting trajectories, we train FORT‑Searcher with supervised fine‑tuning (SFT) only, and it achieves the best overall performance among comparable‑size open‑source search agents on challenging deep search benchmarks. Relevant resources will be made available at https://github.com/RUCAIBox/FORT‑Searcher.
Authors:Balázs Gyenes, Nikolai Franke, Paul Maria Scheikl, Pit Henrich, Rayan Younis, Gerhard Neumann, Martin Wagner, Franziska Mathis-Ullrich
Abstract:
High‑risk applications in robotics, such as robot‑assisted surgery, present unique challenges. These systems must be both highly precise and interpretable in order to be deployed in environments with very low tolerance for error or unsafe exploration. We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery, one of the most common interventions in general surgery. After segmentation of a colorless point cloud from a single camera, target positions for the clips are extracted using spline interpolation, and can then be adjusted by the human operator. The segmentation model is trained on only 60 hand‑labeled real point clouds, reflecting data scarcity in the surgical domain. We overcome this with a combination of pre‑training on 128,000 synthetic point clouds and two novel data augmentation techniques. The motion of the end‑effector to each target is visualized for the operator, satisfying the unique motion constraints of minimally‑invasive surgery while ensuring that the robot's actions are verifiable and interpretable. In real robot experiments, our system localizes targets with the required precision of 0.75mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate. We provide insights that are applicable to many other surgical and non‑surgical tasks that require identifying and navigating to a precise target. Source code and project page: https://github.com/balazsgyenes/kirurc
Authors:Wanting Wang, Xiye Ma, Yuyang He, Minghui Cheng, Ran Cao
Abstract:
The design of reinforced concrete highway barriers is a safety‑critical process that requires strict compliance with regulatory provisions such as the AASHTO‑LRFD bridge design guidelines. Current engineering practice relies heavily on manual, iterative, and heuristic calculations to satisfy complex nonlinear material and mechanics constraints. Although Large Language Models (LLMs) demonstrate strong generative capabilities, their direct application to structural engineering remains limited by hallucination risks and insufficient physical grounding. To address these challenges, this study proposes a novel "generation‑evaluation‑optimization" closed‑loop framework for automated concrete barrier design using the multi‑agent orchestration capabilities of AutoGen. Experimental results demonstrate that the proposed agentic framework achieves over 98% design accuracy, significantly outperforming standalone general‑purpose LLMs. More importantly, the study reveals that design performance is not necessarily correlated with model scale, where an 8B‑parameter lightweight model could outperform unconstrained 631B‑parameter flagship models. This finding highlights the potential to substantially reduce computational costs while improving the accessibility of AI‑assisted engineering tools for industry applications. The source code for the proposed multi‑agent design framework is available at the project GitHub repository: https://github.com/MXY820/barrier‑design. Keywords: Structural Engineering; Multi‑Agent Systems; Large Language Models; Concrete Barrier Design; AutoGen; Design Automation.
Authors:Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira, Naser Damer, Fadi Boutros
Abstract:
Recent advances in Vision Transformers (ViTs) for face recognition (FR) have moved beyond the standard CLS‑token paradigm. In this paradigm, a special classification token (CLS) is prepended to the patch embeddings and used as a representation of the input for downstream tasks. An alternative approach, Concatenated Patch Embeddings (CPE), instead leverages all patch tokens by concatenating them into a single vector, which is then projected into a compact face representation. CPE has been shown to improve recognition performance in comparison to CLS‑based ones, but our qualitative analysis of attention maps showed the presence of artifacts that limit their interpretability. To address this issue, we incorporate register tokens, learnable tokens concatenated to the initial patch embeddings, and processed jointly through the ViT encoder blocks. This mechanism has been shown to produce more structured and interpretable attention maps compared to baseline ViT. We empirically demonstrate that these artifacts consistently appear across various ViT backbones, including small and large models, and that introducing register tokens effectively mitigates them. Adding four or eight registers significantly enhances interpretability, with eight registers providing the highest verification accuracies and smoothest attention structures. Our resulting model, ViT‑8R, corresponds to a CPE‑based ViT‑B architecture augmented with eight register tokens achieves state‑of‑the‑art performance among ViT‑based FR models on large‑scale IJB‑B and IJB‑C benchmarks. Also, ViT‑8R produces substantially clearer attention maps compared with the baseline model, which offer deeper insight into the model's attention behavior (https://github.com/TaharChettaoui/ViT‑FR‑Registers)
Authors:Min Yang, Mi Zhou, Limin Wang
Abstract:
Video understanding is a crucial part of computer vision, with numerous application scenarios. With the increasing popularity of mobile devices, an increasing number of efforts are trying to deploy video understanding models on them. However, existing video understanding models are difficult to deploy due to their large size and prohibitive power consumption. Spiking Neural Networks (SNNs) have shown bioplausibility and low power advantages over Artificial Neural Networks (ANNs), especially on neuromorphic chips which are regarded as essential components of future mobile devices. However, excessively long conversion time‑steps and severe performance degradation problems limit their application. To solve the problems above, we explore the application of SNNs on temporal action detection (TAD), which is an important task in video understanding, and propose the first SNN‑based end‑to‑end TAD architecture coined as SpikeTAD. While maintaining extremely low power consumption, SpikeTAD achieves an average mAP of 67.2% in THUMOS14 and 37.42% in ActivityNet‑1.3, demonstrating the feasibility of a low‑power TAD model. Our code is available at https://github.com/MCG‑NJU/SpikeTAD.
Authors:Song Chen, Linyan Xiang, Ying Zhou, Liu Yang
Abstract:
Generalizable robot manipulation requires not only task‑level reasoning over unseen scenes, but also reliable grounding of visual plans into embodiment‑specific execution. To bridge this gap, we propose VICX (Video generation and In‑Context eXecution), a decoupled closed‑loop manipulation framework. In VICX, a frozen video generation model produces vision‑language‑conditioned high‑level visual plans, while a Video‑to‑Trajectory In‑Context Operator Network (V2T‑ICON) serves as the task‑agnostic interface that grounds these plans into executable robot‑state trajectories. To improve execution generalization, V2T‑ICON operates on segmentation‑extracted arm‑only frame observations and uses retrieved image‑state pairs as in‑context prompts, allowing a robust and generalizable visual‑to‑state mapping at inference time without parameter updates. Experiments on Meta‑World show that VICX supports cross‑task generalization, closed‑loop self‑correction, and cross‑embodiment transfer, demonstrating dual generalization across both task semantics and robot execution. The project webpage can be found here: https://scaling‑group.github.io/vicx/.
Authors:Quankai Wang, Yulin Xie, Tongfei Yang, Minghui Cheng, Ran Cao
Abstract:
Finite element (FE) modeling of safety‑critical infrastructure such as bridge barriers requires high‑fidelity nonlinear dynamic analysis, yet the current FE modeling process remains labor‑intensive and lacks automation. This paper presents the Human‑Enhanced Loop Modeling (HELM) framework, a collaborative human‑agent protocol that decomposes long‑sequence finite element modeling into discrete, visually verifiable checkpoints across geometry generation, boundary condition definition, and material assignment. The framework is demonstrated through a 20‑case matrix of reinforced concrete bridge barriers under MASH TL‑4 and TL‑5 lateral loading conditions, interfacing specialized agents with two widely used commercial FE softwares, i.e., ANSYS and LS‑PrePost. Experimental results show that HELM improves the baseline autonomous modeling success rate from 20% to 75%, with agent‑level pass rates for geometry and boundary condition tasks approximately doubling. Error analysis reveals that spatial reasoning and algebraic logic limitations constitute the primary failure modes, underscoring the value of structured human‑in‑the‑loop intervention for modeling automation. The complete agent design code and prompts are open‑sourced and can be accessed at: https://github.com/SimAgentDev/Ansys‑LSPP‑AgentKit.
Authors:Shang Ma, Jisheng Dang, Wencan Zhang, Yifan Zhang, Bimei Wang, Hong Peng, Bin Hu, Qi Tian, Tat-Seng Chua
Abstract:
We propose a multi‑agent collaborative framework built upon a lightweight Multimodal Large Language Model (MLLM), specifically designed for social intelligence reasoning. A key feature of our approach is that both the training and inference phases are augmented via knowledge distillation. Within this architecture, multi‑modal data pertinent to social intelligence is precisely localized. Furthermore, relevant long‑tail events are identified, extracted, and rendered as formatted, explicit text. This formatting strategy prevents critical long‑tail information from being overshadowed by head events and environmental noise during the tokenization process. Specifically, we integrate Test‑Time Adaptation (TTA) across the entire reasoning pipeline, encompassing the extraction and representation of long‑tail events, Chain‑of‑Thought (CoT) prompting, and self‑reflection. This TTA mechanism is also distillation‑enhanced, utilizing Low‑Rank Adaptation (LoRA) to fine‑tune the foundation model exclusively for instance‑level reasoning. Extensive evaluations against various open‑source and proprietary AI models across multiple benchmarks demonstrate the effectiveness of the proposed framework. With around 30% of training data from IntentTrain, we achieve state‑of‑the‑art results. Codes are available at https://github.com/eeee‑sys/MODF‑SIR, demo is available at https://huggingface.co/spaces/Harry‑1234/MODF‑SIR, LoRA is available at https://huggingface.co/Harry‑1234/MODF‑SIR and the dataset for training router is available at https://huggingface.co/datasets/Harry‑1234/IntentRouterTrain.
Authors:Yiqun Ning, Ao Shen, Chenhang He, Lei Zhang
Abstract:
While diffusion‑based virtual try‑on has achieved impressive visual realism, most methods treat the task as 2D inpainting, prioritizing texture preservation over physical plausibility. Consequently, they often produce plausible‑looking images that fail to reflect authentic garment fit across diverse body shapes. We present FitVTON, a Fit‑aware virtual try‑on model on different bodies in the wild. FitVTON encodes garment‑body size through structured text prompts, and learn from simulated try‑on triplets from parameterized garment model. To improve the fitting effects over garment silhouettes, we introduce two auxiliary head to predict the masks for both the garment and the exposed body. We further introduce a texture rectification stage to improve realistic appearance from simulated data. To evaluate the fitting fidelity, we curate a real‑world dataset, FittingEffect3K, combining VLM‑based scoring protocol. Both subjective and quantitive experiments show that FitVTON demonstrate authentic fitting fidelity, with significant sizing accuracy and shape preservation over state‑of‑the‑art methods while maintaining competitive image quality. Project Page: https://zenoning.github.io/FitVTON/.
Authors:LeKai Yu, Hao Liu, Kun Wang, Zhiran Li, Ruping Cao, Fan Liu, Yupeng Hu
Abstract:
In this report, we present our third‑place solution for the DataMFM Challenge Track 1: Document Parsing. This track requires models to recover structured Markdown documents from document page images while preserving textual content and document structure. To address the complementary requirements of accurate content recovery and faithful structure reconstruction, we propose ParseFixer, an agentic framework for backbone parsing and selective correction. ParseFixer consists of two key modules: Full‑Page Backbone Parsing (FBP) and Agentic Selective Correction (ASC). FBP produces stable initial Markdown outputs with MinerU2.5 Pro, while ASC detects high‑value parsing failures and repairs them through a verify‑and‑rollback correction process. By placing selective multimodal correction after open‑source backbone parsing, ParseFixer improves the recovery of key document elements without rewriting reliable backbone predictions. On the test set, our final system achieves an overall score of 61.78 and ranks third in Track 1, demonstrating its effectiveness for accurate document parsing. Our code will be released at: https://github.com/iLearn‑Lab/CVPRW26‑ParseFixer.
Authors:Mostafa Bamdad, Mohammad Sadegh Eshaghi, Timon Rabczuk
Abstract:
Neural operators provide a powerful framework for learning solution mappings of partial differential equations directly in function space. However, many existing architectures still struggle to represent nonlinear time‑dependent systems that involve multi‑scale structures, long‑range interactions, and stable long‑time evolution. In this work, we introduce the Hierarchical Adaptive Multi‑scale Neural Operator (HAMNO), a neural‑operator architecture that combines local convolutional representations, global spectral operators, and hierarchical encoder‑decoder processing. The central component of HAMNO is a data‑dependent gating mechanism that adaptively balances local and global information at each spatial location, allowing the model to resolve fine‑scale features while preserving long‑range dependencies. We further develop a physics‑informed extension, PI‑HAMNO, based on a multi‑objective loss strategy that combines data fitting with strong‑ and weak‑form physics constraints. The strong‑form term penalizes the domain‑integrated squared PDE residual in physical coordinates, while the weak‑form term is constructed by multiplying the governing residual by finite‑element test functions and evaluating the resulting element integrals using centroid‑based tetrahedral quadrature. The framework is evaluated on non‑periodic Allen‑Cahn (AC), Cahn‑Hilliard (CH), and Swift‑Hohenberg (SH) equations defined on cubic domains. Across long‑horizon rollout, data‑limited training, out‑of‑distribution initial‑condition shifts, and random‑seed variations, HAMNO improves predictive accuracy over standard neural‑operator baselines, while PI‑HAMNO further enhances stability, physical consistency, and data efficiency. The implementation is publicly available at https://github.com/MBamdad/HAMNO .
Authors:Jun Wen Leong
Abstract:
We present an online monitoring system for distributional shift in deployed safety classifiers, using calibrated sequential statistics to detect when a classifier has moved out of distribution. Upon detection, a conformal abstention layer adapts decision thresholds to recover a target error rate epsilon=0.1. In a pre‑registered factorial evaluation (4 classifiers x 5 shift conditions x 20 seeds x 2 window sizes, 800 cells), the system achieves 86.6% valid detection (693/800, 95% CI [84.1%, 88.8%]) with mean latency of 39.5 steps. Detection holds across three ground‑truth regimes: synthetic onset (86.6%), real temporal jailbreaks (85%, 17/20), and GCG adversarial attacks. Weighted conformal prediction recovers up to 39 pp of lost coverage for DeBERTa (ESS=46/300) but collapses for all other classifiers (ESS~300): logistic density ratio estimation achieves perfect source/target separability in high‑dimensional embedding spaces, clipping all importance weights to the floor. DeBERTa shows a gradient from effective correction (paraphrase, ESS=46) to near‑total collapse (adversarial suffix, ESS=206). PCA to 32 dimensions breaks the collapse, recovering 33 pp for Llama Guard and 21 pp for ShieldGemma. Variance decomposition reveals classifier (eta^2=0.243), shift type (eta^2=0.237), and their interaction (eta^2=0.185) all contribute substantially to detection latency variance (all p<0.001), indicating per‑classifier monitoring profiles are necessary.
Authors:Zsolt Robotka, Ádám Rák, Jalal Al-Afandi, András Horváth, György Cserey
Abstract:
Sign language translation (SLT) converts sign language video into spoken language text and holds significant promise for improving accessibility and enabling communication between signing and non‑signing communities. While large weakly‑aligned datasets have enabled pre‑training at scale and gloss‑free methods have reduced reliance on expert annotation, high‑quality parallel sign video‑text pairs for fine‑tuning remain scarce, limiting generalisation on long‑tail vocabulary and unseen constructions. We propose a corpus augmentation approach that requires no additional human annotation, external sign‑language video corpora, or generative video models, relying only on the existing gloss‑annotated training corpus and an LLM for sentence generation: per‑gloss clips are extracted from training videos via CTC forced‑alignment, novel gloss‑sentence pairs are generated by a corpus‑anchored LLM, and synthetic sequences are assembled through random sentence sampling and clip assignment. The resulting synthetic RGB video‑text pairs are architecture‑agnostic at the downstream training stage and can be consumed directly by RGB‑based SLT models, or converted into pose or feature representations by pipelines that derive such inputs from video. Sincan et al. re‑evaluated five recent gloss‑free methods under strictly identical conditions; the largest verified gain over the GFSLT‑VLP baseline was only 0.98 BLEU‑4. Our augmentation, applied within the same framework, achieves +2.92 BLEU‑4 without any change to architecture or training protocol. We further identify that synthetic data harms vision‑language pretraining despite improving its objectives, and that optimising clip transitions for visual smoothness is counter‑productive under L2‑based criteria; we propose that abrupt boundaries may act as a form of implicit regularisation. Code is available at https://github.com/robizso/slt‑datagen.
Authors:Hemansh Shridhar, Miika Toikkanen, June-Woo Kim
Abstract:
Recent respiratory sound classification (RSC) studies largely rely on CLS‑token driven self‑attention architectures such as the Audio Spectrogram Transformer (AST). While effective at modeling global context, recent analyses suggest a low‑pass filtering behavior that may reduce sensitivity to localized abnormal patterns. In this work, we investigate State Space Models (SSMs) as an alternative backbone for RSC. Using the Distilled Audio State Space model, we analyze intermediate representations through spectral response curves and observe stronger preservation of mid‑to‑high spatial‑frequency components. Based on these observations, we introduce spectral‑aware layer regularization using Gaussian convolution applied to selected layers. We further propose Dual‑Axis Patch‑Mix contrastive learning tailored to SSM‑based audio models for robust representation learning. Experiments on the ICBHI benchmark show that our approach achieves 64.48% score, outperforming the AST baseline by 5%. Code is available at https://github.com/RSC‑Toolkit/Lung‑SRAD.
Authors:Yoon Tae Kim, Heejoon Koo, Miika Toikkanen, June-Woo Kim
Abstract:
We present a quality‑adaptive angular‑margin learning framework that improves feature generalization by enforcing intra‑class compactness and inter‑class separability. Our framework, titled QLung, introduces a no‑reference audio quality margin derived from spectral entropy and root‑mean‑square energy, which adaptively scales angular margins based on recording quality. To this end, we propose a log‑scaled angular margin that stabilizes training under severe class imbalance. We also use an angular classifier that normalizes features and class weights, ensuring margin penalties are applied consistently on the unit hypersphere. Our approach improves in‑distribution performance on the ICBHI dataset by 2.46% over the cross‑entropy baseline, and most significantly, achieves the strongest out‑of‑distribution performance on the SPRSound dataset compared to prior state‑of‑the‑art methods. Code is available at https://github.com/RSC‑Toolkit/QLung.
Authors:Hengyi Feng, Zeang Sheng, Meiyi Qiang, Li Yang, Wentao Zhang
Abstract:
Research on Text‑Attributed Graphs (TAGs) has gained significant attention recently due to its broad applications across various real‑world data scenarios, such as citation networks, e‑commerce platforms, social media, and web pages. Inspired by the remarkable semantic understanding ability of Large Language Models (LLMs), there have been numerous attempts to integrate LLMs into TAGs. However, existing methods still struggle to generalize across diverse graphs and tasks, and their ability to capture transferable graph structural patterns remains limited. To address this, we introduce the GraspLLM, a framework that combines Graph structural comprehension with semantic understanding prowess of LLMs to enhance the cross‑dataset and cross‑task generalizability. Specifically, we represent node texts from different graphs in a unified semantic space with a frozen general embedding model, on top of which we perform motif‑aware contrastive learning across multiple motif‑induced adjacency matrices to extract dataset‑agnostic structural information. Then, with our proposed optimal contextual subgraph, we extract the most contextually relevant subgraph for each target node and align these subgraphs to the token space of LLM via an alignment projector. Extensive experiments on TAG benchmark datasets spanning diverse domains reveal that GraspLLM consistently outperforms previous LLM‑based methods for TAGs, especially in zero‑shot scenarios, highlighting its strong generalizability across different datasets and tasks. Our code is available at https://github.com/Heinz217/GraspLLM.
Authors:Yuto Furutani, Takashi Otonari, Kaede Shiohara, Toshihiko Yamasaki
Abstract:
Feed‑forward 3D Gaussian Splatting (3DGS) removes the need for time‑consuming per‑scene optimization required by traditional 3DGS. However, existing feed‑forward approaches struggle with real‑world photo collections that include diverse lighting conditions and transient objects. In this paper, we present Wild3R, a feed‑forward approach for unconstrained sparse photo collections. The main bottleneck is the lack of training data that provides multiple viewpoints, a variety of illuminations, and transient variations necessary for learning robust scene representations. To address this, we introduce the WildCity dataset, which comprises 200 scenes, 170 lighting conditions, and transient objects, resulting in 337,500 images in total. By leveraging the dataset, our model learns appearance consistency across viewpoints conditioned on reference views, while removing transient content. Extensive experiments demonstrate that our method outperforms existing feed‑forward approaches and achieves results competitive with prior per‑scene optimization‑based methods.
Authors:Nicole Damblon, Olga Vysotska, Federico Tombari, Marc Pollefeys, Daniel Barath
Abstract:
Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per‑patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per‑patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real‑world datasets. The code will be available at https://github.com/DmblnNicole/sg2loc.
Authors:Henok Daniel, Omar Alhussein, Cheng Li, Jie Liang, Ernesto Damiani
Abstract:
The Network Data Analytics Function (NWDAF) is central to enabling zero‑touch network management in fifth‑generation (5G) networks by supporting real‑time analytics and closed‑loop automation. Despite its critical role, open‑source NWDAF implementations remain limited in scope and accessibility. In this paper, we develop an open‑source NWDAF, compatible with the open‑source core network Free5GC, that collects network data via subscriptions to Network Functions (NFs), and also includes an integrated Large Language Model (LLM) interface that enables natural language interaction with human operators. The interface processes user intents, encodes them using a semantic embedding model, and maps them to one of seven predefined intent categories to trigger analytics queries or event subscription commands. This architecture abstracts the complexity of traditional interfaces, allowing non‑expert users to manage network analytics and subscriptions with ease. The system supports Access and Management Function (AMF) and Session Management Function (SMF) event subscriptions, real‑time monitoring, and analytics retrieval via Prometheus, all accessible through a conversational interface. By bridging AI‑driven intent recognition with standardized network analytics, our implementation enhances operator usability and provides a foundation towards AI‑native 6G networks. The source code and datasets generated during the current study are available in the github repository, https://github.com/HenokDanielbfg/testbed.
Authors:Aarchi Singh Thakur, Abhijoy Sarkar
Abstract:
Circulating‑tumour DNA (ctDNA) carries evidence of drug resistance months before imaging shows it, but the earliest evidence lives below the assay's limit of detection (LoD): a nascent subclone is detected only intermittently, producing a flickering sequence of faint detects and non‑detects. Commercial liquid biopsies treat each draw as an independent snapshot and a non‑detect as nothing. We argue a non‑detect is a left‑censored observation, and the pattern of non‑detects and faint detects over time carries actionable evidence of growth before any single value is trustworthy. We introduce Span, a censored‑Poisson Bayesian latent‑growth change‑point detector that models the binary detection process, accumulates a sequential generalised‑likelihood‑ratio statistic for an upward change‑point in the per‑variant detection rate, and raises a competing‑risks alarm with calibrated false‑alarm control. Span has no learned weights, so there is nothing to overfit. On a synthetic cohort of HR+/HER2‑ metastatic breast cancer on first‑line CDK4/6‑inhibitor plus endocrine therapy, at a matched 10% false‑alarm rate, Span roughly doubles the fraction of impending progressions caught three months ahead (indolent regime: 25% vs 11% for the snapshot), with a falsifiable dose‑response: large for indolent emergence, vanishing for fast emergence. A value‑trajectory baseline performs identically to the snapshot, isolating the gain to the censored detection model. The survival backbone matches a Cox baseline on real breast‑cancer data (GBSG‑2, n=686; C‑index 0.67 vs 0.68), and on a real longitudinal cohort with clean biomarkers (PBC2, n=312) the same pipeline correctly declines to win, a falsifiable boundary test confirming the mechanism is regime‑specific. All ctDNA trajectories are synthetic.
Authors:Zi Haur Pang, Yahui Fu, Koji Inoue, Tatsuya Kawahara
Abstract:
Emotional validation ‑ explicitly acknowledging that a user's feelings make sense ‑ has proven therapeutic value but has received little computational attention. Emotional validation in dialogue systems can be decomposed into (i) validating response identification, (ii) validation timing detection, and (iii) validating response generation. To support research on all three subtasks, we release M‑EDESConv, a 120k English‑Japanese multilingual corpus created through hybrid manual and automatic annotation, and M‑TESC, a multilingual spoken‑dialogue test set. For timing detection, we propose MEGUMI, a Multilingual Emotion‑aware Gated Unit for Mutual Integration, that fuses frozen XLM‑RoBERTa semantics with language‑specific emotion encoders via cross‑modal attention and gated fusion. MEGUMI shows superior performance on both the M‑EDESConv and M‑TESC datasets, both objectively and subjectively. Finally, our EmoValidBench benchmarks of GPT‑4.1 Nano and Llama‑3.1 8B indicate that current LLMs generate contextually similar and diverse validating responses, but emotional understanding remains a major area for improvement. Project page: https://github.com/zihaurpang/Multilingual‑Emotional‑Validation
Authors:Dongxin Lyu, Jingbo Zhou, Hongxin Xiang, Yuqiang Li, Jun Xia
Abstract:
De novo peptide sequencing from tandem mass spectrometry is pivotal in proteomics, enabling identification of novel peptides without reference databases. While recent Transformer‑based encoder‑decoder models have achieved remarkable performance, we uncover a critical pathology in their inference dynamics. Through comprehensive feature scaling experiments, we demonstrate that existing auto‑regressive peptide decoders tend to over‑rely on generated‑sequence priors while progressively under‑utilizing fine‑grained physical evidence from the input mass spectrum. This phenomenon leads to suboptimal results, where generated peptide sequences are biologically plausible yet not faithful to the input spectrum. To rectify this, we propose MemNovo, a training‑free and plug‑and‑play mechanism that re‑balances peptide and spectral contributions at inference time. MemNovo alleviates the information bottleneck by establishing a persistent spectral memory bank and injecting retrieved features directly into the final decoding stage via an ultra‑conservative residual connection. Theoretical analysis confirms that this mechanism restores the mutual information between the decoder state and the raw spectrum. Extensive experiments on the Nine Species benchmark with two representative baselines, Casanovo and InstaNovo, demonstrate that MemNovo consistently improves both amino acid precision and peptide precision, achieving up to 39.1% relative improvement in peptide precision for Casanovo and up to 3.9% for InstaNovo, with negligible computational overhead.
Authors:Mingzhe Lyu, Jinqiang Cui, Hong Zhang
Abstract:
Low‑light novel view synthesis is challenging because dark multi‑view images contain noise, weak structural detail, and compressed dynamic range. Recent 3D Gaussian Splatting (3DGS) methods address these challenges by generating pseudo ground‑truth (pseudo‑GT) images as supervision targets when paired normal‑light references are unavailable. Existing pseudo‑GT methods apply a uniform linear gain to all pixels, which clips bright regions while providing insufficient enhancement in dark regions, limiting reconstruction quality. We observe that nonlinear tone mappings, long established in 2D low‑light enhancement, have not been explored for pseudo‑GT generation in 3D reconstruction. Accordingly, we propose a scene‑adaptive nonlinear tone‑curve framework that replaces linear pseudo‑GT with nonlinear alternatives. The framework introduces percentile‑based normalisation for scene‑agnostic curve application, a scene‑adaptive offset for automatic black‑level adjustment, and two complementary curves: Adaptive SoftExp (ASE), a bounded exponential curve, and Adaptive Poly3 (AP3), a data‑driven cubic polynomial. The module changes only the pseudo‑GT computation and leaves the 3DGS backbone unchanged. Experiments on three benchmarks covering 21 scenes show that both curves consistently outperform the linear baseline with PSNR improvements up to +4.34 dB on LOM and +3.25 dB on RealX3D. Both curves achieve similar performance despite their different mathematical forms, suggesting the improvement is curve‑agnostic. Code is available at https://github.com/lvmingzhe/adaptiveToneCurve
Authors:Sam Gijsen, Michał Łukomski, Marc-André Schulz, Kerstin Ritter
Abstract:
Flow matching and diffusion models enable conditional generation across domains ranging from images to proteins, with recent extensions to out‑of‑distribution contexts. Yet generative models of neural time series have largely remained restricted to categorical conditioning, precluding compositional and zero‑shot generalization. In this work, we propose a per‑timestep conditioned diffusion transformer for generating realistic fMRI brain dynamics during unseen cognitive tasks by injecting both compositional language and optional spatial priors in‑context. Such zero‑shot generation could enable counterfactual neuroscience by supporting in‑silico design and evaluation of novel cognitive experiments before empirical validation. Leveraging this model, we evaluate across hundreds of held‑out task conditions and characterize predictive performance in relation to the training manifold. From language alone, the model recovers region‑specific recruitment across tasks and held‑out spatial activation patterns. Spatial priors, when available, complement the text pathway by anchoring generation in regions of task space where language alone degrades, while retaining the compositional structure needed for counterfactual task specification. To our knowledge this is the first generative model of whole‑cortex fMRI dynamics for unseen cognitive tasks, advancing counterfactual neuroscience and data‑driven experimental design.
Authors:Qi Shao, Hao Guo, Jiawen Chen, Duxin Chen, Wenwu Yu
Abstract:
Neural relational inference (NRI) methods discover interaction graphs from trajectories through variational reasoning on discrete potential edges. However, these methods typically rely on oversimplified, factorized graph priors. Such priors, typically nearing uniform distributions, treat edges as independent entities. This systemic misalignment does not match the real‑world systems and yields diffuse and indecisive edge posteriors limiting the reliability of structural discovery. To address this, we propose Diff‑prior, a diffusion‑parameterized adaptive prior used to calibrate latent graph distribution rather than generate graphs. Our core insight is to reframe prior integration as a learnable denoising‑style calibration that organizes scattered, uncertain edge posteriors into a more reliable overall structure which can be trained by the diffusion model. Diff‑prior learns an adaptive structure prior that performs structured calibration on the edge posteriors during inference, guiding it towards a distribution closer to the underlying structure. The diff‑prior operates before structural sampling and acts as a denoising calibrator directly on the encoder edge distribution, which provides a generic training paradigm over structured variables. Experiments on standard benchmarks validated our framework, and the results indicate that Diff‑prior improves the performance of structure inference and generates more decisive edge posteriors across multiple NRI‑family architectures. The code is available on https://github.com/Hardy158118/Diffprior.
Authors:Victor Dods
Abstract:
Originally motivated by creating first‑person computer visualizations within Riemannian manifolds ‑‑ the author was led to study deformable‑body mechanics, as rigid‑body mechanics is not available in a generic Riemannian manifold due to its lack of nontrivial isometry group. Hyperelasticity is a particularly nice sub‑category of continuum mechanics in which a deformable, elastic body's behavior is determined by a stored energy density function. This allows problems to be posed variationally, and powerful tools brought to bear on studying and solving them. This article presents numerical simulations of static solutions to a particular class of problems in hyperelastic mechanics in 2‑dimensional Riemannian manifolds in which a flat hyperelastic body B is embedded into a region Ω in a nowhere‑flat surface S of revolution z=z\left(r\right) such that \left|K\left(r\right)\right| decreases as r\to\infty, where K denotes the Gaussian curvature of S. For example, the funnel z=‑r^‑1 or the paraboloid z=\frac12r^2. Because B is flat, the body can't achieve a zero‑stored‑energy configuration, and restorative forces arise in the body to move it toward a region of lower stored energy ‑‑ meaning, toward a flatter configuration. With the addition of a gravitational potential U\left(r\right)=z\left(r\right) on S, forces act on the body to pull it toward r=0. If the body has sufficient stiffness and remains within the region Ω, then the body has an equilibrium configuration in which the body's deformation‑response forces perfectly cancel the gravitational forces. Such a configuration represents a kind of "levitation" phenomenon within this surface. The numerical implementation of this problem will be detailed and the resulting numerical solutions and various consequences discussed.
Authors:Maximilian Poretschkin, Tabea Naeven
Abstract:
The European AI Act is the first comprehensive regulation of artificial intelligence (AI), setting out extensive obligations, particularly for so‑called high‑risk and general‑purpose AI systems. A key distinguishing feature of AI systems under the AI Act is the capability to infer. Since the AI Act does not clearly define what inference is, there is a gray area for certain data‑driven systems. A specific example is credit scoring systems, which are listed by Annex III of the AI Act. At the same time, however, these are often implemented using statistical models for which it is unclear whether they have the capability to infer and thus fall under the AI definition of the AI Act at all. Motivated by statistical learning theory, this work develops a framework for grading different levels of the capability to infer. Based on the AI Act and the Commission Guidelines on the definition of an artificial intelligence system, we analyze which levels constitute sufficient capability to infer within the meaning of the AI Act and where further regulatory clarity is needed. We illustrate the framework by creating two realistic credit scoring workflows and show whether and where inference occurs in them. Our analysis illustrates that not only individual models but the entire data processing workflow must be considered. It also shows that the involvement of human experts during development can have significant influence on the capability to infer. Code can be found at https://github.com/fraunhofer‑iais/inference‑framework‑creditscorecards.
Authors:Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya, Mohor Banerjee, Swaagat Bikash Saikia, Alvin Chan
Abstract:
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks. However, most existing creativity metrics are tightly coupled to specific tasks, embedding domain assumptions into the evaluation process, and limiting scalability and generality. To address this gap, we introduce an automated, domain‑agnostic framework for quantifying LLM creativity across open‑ended tasks. Our approach separates the measurement apparatus from the creative task itself, enabling scalable, task‑agnostic assessment. Divergent creativity is measured using semantic entropy, a reference‑free and robust metric for novelty and diversity, validated against human annotations, LLM‑based novelty judgments and baseline diversity measures. Convergent creativity is assessed via a novel retrieval‑based multi‑agent judge framework that delivers context‑sensitive evaluation of task fulfilment with over 60% improved efficiency. We validate our framework in three qualitatively distinct domains: problem‑solving (MacGyver), research ideation (HypoGen), and creative writing (BookMIA), using a broad suite of LLMs. Empirical results show that our framework reliably captures key facets of creativity, including novelty, diversity, and task fulfilment, and reveal how model properties, such as size, temperature, recency, and reasoning, impact creative performance. Our work establishes a reproducible and generalizable standard for automated LLM creativity evaluation, paving the way for scalable benchmarking and accelerating progress in creative AI.
Authors:Atif Hassan, Swanand Khare, Jiaul H. Paik
Abstract:
Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training. However, existing methods often struggle to maintain strong worst‑group accuracy, particularly at high pruning rates, across balanced and imbalanced datasets. To address this challenge, we propose RCAP, a Robust, Class‑Aware, Probabilistic dynamic dataset pruning algorithm for classification tasks. RCAP applies a closed‑form solution to estimate the fraction of samples to be included in the training subset for each individual class. This fraction is adaptively adjusted in every epoch using class‑wise aggregated loss. Thereafter, it employs an adaptive sampling strategy that prioritizes samples having high loss for populating the class‑wise subsets. We evaluate RCAP on six diverse datasets ranging from class‑balanced to highly imbalanced using five distinct models across three training paradigms: training from scratch, transfer learning, and fine‑tuning. Our approach consistently outperforms state‑of‑the‑art dataset pruning methods, achieving superior worst‑group accuracy at all pruning rates. Remarkably, with only 10% data, RCAP delivers >1% improvement in performance on class‑imbalanced datasets compared to full data training while providing an average 8.69× speedup. The code can be accessed at https://github.com/atif‑hassan/RCAP‑dynamic‑dataset‑pruning
Authors:Hang Xu, Xiaoxiao Ma, Guohui Zhang, Yu Hu, Siming Fu, Jie Huang, Lin Song, Haoyang Huang, Nan Duan, Feng Zhao
Abstract:
Multi‑turn image editing is essential for iterative design, yet current models often struggle with identity drift and error accumulation over successive steps. While existing research leverages video priors for consistency, their reliance on bidirectional attention is fundamentally misaligned with the causal, sequential nature of interactive editing. In this paper, we propose AnchorEdit, the first autoregressive (AR) diffusion‑based framework designed specifically for high‑resolution, long‑term multi‑turn editing. AnchorEdit bridges the gap between video priors and causal inference through a three‑stage training curriculum: identity‑preserving sing‑turn pretraining, causal AR forcing fine‑tuning with a novel self‑rollout strategy to mitigate exposure bias, and consistency distillation for efficient 4‑step generation. During inference, we introduce a memory mechanism to anchor the initial subject identity and ensure stable extrapolation across extended editing trajectories. To evaluate performance, we provide a new high‑resolution multi‑turn editing benchmark designed to stress‑test long‑horizon stability. Extensive experiments demonstrate that AnchorEdit achieves state‑of‑the‑art results, maintaining exceptional subject fidelity and instruction following even over 10+ interaction rounds.
Authors:Mengzhuo Chen, Yan Shu, Chi Liu, Hongming Piao, Xidong Wang, Derek Li, Bryan Dai
Abstract:
We study whether grounded reasoning supervision from abundant 2D medical images can improve 3D medical VQA when both input types are aligned through a common reasoning interface. We introduce UniReason‑Med, a single‑checkpoint framework that processes either a 2D image or a slice‑serialized 3D volume at inference time, generating interleaved textual reasoning and localized visual evidence through shared box syntax, region‑token injection, and a common grounded reasoning policy. To train this interface, we construct UniMed‑CoT, a 220K instruction‑tuning dataset with interleaved textual reasoning and grounded visual evidence, including 170K 2D and 50K 3D samples. Through supervised fine‑tuning followed by outcome‑level reinforcement learning, UniReason‑Med learns to generate grounded reasoning traces without IoU/Dice‑based localization rewards during RL. Data‑mixture and component ablations show that joint 2D+3D grounded supervision substantially improves 3D reasoning over 3D‑only training, while grounding and region‑token injection consistently benefit both 2D and 3D tasks. These results suggest that a shared grounded reasoning interface can transfer reasoning structure from 2D images to slice‑serialized volumetric medical understanding. The code and data are publicly available at https://github.com/IQuestLab/unireason‑med.
Authors:Evgeny Gorelik, Kenny Dean Karrow, Fikret Sivrikaya, Sahin Albayrak, Christian Baumann
Abstract:
We introduce a multi‑view in‑cabin monitoring dataset for public transportation with synchronized RGB and depth images from four inward‑facing cameras and a rotating LiDAR covering the vehicle interior of a digitalized and partly automated German city bus. The dataset contains 9.136 synchronized samples with annotations and is accompanied by a calibration and pseudo‑labeling pipeline that generates 3D human pose estimates and oriented 3D bounding boxes for occupants. We further provide a nuScenes‑format conversion and benchmark representative multi‑view 3D detection models (e.g., Lift‑Splat‑Shoot and BEVFusion), supporting comparative evaluation and small‑scale training of multi‑view in‑cabin perception models. The dataset and tools are available at https://github.com/EvgenyGorelik/multiview_incabin_dataset.
Authors:Youwang Deng
Abstract:
User‑side memory in LLMs is typically scored as a single "personalization" capability: given a user's history, is the output more user‑aware? We show this aggregate metric hides opposite‑direction failures. Memory factorises into at least three orthogonal axes ‑‑ behavioral consistency (style, voice), factual presence (recall facts in history), and factual absence (abstain when a fact is absent) ‑‑ and no single substrate wins all three. Comparing per‑user gamma‑LoRA (a small LoRA adapter trained on each user's history; gamma denotes per‑user, not per‑task) against BGE‑large dense top‑K retrieval on a controlled 50‑user synthetic corpus and a real‑data probe (LaMP‑3), we find gamma‑LoRA decisively wins behavioral style while RAG decisively wins factual absence ‑‑ and the same query‑projection cells in attention layers 21‑35 causally load‑bear both effects in opposite directions (zeroing those LoRA weights raises absence‑probe TPR by +33 pp and drops presence‑probe TPR by 20 pp). On the more heavily RLHF‑tuned Llama‑3.1‑8B‑Instruct the asymmetry strengthens, not heals: parametric memory's behavioral advantage collapses while its absence‑calibration deficit against retrieval widens ‑‑ an alignment tax on parametric user‑memory. On real‑data LaMP‑3, gamma‑LoRA underperforms a majority baseline; a 9‑condition mitigation sweep diagnoses this as instruction‑following collapse, not substrate failure (a 9x2 cross‑product shows the eval‑time 1..5 logit mask drives main_acc to >=0.995 on every recipe), and the best training‑time fix replicates bit‑identically on Llama. Finally, substrate‑selection routing is question‑classification, not calibration: a 110M DistilBERT on the question text alone beats every logit‑based router. We contribute the diagnostic framework, the diagnosed real‑data negative, the alignment‑tax replication, and the routing‑as‑classification finding.
Authors:Tajamul Ashraf, Hyewon Jeong, Fida Mohammad Thoker, Bernard Ghanem
Abstract:
To make clinically grounded decisions, medical AI agents are expected to go beyond simple recognition and be capable of tool retrieval, evidence acquisition, and integration. Existing benchmarks largely evaluate isolated perception or single‑turn question answering, and therefore provide limited visibility into failures of planning, tool recruitment, and rollout reliability. We introduce MedCTA, a benchmark for evaluating medical tool agents on clinician‑validated, step‑implicit tasks grounded in realistic multimodal clinical inputs, including radiology images, pathology slides, and reports. MedCTA comprises 107 real‑world clinical tasks with clinician‑verified executable trajectories over 5 deployed tools, and supports process‑aware evaluation of tool selection, argument validity, execution stability, trajectory fidelity, and outcome quality. We benchmark 18 open‑ and closed‑source multimodal models and find that even frontier systems remain brittle in multi‑step clinical tool use: autonomous rollouts are dominated by protocol failures, premature stopping, and incorrect tool recruitment, while gold‑standard tool routing yields large but still incomplete gains. These results show that strong backbone perception does not translate into reliable agentic behavior in clinical settings. MedCTA provides a rigorous testbed for auditing, diagnosing, and advancing trustworthy medical AI agents. The dataset and evaluation suite are available at https://ivul‑kaust.github.io/MedCTA/
Authors:Chitral Patil
Abstract:
Every public LLM cost calculator we surveyed treats GPU utilization as a fixed input ‑‑ entered by the user, baked in as a preset, or silently assumed at 100% ‑‑ never measured against the operator's actual load. We show that this assumption is the dominant source of error: on identical H100 hardware, effective cost spans \0.21 to \15.25 per million output tokens, an underutilization penalty of 2.5‑24x across low‑to‑moderate enterprise loads (1‑10 rps) and up to 36.3x near idle ‑‑ driven by one operator‑controlled variable, offered request rate lambda, which sets in‑flight concurrency via Little's Law and which no open‑source calculator exposes. Because calculators take utilization as a user‑supplied input, any utilization‑naive estimate understates true cost by exactly 1/U, systematically mispricing self‑hosting ‑‑ most severely over‑selling it for low‑traffic workloads. We propose a measurement methodology that parameterizes the relationship as C_eff = f(H, M, Q, lambda, L), validate it with 42 benchmarks across dense, ultra‑sparse MoE, and sparse MoE models, and release vllm‑cost‑meter, an open‑source cost meter that attaches to a live vLLM server and reports real \/M‑tokens against the operator's own traffic. We further show that FP8 quantization benefits the MoE architectures we tested roughly 2.2‑2.4x more than the dense model (+69 to +74% vs. +31% peak throughput; n=3, broader validation needed), and our data are consistent with active parameter count, not total model size, being a primary predictor of saturation economics. To rule out single‑hardware confounding we repeat the core sweep on A100 80GB PCIe (56 runs): the load‑driven spread reproduces at 7.0‑11.4x, the active‑parameters ordering survives at FP8, and the dense‑FP8 advantage inverts on silicon without native FP8 tensor cores ‑‑ a hardware‑conditional caveat the framework already accommodates.
Authors:Youwang Deng
Abstract:
Long‑horizon LLM agents are not trusted to run unattended: with no human watching, they confidently report success they never verified. We treat honesty ‑‑ bounding what an agent may claim at termination ‑‑ as a first‑class metric for unattended autonomy, distinct from capability. We present Autopilot, an execution model that makes silent fabricated success structurally impossible rather than merely rarer. Autopilot externalizes all working state into a durable, gated finite‑state machine that a scheduler advances one stateless tick at a time; a hard floor forbids any terminal "done" claim whose falsifiable gate did not actually execute and pass. We prove a No‑False‑Success theorem ‑‑ under gate soundness, floor enforcement, and plan coverage, termination implies the goal holds ‑‑ whose only trust points are empirically measurable, and show the worst case degrades to an honest stall, never a fabricated success. Because each tick rehydrates only the state machine, per‑step context cost is constant in the horizon. Across a 3,150‑cell paired corpus (70 tasks × 3 systems × 3 models × 5 seeds, including 50 SWE‑bench Lite tasks across 11 OSS repos), Autopilot fabricates on 0.95% of cells [95% CI 0.38‑‑1.62] while Reflexion and StateFlow baselines fabricate on 8.10% [6.48‑‑9.81] and 25.05% [22.48‑‑27.62] respectively. The headline contrast lives in the hard regime: on SWE‑bench Lite, the firewall reduces fabrication from 33.7% (StateFlow) to 0.67%, a paired difference of ‑33.07 pp [95% CI ‑36.53, ‑29.73]. The mechanism is the gate, not the model: all ten Autopilot fabrications come from the strongest model, while two weaker mid‑tier models never fabricate across 700 paired cells. The firewall trades coverage for honesty by design ‑‑ an honest stall is recoverable; a confident wrong output shipped downstream is not.
Authors:Marius Bayizere
Abstract:
Unmanned Aerial Vehicle (UAV) threats have emerged as a defining security challenge of the 21st century. This paper presents DroneShield‑AI, a unified open framework integrating six processing layers: RF signal classification, acoustic motor‑signature detection, YOLOv8‑based visual detection, evidence‑weighted sensor fusion, a Behavioral Intent Classification Engine (BICE), and a Graph Neural Network Swarm Intelligence Module (GNN‑SIM). BICE introduces the first systematic six‑class threat taxonomy for drone flight patterns, enabling predictive operator alerts with a 30‑second advance‑warning horizon. GNN‑SIM is the first open framework for adversarial multi‑drone formation analysis using Graph Attention Networks. Evaluated on three publicly available real‑world datasets, the fused pipeline achieves 96.1% detection accuracy, 3.2% false alarm rate, AUC‑ROC: 0.981, and 142ms end‑to‑end latency on commodity CPU‑class hardware at approximately 500‑780 USD total system cost. All code, model weights, and simulation datasets are publicly released at submission.
Authors:Chaofan Ma, Zhenjie Mao, Yuhuan Yang, Fanqin Zeng, Yue Shi, Yingjie Zhou, Xiaofeng Cao, Jiangchao Yao
Abstract:
Spatial reasoning from egocentric videos is inherently challenging because the observable evidence is constrained by the camera trajectory. Existing methods rely on single‑turn inference, forcing models to resolve geometric ambiguity through semantic priors rather than verifiable evidence. We argue that spatial reasoning should be revisitable: conclusions formed under limited evidence should remain open to revision when complementary viewpoints become available. Building on this insight, we propose Reason, then Re‑reason (ReRe), a training‑free, inference‑time framework with two phases: in the Reason Phase, an MLLM forms a spatial hypothesis from the original video; in the Re‑reason Phase, it verifies or revises the hypothesis by observing a synthesized novel‑view video. To enable effective cross‑view revisiting, we design a Geometry‑to‑Video pipeline that renders strategically complementary novel views from predicted 3D geometry. These views feature an elevated, oblique perspective with scene‑spanning coverage, while preserving the MLLM's native video interface without architectural modifications. Extensive evaluations on VSI‑Bench and STI‑Bench demonstrate that ReRe substantially boosts open‑source MLLMs to rival proprietary state‑of‑the‑art performance. Project page: https://zhenjiemao.github.io/ReRe/
Authors:Sangmin Lee, Eekgyun Ahn, Woongjib Choi, Hong-Goo Kang
Abstract:
We propose UR‑BERT, a Romanized transcription‑based text‑to‑speech (TTS) encoder for massively multilingual TTS systems. Conventional grapheme‑to‑phoneme (G2P)‑based approaches are limited to around 100 languages due to the availability of reliable G2P resources. In contrast, UR‑BERT scales to 495 languages by unifying diverse writing systems into a shared Romanization representation. To further enhance phonetic fidelity and text‑speech alignment, we introduce a speech token prediction objective during training, which encourages the encoder to learn speech‑aware phonetic representations in a data‑efficient manner. Experiments show that TTS systems built on UR‑BERT consistently outperform recent text encoder baselines across a wide range of languages and resource conditions, and demonstrate strong generalization to unseen languages.
Authors:Lucia Quirke, Louis Jaburi, David Johnston, William Z. Li, Gonçalo Paulo, Guillaume Martres, Girish Gupta, Stella Biderman, Nora Belrose
Abstract:
Data attribution is a promising field in interpretability that aims to explain model behavior through the influence of its training data, with applications including debugging undesirable model behavior and training dataset curation. However, significant engineering effort is required to perform it at scale, and many cutting edge techniques lack open‑source tooling and support. Bergson is an open source library that aims to enable faster progress in the field by providing a host of techniques that scale to very large language models and pre‑training datasets. The library natively supports on‑disk gradient stores and multi‑node distributed training, and provides quality of life tools for researchers. Finally, we introduce the first open‑source implementations of three leading data attribution methods: MAGIC, SOURCE, and TrackStar. The library is available at https://github.com/EleutherAI/bergson .
Authors:Kailin Lyu, Di Wu, Pengwei Zhang, Yuhang Zheng, Yingxin Lai, Long Xiao, Kangyi Wu, Pengna Li, Chen Gao, Lianyu Hu, Xiaobin Hu, Jie Hao, Ce Hao, Weihao Yuan, Shuicheng Yan
Abstract:
Touch is a key modality for embodied agents to understand the physical world. Although recent work has incorporated tactile signals into language systems for tactile commonsense reasoning, scaling such systems to realistic open‑world settings remains challenging due to two key bottlenecks: (1) current tactile reasoning datasets remain limited in format and scale, providing insufficient supervision for reasoning from tactile observations to physical commonsense and hindering the learning of transferable tactile commonsense; (2) Tactile signals are inherently redundant and action‑specific, yet existing methods often overlook these properties, resulting in inefficient representations with limited semantic expressiveness. To address these limitations, we propose TouchThinker, a tactile‑language framework that scales tactile commonsense reasoning to the open world from both data and representation perspectives. First, we construct TouchThinker‑1M, a million‑scale, multi‑source tactile reasoning dataset covering 415 objects, 8 scenarios, and 7 sensor types, providing a solid data foundation for open‑world generalization. We further introduce TouchThinker‑Bench, an open‑world benchmark with more realistic and diverse tasks. Then, we propose action‑aware modeling mechanism to improve tactile representation efficiency and enable efficient reasoning. Experimental results demonstrate that TouchThinker achieves competitive performance against state‑of‑the‑art models across multiple datasets. Our code and dataset will be made available at: https://github.com/lvkailin0118/TouchThinker.
Authors:Jun Xu, Zhengxue Cheng, Fengxi Zhang, Yuhan Liu, Li Song, Wenjun Zhang
Abstract:
Learning‑based speech compression has achieved promising low‑bitrate performance, but many neural speech codecs still describe quantized latents with preset‑rate discrete symbols or apply entropy coding only after symbol generation. Such designs decouple representation learning from probability modeling, limiting their ability to exploit the non‑uniform usage and temporal dependencies of learned speech latents. In this paper, we benchmark neural speech compression from a rate‑‑distortion perspective and further investigate entropy‑constrained coding for low‑bitrate speech compression. We first formulate a unified learning‑based speech coding pipeline and provide a benchmark‑style analysis of recent neural speech codecs, showing that explicit probability modeling remains underexplored in learned speech compression. We then propose ECC, an Entropy‑Constrained Codec that combines scalar quantization with a learned entropy model. ECC integrates hyperprior‑based side information, channel‑wise context modeling, latent residual prediction, and lightweight temporal modeling to estimate latent likelihoods for rate estimation during training and arithmetic coding during inference. To further improve low‑bitrate efficiency, ECC introduces entropy skip, which omits highly predictable residual symbols using decoder‑available scale estimates without transmitting additional skip masks. Extensive experiments show that ECC achieves a favorable low‑bitrate rate‑‑distortion trade‑off over conventional and neural codec baselines, reducing BD‑rate by 39.9% on ViSQOL and 76.3% on PESQ on average over two widely‑used test sets. Ablation and diagnostic studies further validate the effectiveness of entropy modeling. Project Page: https://avery‑xu.github.io/ECC‑demo/
Authors:Harsh Gupta, Guanya Shi, Wenzhen Yuan
Abstract:
The most widely‑adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments. In contrast, unstructured human videos provide a scalable alternative. They contain diverse manipulation demonstrations across objects, scenes, and strategies, but are not directly connected to robot action. We propose LUCID, a two‑stage framework that learns task intent from unstructured human videos drawn from internet‑scale datasets and learns robot control in massively‑parallel simulation. The intent model predicts short‑horizon intent (what should happen next in the scene) from the current observation in closed loop. An embodiment‑specific sensorimotor policy converts this intent into robot actions. The intent interface is shared across controllers, so the same intent model can be applied to different embodiments, from our primary dexterous hand to a parallel‑jaw gripper. We evaluate LUCID on five real‑world manipulation tasks: stirring, wiping, and binning supervised by only internet video, with zero‑shot transfer to novel scenes and object instances; and push‑T and cable routing supervised by 1 hr each of self‑collected smartphone video. Project page: https://lucid‑robot.github.io/.
Authors:Cheng Chen, Jingyu Zhou, Yifan Zhao, Jia Li
Abstract:
Understanding multi‑label images remains a challenging task in computer vision. With the rapid progress of vision‑language multimodal learning, vision‑language models (VLMs) enable zero‑shot recognition without labeled data. However, due to their intrinsic design, these models often prioritize the most iconic object and omit other contextual positives. This intrinsic bias conflicts with the nature of multi‑label learning, thereby limiting their applicability. In this work, we propose an unsupervised framework that adapts VLMs from iconic recognition toward inclusive understanding, enabling label‑free multi‑label image recognition. Our approach consists of two key stages, ``cutting'' and ``sewing'': In the cutting stage, we present the multi‑sampling response estimator to prevent the model from concentrating only on one single object. In the second sewing stage, the multi‑object blend adaptation is introduced to adjust the labels to better conform to the multi‑label distribution while preserving the intrinsic characteristics of the original model within only one epoch. Extensive experiments show that our framework significantly outperforms existing unsupervised approaches on four public datasets, even surpassing several representative weakly supervised baselines. These results demonstrate the potential of adapting pre‑trained VLMs for more comprehensive visual understanding without manual annotations. Our code is publicly available at https://github.com/iCVTEAM/TailorCLIP.
Authors:Kanghui Ning, Yushan Jiang, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Dongjin Song
Abstract:
Time‑series foundation models (TSFMs) are increasingly explored as predictive experts within emerging agentic time‑series systems. However, TSFMs exhibit heterogeneous inductive biases, and no single model consistently dominates across forecasting regimes, making expert selection a critical challenge. Existing systems often delegate this decision to LLM‑based controllers, incurring substantial inference overhead. We present TimeRouter, an efficient routing framework that leverages empirical complementarity across a pool of pretrained TSFMs through lightweight discriminative routing, selective gating, and ensemble fallback. Concretely, TimeRouter combines a learned routing head, a selective gate, and an ensemble fallback, enabling adaptive expert selection without invoking an LLM at inference time. TimeRouter achieves state‑of‑the‑art performance on the GIFT‑EVAL leaderboard, with an LB MASE of 0.6765. Beyond benchmark performance, our ablation studies provide empirical insights into TSFM routing design, highlighting the importance of pool composition and selective gating. Taken together, these results position TimeRouter as a modular and lightweight routing layer for future agentic time‑series systems built upon foundation‑model pools. Our code is available at https://github.com/UConn‑DSIS/TimeRouter.
Authors:Jiale Deng, Yanyan Shen, Xiaogang Shi, Chai Junjun
Abstract:
High‑quality training data is essential for the success of machine learning models. However, real‑world datasets often contain mixed types of errors arising from systematic flaws in data preparation pipelines, including label errors, feature errors, and spurious correlations. Effective debugging of training data requires both detecting erroneous samples and identifying their specific error types to enable targeted repair, yet existing data cleaning and attribution methods fail to adequately address this dual requirement. In this paper, we propose DeMix, a novel framework that simultaneously diagnoses erroneous samples and their error types. Our key insight is that different error types produce distinct patterns on model behavior. DeMix captures such error‑specific patterns by influence vectors that characterize how each training sample affects model predictions across all validation samples. We formulate training data debugging as a multi‑label classification problem where a classifier is developed to predict error types directly from influence vectors. We further introduce an intervention‑based learning strategy that guides the classifier to capture invariant rationales specific to each error type, ensuring the learned classifier generalizes effectively. Empirical evaluations on 11 tasks across tabular data prediction, recommendation systems, and LLM alignment demonstrate that DeMix significantly outperforms state‑of‑the‑art approaches, achieving a 22.61% improvement in data debugging F1‑score and a 9.32% gain in task model performance after data repair. Code is available at: https://github.com/SJTU‑DMTai/DeMix.
Authors:Zequn Yang, Yake Wei, Haotian Ni, Zhihao Xu, Di Hu
Abstract:
Multimodal learning hinges on capturing redundant, unique, and synergistic information across modalities, which collectively constitute multimodal interactions. A critical yet underexplored challenge is that these implicit interactions vary dynamically across samples. In this work, we present the first systematic, information‑theoretic analysis highlighting why learning these dynamic, sample‑specific interactions is critical for effective multimodal learning. Our analysis further reveals deficits in conventional paradigms at learning these distinct interaction types: modality ensemble approaches struggle to capture synergy, while joint learning paradigms often under‑utilize redundant information. This highlights the need for an approach that can adaptively learn from different interaction types on a per‑sample basis. To this end, we propose Decomposition‑based Multimodal Interaction Learning (DMIL), a novel paradigm that explicitly models and learns from sample‑specific interactions. First, we design a variational decomposition architecture to isolate the constituent interaction components. Second, we employ a new learning strategy that leverages these explicit interaction components in a fine‑tuning process to achieve comprehensive interaction learning. Extensive experiments across diverse tasks and architectures demonstrate that DMIL consistently achieves superior performance by adapting to holistic sample‑specific interactions. Our framework is flexible and broadly applicable, establishing an interaction‑centric paradigm for multimodal learning. The code is available at https://github.com/GeWu‑Lab/DMIL.
Authors:Zhuoyi Peng, Hanlin Gu, Lixin Fan, Yi Yang
Abstract:
Text‑attributed graphs (TAGs) underlie real‑world applications such as citation networks, social media, and e‑commerce. Few‑shot graph learning on TAGs is hard: with only a handful of labels per class and the rest of the graph unannotated, neither GNNs nor LLMs can learn well on their own. GNNs read topology and fail on cold nodes; LLMs read text and fail on text‑ambiguous nodes. Existing LLM‑GNN methods all follow the same recipe: designate one model as the golden teacher and use its outputs (e.g., features or pseudo‑labels) to supervise the other. We argue this golden‑teacher assumption breaks under sparse supervision: neither model is golden, and treating either as such transfers its blind spots into the student. We therefore ask: can we avoid designating either model as the golden teacher, and still perform effective graph learning? We answer with LLM‑GNN Co‑Teaching, a bidirectional co‑teaching framework in which neither model is fixed as teacher. The GNN and LLM exchange their most confident pseudo‑labels under an architecture‑specific small‑loss criterion, and both update every round. Supervision is then mined from the trajectory: whenever a node moves from cross‑model contradiction at round t to cross‑model agreement at round t+1, the LLM's two answers on the same input form a preference pair (old contradicting self < new peer‑endorsed self) for DPO training. We call this Round‑based Pseudo‑Label Preference Optimization (RPL‑PO). On six benchmarks, LLM‑GNN Co‑Teaching consistently outperforms GNN‑as‑Judge and all prior methods, with absolute 3‑shot gains of 7.86% on Cora and 7.73% on ogbn‑arxiv; improvements carry over to 5‑shot and to zero‑shot cross‑dataset transfer. Error‑structure analysis further shows that abandoning the golden‑teacher assumption substantially improves the LLM's graph learning capability on challenging samples.
Authors:David Hall, Joshua Knights, Mark Cox, Peyman Moghadam
Abstract:
Natural environments present a complex challenge to robotics perception systems. Current models, particularly vision foundation models, are largely trained on structured, urban environments leading to weaknesses in their perception for field robotics tasks. We showcase the limitations of current models using our recently released WildCross benchmark, a new cross‑modal benchmark for place recognition and metric depth estimation in large‑scale natural environments. WildCross comprises over 476K sequential RGB frames with semi‑dense depth and surface normal annotations, each aligned with accurate 6DoF pose and synchronized dense lidar submaps. In this work, we provide an expanded analysis of the benchmark results from the recent WildCross benchmark, with particular emphasis on expanded metric depth estimation experiments. Access to the code repository and dataset for this work can be found at https://csiro‑robotics.github.io/WildCross.
Authors:Zhuoyi Peng, Jingzhou Jiang, Hanlin Gu, Lixin Fan, Yi Yang
Abstract:
Graph analysis underlies many applications whose answers cannot be looked up in a single record or retrieved along a path: laundering rings, drug repurposing, user preference, and scientific theme are all inferred from a node together with its neighbourhood. We introduce GraphInfer‑Bench, a benchmark for whether LLMs can perform this graph inference: producing an open‑ended answer that no single node supports and no path retrieves. Existing graph‑QA protocols cannot test this capability: algorithm simulation, node classification, single‑node description, KG‑QA, and GraphRAG all admit answers retrievable from one node or along a path. GraphInfer‑Bench defines five tasks along Description (what a region is) and Comparison (how regions differ), each constructed so the ground truth lives in no single node. The release contains 42,000 samples across six real‑world graphs, produced automatically and screened by a four‑layer quality‑control protocol. We evaluate four method families against the same tasks: graph‑token alignment models, zero‑shot frontier closed‑source LLMs, Graph2Text supervised fine‑tuning, and plain GNNs as a structural reference. No method family closes the gap. Graph‑token alignment partially handles description tasks (relational, theme) but collapses on comparison tasks. Frontier LLMs lead on outlier detection and community partition among LLM‑based methods but lag on masked‑node prediction. Graph2Text SFT is the strongest LLM‑based method on the description side yet falls behind frontier LLMs on comparison. Across every task, plain GNNs match or beat the strongest LLM‑based row, with the largest margin on community detection. GraphInfer‑Bench surfaces graph inference as an open capability gap rather than a property of any one architecture.
Authors:Zhiyu Chen, Zihan Guo, Bo Huang, Bingwei Lu, Jianghao Lin, Yuanjian Zhou, Weinan Zhang
Abstract:
Agent Skills augment large language model (LLM) agents with procedural knowledge at inference time, but current benchmarks rarely distinguish what a Skill says from how it is organized. We study this distinction through Progressive Disclosure, where a concise root file points agents to supporting resources on demand, and compare it with a normalized flat baseline. We present SkillJuror, a framework for evaluating Skill writing paradigms through semantically controlled variants, matched multi‑trial evaluations, and trajectory evidence while holding task knowledge fixed. In an 82‑task SkillsBench study, Progressive Disclosure changes runtime behavior before aggregate outcomes: distinct Skill resources touched per trajectory rise from 1.18 to 3.85, and effective uptake events rise from 1.33 to 3.92. It also yields 17 additional verifier‑passing trials out of 410 matched trials (+4.1%) over the normalized flat baseline. The benefit is task‑dependent. Progressive Disclosure helps when supporting resources guide implementation, checking, or repair, but is weaker when success hinges on exact output conventions, numerical thresholds, or long artifact‑generation pipelines. These results show that Skill organization is not mere presentation: it can change how agents search and apply procedural knowledge, while outcome gains depend on whether the exposed resources are actionable for the task. Code is available at https://github.com/zhiyuchen‑ai/skill‑juror.
Authors:Abdelrahman Abdallah, AbdelRahim A. Elmadany, Sameh Al Natour, Hasan Cavusoglu, Adam Jatowt, Muhammad Abdul-Mageed
Abstract:
Financial and tabular question answering requires more than fluent reasoning: answers must be grounded in the exact facts, formulas, units, signs, and scales that support them. A single misread cell or incorrect operation can silently produce a plausible but wrong result. We introduce \textscMOCA‑Agent, a market‑of‑claims code agent that replaces free‑form multi‑agent debate with claim‑level verification. The system decomposes each question into typed atomic claims, asks specialist trader agents to buy or sell those claims, clears their orders into confidence‑weighted accept/reject decisions, and synthesizes an executable Python program from market‑supported evidence. A code‑aware verifier then checks the program for execution, structural consistency, and common financial reasoning errors, with at most one market‑aware repair round. Across ten public benchmarks spanning financial numerical reasoning, general tabular reasoning, ESG question answering, and multimodal chart reasoning, \textscMOCA‑Agent achieves strong performance using a fixed Qwen3.6‑27B backbone, including 78.3% on FinQA, 76.0% on FinanceMath, 71.2% on MultiHiertt, 86.9% on ESGenius, and 85.6% average on FinChart‑Bench. These results show that aggregating evidence at the level of atomic claims, rather than whole answers, improves robustness in high‑stakes numerical reasoning.\footnoteThe code and data are available: https://github.com/UBC‑NLP/MoCA‑Agent.
Authors:Hongyi Liu, Frederic Sala, Thomas Reps, Adithya Murali
Abstract:
LLMs and LLM agents should improve when given feedback, but identifying when they are able to do so is difficult: feedback is heterogeneous, domain‑specific, and difficult to control. We approach this challenge by asking LLMs to perform regular‑expression induction, a classical symbolic learning problem where precise mechanisms for feedback exist in the form of counterexamples. In counterexample‑guided learning, a learner (LLM) proposes candidate regular expressions from positive/negative‑labeled strings, and the teacher (verifier) returns counterexamples showcasing the difference between the candidate and target languages. We identify novel counterexample‑guided refinement strategies that enable effective regex learning, such as regularization and symbolic counterexample clusters. We also explore agentic strategies such as reflection and repair loops. Empirically, we find that verifier feedback substantially improves sample efficiency on challenging regex‑induction tasks, reducing the number of labeled examples required and enabling learning of complex target expressions where standard prompting fails. For example, on the hardest task groups, our counterexample‑guided framework improves success from 3.2% to 38.1% and from 38.9% to 74.1% on two different regex domains. These results suggest that LLMs can benefit from rich feedback beyond treating it as additional data, opening the door for robust verifier‑guided methods for LLM‑based program synthesis and formal reasoning.
Authors:Siyuan Luo, Nairong Zheng, Lin Zhou, Tiankuo Yao, Shengyou Yuan, Haojia Yu, Cong Pang, Jiapeng Luo, Lewei Lu
Abstract:
Training capable OS agents requires data that simultaneously captures structured user intents, multi‑turn task delegation, and grounded tool execution‑‑properties absent from existing datasets. We propose ISE (Intent ‑> Simulate ‑> Execute), a three‑stage synthesis paradigm that addresses these gaps jointly. Stage 1 constructs roughly 50000 structured intents via a 4D framework (Persona x Domain x Task x Complexity); after deduplication the pool contains 43956 unique intents and attains a Vendi Score of 61.57 over the entire pool on mpnet‑base‑v2 embeddings (cosine kernel, q=1). Stage 2 drives multi‑turn user‑agent interaction through a role‑locked user simulator that grounds each user turn in actual execution outcomes, producing 23132 complete trajectories averaging 8.12 user turns and 68.24 total dialogue turns. Stage 3 runs every tool call inside a live, isolated OS workspace, generating authentic failure‑recovery dynamics instead of simulated responses. Fine‑tuning on ISETrace improves ClawEval pass@1 from 19.3 to 37.7 using Qwen3‑8B on agent tool‑use tasks with a standard protocol. This result outperforms zero‑shot GPT‑4o and the larger Qwen3‑32B base model which is four times bigger. An ablation on Stage 2 proves multi‑turn simulation brings a large portion of the performance gain. We release all source code and dataset at https://github.com/Valiere01/ISE‑Trace.
Authors:Mo Wang, Wenhao Ye, Junfeng Xia, Minghao Xu, Hongkai Wen, Quanying Liu
Abstract:
The success of large‑scale deep learning models in neuroscience is fundamentally constrained by severe data heterogeneity. Native fMRI data aggregated from diverse sources exhibit substantial variation in both spatial and temporal resolutions. Consequently, most existing frameworks rely on lengthy, rigid preprocessing pipelines that enforce uniformity across datasets. This practice introduces two critical limitations: (1) potential degradation of subject‑specific anatomical information; (2) significant computational overhead, often requiring hours of processing per subject. Here, we propose FlexiBrain, a resolution‑agnostic voxel‑level encoding framework for native fMRI based on Mamba‑JEPA. FlexiBrain defines patch sizes in real‑world physical units and employs a dynamic patch resizing, thereby bypassing destructive spatial standardization while enabling direct ingestion of data in native space. We instantiate the framework using an efficient Mamba‑JEPA backbone to model high‑dimensional 4D fMRI signals. Across five diverse downstream neuroscience tasks, FlexiBrain consistently outperforms recent state‑of‑the‑art methods, achieving gains of up to 12 percentage points without external data augmentation. Importantly, FlexiBrain functions as a seamless plug‑in module, substantially reducing preprocessing costs and accelerating the development of robust voxel‑level fMRI foundation models. Code is available at https://github.com/OneMore1/FlexiBrain.
Authors:Victor V. Albert, Philippe Faist
Abstract:
Barcode scans, clear phone calls, reliable data storage, satellite communication, and large‑scale quantum computation are all made possible by error correction. We present a handbook version of The Error Correction Zoo, a curated reference of methods for protecting classical or quantum information from errors during storage and transmission. The handbook includes descriptions of these error‑correcting codes and a classification according to the symbols they use. It also catalogues relations among codes and related objects such as sphere packings, lattices, designs, groups, and classical and quantum phases of matter. The collection is intended both as a rigorous reference and as a practical aid for tracing the web of code relationships and uncovering new connections.
Authors:Jebacyril Arockiaraj, Dhruv Parikh, Jayashree Adivarahan, Rajgopal Kannan, Viktor Prasanna
Abstract:
Federated continual learning (FCL) must learn from distributed task streams under limited resources, such as communication, computation, memory, and label availability. Existing FCL methods often rely on repeated local optimization, replay, and full supervision. Analytic alternatives avoid iterative training and replay, but using high‑dimensional random features to improve accuracy requires a second‑order feature statistic, the Gram matrix, which has a quadratic communication cost in the random feature size M. We propose FedRAN, a resource‑aware analytic FCL framework that replaces gradient‑based updates with compact random feature statistics. Each client transmits a truncated‑SVD summary of its Gram matrix, reducing the dominant second‑order upload from quadratic to linear in M for fixed rank. The server performs a two‑level QR‑SVD subspace merge, spatially across clients and temporally across tasks, and solves a ridge classifier in closed form. FedRAN further supports label scarcity through prototype‑based pseudo‑labeling. Across CIFAR‑100, ImageNet‑R, and VTAB datasets, FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6‑121.8× less per‑client communication than optimization‑based FCL, and is 190.3× faster on average than gradient‑based baselines; with only 20% labels, pseudo‑labeling improves average accuracy by up to 6.61 points. These results show that FedRAN enables accurate and resource‑efficient FCL under communication, computation, and label constraints. The source code is available at https://github.com/JebacyrilArockiaraj/Fed‑RAN‑SSL.
Authors:Shengkai Sun, Zhiyong Cheng, Zefan Zhang, Jianfeng Dong, Zhihui Li, Meng Wang
Abstract:
Recently, masked skeleton reconstruction models have emerged as strong action representation learners, driving significant progress in self‑supervised skeleton‑based action recognition. However, existing state‑of‑the‑art methods must predict an exceedingly large number of spatiotemporal patches, significantly prolonging training time. Besides, by treating all spatiotemporal regions equally during reconstruction, these models are distracted from learning the critical motion patterns that underlie action semantics. To address these challenges, we propose Adaptive Masked Reconstruction (AMR), a faster and stronger pre‑training framework. We first decouple the decoder from the encoder, enabling flexible prediction of larger spatiotemporal patches and dramatically reducing reconstruction complexity. Given that larger patches contain more complex information, which is challenging to predict and consequently degrades performance, we accordingly introduce an adaptive guidance module. This module identifies regions of high motion informativeness, guiding the model to focus on the most discriminative parts of each patch and alleviating reconstruction difficulty. Experiments on NTU RGB+D 60, NTU RGB+D 120, and PKU‑MMD datasets demonstrate that AMR not only accelerates pre‑training substantially but also improves downstream recognition accuracy, surpassing current state‑of‑the‑art approaches.
Authors:Kexin Ding, Yang Zhou, Can Jin, Feng Tong, Mu Zhou, Dimitris N. Metaxas
Abstract:
The growth of agent skills has transformed how agentic systems are built, evaluated, and deployed. As skill libraries continue to scale, rigorous evaluation becomes critical to ensuring their utility, quality, and safety in real‑world applications. Consequently, the field is undergoing an emerging paradigm shift from isolated skill creation to automated, evaluation‑driven skill evolution. In this survey, we systematically examine the landscape of skill evolution and evaluation beyond foundational skill creation. We categorize evolution into four distinct paradigms, spanning execution feedback, trajectory distillation, compression, and reinforcement learning, showing how each element contributes to improving skill utility and reliability. We also provide an analysis of six skill‑centric benchmark categories, identifying structural gaps in benchmark coverage, trade‑offs, and metric richness to advance skill research. Finally, we identify open directions for building skill ecosystems that are generalizable, efficient, and verifiably safe. The project URL is https://github.com/Cassie07/AgentSkill_Survey
Authors:S. H. Mirtajadini, N. Rublein, R. M. Ramakrishnan, G. ter Maat, M. Aldibaja, A. Y. Mersha
Abstract:
Most existing drone‑based inspection systems require the drone to fly dangerously close to the target or follow complex flight paths to capture small details. In addition, drone flight is affected by disturbances and localization inaccuracies, which can cause the drone to lose sight of its supposed target when it has a narrow view. Furthermore, trajectory planning often requires prior information about the target's geometry, position, and orientation, which is not always available for non‑structural targets such as trees, vehicles, or people. To address these challenges, this paper presents aerial_micro_inspection, a generic pipeline for aerial micro‑inspection across different use cases. The pipeline assumes a PX4‑powered drone equipped with two cameras: (i) a zoomed, gimbal‑mounted inspection camera that captures fine details without requiring the drone to fly very close to the target, and (ii) a wide‑field‑of‑view stereo navigation camera that acquires the target surface on site, estimates its range, and partitions it into smaller inspection regions. In addition, a vision‑based feedback loop compensates for drone motion while the inspection camera visits small partitions of a larger surface. We evaluate the pipeline in simulation and real‑world experiments, mainly in two use‑case scenarios: tree inspection for detecting oak processionary caterpillars and their eggs, and greenhouse inspection of sticky traps for detecting whiteflies. The results show improved coverage robustness under drone disturbances in simulation, as well as effective detection of caterpillars and eggs and high‑detail imaging of insects in real‑world experiments. The pipeline is open‑source, developed in ROS 2, and can be adapted to new applications by replacing the surface‑segmentation and micro‑target detection checkpoints. The code is available at: https://github.com/SaxionMechatronics/aerial_micro_inspection
Authors:Ayush Mittal, Dhruv Gupta
Abstract:
Compression progress is a long‑standing proposal for intrinsic motivation: reward an agent when its world model becomes better at predicting or compressing experience. The folk claim is that this reward is "credible" because it is paid only for learning. We make this precise and prove it. If intrinsic reward is the signed decrease of a fixed sealed‑audit loss, r_t = E(theta_t‑1) ‑ E(theta_t), then cumulative reward telescopes exactly to endpoint audit improvement, so no policy can push reward up indefinitely while true audit performance stagnates or degrades. For finite audit panels the same result holds with a sharp false‑positive budget: cumulative empirical reward is at most true audit improvement plus 2 Delta_n(F, delta), the uniform audit deviation of the model class. This is horizon‑free: adaptivity over time costs nothing once the sealed panel uniformly controls the class. The theorem also identifies the failure modes: the guarantee disappears if progress is clipped, scored on the agent's own stream, exposed to a high‑capacity model on a reusable panel, or applied to a neural class that makes Delta_n vacuous. We give a Lean 4 mechanization of the structural core (telescoping, the finite‑audit bound, finite Gibbs, and the entropy floor) and an experiment suite on ARC‑TGI grid‑transformation generators with adaptive holdout attacks. Experiments confirm the theory: finite‑audit deviation scales as n^‑0.527; signed progress resists clip‑farming, stream leakage, and noisy‑TV curiosity; naive reusable audits are exploitable by black‑box scalar feedback, while standard release defenses keep the attack below the 2 Delta_n threshold. Signed compression progress on a sealed audit is an accounting signal of genuine improvement.
Authors:Emily Nguyen, Yongchan Hong, Harsh Toshniwal, Yan Liu, Andreas Luttens
Abstract:
Deep learning models facilitate the discovery of molecules with tailored properties among billions of candidate compounds. However, the computational burden to develop and deploy state‑of‑the‑art models continuously increases, limiting their scalability. Most large‑scale models are unimodal in nature and overlook the potential to leverage complementary molecular data modalities. To address these shortcomings, this paper introduces the Graph‑Language Alignment for Chemical Inference and Exploration using Representations (GLACIER) model, a student‑teacher framework that integrates molecular graphs, SMILES strings, and physicochemical descriptors to learn rich molecular embeddings. Our framework consists of three stages: (1) we pretrain three student encoders on 100,000 drug‑like molecules: a message‑passing neural network for molecular graphs, a transformer‑based encoder for SMILES strings, and a multilayer perceptron for physicochemical descriptors, (2) we fuse these student modalities using a novel Finsler geometry‑aware module, and (3) distill complementary knowledge from large teacher models, including MiniMol and MolFormer, into a single lightweight model via contrastive learning. We demonstrate that GLACIER is a robust framework that delivers high predictive performance and computational efficiency in complex molecular property prediction tasks. Our code is publicly available at https://github.com/eemokey/glacier.
Authors:Orion Reblitz-Richardson
Abstract:
Standard linear probing declares a property "encoded" when a classifier on hidden states achieves high accuracy. The protocol works well on a snapshot but breaks across pre‑training: probe accuracy saturates within the first few thousand steps, leaving most of training invisible to the instrument. We introduce fragility, a complementary per‑layer metric defined as the activation‑noise level at which probe accuracy collapses. Fragility is sensitive to both the margin of separability and the redundancy of representation, both of which keep evolving long after accuracy plateaus. Applied to open‑checkpoint language models, fragility recovers structure that accuracy alone cannot see. Moralized representations emerge along a lexical \to compositional gradient: lexical moral detection first, compositional moral encoding later. Because probe accuracy on its own tracks how lexically separable a dataset is, we establish the compositional encoding directly, by showing it transfers across construction types that share no contrast tokens. A layer‑depth robustness gradient develops monotonically across training while accuracy stays flat. And matched fine‑tuning corpora that produce identical probing accuracy leave distinct fragility fingerprints, showing that data curation reshapes probe robustness without changing probe accuracy. In every comparison we test, where probing accuracy returns a flat answer, fragility returns a structured one.
Authors:Chia-Hsuan Chang, Haerin Song, Brian Ondov, Hua Xu
Abstract:
Structured abstracts are important for biomedical literature processing, by facilitating information retrieval, text mining, and knowledge synthesis. However, a vast portion of abstracts indexed in PubMed remain unstructured, presenting a significant bottleneck for downstream text‑processing workflows and applications. To resolve this limitation, we introduce Structured PubMed, a comprehensive corpus of section‑labeled biomedical abstracts compiled from the complete PubMed database, encompassing over 23.2 million research‑article records. The corpus is divided into two distinct subsets: a collection of 5.9 million author‑structured abstracts parsed from official XML files, and an automatically labeled collection of 17.2 million originally unstructured abstracts structured via a verbatim‑extraction Large Language Model pipeline. Every record is harmonized under a unified five‑section schema and mapped to its original PubMed identifier, publication type, and publication date. This dataset can be utilized to train sentence‑classification models, benchmark text‑segmentation architectures, and perform large‑scale, section‑specific information extraction at an unprecedented PubMed‑wide scale.
Authors:Barsat Khadka, Kawsher Roxy, Md Rubel Ahmed
Abstract:
Clock Tree Synthesis (CTS) is a computationally expensive stage in the physical design flow, requiring iterative EDA tool invocations to navigate a vast configuration space for optimal power, wirelength, and timing skew. Existing machine learning approaches require computationally expensive retraining or fine‑tuning cycles to adapt to unseen macro architectures and are architecturally mismatched to the millions of evaluations demanded by exhaustive combinatorial search. We present SwiftCTS, a physics‑informed surrogate framework that addresses both limitations simultaneously. By coupling lightweight, physics‑grounded statistical features with gradient‑boosted ensembles, SwiftCTS trains in under five seconds on a CPU and delivers sub‑millisecond inference without GPU support. To handle out‑of‑distribution (OOD) designs without retraining or fine‑tuning, we introduce a K‑shot multiplicative calibration mechanism that anchors predictions to just one or two physical reference runs, reducing power prediction error from 24.5% to 3.3% and wirelength error from 56.6% to under 1% on unseen macros. Integrating this engine with an evolutionary optimizer, SwiftCTS evaluates 100,000 CTS configurations in under ten seconds, yielding Pareto‑optimal frontiers that are physically validated within the OpenROAD flow. Closed‑loop validation confirms prediction errors below 0.5% for power and wirelength, and timing skew predictions within five picoseconds on an OOD benchmark, consistently outperforming default tool heuristics across all target metrics. Code publicly available at: \hrefhttps://anonymous.4open.science/r/SwiftCTS‑7E6Ehttps://github.com/BarsatKhadka/SwiftCTS
Authors:Kiril Georgiev, Yuxia Wang, Dimitar Iliyanov Dimitrov, Preslav Nakov, Ivan Koychev
Abstract:
Large language models are increasingly deployed across professional domains, bringing hard‑to‑predict risks, including the generation of harmful or disrespectful content. Although substantial progress has been made in developing safety evaluation datasets, existing resources remain overwhelmingly English‑ and Chinese‑centric. This limitation is particularly pronounced when evaluating languages that operate within shared sociocultural, legal, and ethical contexts. To address this gap, we introduce Schützen: a German‑‑Bulgarian safety dataset designed to assess model answerability under risk, covering both a low‑resource language (Bulgarian) and a high‑resource language (German). Experiments with multilingual and language‑specific LLMs reveal pronounced cross‑language differences in safety behavior, highlighting the necessity of tailored, region‑specific evaluation resources to support the responsible deployment of LLMs in Germany and Bulgaria. Datasets and code are available at https://github.com/xnlp‑lab/Schutzen. Warning: this paper contains examples that may be offensive, harmful, or biased.
Authors:Boya Zeng, Tianze Luo, Shu Pu, Jucheng Shen, Taiming Lu, Gabriel Sarch, Zhuang Liu
Abstract:
Diffusion models have consistently driven progress in text‑to‑image generation. However, it is challenging to attribute recent progress to specific modeling and data choices: state‑of‑the‑art open‑weight models provide limited ablations, and do not disclose their training data and full training details. The research community needs fully open (weights, data, and code) models as a foundation for further research; yet existing fully open models still fall significantly short of leading models in performance. In this project, we conduct a systematic investigation of the modeling and data design choices in text‑to‑image diffusion training and inference with 300+ controlled experiments totaling 700K+ TPU v6e hours. Our experiments highlight several empirical findings (e.g., equal weighting is a strong default for mixing curated datasets) and simple design decisions (e.g., larger text encoder adapters improve performance with minimal added parameters) for training strong models. Guided by these insights, we train i1, a 3B‑parameter text‑to‑image diffusion model using only publicly available datasets. i1 is competitive with leading models on five representative benchmarks (GenEval, DPG, PRISM, CVTG‑2K, and LongText), and outperforms the best existing fully open model by 29.5 absolute percentage points on average. We provide the i1 checkpoints, training and inference code, and the data processing pipeline. Together, our findings and the i1 recipe establish a practical foundation for future open research in text‑to‑image diffusion models. Our code is available at https://github.com/zlab‑princeton/i1.
Authors:Xurui Wang, Qin Ren, Jun Ma, Haibin Ling, Chenyu You
Abstract:
High‑content imaging assays quantify cellular responses to chemical and genetic perturbations, yet continuous trajectories of individual cells are unobservable because cells are chemically fixed at acquisition. Perturbation modeling therefore reduces to inferring stochastic transport between control and treated populations observed only as separate marginals. While recent generative models achieve strong end‑point alignment, boundary consistency does not determine intermediate evolution: multiple stochastic processes may connect identical marginals while traversing regions unsupported by observed single‑cell morphologies. We introduce FreeBridge, a Schrödinger Bridge formulation for single‑cell transition modeling under endpoint‑only supervision. FreeBridge defines atomic states as instance‑segmented single‑cell representations, establishing a fixed cellular manifold, and learns stochastic transport constrained within this geometry via empirical latent support regularization. Across BBBC021, RxRx1, and JUMP, FreeBridge maintains competitive or improved endpoint fidelity and mechanism‑of‑action retention under a unified evaluation protocol; on BBBC021, it further reduces intermediate support violations. These findings highlight the importance of geometric grounding for biologically interpretable perturbation dynamics. Project page: https://y‑research‑sbu.github.io/FreeBridge/.
Authors:Jia Li, Qian Chen, Wei Wang, Xinyu Li, Zhenzhen Hu, Dongsheng Shao, Richang Hong, Meng Wang
Abstract:
Personality assessment aims to infer stable personality traits from dynamic behaviors across language, voice, and facial cues. Since different personality dimensions are revealed through distinct behavioral perspectives, modeling trait‑specific evidence is challenging. However, most existing approaches adopt a uniform multimodal fusion strategy across all dimensions, assuming identical modality contributions. This overlooks trait‑specific modality preferences and introduces cross‑modal interference. To address this issue, we propose a novel personality assessment framework called Traits Run Deeper, which consists of three components. Specifically, the Multimodal Foundation Representation (MFR) module constructs personality‑oriented multimodal inputs and leverages psychology‑informed semantic templates as anchors, enabling foundation models to capture trait‑relevant information. Building upon MFR, the Trait‑Specific Modality Fusion (TSMF) module acts as an asymmetric fusion mechanism, allowing each dimension to selectively exploit different modality pathways from modality‑specific modeling to complementary fusion. Thus, TSMF captures heterogeneous modality preferences while reducing cross‑modal contamination. Furthermore, the Distribution‑Calibrated Personality Regression (DCPR) module mitigates label imbalance and central tendency bias through target distribution calibration, improving robustness and stability. Experimental results on the AVI Challenge 2026 validation set demonstrate the effectiveness of the proposed framework, reducing mean squared error (MSE) by approximately 25% compared with the baseline. Consistent improvements are observed on the official test set, where our method achieves the best performance and ranks first in the Personality Assessment Track. The source code will be made available at https://github.com/MSA‑LMC/AVI2026.
Authors:Yechan Kang, Yongjin Kweon, Mingyeong Seo, Sohee Park, Yeonguk Jeon, Jongkil Park, Hyun Jae Jang, Jaewook Kim, YeonJoo Jeong, Suyoun Lee, Seongsik Park
Abstract:
Training deep spiking neural networks (SNNs) remains challenging due to sharp loss landscapes and temporal inconsistency caused by surrogate gradients. To address these challenges, we propose a unified framework: adaptive and asymmetric surrogate gradients A2SG. The adaptive gradients adjust an effective window for spatio‑temporal adaptation, reducing spatial gradient variation and maintaining directional consistency of gradients over time. The asymmetric gradients reflect neuronal dynamics by assigning larger gradients to neurons with higher membrane potentials, and we prove that they yield lower variation than symmetric surrogates. Our analysis further establishes a direct connection between local gradient variation and the curvature of the loss landscape, providing a principled explanation for how A2SG promotes convergence to flatter minima and improves generalization. We conduct extensive experiments on diverse models, including CNN‑based and Transformer‑based SNNs, across various tasks such as image classification using both static and neuromorphic datasets, as well as segmentation. The results demonstrate that A2SG consistently improves accuracy and energy efficiency, establishing it as a general and reliable solution for training deep SNNs. Our code is available at https://github.com/KIST‑NCL/A2SG.git.
Authors:Suhang Li, Osamu Yoshie, Yuya Ieiri
Abstract:
Vision‑language reinforcement learning has recently shown strong target‑present localization for camouflaged object detection (COD). Yet localization is only one side of the decision: when the agent faces an ordinary image with no camouflaged target, will it still claim that a camouflaged object exists? Standard COD training and evaluation data are positive‑only, so agents optimized under this setting can acquire an over‑detect bias, a task‑specific form of object hallucination that standard COD evaluation leaves unmeasured. To quantify this target‑absent behavior, we construct Counterfactual COD (CF‑COD), a paired benchmark that removes the camouflaged target from each held‑out COD evaluation image while preserving a plausible background. CF‑COD evaluates whether a model detects the target on the original image and abstains on the target‑absent counterfactual, summarized by Pair Accuracy (PA). We further introduce CFCamo, a paired counterfactual framework for COD with abstention. For training, CFCamo optimizes a Qwen3‑VL‑4B‑Instruct agent with Counterfactual Sequence Policy Optimization (CSPO), which samples paired original‑counterfactual rollouts and uses a Counterfactual Paired Reward (CPR) to couple original‑image detection with counterfactual abstention. On CAMO‑test, CFCamo improves S_alpha by +3.7 pp over the prior RL‑based COD baseline; across CF‑COD, it reaches 80.0‑90.8% PA. Ablations show that removing counterfactual coupling reduces PA to 1.4‑5.2% despite strong target‑present COD scores, showing that target‑present evaluation alone does not characterize detect‑or‑abstain behavior. Overall, these results indicate that CFCamo improves COD agents by coupling target‑present detection with target‑absent abstention, rather than merely strengthening target‑present localization. Code and data are available at https://github.com/suhang2000/CFCamo.
Authors:Dmitriy Kompaneets
Abstract:
How much meaning does a text carry? Shannon's theory measures uncertainty over symbols and is intentionally indifferent to meaning, while pairwise metrics such as BERTScore compare two texts rather than characterizing one. We develop a geometric framework that measures semantic content from the structure of a text's sentence embeddings. The framework has three parts. First, within a fixed embedding and baseline, six natural axioms uniquely determine a scalar measure up to scale, a frame‑conditional uniqueness theorem. The resulting scalar is empirically too coarse, motivating a richer representation. Second, we propose a three‑coordinate semantic profile capturing novelty (displacement from generic discourse), breadth (diversity of distinct ideas), and integration (connectedness among them), together with a discrete minimal unit (the semantic quantum) whose resolution is fixed by a clustering threshold τ. Third, we prove a no‑go theorem: no scalar summary of the profile can simultaneously satisfy analytic stability under paraphrase and concatenation, ordinal robustness across text scales, and cross‑representation comparability. We exhibit two practical scalars, S_\mathrmminmax and S_\mathrmrank, each occupying a distinct corner of this trade‑off triangle. Validation across 23 synthetic categories, 5 Project Gutenberg novels, and 3 embedding models confirms the trade‑off. The recommended rank‑normalized configuration passes 25 of 28 ordinal checks as point estimates (21 of 28 after Benjamini‑Hochberg correction), outperforming seven baselines including unigram entropy and a BERTScore‑based novelty signal. A separate variational result connects the breadth coordinate to the log‑determinant of a determinantal point process (Spearman ρ= 0.985 over 507 Gutenberg chapters), giving an optimization‑theoretic foundation for breadth.
Authors:Jaspreet Singh Nahal
Abstract:
Standard Retrieval‑Augmented Generation (RAG) pipelines route every query through retrieval and generation unconditionally, incurring unnecessary computation and propagating low‑quality context to the generator. We introduce EverydayGPT, a lightweight conversational QA system built around a Confidence‑Gated Routing (CGR) mechanism that formalises the routing decision as a joint policy over retrieval distance and extraction adequacy. The backbone is a 205M‑parameter GPT trained from scratch on 10B tokens of FineWeb‑Edu. CGR avoids invoking the costly GPT pathway (~5.9s) for 85 percent of queries by resolving them via fast RAG extraction (~45 ms), yielding over 120x latency reduction on the majority of queries while maintaining answer quality. On a 500‑question in‑domain benchmark, the system achieves F1 = 0.226 +/‑ 0.004 compared to 0.171 for GPT‑only and 0.210 for unconditional RAG. Gains over strong baselines are modest but consistent, while efficiency improvements are substantial (6.3x mean latency reduction). A structured grounding audit finds no unsupported claims in the sampled set, with explicit scope limitations. We position this work as a study of routing strategies under resource constraints rather than a claim of state‑of‑the‑art performance.
Authors:Faruk Alpay, Bugra Kilictas
Abstract:
Structured sequence generation often requires a model to satisfy several input‑derived constraints in a single output. Standard decoding methods may assign high probability to fluent continuations while placing low mass on continuations that realize all required anchors jointly. We study this regime as a rare‑event sequential inference problem. LatticeBridge combines a compact prefix language model, instance‑compiled surface automata, and a twisted sequential Monte Carlo (SMC) decoder with resampling, multilevel splitting, and a source‑support proposal term derived from instance‑provided phrases. The constraint representation is compiled from each input instance and does not rely on manually curated lexical classes. On 2,610 attainable validation tasks spanning CommonGen, E2E NLG, and WikiBio, the particle decoder improves exact anchor satisfaction and mean anchor coverage over greedy, beam‑filtered, and best‑of‑k ancestral baselines under a shared proposal model. Since exact anchor satisfaction alone does not rule out unsupported attribute substitutions, the evaluation reports required‑anchor coverage, source coverage, source‑intrusion diagnostics, overlap, runtime, and particle statistics jointly. The benchmark characterizes the faithfulness‑overlap‑latency frontier under a fixed proposal model.
Authors:Jin Gan, Xin Li, Jun Luo
Abstract:
The wide deployment of LLMs has made model alignment necessary to make newly trained models safely and effectively respond to user instructions. Among different methods, inference‑time alignment is often cheaper as it intervenes (i.e., offers guidances) only during output generation. Existing proposals apply guidances extracted from certain aligned models without properly assessing their reliability. Nonetheless, our systematic evaluation reveals that guidance effectiveness varies drastically across models; since ineffective guidances lead to further confusion and thus further interventions, the resulting excessive interventions typically indicate poor performance. To make interventions more effective and thus more efficient, we introduce BlendIn, an inference‑time alignment framework that shifts from binary decisions to creating hybrid distributions integrating both models' knowledge. BlendIn stabilizes inference‑time alignment by performing quality‑aware alignment and proportionally weighting each model's contribution based on reliability. Compared with existing works, it preserves beneficial guidance while downweighting unreliable suggestions. BlendIn provides both diagnostic signals and mitigation strategies for misaligned guidance, achieving consistent and up to 50% performance improvement on challenging model pairs. Our code is available at: https://github.com/DecayingSeart/BlendIn.
Authors:Ilay Kamai, Hugues Van Assel, Aviv Regev, Hagai B. Perets, Randall Balestriero
Abstract:
Cross‑modal alignment (CA) and cross‑modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic understanding of when each succeeds, when each fails, and when cross‑modal training helps at all ‑‑ a gap that leaves practitioners, especially in scientific domains like biomedicine or astrophysics, with heterogeneous instruments and multiple levels of organization and measurement, unable to diagnose why standard methods underperform the best single modality. We develop a unified linear framework that addresses both questions. Under a spiked signal‑plus‑noise model with structured cross‑modal nuisance correlation, we derive separation ratios for both objectives that expose complementary failure modes: alignment whitens each modality and fails when nuisance is strongly correlated across views; prediction encodes whatever is cross‑predictable through a one‑sided whitening, with recovery governed by source‑modality quality. The resulting phase diagram partitions multimodal problems into four regimes: Both, CA only, CP only, and Neither. We present a data‑driven procedure to locate real‑world datasets in this diagram using a small labeled subsample, identifying the preferred objective and prediction direction before any cross‑modal training. Experiments on synthetic data, stereo‑vision benchmarks, image‑caption pairs, and real astrophysical data validate the predictions in the nonlinear regime, including the Neither regime where cross‑modal training is actively harmful. Our framework lets practitioners diagnose their multimodal problem and choose the right objective before committing to training. Code to reproduce the results is available at https://github.com/IlayMalinyak/mm_align_vs_pred.
Authors:Junke Wang, Xiao Wang, Jiacheng Pan, Xuefeng Hu, Feng Li, Jingxiang Sun, Chaorui Deng, Zilong Chen, Yunpeng Chen, Kaibin Tian, Matthew Gwilliam, Hao Chen, Danhui Guan, Kun Xu, Weilin Huang, Zuxuan Wu, Haoqi Fan, Yu-Gang Jiang, Zhenheng Yang
Abstract:
This paper introduces ARM, a discrete representation‑based AutoRegressive Model that unifies image understanding, generation, and editing within a next‑token prediction framework. ARM is built on three efforts: first, we train a discrete semantic visual tokenizer that maps images into compact token sequences. Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment and faithful reconstruction, thereby supporting diverse tasks in a shared latent space. With this, we train a 7B autoregressive model over large‑scale text and image token sequences, seamlessly developing vision‑language perception and generation capabilities. Finally, to further improve preference‑aligned behavior for text‑to‑image generation and instruction‑guided editing, ARM applies reinforcement learning (RL) to optimize task‑level objectives such as visual quality, instruction adherence, and edit consistency. Surprisingly, the results show that RL not only substantially improves performance on the target tasks (e.g., raising WISE overall from 0.50 to 0.56, GEdit‑Bench‑EN G_O from 5.75 to 6.68), but also induces cross‑task synergy between text‑to‑image generation and editing. Collectively, these findings highlight autoregressive modeling, when paired with strong representations and preference optimization, as a scalable foundation for multimodal intelligence. Code: https://github.com/wdrink/ARM.
Authors:Gangwei Xu, Qihang Zhang, Jiaming Zhou, Xing Zhu, Yujun Shen, Xin Yang, Yinghao Xu
Abstract:
Autoregressive video generation has emerged as a powerful paradigm for World Action Models (WAMs). However, existing approaches suffer from slow training convergence and limited converged accuracy, particularly at high frame rates, as the training supervision is confined to the current chunk without explicit signals about future dynamics; they also suffer from slow inference due to iterative video denoising. In this paper, we present Next Forcing, a multi‑chunk prediction (MCP) framework for causal world modeling that enables faster training, higher accuracy, and accelerated inference. Inspired by multi‑token prediction in large language models, Next Forcing introduces an MCP training objective that augments the main model with lightweight auxiliary MCP modules to simultaneously denoise video chunks at multiple future temporal horizons (next^1, next^2, next^3 chunks). These MCP modules form a causal chain across prediction depths, where intermediate features fused from multiple layers of the main model are leveraged to predict future dynamics, allowing near‑future predictions to inform farther‑future ones and providing dense multi‑scale temporal supervision back to the main model. During training, the MCP modules significantly accelerate convergence and improve converged accuracy, especially at high frame rates: at 50 fps, Next Forcing achieves a 93.1% relative improvement over LingBot‑VA at 5k training steps and 2.3x faster convergence, and establishes new state‑of‑the‑art results on the RoboTwin benchmark (94.1/93.5% on Clean/Random). At inference, the MCP modules can be retained to predict the next video chunk in parallel with the current one, achieving 2x inference acceleration. Next Forcing also demonstrates significant improvements on PhyWorld, a benchmark evaluating adherence to physical laws in video generation, and over 50% FVD reduction on general video pretraining.
Authors:Hangfeng Liang, Yutao Hu, Yanhan Hu, Xiaohan Wu, Wenqi Shao, Ying Fu
Abstract:
Low‑light video enhancement (LLVE) remains a challenging task due to severe information degradation under low‑illumination conditions. Recent multimodal approaches have significantly improved enhancement performance by incorporating auxiliary modalities, such as event streams and infrared images. However, these methods typically assume the availability of these modalities at inference, which is often not feasible in real‑world scenarios. To solve this problem, in this work, we propose AMNet, a unified multimodal framework for LLVE, to support flexible modality‑agnostic inference, where auxiliary modalities may be unavailable. To address the issue of modality absence, we introduce a Spatial‑Spectral Dual‑Gated Translator that learns the correspondence between auxiliary modalities and RGB inputs, producing implicit auxiliary representations to support the robust enhancement. Additionally, to fully facilitate the learning of cross‑modal correspondence, we conduct large‑scale multimodal pretraining based on the RGB‑only dataset with synthetic auxiliary modalities. Extensive experiments demonstrate that AMNet could handle arbitrary inference‑time modality combinations and exhibits superior performance for LLVE under modality absence conditions. Code and models are available on the project page.
Authors:Yujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang, Shuicheng Yan, Wenchao Ding
Abstract:
Contact‑rich manipulation requires robots to continuously perceive and regulate evolving physical interactions under dynamic contact transitions or complex surface geometries. Recent imitation learning methods improve contact‑aware control by incorporating tactile or force feedback, but they rarely model the asymmetric spatiotemporal roles of global force and local tactile sensing. To address this, we propose TacForeSight, a lightweight force‑conditioned tactile foresight framework for real‑time manipulation. The core component is TacForceWM, a tactile world model that predicts short‑horizon tactile latent dynamics from dual‑finger tactile observations conditioned on high‑frequency wrist force and torque signals. Another key component, the Predictive Tactile‑Conditioned Policy, leverages the predicted latents as anticipatory contact priors, models the current‑to‑future tactile evolution via cross‑attention, and adaptively fuses visuo‑tactile features through a tactile‑guided gating module. By forecasting purely within a compact latent space, TacForeSight enables proactive contact reasoning with efficient real‑time inference suitable for high‑frequency manipulation control. Real‑robot experiments on five representative tasks and three in‑process perturbation settings show that TacForeSight consistently outperforms existing baselines, particularly under dynamic contact disturbances. All models and datasets will be made publicly available on the project website at https://tacforesight.github.io/ProjectPage.
Authors:Paul Hyunbin Cho, Jinhyuk Jang, SeokYoung Lee, Joungbin Lee, Siyoon Jin, Heeseong Shin, Jung Yi, Yunjin Park, Chulmin Park, Seungryong Kim
Abstract:
Diffusion‑based lip synchronization models achieve strong visual quality and audio‑visual alignment, but full‑sequence bidirectional attention and many denoising steps make them impractical for real‑time inference. We present Lip Forcing, to our knowledge the first autoregressive diffusion method for video‑to‑video (V2V) lip synchronization, which distills a 14B audio‑conditioned bidirectional video diffusion teacher into causal students. At inference, the students generate each chunk in only two denoising steps without inference‑time CFG, enabling real‑time lip synchronization. A lip‑sync‑specific teacher‑trajectory analysis reveals a CFG fidelity‑sync tradeoff: no‑CFG predictions favor reference fidelity, whereas CFG‑guided predictions favor synchronization within a mid‑trajectory band. Lip Forcing translates this finding into three analysis‑derived components: Sync‑Window DMD, a two‑step inference schedule, and a SyncNet‑based reward. We validate Lip Forcing at two student scales, both distilled from the 14B teacher. The 1.3B student crosses into real‑time streaming at 31 FPS, 17.6× faster than its same‑scale bidirectional model. The 14B student, the largest diffusion model reported for V2V lip synchronization, runs 39.8× faster than its teacher at comparable reference fidelity. Time‑to‑first‑frame is sub‑millisecond at both scales, far below every diffusion baseline.
Authors:Kevin Qinghong Lin, Batu EI, Yuhong Shi, Pan Lu, Philip Torr, James Zou
Abstract:
Data tells stories that shape society; the data journalist's job is to turn raw information into stories non‑experts can trust. A high‑quality news feature takes a newsroom team weeks: hunting for context, running statistics, choosing an angle, and designing visuals. Recent agents handle individual steps well: data‑science agents close the analysis loop, while design agents synthesize beautiful websites. But can an agent serve as a data journalist end to end? We introduce Data Journalist Agent (Data2Story), a multi‑agent framework that orchestrates specialized roles into a single virtual newsroom. Data2Story contributes two innovations. (i) Claims are evidence‑grounded: an Inspector links every number, angle, and asset back to data, code, or an external reference. (ii) Articles are multimodally generative: rather than defaulting to plain text and static charts, Data2Story reasons about what readers will want to see, then deploys multimodal tools, such as interactive maps for geography and audio for music. We evaluate Data2Story on 18 articles, each paired with the originally published expert piece, along four axes: (a) human‑agent angle coverage; (b) rubric evaluation with 53 participants across five dimensions; (c) computer‑use agents as judges, a cost‑saving proxy for how readers navigate interactive articles; and (d) verifiability, where a coding verifier re‑executes statements against the data and checks claims against references. Data2Story produces competitive, evidence‑traceable multimedia stories, with particular strength in transparency and auditability. Human articles retain an edge in editorial angle, creative design, and presentation. We position Data2Story as a collaborator for journalists, enabling more evidence‑based, transparent, and verifiable reporting. Code and demos are available at https://data2story.github.io.
Authors:Yikang Yang, Zhanpeng Hu, Youtian Lin, Mengqi Zhou, Jingxi Xu, Feihu Zhang, Jiaheng Liu, Yao Yao
Abstract:
Multimodal large language models can write code to produce complex programs as well as use programs to do 3D modeling, which opens up a new avenue for 3D generation powered by their priors, world knowledge and reasoning. Yet existing benchmarks rarely evaluate 3D modeling through code. Such modeling demands more than runnable code: from a text or visual specification, a model must generate a parametric 3D program that is geometrically precise, semantically aligned and assembly‑consistent. We introduce P3D‑Bench, a benchmark for parametric 3D generation. Unlike a 3D mesh, a parametric 3D program exposes explicit dimensions, construction operations and part relations, revealing whether a model recovers a design's structure, not just its appearance. Under a unified protocol, P3D‑Bench covers three task families (Text‑to‑3D, Image‑to‑3D and Assembly‑3D) and scores each output for executability, geometric fidelity, topology, text‑grounded constraints, multiview semantic alignment and part‑level structure. We evaluate frontier MLLMs and text‑only LLMs on 400 text cases, 400 image cases and 203 annotated assemblies, with domain‑specific models as reference points. Our extensive evaluation yields three findings. First, assemblies are the hardest setting, where models still fail to compose multiple parts into a coherent structure. Second, models can often recover the global shape and semantic identity of the target object, yet fail to reproduce the precise parametric geometry specified by the input. Third, part‑level modeling remains weak on assemblies, where models recover neither the geometry of each part nor the right number of parts. These results position P3D‑Bench as a benchmark for evaluating precise parametric geometry and part‑level structure in parametric 3D generation.
Authors:Abhijoy Sarkar, Aarchi Singh Thakur
Abstract:
Resistance to first‑line osimertinib in EGFR‑mutant non‑small‑cell lung cancer (NSCLC) is the canonical example of predictable clonal evolution under therapeutic pressure, yet no public benchmark exists for training or evaluating computational models on the corresponding longitudinal patient trajectories. We introduce OncoTraj, a public benchmark of 813 EGFR‑mutant NSCLC patients receiving first‑line osimertinib, harmonized from three real‑world clinical‑genomic sources: MSK‑CHORD (672 patients), AACR Project GENIE BPC NSCLC (34 patients), and the FLAURA molecular‑resistance supplement (107 patients). OncoTraj defines three locked tasks: (A) binary classification of progression by a fixed 12‑month landmark, (B) regression of time‑to‑first‑progression in days, and (C) six‑class classification of the dominant resistance mechanism. We release the harmonized dataset, patient‑level train/validation/test splits with an audited no‑leakage guarantee, an open‑source evaluation harness, and six reference baselines spanning a majority‑class predictor, logistic regression, random forest, XGBoost, an LSTM, and a multi‑task transformer. With v1's single‑timepoint snapshot features, no task clears chance on clean within‑source evaluation: the uniformity of this ceiling across every model class localizes the limit to the input modality (single‑snapshot tissue NGS rather than serial ctDNA), not the algorithm. The benchmark does recover a reproducible literature‑consistent association: TP53 co‑mutation raises the 12‑month progression rate from 29% to 59% cohort‑wide. OncoTraj establishes a reproducible, leakage‑audited baseline and converts the modality limit into concrete design requirements for a serial‑ctDNA‑enriched v2.
Authors:Yuke Zhao, Wangbo Zhao, Weijie Wang, Zeyu Zhang, Dakai An, Akide Liu, Yinghao Yu, Jiasheng Tang, Fan Wang, Wei Wang, Bohan Zhuang
Abstract:
We introduce WorldOlympiad, a benchmark for diagnosing video‑based world models across physical faithfulness, geometric consistency, and interaction fidelity. While existing benchmarks often focus on visual quality, semantic alignment, or short‑term temporal coherence, they provide limited insight into whether generated videos obey physical rules, preserve coherent 3D structure, and sustain controllable interactions over long horizons. To address this gap, WorldOlympiad decomposes world‑model evaluation into three complementary dimensions. The physical track uses object segmentation and MLLM‑as‑judge to assess whether generated videos follow interpretable rules in mechanics, thermal phenomena, and material properties. The geometry track reconstructs generated videos with Gaussian splatting and evaluates structural consistency, cross‑view coherence, and camera‑trajectory alignment. The interaction track assesses whether generated rollouts follow complex action prompts and maintain smooth, coherent transitions across consecutive video chunks. WorldOlympiad further covers three major downstream scenarios, including gaming, robotics, and general real‑world videos, capturing diverse challenges from interactive control and embodied manipulation to open‑domain motion and camera dynamics. Together, these tracks and scenarios form a scalable and interpretable evaluation suite that exposes failure modes beyond generic video quality. Experiments on state‑of‑the‑art models reveal substantial gaps in physical reasoning, 3D consistency, and long‑horizon interaction, underscoring the need for more structured evaluation protocols for generative world models.
Authors:Fathima Mashood, Mohamed Nabeel
Abstract:
We present a longitudinal study of approximately 1.52 million malicious domains observed on VirusTotal (VT) between January and May 2026. Domains were selected on the basis of detection by at least five independent VT scanning engines and a first‑seen date within the study window. We group the dataset into compromised domains and attacker created domains, which account for approximately 89.3% of the dataset. Combining WHOIS registration records and passive DNS (PDNS) data with the VT dataset, we characterise attacker behaviour across eight dimensions: temporal distribution, compromisedvs.attack classification, domain age at first detection, registrar and TLD preferences, DNS query volume as a damage proxy, hosting infrastructure concentration (IP and ASN level), bulk registration patterns, and brand impersonation. Key findings include: the majority of attacker created domains are short lived registrations used within weeks of creation; a small number of registrars and TLDs account for most abuse; Cloudflare infrastructure is heavily exploited for domain fronting; bulk registration events involving thousands of domains from a single registrar on a single day are widespread; and several global brands, particularly WhatsApp and Google, are heavily impersonated. We share the annotated dataset in the GitHub repo https://github.com/mufimash/malicious_domains for further research.
Authors:Mahmood Alzubaidi, Uzair Shah, Raden Muaz, Ines Abbes, Nader Mohammed, Abdullatif Magram, Khalid Alyafei, Mowafa Househ, Marco Agus
Abstract:
A global shortage of trained sonographers limits prenatal ultrasound screening in low‑ and middle‑income countries, where over half of pregnant women receive no skilled sonography. Current deep learning approaches address detection, segmentation, or classification in isolation, each demanding a separate model and expert‑specified labels at inference. We present FADA, a unified vision‑language model built on Qwen3.5‑VL that performs clinical interpretation, classification, detection, and segmentation through a single interpretation‑first pipeline without external labels. FADA distills knowledge from four domain‑specific foundation models (FetalCLIP, UltraSAM, USF‑MAE, UltraFedFM) via offline pre‑computed feature caching. Selective distillation, which applies feature alignment only to annotation tasks while interpretation relies on standard fine‑tuning, consistently outperforms full distillation across most evaluation axes. The recommended variant, FADA‑SKD, achieves 0.8820 mean Dice for segmentation, 0.7671 mAP@0.50 for detection, and 100% structured interpretation compliance. Expert sonographer validation across 237 images confirms clinically acceptable outputs in both autonomous and human‑in‑the‑loop modes, with 73.5% of interpretations scoring perfectly under clinician guidance. The system is trainable on a single consumer GPU and deployable without cloud connectivity. We validate edge deployment by running the compressed 0.8B model on a commodity smartphone (Qualcomm Snapdragon 7 Gen 1, 12 GB RAM) using llama.cpp with GGUF quantization, completing the full 5‑phase pipeline in approximately 60 seconds entirely offline. This establishes a practical pathway for integrating AI‑assisted fetal assessment with portable ultrasound devices, directly addressing diagnostic access gaps in resource‑constrained settings. Code, models, and data are available at https://github.com/mahmoodphd/FADA.
Authors:Zach Moczkodan, Hany Ragab
Abstract:
Recent deep learning approaches for network intrusion detection increasingly incorporate temporal architectures such as recurrent networks and Transformers, often reporting near‑perfect performance on CIC‑IDS2017. However, many existing studies neither supply their temporal modules with genuine sequence inputs nor evaluate under realistic, leakage‑free conditions, making it unclear whether reported gains arise from true sequence‑modeling capability. In this work, we reformulate CIC‑IDS2017 as a temporal intrusion‑detection task by constructing ordered flow sequences from network conversations and benchmarking nine classical and deep learning architectures under a random split, two leakage‑free splits, and a padding‑scheme ablation. The central finding is that padding convention, not architecture, determines the Transformer's performance: on genuinely sequential (non‑padded) windows the Transformer achieves the highest macro‑F1 of any model in the experiment (0.89); under zero‑pad+mask evaluation it drops markedly (‑0.24 macro‑F1), while LSTM, GRU, and 1D‑CNN remain stable. Under leakage‑free group evaluation the Random Forest is the most robust model (+0.009), while the Transformer's false‑alarm rate grows from 0.04% to 2.7%, a 67‑fold increase invisible under conventional protocols. These findings demonstrate that evaluation methodology ‑‑ specifically padding convention and split protocol ‑‑ has a larger effect on reported performance than architectural choice, and that widely used random splits with repeat‑last padding can overestimate model robustness by up to 0.24 macro‑F1. We advocate leakage‑free splits, explicit padding disclosure, and sequence‑aware benchmarking as standard practice in future IDS research. Code and implementation details are available at https://github.com/zachmocz/temporal‑ids‑bench.
Authors:Yitong Chen, Zijie Diao, Junke Wang, Lingyu Kong, Yixuan Ren, Bo He, Yu-Gang Jiang, Zuxuan Wu
Abstract:
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine‑grained visual detail. This limitation becomes even more severe after discretization, where missing low‑level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high‑level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In‑depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.
Authors:Jaewoo Lee, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen, Supriyo Chakraborty, Kartik Balasubramaniam, Sambit Sahu, Elias Stengel-Eskin, Hyunji Lee, Mohit Bansal
Abstract:
Various test‑time interventions for Computer Use Agents (CUAs), including critic models, have been developed to improve performance through pre‑execution action evaluation in complex Graphical User Interface (GUI) environments. However, existing critics suffer from two key limitations: they (1) focus primarily on short‑sighted decision loops (e.g., forgetting earlier actions) and (2) lack the visual grounding needed to detect flawed actions (e.g., clicking wrong UI elements). To address these, we introduce HiViG, a History‑aware Visually Grounded test‑time framework, built around a multimodal critic trained on real GUI trajectories to abstract past interactions into a compact record and to evaluate actions with visual grounding. At test time, HiViG integrates the critic into the policy decision loop to provide macro‑action history, which summarizes the policy's completed achievements, and visually grounded critique, which verifies raw execution coordinates against the current screenshot to intercept errors before execution. Across web, mobile, and desktop benchmarks, HiViG consistently outperforms existing scalar and verbal critics, improving average success rates over the strongest baseline by 5.8% for Qwen3‑VL‑32B and 9.0% for Gemini‑3‑Flash, and demonstrates strong cross‑platform generalization. Ablations show that macro‑action history mitigates short‑sighted planning and visually grounded critique reduces execution errors, with both components being critical for test‑time scaling in long‑horizon GUI tasks.
Authors:Shuang Liang, Jubo Xu, Giulio Bassanino, Qianzhou Wang, Yidong Zhou, Yuncheng Lu, Zhiwen Mo, Paul H. J. Kelly, Bo Yuan, Wayne Luk, Hongxiang Fan
Abstract:
Reliable large‑scale quantum computation relies on fault‑tolerant architectures, where quantum error correction (QEC) continuously extracts and decodes error syndromes in real time. A critical component in QEC is the decoder, a classical subsystem that must simultaneously deliver high logical accuracy and ultra‑low latency. This paper presents a novel algorithm‑hardware co‑design that improves the accuracy‑latency trade‑off over existing approaches such as vanilla Minimum‑Weight Perfect Matching (MWPM) and Union‑Find (UF) decoders. At the algorithmic level, we introduce coset ensemble decoding, which improves UF decoding by explicitly exploiting logically equivalent cosets. Our method performs ensemble forest exploration to generate multiple coset‑consistent candidates and aggregates them to approximate coset‑level maximum‑likelihood decoding. We further reduce computational and memory complexity via reverse‑order elimination and lossless graph compression, without sacrificing accuracy. At the hardware level, we design a domain‑specific architecture that temporally reuses resources, avoiding the code‑distance‑proportional resource growth in prior spatial architectures. Several optimizations, such as multi‑bank memory hashing and hierarchical ID mapping, are proposed to mitigate pipeline stalls and memory conflicts under highly concurrent access patterns. Under a circuit‑level depolarizing noise model, our co‑design approach achieves a better accuracy‑latency trade‑off than prior MWPM‑ and UF‑based decoders, while reducing FPGA LUT consumption by up to 8.2 times compared with reported UF‑based decoder resources. The tunable candidate number further exposes a flexible design knob, enabling users to tailor decoding performance to the requirements of different fault‑tolerant workloads. Our implementation is publicly available at https://github.com/IMSeonL/coset‑ensemble‑decoder.
Authors:Zhiwen Yang, Jiayin Li, Hao Lu, Hui Zhang, Zihua Wang, Bingzheng Wei, Yan Xu
Abstract:
Existing deep learning models for Positron Emission Tomography (PET) image denoising often suffer from severe performance degradation under distribution shifts, fundamentally restricting their robust clinical deployment. This lack of generalization stems from the conventional paradigm of fixed‑parameter models that cannot adapt to variations in test data (e.g., dose levels or scanner types) after training. To overcome this limitation and achieve robust generalization, we introduce U‑TTT, a novel U‑shaped model that integrates Test‑Time Training (TTT) layers to dynamically adjust model parameters during inference through self‑supervision, thereby adapting to the specific characteristics of each test instance. Furthermore, to comprehensively capture the complex degradations of 3D PET data, U‑TTT features a dual‑domain adaptation mechanism comprising a Spatial Test‑Time Training (S‑TTT) layer and a Frequency Test‑Time Training (F‑TTT) layer. The S‑TTT layer captures and corrects spatial structural degradations, while the F‑TTT layer suppresses global noise spectra and restores delicate high‑frequency details. Extensive experiments demonstrate that U‑TTT achieves state‑of‑the‑art PET denoising performance and exhibits superior generalization under challenging distribution shifts, including both unseen dose levels and unseen scanners. Our code will be available at https://github.com/Yaziwel/U‑TTT.
Authors:Bowen Ping, Xiangxin Zhou, Penghui Qi, Minnan Luo, Liefeng Bo, Tianyu Pang
Abstract:
Recent work has demonstrated that online reinforcement learning (RL) can substantially improve the quality and alignment of flow matching models for image and video generation. Methods such as Flow‑GRPO and CPS cast the denoising process as a Markov Decision Process and apply PPO‑style ratio clipping to enforce a trust region. However, we argue that ratio clipping is structurally ill‑suited for flow models: the probability ratio between new and old policies is a noisy, single‑sample estimate of the true policy divergence, leading to over‑constraining in some regions of the trajectory and under‑constraining in others. We propose Flow‑DPPO (Flow Divergence Proximal Policy Optimization), which replaces ratio clipping with a divergence proximal constraint. A key observation is that the per‑step policy in flow models is Gaussian, enabling exact and cheap computation of the KL divergence between old and new policies. Flow‑DPPO employs an asymmetric divergence mask that blocks gradient updates only when they simultaneously move away from the trusted region and violate the divergence threshold. Experiments show that Flow‑DPPO achieves higher rewards with better KL‑proximal efficiency, alleviates catastrophic forgetting, promotes balanced multi‑objective optimization, and enables stable multi‑epoch training where ratio clipping degrades. Code and models are available at https://github.com/Tencent‑Hunyuan/UniRL/tree/main/FlowDPPO.
Authors:Yifan Li, Jiahong Liu, Xinni Zhang, Hao Chen, Yankai Chen, Wenhao Yu, Jianting Chen, Irwin King
Abstract:
Sequential recommendation aims to predict users' next interaction with items by analyzing their historical behavior. However, the limited quality of item representations remains a critical bottleneck. While pre‑trained large language models (LLMs) can provide rich semantic representations, existing approaches only rely on static encoding of fixed attributes, overlooking the crucial role of target audiences in defining item identity. Moreover, the semantic space struggles to reflect actual user behavior, resulting in a significant gap between semantic representations and behavioral patterns. To address these limitations, we propose GenAIR, a general framework that empowers sequential recommendation with Generative Archetype‑grounded Item Representations. Specifically, we first leverage an LLM to analyze item metadata and infer textual description of the Archetype, which represents the conceptual profile of the item's ideal target audience. We then extract the corresponding embeddings in a single forward pass. Further, to ground these generative archetypes in real‑world behavior, we introduce a behavioral calibration objective, which explicitly incorporates behavioral signals from actual interactions. This objective adjusts the structure of the embedding space to reflect empirical patterns. GenAIR enables seamless integration with most existing models while maintaining high efficiency. Comprehensive experiments conducted on three real‑world datasets demonstrate that GenAIR significantly improves the performance of various sequential recommendation models and consistently outperforms state‑of‑the‑art baseline approaches. Implementation codes are available at https://github.com/AI‑Santiago/GenAIR.
Authors:A Pranav, Vagrant Gautam, Martin Mundt, Jordan Taylor, Arjun Subramonian, Franziska Sofia Hafner, Daniel Chechelnitsky, William Agnew, Anne Lauscher
Abstract:
In academic publishing, names connect scholars to their work. When scholars change their names, including for marriage, academic recognition, or gender transition, they may lose credit for past publications. However, despite significant impacts on citation accuracy and researcher well‑being, no existing studies examine how naming policies in computer science serve researchers who change their names. We use a mixed‑methods approach combining surveys, interviews, and large‑scale citation analysis of papers from eight major computer science venues from 2019‑2025. We document the multi‑year advocacy effort that established the first name change policies, identify implementation barriers including incomplete publisher updates and months‑long processing delays. Researchers continue being cited with misparsed and incorrect names despite publisher updates. When these citation errors happen, interviewees report significant mental health impacts, including stress, anxiety, and safety risks. Empirically, we find that venues with accessible and visible name change policies have significantly fewer citation errors compared to inaccessible policies (899 vs. 996 errors per 1,000 papers). Our annotation analysis shows that deadnaming of transgender researchers in citations decreased by 92% from 2019 to 2024. Our findings demonstrate the importance of inclusive publishing policies, for which name change policy advocacy led by trans researchers has been a significant driver. We recommend that venues adopt proactive visible name change policies, support queer advocacy groups, and improve publication infrastructure to build an inclusive publishing landscape. The accompanied toolkit to check errors in bibliographic latex file is available here https://github.com/pranav‑ust/cite‑updater.
Authors:Chirag Shinde
Abstract:
We present Density Field State Space Models (DF‑SSM), a framework for compressing SSMs to a 1‑bit scaffold with int8 low‑rank correction. Applied to Mamba‑2 1.3B, we achieve a 278 MB model (9.7x smaller than the 2.7 GB FP16 teacher) that runs at 21.4x faster inference on GPU (batch=1, relative to the mamba‑ssm reference implementation) while maintaining downstream task performance within 2‑4 percentage points of BitMamba‑2, a 1.58‑bit model trained from scratch on 150B tokens. The distillation itself requires only 32M tokens and 6 hours on a single A100 GPU, though it presupposes a pretrained FP16 teacher. We develop an optimized inference pipeline combining cuBLAS INT8 tensor cores for the scaffold matmul, custom CUDA kernels for stateful SSM and convolution operations, and an AVX‑512 CPU backend for efficient deployment on both GPU and CPU. Beyond compression, we investigate the internal knowledge organization of the resulting model, discovering three distinct processing phases: intent classification (layers 0‑3, operating in an abstract space with no vocabulary alignment), knowledge retrieval (layers 25‑35, where factual associations localize to a 5‑layer window), and output formatting (layers 36‑47, where category structure dissolves). Through systematic analysis of 445 factual prompts across 19 categories, we find that early‑layer classification is syntactic (driven by template structure) rather than semantic, and that the model exhibits well‑organized knowledge representations despite weak factual recall‑‑suggesting that representational structure may precede factual strength.
Authors:Hang Yin, Yinan Liang, Jiazhao Zhang, Jiahang Liu, Minghan Li, Zhizheng Zhang, He Wang
Abstract:
Lifelong embodied navigation in dynamic environments requires robots to form persistent scene understanding from fragmentary observations, which remains difficult for existing methods that rely on explicit maps or scene graphs and struggle to generalize beyond structured settings. We propose AllDayNav, a lifelong self‑learning navigation framework that implicitly encodes scene dynamics into the billion‑scale parameters of a large model via reinforcement learning, powered by a self‑evolving multimodal memory that maintains and updates visual keyframes, semantic descriptions, and temporal context while autonomously generating open‑vocabulary instructions, image goals, and structured rewards. Experiments in both synthetic and real‑world environments across cross‑room, cross‑episode, and cross‑task scenarios show that AllDayNav achieves success rates approaching 100% and consistently surpasses strong map‑based, VLM, and RL baselines in path efficiency and robustness, demonstrating implicit, memory‑driven reinforcement learning as a scalable alternative to explicit mapping for reliable lifelong navigation.
Authors:Artur Kuramshin, Özgür Aslan, Cyrus Neary, Glen Berseth
Abstract:
The recent trend in scaling models for robot learning has resulted in impressive policies that can perform various manipulation tasks and generalize to novel scenarios. However, these policies continue to struggle with following instructions, likely due to the limited linguistic and action sequence diversity in existing robotics datasets. This paper introduces Task Robustness via Re‑Labelling Vision‑Action Robot Data (TREAD), a scalable framework that leverages large Vision‑Language Models (VLMs) to augment existing robotics datasets without additional data collection, harnessing the transferable knowledge embedded in these models. Our approach leverages a pretrained VLM through three stages: generating semantic sub‑tasks from original instruction labels and initial scenes, segmenting demonstration videos conditioned on these sub‑tasks, and producing diverse instructions that incorporate object properties, effectively decomposing longer demonstrations into grounded language‑action pairs. We further enhance robustness by augmenting the data with linguistically diverse versions of the text goals. Evaluations on LIBERO demonstrate that policies trained on our augmented datasets exhibit improved performance on novel, unseen tasks and goals. Our results show that TREAD enhances both planning generalization through trajectory decomposition and language‑conditioned policy generalization through increased linguistic diversity.
Authors:Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao
Abstract:
Large language models (LLMs) rely on tool use to act as autonomous agents, yet often fail in multi‑step execution due to insufficient tool‑related knowledge and ineffective knowledge activation. Therefore, we present a systematic study on how knowledge influences tool‑use performance, covering the stages of knowledge acquisition, activation, and internalization. In the knowledge acquisition stage, we acquire and evaluate various forms of experiential knowledge, and our analysis shows that simple instance‑level knowledge can already provide strong and reliable gains, while abstract intent‑level knowledge offers limited benefits. At inference time, to activate knowledge, we find that prompting LLM to expand the depth of reasoning yields diminishing returns, whereas expanding the width of reasoning by parallel sampling with aggregation more effectively activates latent experiential knowledge. At training time, for knowledge internalization, post‑training with knowledge‑augmented data further improves performance, with reinforcement learning outperforming supervised fine‑tuning. Based on these insights, we propose the Knowledge‑Augmented Tool Execution (KATE), a knowledge‑augmented tool execution framework that integrates experiential knowledge with reasoning‑width‑expanded inference and knowledge‑aware training. Experiments on BFCL‑V3 and AppWorld demonstrate consistent and substantial improvements over strong baselines across model scales. Our Code is available at https://github.com/hypasd‑art/KATE.
Authors:Taishan Li, Jiwen Zhang, Siyuan Wang, Xuanjing Huang, Zhongyu Wei
Abstract:
Vision‑Language‑Action (VLA) models achieve strong performance on standard manipulation benchmarks, but most evaluations assume that task‑relevant objects are fully visible. This assumption often fails in realistic settings, where occlusion makes manipulation partially observable. In this paper, we study scene‑induced occlusion as a fundamental challenge for VLA models and introduce LIBERO‑Occ, an occlusion‑oriented extension of LIBERO. Experiments show that state‑of‑the‑art VLAs suffer substantial performance degradation under occlusion. To address this issue, we propose Viewpoint Imagination (VIM), which generates a complementary view from an occluded primary observation and conditions action prediction on both observed and imagined evidence. VIM improves robustness across task suites, occlusion types, and severity levels without requiring additional cameras at deployment time, suggesting that viewpoint imagination is an promising mechanism for perception completion in partially observable manipulation. Our benchmark and corresponding code are available at: \hrefhttps://github.com/litsh/Libero‑Occhttps://github.com/litsh/Libero‑Occ.
Authors:Cong Wang, Zhentao Yu, Hongmei Wang, Weicong Liang, Zixiang Zhou, Zilin Yang, Jiarong Ou, Rui Chen, Yuan Zhou, Qinglin Lu
Abstract:
Current identity‑consistent video generation methods struggle to preserve appearance fidelity under large viewpoint changes. While introducing multi‑view reference input offers a natural solution, progress remains constrained by the lack of effective frameworks for multi‑view inputs and the scarcity of multi‑view data. We address these challenges by proposing HarmoView, a robust framework for identity‑consistent video generation that effectively integrates multi‑view cues through three architectural refinements complemented by a staged training curriculum. Specifically, we first introduce Multi‑level Feature Injection to anchor identity fidelity; by injecting raw ViT features from frontal references alongside text tokens via cross‑attention, MFI provides persistent low‑level appearance anchors that complement the high‑level identity features within DiT blocks, leading to enhanced identity preservation. Then, we employ learnable proxy tokens to unify heterogeneous reference layouts across single‑/multi‑view settings while simultaneously resolving the reference‑view mismatch problem. Jump‑RoPE is further developed for identity‑wise feature isolation to reduce identity crosstalk. To activate these structural capabilities while preserving the original generative priors, we propose the Progressive View Curriculum. This four‑stage training strategy employs view dropout to facilitate a stable transition from vanilla T2V generation to high‑fidelity, identity‑persistent spatial reasoning. Furthermore, we construct a large‑scale multi‑view dataset to address the issue of data scarcity. Extensive evaluation on our multi‑view benchmark, comprising 100 manually‑curated cases spanning 52 unique identities, demonstrates that HarmoView significantly outperforms open‑source baselines and matches leading closed‑source engines, achieving state‑of‑the‑art performance in identity‑consistent video generation.
Authors:Zhiwei Tang, Yuanyu He, Yizheng Han, Wangbo Zhao, Jiasheng Tang, Fan Wang, Bohan Zhuang
Abstract:
Autoregressive (AR) language modeling is the dominant paradigm for text generation, yet its sequential token‑by‑token decoding makes inference memory‑bound and inefficient. Existing acceleration approaches, such as speculative decoding and diffusion language models, can yield speedups under certain conditions but do not directly address high‑load batch serving‑‑the scenario most critical for industrial‑scale deployment. We introduce K‑Forcing, a push‑forward language modeling paradigm for joint next‑k‑token decoding. K‑Forcing distills an existing AR model into a conditional push‑forward mapping‑‑one that transforms independent uniform noise variables into a joint sample of multiple future tokens in a single forward pass. This design preserves fixed‑length outputs, reuses the AR teacher backbone, and remains compatible with standard AR serving infrastructure. We train this mapping via progressive self‑forcing distillation, which gradually expands the prediction window while enabling the student to closely match the sequence distribution of the AR teacher. We evaluate K‑Forcing on LM1B and OpenWebText using a standard causal Transformer backbone. When aggressively configured to generate k = 4 tokens per forward pass, K‑Forcing delivers approximately 2.4‑3.5x speedup across different batch sizes, while incurring modest quality degradation relative to its AR teacher. As inference increasingly dominates the lifetime compute cost of modern LLMs, K‑Forcing offers a promising route toward accelerating AR generation under real‑world high‑load deployment.
Authors:Shuwen Xu, Zhitao He, Yi R. Fung
Abstract:
Users rely on execution traces to observe agent behavior, diagnose failures, and ensure accountability. These traces contain rich procedural detail, including tool invocations, intermediate decisions, and error‑recovery logic. Yet this detail can expose private procedural skills, allowing downstream methods to recover key formulas, thresholds, and strategies without access to model weights or skill files. To quantify this risk and evaluate protection, we construct \textscCapTraceBench, a benchmark of 75 specialized long‑horizon tasks and 154 curated skills across seven domains. We also introduce \textscRedAct https://github.com/XuShuwenn/RedAct, a protected trace release framework that localizes protected key information, rewrites traces while preserving verifier‑critical evidence, and embeds behavioral watermarks for downstream provenance analysis. Across representative trace reuse methods, \textscRedAct reduces normalized skill transfer (NST) from 44.7‑‑67.1% on raw traces to below the no‑skill baseline, while preserving audit evidence. Its standalone behavioral watermarks reach 93.6‑‑100.0% true detection with a false alarm rate of at most 1.9%. These results frame public agent traces as security interfaces and show that selective redaction can reduce procedural capability leakage without removing audit evidence.
Authors:Wenhao Yan, Fengjia Guo, Zhuoyi Yang, Jie Tang
Abstract:
Controlled character animation requires transferring motion from a driving sequence to a reference character. Prior works heavily rely on intermediate representations, including pose skeletons to represent motion or masked background to represent environment, which inevitably leads to information loss. To address this, we present SCAIL‑2, a framework that bypasses those intermediates and achieves end‑to‑end character animation. By directly concatenating driving videos to the sequence, the model can obtain all the required visual information from the input video. To address the lack of end‑to‑end data, we unify sub‑tasks of character animation with decoupled conditions and then curate a pipeline to synthesize MotionPair‑60K, an end‑to‑end motion transfer dataset containing heterogeneous tasks of character animation. To achieve the unification, we utilize in‑context mask conditioning and mode‑specific RoPE as soft guidance beyond textual instructions and raw visual information. To address synthetic discrepancy in detailed regions, we propose Bias‑Aware DPO to construct preference items to mitigate the errors. Extensive experiments demonstrate that our method substantially outperforms existing state‑of‑the‑art approaches in various character animation tasks. A large subset of synthetic data as well as model weights will be released at our project page: https://teal024.github.io/SCAIL‑2/.
Authors:Xukun Zhu, Hang Yu, Peng Di, Linchao Zhu
Abstract:
The success of Large Language Models in mathematical reasoning relies heavily on the generation of diverse and valid solution paths during the rollout phase. However, current rollout techniques face a fundamental trade‑off: token‑level sampling often yields redundant trajectories that differ only in rephrasing, while embedding‑level methods utilizing random noise frequently disrupt semantic consistency. To resolve this, we introduce N‑GRPO, a novel exploration strategy integrated into the Group Relative Policy Optimization (GRPO) framework. Rather than relying on token‑level sampling or native embedding‑level noise, our approach leverages Semantic Neighbor Mixing. This mechanism dynamically constructs input representations by mixing the embeddings of an anchor token and its nearest semantic neighbors, thereby injecting diversity while strictly adhering to the local semantic manifold. Experimental evaluations on the DeepSeek‑R1‑Distill‑Qwen models across different sizes show that N‑GRPO not only achieves consistent improvements over strong baselines on math reasoning benchmarks but also exhibits robust generalization capabilities on out‑of‑distribution tasks.
Authors:Qiaoxin Li, Caini Pan, Pierre-Antoine Comby, Chaithya Giliyar, Philippe Ciuciu
Abstract:
Accelerated acquisition of fMRI enables enhanced detection of neurovascular (BOLD) activity in the brain, but image reconstruction becomes challenging with high k‑space undersampling: Task‑evoked BOLD signals are small in magnitude, which traditional anatomical MRI reconstruction methods fail to recover, as they favor spatial accuracy over temporal fidelity. We present DD‑INR, a Dynamics‑Driven Implicit Neural Representation framework tailored for accelerated fMRI that benefits from incoherent time‑varying sampling and a tailored spatiotemporal prior, outperforming traditional methods, demonstrated in simulation and in‑vivo acquisition, both in terms of image quality and retrieval of activation patterns. DD‑INR achieves this by splitting the fMRI data into a static background and a temporally varying dynamic component, representing only the dynamics with a dedicated INR, thereby focusing the model's capacity on activation‑relevant changes while remaining compact. In general, DD‑INR provides a promising framework for accelerated fMRI reconstruction, with the potential to improve the sensitivity and robustness of fMRI studies within practical scan time limits. The source code is available at https://github.com/JoosenLi/DD‑INR.
Authors:Filippo Tonini, Federico Torrielli, Anton Danholt Lautrup, Peter Schneider-Kamp, Mustafa Mert Çelikok, Lukas Galke Poech
Abstract:
As AI systems built from multiple language‑model agents become more common, they are increasingly used to make decisions together: discussing, negotiating, and acting on shared tasks. While individual agents may appear well‑aligned when tested on their own, problems can arise from how they interact with one another. We introduce the Arbiter, an agent designed to monitor multi‑agent conversations in real time and identify which participants may be behaving in misaligned ways. The Arbiter operates under a limited "inspection budget", meaning it must decide carefully how to use its resources. As it observes a conversation step by step, it can choose to wait, question a participant, examine internal information such as system prompts or reasoning traces, or log concerning behavior. At the end, it produces a report identifying the likely source of misalignment. We evaluate the Arbiter across five conversation conditions, ranging from risky financial advice model organisms to evaluation‑aware and colluding agents, we test five tool configurations of increasing capability and two backbone models. We find that the Arbiter reliably detects misaligned agents well before the end of the conversation, with active inspection tools improving both detection accuracy and speed. Weight‑induced misalignment proves hardest to detect, while instruction‑induced misalignment is identified reliably even under passive observation. The logging tool exhibits a dual effect, improving recall at the cost of precision. These results suggest that continual, budget‑aware monitoring can effectively catch misalignment, and that overseeing multi‑agent systems may require treating the auditor as an active participant in the process. The code is available at https://github.com/aisilab/arbiter.
Authors:Zhiyuan Zhu, Yixuan Chen, Yiwen Shao, Wenxiang Guo, Changhao Pan, Yu Zhang, Yuxiang Wang, Wei Liu, Houhua Zhang, Chengkuan Zeng, Wenbo Cheng, Yunxi Liu, Rui Yang, Steve Yves, Liefeng Bo, Zhou Zhao
Abstract:
Recent multimodal large language models mainly process audio as monaural signals, thereby discarding the spatial cues contained in spatial audio for sound localization, spatial relation reasoning, and spatial scene understanding. We propose Spatial‑Omni, a lightweight method that implements SO‑Encoder to inject First‑Order Ambisonics (FOA) spatial audio into existing Omni LLMs as an independent modality, without modifying their original audio encoders. SO‑Encoder provides spatial tokens with limited additional context cost and improves spatial audio understanding through efficient staged training. To support training and evaluation, we construct SO‑Dataset, SO‑QA, and SO‑Bench from open‑source data, real recordings, and simulations, containing 400K FOA spatial audio clips and 2.1M spatial question answering pairs. SO‑Bench covers 16 spatial audio understanding subtasks, including basic detection and location estimation, spatial relation understanding, and complex spatial reasoning. Experiments show that Spatial‑Omni outperforms existing open‑source Large Audio‑Language Models (LALMs) and Omni LLM models on spatial audio understanding tasks while retaining a reasonable level of general audio understanding. Code and data are available at https://github.com/dieKarotte/Spatial‑Omni.
Authors:Ana Sofia Santos, André Ferreira, Gijs Luijten, Naida Solak, Lisle Faray de Paiva, Behrus Hinrichs-Puladi, Jens Kleesiek, Jan Egger, Victor Alves
Abstract:
The nnU‑Net has demonstrated continuous success in medical segmentation tasks, which heavily rely on the availability and diversity of annotated biomedical data. However, assembling medical imaging cohorts remains challenging due to numerous factors such as privacy regulations and annotation costs. As a result, data augmentation plays a crucial role in increasing data availability while maintaining anatomical feasibility. Hence, we propose the ++nnU‑Net, a novel data augmentation module based on image registration that operates prior to preprocessing and training take place. Our framework was evaluated across five different 2D datasets. In this workflow, image data go through a two‑stage registration process, generating new warped images. The transformations are then applied to the respective segmentation. In addition, the pipeline computes available disk space, generates supplementary binary synthetic masks and generates checkpoints. We demonstrate that the ++nnU‑Net outperforms the nnU‑Net baseline, yielding improvements in Dice Similarity Coefficient scores. In the most prominent cases, we observe performance gains of approximately 22%. These findings highlight the effectiveness of registration‑based data augmentation, particularly for 2D medical imaging datasets and suggest that the ++nnU‑Net provides a practical and scalable approach for enhancing segmentation performance in data‑limited settings. The source code for the ++nnU‑Net is available at: https://github.com/sofia‑adelie/plusplusnnunet.git
Authors:Yinglong Yan, Yunkai Yang, Haoyi Wang, Wei Fu, Linshan Wu, Honghu Pan, Shaobo Xia, Shanghang Zhang, Hao Chen, Leyuan Fang
Abstract:
Remote sensing vector mapping aims to generate structured maps of geospatial entities, such as buildings, roads, and water bodies, from remote sensing imagery. In practice, vector maps usually contain multiple category layers and heterogeneous entity structures, requiring a unified model for diverse mapping needs. However, existing methods typically represent vector objects as polygons or graphs, making them suitable only for specific categories: polygons poorly capture topological relations, while graphs often blur instance boundaries. We observe that language, as a natural medium for human communication, offers a flexible and expressive representation that can accommodate heterogeneous map elements, including geometry, semantics, and topolog. Motivated by this insight, we propose Vector Map as Language (VecLang), a unified paradigm that reformulates multiclass vector mapping as structured text generation. VecLang encodes the common elements of different geospatial entities into a GeoJSON‑like vector language, enabling cross‑category modeling within a shared textual format. To generate this language reliably, we design a progressive vision‑language mapping framework that first localizes vectorization units and then generates structured map elements. We further introduce Hierarchical Vector Language Optimization, which uses reinforcement learning to improve syntax validity, content fidelity, and map executability. We also build VecMap‑Bench with 54K images and 800K instances, supporting training and evaluation across standard and generalization settings. Extensive experiments demonstrate that VecLang handles both single‑class and multiclass vector mapping while achieving strong cross‑dataset and open‑vocabulary generalization. The model and dataset are publicly available at https://github.com/yyyyll0ss/VecLang.
Authors:Justin Berman, Francois Charton, Andres Luna, Matthias Wilhelm, Mao Zeng
Abstract:
In this paper, we use machine learning to discover a new seeding strategy for integration‑by‑parts reduction of Feynman integrals, which is a frequent bottleneck in state‑of‑the‑art calculations in theoretical particle and gravitational‑wave physics. Our strategy allows us to reduce multi‑loop integrals with large numerator powers via essentially the standard Laporta algorithm but with a sparse selection of seed integrals that grows only linearly with the numerator power, whereas existing strategies lead to growth with a polynomial power that increases with the complexity of the integral being reduced. The seeds are restricted to a thin tube‑like region that connects the target integral to the master integrals along a zigzag path. We demonstrate the power of our approach by reducing non‑planar 2‑loop 5‑point integrals of rank 20 with numerical kinematics over a finite field, which is prohibitively difficult for the Laporta algorithm with conventional seeding. Going beyond individual integrals, we further demonstrate the reduction of a complete set of top‑level rank‑10 integrals by dividing the target integrals into several chunks, each of which can be solved by our sparse seeding strategy with considerably less time and a significantly lower memory footprint than other state‑of‑the‑art strategies, making the approach well‑suited for phenomenological applications. We provide a proof‑of‑principle implementation on GitHub at https://github.com/andreslunagodoy/tube_seeding.
Authors:David P. Hofmeyr
Abstract:
Although some very common test beds exist for assessing the performance of clustering methods, large scale benchmarking is typically limited to relatively simplistic simulation set‑ups. Here we describe the production and curation of close to 3000 synthetic data sets, derived from more than 200 publicly available data sets; the majority of which arose from real‑world applications. By fitting a flexible non‑parametric distribution to each base data set we are able to retain much of the nuance in real‑world data which is difficult to reproduce in standard simulations, while also producing data sets whose sizes are sometimes substantially greater than the data sets from which they are derived. The synthetic data sets, plus an accompanying R package, are available for download from https://github.com/DavidHofmeyr/ClusBench.
Authors:Yuzhen Mao, Azalia Mirhoseini
Abstract:
Multi‑agent systems (MAS) can scale large language model reasoning at test time by decomposing complex problems into parallel subtasks. However, most existing MAS rely on centralized orchestration, where a main agent assigns work, collects outputs, and merges results. As the number of subtasks grows, this controller becomes a communication and integration bottleneck. We propose Decentralized Language Models (DeLM), a MAS framework that decentralizes coordination through parallel agents, a shared verified context, and a task queue. Agents asynchronously claim subtasks, read accumulated progress, perform local reasoning, and write back compact verified updates. The shared context acts as a common communication substrate, enabling agents to build on one another's verified progress without routing every update through a central controller. Empirically, DeLM improves both software‑engineering test‑time scaling and long‑context reasoning. On SWE‑bench Verified, DeLM achieves the best performance across Avg.@1, Pass@2, and Pass@4, with gains of up to 10.5 percentage points over the strongest baseline, while reducing cost per task by roughly 50%. On LongBench‑v2 Multi‑Doc QA, DeLM achieves the highest average accuracy across four frontier model families, improving over the strongest baseline by up to 5.7 percentage points. The code is available on our project website at https://yuzhenmao.github.io/DeLM/.
Authors:Qi Song, Yifei He, Chi Zhang, Zheng Fu, Xuhe Zhao, Mengmeng Yang, Kun Jiang, Rui Huang, Diange Yang
Abstract:
Forecasting the future evolution of dynamic scenes is crucial in autonomous driving. However, existing feed‑forward paradigms are primarily designed for interpolation. When extended to future extrapolation, they suffer from ghosting artifacts under large displacements and are constrained by simplified motion assumptions or strict future priors. To overcome these challenges, we propose Envision4D, a fully self‑supervised feed‑forward framework for pose‑free future extrapolation. Specifically, we introduce a Future Pose Prediction module that infers future camera parameters via an iterative denoising process. Furthermore, to capture non‑linear dynamics, we propose In‑layer Temporal Attention and employ Conditioned Motion Lifting, which transforms the highly uncertain extrapolation process into robust relational mappings. Finally, a Progressive Training Strategy is utilized to stabilize unsupervised motion learning against error accumulation. Extensive experiments demonstrate that Envision4D achieves state‑of‑the‑art performance, significantly outperforming existing methods in future view synthesis.
Authors:Emma Kasteleyn, Timo Maier, Axel Lauer, Veronika Eyring, Pierre Gentine, Ana Lucic
Abstract:
Machine learning weather prediction (MLWP) models have achieved impressive forecasting performance at a small fraction of the computational costs required for traditional physics‑based methods. However, they are primarily (1) data‑driven and (2) evaluated using pixel‑wide error metrics (e.g., RMSE), so there are no guarantees that their forecasts are consistent with known physical laws. We introduce PhysMetrics.Weather, an evaluation framework that assesses the physical realism of MLWP models across three types of metrics: conservation, spectral, and dynamical. By quantifying physical realism, this tool guides the development of physics‑informed architectures and helps evaluate whether MLWP models are reliable for operational use. Our framework is available on Github at https://github.com/Emmakast/PhysMetrics.Weather.
Authors:Hao Liu, Ruping Cao, Kun Wang, Zhiran Li, Fan Liu, Yupeng Hu, Liqiang Nie
Abstract:
In this report, we present our champion solution for the DataMFM Challenge Track 2: Chart Understanding. This track requires models to recover structured chart data and generate faithful natural‑language summaries from chart images. To address the complementary requirements of accurate data extraction and factual narration, we propose ChartLens, a dual‑branch framework for chart data correction and summary refinement. ChartLens consists of two key modules: Structure‑Aware CSV Verification and Correction (SAVC) and Text‑Retention‑Guided Summary Refinement (TRSR). SAVC improves the reliability of structured data extraction through verification and correction, while TRSR enhances summary generation by preserving critical textual and numerical evidence from charts. By combining model adaptation, correction‑based generation, and OCR‑assisted evidence grounding, ChartLens improves both structured data recovery and summary factuality. On the test set, our final system achieves an overall score of 69.10 and ranks first in Track 2, demonstrating its effectiveness for accurate chart understanding. Our code will be released at: https://github.com/iLearn‑Lab/CVPRW26‑ChartLens.
Authors:Zhengxuan Wei, Yi Dong, Zonghui Li, Xianhui Lin, Xing Liu, Hong Gu, Shaofeng Zhang, Wenbin Li, Qi Fan
Abstract:
Low‑Rank Adaptation (LoRA) merging can efficiently combine diverse generative capabilities from multiple trained LoRAs for a diffusion model. However, existing LoRA merging techniques often suffer from severe parameter interference, causing destructive collisions in the shared parameter space. To address this, we propose Subspace Signal Routing (SSR), which resolves interference by routing internal signals instead of performing parameter‑space merge. Specifically, SSR first constructs a unified subspace by concatenating candidate LoRAs along the rank dimension. Next, SSR employs an inverse correlation matrix to decorrelate mixed signals within this space. Finally, a directional guide matrix steers these purified signals into their respective task‑specific subspaces. We provide a rigorous theoretical analysis proving that SSR aligns with the Ordinary Least Squares (OLS) solution, thereby ensuring mathematical optimality. We utilize the additivity of sufficient statistics to design a streaming algorithm. This enables on‑the‑fly updates that significantly reduce memory overhead and computation time. Extensive experiments validate that SSR significantly outperforms state‑of‑the‑art methods while maintaining comparable efficiency. Code is available at https://github.com/nagara214/SSR‑Merge.
Authors:Haoliang Han, Ziyuan Luo, Renjie Wan
Abstract:
3D Gaussian Splatting (3DGS) is a powerful technique for creating high‑fidelity 3D assets. However, the widespread sharing and iterative modification of 3DGS models across digital platforms create pressing challenges for intellectual property protection and forensic traceability. To address this, we propose GaussTrace, a novel framework for constructing directed provenance graphs for 3DGS models. GaussTrace formulates provenance analysis as an evidence‑based reasoning problem. It builds upon attribute‑wise statistical profiling of 3DGS parameters to capture intrinsic properties. Moreover, we introduce hypothesis‑driven editing simulations of common operations to provide auxiliary evidence for plausible transformation pathways. These statistical and simulated cues jointly enable a Large Language Model (LLM) to perform structured Chain‑of‑Thought (CoT) reasoning, yielding directional provenance inferences and explainable edge reasons. Experimental results demonstrate that GaussTrace effectively constructs evolutionary relationships among diverse 3DGS models, delivering accurate, interpretable, and robust provenance graphs without requiring model training or access to editing histories. Project page: https://haolianghan.github.io/GaussTrace.
Authors:Mao Chen, Xu Yang, Chuankai Liu, Xiangkai Zhang, Xiaoxue Wang, Zheng Bo, Zuoyu Zhang, Zhiyong Liu
Abstract:
Precise rover localization is a prerequisite for autonomous lunar exploration, yet the absence of Global Navigation Satellite System (GNSS) signals and the cumulative drift of local localization methods severely constrain long‑range missions. Cross‑view localization provides a promising drift‑free global solution by matching rover‑view and satellite‑view imagery. However, the lunar environment poses unique challenges for correspondence alignment, including inter‑entity entanglement, inter‑viewpoint divergence, and simulation‑to‑real domain shift. To address these challenges, we propose Warped Alignment of Reprojected Graphs (WARG), a framework that leverages unified graph learning and reprojected graph matching for robust cross‑view alignment. Pretrained on the synthetic LuSNAR dataset, WARG achieves an average test error of 0.32 m and demonstrates robust zero‑shot generalization to the synthetic lunar south pole region with an error of 3.63 m. More importantly, when validated on real‑world data from the YuTu‑2 rover, WARG achieves a localization error of 1.68 m within a 100 m x 100 m search area, corresponding to nearly one‑pixel precision in low‑resolution satellite imagery with a spatial resolution of 1.40 m/pixel. Beyond accuracy, WARG is computationally efficient, containing only 1.56M parameters, corresponding to 16.12% of previous lightweight models, and operating at 5.49 Hz on an NVIDIA RTX A6000 GPU, approaching GNSS‑level update frequency. Finally, we observe that WARG naturally develops low‑level spatial awareness, including semantic segmentation and structural reasoning, through cross‑view localization learning, highlighting its potential as a promising paradigm for spatial intelligence with minimal annotation cost. The source code is available at https://github.com/maochen‑casia/warg.
Authors:Zhi Zheng, Ziqiao Meng, Hao Luan, Wei Liu, Wee Sun Lee
Abstract:
External memory effectively grounds large language models (LLMs) and vision‑language models (VLMs)‑based question answering (QA) in relevant multimodal evidence. However, existing memory paradigms represent each memory item in raw text and image forms, so retrieval‑based systems must pass the retrieved text or images to the generation LLMs/VLMs, resulting in high token consumption and storage pressure, making it unaffordable for resource‑constrained applications. We propose Latent Memory, a latent‑space memory paradigm that replaces each raw text or image evidence item with a single high‑dimensional latent token produced by a small compressor LLM/VLM. Rather than retrieving raw evidence for generation, Latent Memory operates in a unified latent representation space: the query is embedded into this space to retrieve relevant latent tokens, and the retrieved latent tokens are directly prompted to a pretrained LLM or VLM for answer generation. To make each latent token simultaneously informative for reconstruction, retrieval, and generation, we train the compressor with reconstruction, contrastive, and distillation objectives in a unified end‑to‑end manner. Latent Memory is evaluated on seven text‑only QA benchmarks (e.g., HotpotQA) and multimodal QA benchmarks, where it achieves competitive QA performance compared to advanced RAG baselines while consuming 3x to 10x fewer generator tokens. It can also deliver the strongest image‑grounded QA performance on WebQA. Code is available at https://github.com/zz1358m/Latent‑Memory‑Master.
Authors:Jing Xiong, Qi Han, Shansan Gong, Yunta Hsieh, Chengyue Wu, Chaofan Tao, Chenyang Zhao, Ngai Wong
Abstract:
Diffusion large language models (dLLMs) re‑encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long‑context scenarios. We propose Prefilling‑dLLM, a training‑free prefill‑decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top‑K most relevant chunks with intra‑chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per‑step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling‑dLLM achieves state‑of‑the‑art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non‑contiguously cached chunk KV yields 9.1‑‑28.0x speedup at 8K‑‑32K contexts. We further show that beginning‑of‑sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost‑in‑the‑middle phenomenon. Code is available at https://github.com/menik1126/Prefilling‑dLLM.
Authors:Haodong Lei, Hongsong Wang, Bingxuan Dai, Pan Zhou
Abstract:
The growing need for high‑resolution image generation in autoregressive text‑to‑image models has resulted in extended token sequences, significantly increasing computational costs and inference times. However, existing state‑of‑the‑art methods for accelerating autoregressive text‑to‑image models rely on chain‑structured draft token sequences, leading to inefficient draft token search and limited acceptance lengths. To address this, we propose parallel‑path cross‑relaxed speculative Jacobi decoding (PathSpec), a novel framework that enhances efficiency through a multi‑sequence draft tree structure. Our parallel‑path speculative Jacobi decoding (PathExplore) expands the token search space, achieving a higher speedup ratio without sacrificing image quality. Additionally, we introduce cross‑path relaxed verification (PathRelax) that exploits semantic similarities across sequences to further boost token acceptance rates. Evaluated on the Parti‑Prompts, MSCOCO2017, and T2ICompBench datasets, our method achieves a speedup ratio of 4.14 ×, 3.95×, and 4.18×, respectively. Remarkably, PathExplore, without any relaxed sampling, outperforms relaxed sampling methods in the speedup ratio, such as GSD and LANTERN. Moreover, PathRelax's relaxation mechanism can be seamlessly integrated with other relaxation techniques, enabling further acceleration and providing an efficient solution for real‑time text‑to‑image generation. Our code is available at https://github.com/Haodong‑Lei‑Ray/PathSpec.
Authors:Yifan Zhu, Can Lin, Hangjie Yuan, Zixiang Zhao, Pengfei Zhang, Tao Feng, Zhonghong Ou
Abstract:
Parameter‑Efficient Fine‑Tuning (PEFT) methods provide a streamlined and efficient tool for adapting large models to domain‑specific multimodal downstream tasks. Although these methods proved their tangible effects in practice, their principal aspects remain under‑explored. Therefore we remain curious about the underlying generalization mechanisms in various PEFT methods and how they can be further enhanced. In this paper, we reveal the flatness preference widely present in various PEFTs, where a small fraction of sharp dimensions dominates the generalization of PEFT. This finding suggests an appealing possibility: we may be satisfied with a better generalization by merely attending to this small fraction of sharp dimensions instead of all of them. Furthermore, we propose Flatness Preference Optimization (FlatPO) to flatten these key sharpness dimensions, leading various PEFTs toward better generalization. Extensive experiments demonstrate the effectiveness of our findings and the proposed method. Code is available at https://github.com/Can‑Lin/FlatPO.
Authors:Shunkai Zhang, Haoran Zhang, Yun Luo, Qianjia Cheng, Haodi Lei, Yizhuo Li, Runzhe Zhan, Zhilin Wang, Bangjie Xu, Yucheng Su, Xinmiao Han, Xiaoye Qu, Dongrui Liu, Zhouchen Lin, Yu Qiao, Ning Ding, Yafu Li, Yu Cheng
Abstract:
Combinatorics is central to Olympiad‑level mathematical problem solving, requiring deep discrete reasoning, creative constructions, and rigorous structural insight. Recent evidence suggests that even today's strongest frontier models remain uneven on Olympiad combinatorics, revealing a gap in creative mathematical reasoning. We introduce ComBench, an Olympiad‑level combinatorics benchmark for evaluating and diagnosing the combinatorial reasoning capabilities of large language models. ComBench contains 100 human‑annotated competition‑level problems organized around two complementary settings: analysis‑centric problems, which primarily require rigorous mathematical arguments, and construction‑centric problems, which require explicit constructions in addition to correctness justifications. The evaluation protocol combines rubric‑guided proof grading with deterministic construction verification, exposing cases where proof quality and construction validity diverge. Experiments on frontier open‑ and closed‑source models show that ComBench is far from saturated: the strongest model reaches 65.4% overall Avg. and 75.3% overall Best@4. We further find that Rigorous Proof Reasoning and Constructive Realization are distinct capabilities: Kimi‑K2.6 trails GPT‑5.5 on analysis‑centric proof grading but surpasses it on construction‑centric Best@4, while Existence and Construction problems remain consistently hardest across representative frontier models.
Authors:Zeyu Liu, Xuanzhi Feng, Sing Kwong Lai, Yuanchen Gao, Xiaoyi Pang, Hualei Zhang, Jingcai Guo, Jie Zhang, Song Guo
Abstract:
The financial market is a typical low signal‑to‑noise ratio (SNR) setting, which often destabilizes off‑policy maximum‑entropy methods like Soft Actor‑Critic (SAC). Specifically, noisy state representations may produce unreliable Q‑value estimates, and bootstrapping amplifies these errors, forming a failure mode we call the "Financial Entropy Trap". In this paper, we propose FPQC‑SAC, an efficient and plug‑and‑play SAC variant that places a compact and bounded Parameterized Quantum Circuit (PQC) before the actor and critic networks to constrain feature propagation at the representation level, rather than filtering raw inputs or regularizing Q‑values after bootstrapping. Notably, FPQC‑SAC reduces the impact of extreme market fluctuations on Bellman target estimation, while trainable quantum entanglement preserves flexible cross‑asset interactions. Empirical evaluations on real‑world portfolio management tasks demonstrate that FPQC‑SAC substantially enhances out‑of‑sample stability and cumulative returns by achieving a 66.89% relative gain in cumulative return over standard unconstrained SAC and outperforms the best continuous‑control deep reinforcement learning baseline by approximately 27%. Open‑source code is available at https://github.com/ZeyuLIU‑UST/FPQC‑SAC‑main.
Authors:Muhammad Ahmed
Abstract:
Transformers achieve strong language modeling performance by providing direct token‑to‑token communication paths, but causal self‑attention scales quadratically with context length. Recurrent and state‑space models reduce this cost, yet compress history into sequentially updated fixed‑size states. This paper studies a third primitive: a parallel content‑addressed memory over causal successor records. The proposed Parallel Causal Associative Field (PCAF) writes local records from a context window into hash buckets, retrieves a bounded candidate set for the current query, forms a sparse cache distribution over successor tokens, and mixes that cache with a parametric local language model through a learned gate. The resulting model maintains sparse long‑context access while avoiding a single fixed recurrent state bottleneck. We evaluate PCAF under full autoregressive pretraining on WikiText‑103 and PG‑19 using a distributed Google Cloud TPU v4‑32 pod. At 303M parameters and context length T = 2048, PCAF‑semantic reaches 36.31 perplexity on WikiText‑103 and 52.45 perplexity on PG‑19, compared with 47.49 and 53.84 for a matched dense Transformer. PCAF‑semantic simultaneously processes 0.61‑0.62M tokens/s across the TPU pod, versus 0.43M tokens/s for dense and local attention baselines. Supporting 41M‑parameter multi‑seed sweeps and single‑GPU component ablations show that the associative cache, retrieval capacity, and learned gate materially affect the speed‑quality trade‑off.
Authors:Jayoo Hwang, Xiaowen Zhang, Vedant Padwal
Abstract:
Autonomous web navigation remains challenging for LLM agents, and the strongest generalist systems rely on proprietary reasoning models whose inference cost is prohibitive for the repetitive tasks where such agents would be most useful. We argue this gap stems not from insufficient model capability but from agent architectures that fail to replicate three human cognitive advantages: selective attention to relevant page regions, persistent memory of website structure, and procedural fluency with common interaction patterns. We introduce WebChallenger, a web agent framework that addresses each gap through architecture design rather than model scale, built around PageMem: a structured page representation deterministically constructed from the DOM that exposes each page as a hierarchy of semantic sections with short summaries. On this shared substrate we build three mechanisms that mirror the three cognitive advantages: a divide‑and‑conquer observation pipeline that lets the agent skim section summaries and extract details only from task‑relevant regions; a lightweight exploration and memory system that traverses each website once to build a reusable map of pages and element behaviors; and compound action workflows that collapse common multi‑step interactions into single agent actions, handling partial state changes automatically. Because all three operate over PageMem, the framework generalizes across websites without site‑specific adapters. Using off‑the‑shelf open‑weight models without fine‑tuning, our system achieves 56.3% on WebArena, 48.7% on VisualWebArena, 51.0% on Online‑Mind2Web, and 70.9% on WorkArena, approaching frontier proprietary systems at a fraction of the cost. Our code is released at https://github.com/jayoohwang1/webchallenger
Authors:Sanghee Park, Geewook Kim, Kee-Eung Kim
Abstract:
Math reasoning benchmarks have proliferated, yet most lack a per‑item difficulty signal grounded in actual human performance. We introduce KCSAT‑ML, a decade (2014‑2025) of Korean College Scholastic Ability Test (KCSAT; Suneung) mathematics: 664 problems with a 339‑item core set carrying official per‑item error rates from nationwide cohorts of hundreds of thousands of examinees. We pair the benchmark with Difficulty‑aligned Reasoning Gain (DRG): a score‑orthogonal metric that asks whether a model's mistakes concentrate on the items humans found hard, or on items humans found easy. Together they expose, across a wide range of VLMs (and LLMs via OCR), three patterns: (i) low‑budget accuracy collapses on the high‑human‑error tail at every model size; (ii) test‑time scaling (TTS) raises token use roughly linearly with cohort error rate, while accuracy gains follow a non‑monotonic curve; (iii) within a single family, TTS flips between anti‑scaling on the hardest items and overthinking on easier ones ‑‑ two faces of the same alignment failure. On DRG, models with near‑identical accuracy can sit at near‑opposite values: one model gets wrong what humans also find hard, while another solves the hardest items yet fails on items humans find easy ‑‑ a contrast that aggregate accuracy hides. Our code and dataset builder will be open‑sourced at https://github.com/naver‑ai/KCSAT‑ML.
Authors:Haoran Li, Zengle Ge, Ziyang Zhang, Xiaomin Yuan, Yui Lo, Qianhui Liu, Bocheng An, Dongke Rong, Jiaqun Liu, Annan Li, Jianmin Wu, Dawei Yin, Dou Shen
Abstract:
Recent advances in LLM‑driven code evolution have enabled automated discovery by iteratively generating and improving programs. However, applying these methods to adversarial multi‑agent games introduces a fundamental challenge: the evaluation landscape shifts as strategies improve, causing fixed evaluators to become unreliable and evolution to stagnate. We propose three mechanisms to address this challenge: evaluator co‑evolution, which incorporates discovered champions into the opponent pool; hierarchical deep evaluation, which replaces noisy few‑game scores with statistically reliable assessments; and weakness pressure, which dynamically up‑weights the most difficult opponents to break through plateaus. We implement these mechanisms within FAMOU, a framework built upon the same foundation‑model code‑evolution paradigm as OpenEvolve and ShinkaEvolve. On the MCTF 2026 3v3 maritime capture‑the‑flag task, FAMOU consistently outperforms both baselines under two backbone LLMs, achieving the highest combined score (0.526) and the best generalization to unseen opponents (61.7% win rate), while ablations confirm that each mechanism contributes to performance. Notably, the LLM mutation process generates tactical structures entirely absent from the seed strategies ‑‑ including lookahead search and adaptive interception ‑‑ demonstrating that code‑level evolution can produce nontrivial algorithmic innovations in adversarial settings. The FAMOU‑evolved strategy further achieved 1st place in the hardware round‑robin and 3rd in simulation at the AAMAS 2026 MCTF Competition, validating its real‑world transferability. The optimized implementation and corresponding evaluation codes developed through our evolutionary process are available at: https://github.com/1xiangliu1/FAMOU‑CoEvo
Authors:Wenhao Zhang
Abstract:
On‑policy distillation (OPD) has demonstrated strong empirical gains in enhancing complex reasoning in LLMs by aligning a student model with a teacher's predictive distribution over the student's own trajectories. An emerging variant, Privileged OPD, further strengthens this paradigm by employing a self‑teacher model augmented with privileged information, such as oracle traces, to mitigate teacher‑student capacity gaps while providing dense, answer‑directed supervision. However, current methods treat privileged information as a monolithic imitation target, failing to disentangle locally reachable reasoning steps from future‑conditioned oracle signals. Consequently, the student is encouraged to match a hindsight‑biased distribution that often falls outside its local predictive support. This reachability mismatch incentivizes the student model to skip valid intermediate reasoning in favor of locally unsupported shortcuts. To resolve this, we introduce Anchored Residual On‑Policy Distillation (AR‑OPD), a dual‑view framework that disentangles privileged supervision. Rather than enforcing strict full‑view imitation, AR‑OPD establishes a locally compatible anchor using a partially privileged teacher, isolating and injecting oracle foresight as a controlled residual to provide destination‑directed guidance. Across diverse reasoning tasks, AR‑OPD outperforms full privileged OPD by 2.3 points and SFT by 7.9 points. Crucially, this anchored residual mechanism reduces hindsight leakage by 21.7% and mitigates late‑stage drift, yielding up to a 7.2‑point advantage on challenging long‑horizon trajectories exceeding 768 tokens.
Authors:Xuanchen Li, Tianrui Wang, Yuheng Lu, Zikang Huang, Yu Jiang, Chenghan Lin, Chenrui Cui, Ziyang Ma, Xingyu Ma, Chunyu Qiang, Guochen Yu, Xie Chen, Longbiao Wang, Jianwu Dang
Abstract:
Speech‑to‑text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens. In contrast, continuous‑target language modelling performs generation in a continuous space, yet its potential for S2T remains unexplored. To bridge this gap, we propose ELF‑S2T, an audio‑conditioned continuous‑target generative model for S2T. Built upon the pre‑trained Embedded Language Flows (ELF) backbone, ELF‑S2T processes speech via a frozen Whisper encoder and a single linear projector, prepending the resulting audio condition to the noisy text latent for in‑context, flow‑matching denoising. To prevent the model from over‑relying on its pre‑trained text context, we introduce audio forcing during training, and further amplify the audio condition via classifier‑free guidance at inference. Experiments on LibriSpeech and CoVoST2 show that ELF‑S2T achieves competitive ASR and S2TT performance. Crucially, our error analysis reveals that, although ASR and S2TT errors look very different on the surface, both stem from the same underlying cause, a close distance confusion in the continuous latent space. This finding naturally aligns with the continuous representation generation paradigm, indicating a common semantic mapping process beneath recognition and translation. Our code and pretrained models are publicly available at https://github.com/Sslnon/ELF‑S2T.
Authors:Siqiao Huang, Kun-Ying Lee, Dongming Qiao, Guanqi He, Zhenyu Wang, Yitang Li, Shaoting Zhu, Hang Zhao
Abstract:
Humanoid whole‑body control has made significant progress in recent years, yet existing approaches remain limited to few‑skill policies with heavy reward engineering, or motion trackers that are difficult to extend to new input modalities. We argue that the key to general‑purpose humanoid control is to build a scalable brain, a module capable of reasoning with diverse conditioning modalities, atop a reactive motion tracking cerebellum, mirroring the hierarchical structure of biological motor systems. Two challenges arise in realizing this vision: acquiring a vast amount of high‑quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, extensible multi‑modal inputs. We present OMG, which addresses these challenges with a meticulous data curation, filtering and labeling pipeline, as well as a diffusion‑based motion generation backbone that conditions on language, audio, and human reference motions. Extensive experiments validate OMG as an omni‑modal whole‑body controller exhibiting state‑of‑the‑art performance, model scaling behavior and efficient adaptation to new distributions and modalities, marking a concrete step toward foundation models for humanoid robots.
Authors:Dahye Kim, Jaehyun Choi, Hyun Seok Seong, Seongho Kim, Donghun Lee, Sungwon Yi, Jang-Ho Choi
Abstract:
While existing AI‑generated image detectors report high performance, we identify that this is largely driven by a critical prediction asymmetry: a bias toward the real class that severely limits sensitivity to generated content, especially under standard post‑processing operations such as compression and resizing. We hypothesize that this stems from the model's reliance on spurious features, distracting signals that obscure true generative artifacts. To address this, we propose DEAR (Dissect and Prune), which leverages inpainted images to identify and prune these interfering components. Specifically, we find that features strongly aligned to either inpainted or non‑inpainted regions are less robust to post‑processing. By measuring the alignment between channel activations and inpaint masks, DEAR removes features at both extremes, retaining only those that capture genuine generative artifacts. Experimental results demonstrate that our approach significantly enhances robustness against unseen generators and post‑processing, effectively mitigating the prediction asymmetry. Our code is available at https://github.com/dahyedahye/dear.
Authors:Qianzhong Chen, Hau Zheng, Justin Yu, Suning Huang, Jiankai Sun, Ken Goldberg, Chuan Wen, Pieter Abbeel, Yide Shentu, Philipp Wu, Mac Schwager
Abstract:
Fine‑tuning vision‑language‑action (VLA) policies for long‑horizon manipulation still relies heavily on behavior cloning, which requires costly high‑quality demonstrations and keeps policies near the demonstration distribution. Reward models can reduce this dependence by reweighting demonstrations and providing dense supervision for on‑robot reinforcement learning (RL), but they must be dense, accurate, and general. Existing methods fall short: task‑specific stage‑aware models are accurate but require per‑task annotations, while general vision‑language‑model (VLM) reward models are broadly applicable but too coarse for fine‑grained long‑horizon progress. We introduce RM, a multi‑task stage‑aware reward model that combines an action‑primitive‑based stage estimator with a multi‑gate Mixture‑of‑Experts (MMoE) value head to produce dense per‑step rewards across manipulation tasks. Building on RM, we further propose SPIRAL (Self‑Policy Improvement via Reward‑Aligned Learning), an on‑policy reward‑guided framework that improves VLA policies from cheap autonomous rollouts. On a 10‑task benchmark, RM reduces value‑estimation MSE by 80% over the strongest baselines; when used in SPIRAL, it improves task success from around 50% to near‑perfect performance on Folding Shorts (58% to 100%) and Cleaning Whiteboard (50% to 90%), showing that high‑quality dense rewards are key to a stable robot data flywheel. Project website: https://qianzhong‑chen.github.io/sarm2.github.io/.
Authors:Runze Jiang, Taiqiang Wu, Yan Wang, Bingyu Zhu, Longtao Huang
Abstract:
When large language models generate from retrieved or augmented contexts, conflicts between external context and parametric priors remain a central reliability bottleneck. Existing contrastive decoding methods follow a \emphcontext‑aware paradigm that unilaterally amplifies context over parametric priors, overwriting correct priors when the context is erroneous. We generalize this to the conflict‑aware paradigm that dynamically allocates authority between prior and context based on conflict signals, rather than presupposing context trustworthiness. We show that the affine combination of prior and context logits yields a power family with an inherent regime asymmetry: extrapolation amplifies errors unboundedly when the prior is correct, interpolation under‑corrects when the context is correct, and no static regime covers both. Existing contrastive decoding methods are instances of this family, mostly extrapolative. To evaluate both conflict directions, we propose TriState‑Bench, a model‑aware evaluation protocol that calibrates per‑model prior knowledge to measure three conflict states: correction, resistance, and agreement. To resolve the asymmetry, we propose Adaptive Regime Routing (ARR), which routes between regimes at each step, lifting resistance EM from below 6 to 16‑‑33 without sacrificing correction or agreement. Our code is available at https://github.com/keith‑Jiang/conflict‑aware‑decoding.
Authors:Fen Peng, Taizo Suzuki, Seisuke Kyochi
Abstract:
In this study, we propose an overlapped wavelet diffusion framework for Low‑Light Image Enhancement (LLIE), which incorporates two complementary components to achieve blocking artifact‑free and detail‑preserving enhancement. Although recent diffusion‑based LLIE methods have demonstrated remarkable performance compared with traditional approaches, DiffLL still suffers from blocking artifacts caused by the Haar Wavelet Transform (WT) and blurred edges or over‑smoothed textures due to the limitations of its High‑Frequency Restoration Module (HFRM). To overcome these issues, we introduce an Overlapped WT (OWT) that incorporates correlations across neighboring regions, thereby structurally preventing blocking artifacts. Furthermore, we integrate a low‑frequency‑guided High‑Frequency Enhance Block (HFEBlock) to strengthen detail recovery, yielding sharper edges and more reliable textures. Extensive experiments on the LOLv1 and LOLv2‑real datasets demonstrate that our framework, termed OWDiff, consistently outperforms existing LLIE methods both qualitatively and quantitatively, achieving superior visual quality while maintaining computational efficiency. OWDiff effectively addresses the structural limitations of the Haar WT and the HFRM, achieving an average PSNR gain of 0.58 dB, along with a 1.64% relative improvement in SSIM and a 5.9% relative reduction in LPIPS, compared to DiffLL across both the LOLv1 and LOLv2‑real datasets.
Authors:Yiteng Mao, Kenan Xu, Yijia Lyu, Wenhao Li, Jianlong Chen, Xiangfeng Wang
Abstract:
While Large Language Models (LLMs) have achieved near‑perfect performance in \emphsolving high‑school mathematics, their ability to \emphevaluate the diverse reasoning processes of real human students remains under‑examined. To bridge this gap, we introduce RealMath‑Eval, a rigorously annotated benchmark of 224 real‑world exam responses from high schools. Our initial evaluation reveals that even state‑of‑the‑art LLM judges struggle significantly on this task, exhibiting a high Mean Squared Error (～2.96) against expert human grading. To probe a plausible explanation, we contrast this performance with a control setting where the same judges evaluate synthetic LLM‑generated solutions. We identify a stark ``Evaluation Gap'': judges are considerably more accurate and consistent on synthetic text (MSE ～1.17) but struggle to generalize to authentic student reasoning. Through semantic embedding analysis, we find that synthetic errors suffer from a ``structural collapse'' into predictable, low‑dimensional linear subspaces, whereas human errors form a more diverse error space. Furthermore, generative probability probes suggest that human reasoning involves significantly higher information‑theoretic surprisal, indicating that student reasoning transitions are more out‑of‑distribution for current models. Finally, we find that surface‑level style transfer fails to close this gap. Our findings suggest that current LLM evaluation pipelines relying heavily on synthetic data may not adequately capture the diversity of authentic student mathematical reasoning.
Authors:Yohei Nakajima
Abstract:
Autonomous improvement loops are hard to trust because the improvement process is usually external scaffolding bolted onto the agent: failures go unlogged, diagnoses cannot be replayed, and promote‑or‑discard decisions land in a side database rather than the agent's own history. We show that an event‑sourced agent runtime removes that friction and turns controlled improvement into a first‑class workflow. When the agent's state is a deterministic projection of an append‑only event log, failures are recorded, a run replays exactly from its log, candidate patches scope to typed pipeline seams, gates are auditable, and every promotion or discard is itself an event. We demonstrate this with Regimes, a loop on the ActiveGraph runtime that diagnoses failed evaluations, proposes a repair at a pipeline point, and promotes it only after static checks, sandbox execution, in‑sample evaluation, and held‑out validation. The loop is target‑agnostic: the same control flow runs against different tasks through a common interface. On LongMemEval‑S the dominant failure is not retrieval but reconciliation: the evidence is already in the assembled context, yet the reader answers incorrectly. Across five seeded held‑out splits, Regimes discovers reader‑prompt repairs that improve final held‑out accuracy by +0.05 to +0.10 in four splits and +0.01 in one over‑promotion split; two splits are individually significant (seed 5 unadjusted for its sequential promotion structure), and the pooled count is descriptive only, since the splits share one 500‑question pool. The durable contributions are ActiveGraph as an auditable substrate that makes controlled improvement loops tractable, the held‑out‑gated loop it supports, the failure‑regime taxonomy routing each failure to a pipeline location (whose marginal value over an unrouted baseline is the primary open question), and the prompt‑as‑discovery‑probe hypothesis.
Authors:Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das, Tahira Naseem, Tim G. J. Rudner, Andrew McCallum
Abstract:
Insertion Language Models (ILMs) offer several advantages over left‑to‑right generation and mask‑based generation. However, existing formulations of insertion‑based generation have largely been ad‑hoc. In this paper, we derive a diffusion‑style denoising objective for ILMs from first principles by formulating the noising process as a continuous‑time Markov chain on the space of variable‑length sequences. We show that previous formulations of ILMs can be viewed as special cases of this denoising framework. Through empirical evaluation on a synthetic planning task, we show that the proposed approach retains the benefits of insertion‑based generation over left‑to‑right generation and masked diffusion models. In language modeling, our diffusion‑based approach is competitive with left‑to‑right generation and masked diffusion models, while offering additional flexibility in sampling compared to existing insertion language models.
Authors:Ghodsiyeh Rostami, Po-Han Chen, Mahdi S. Hosseini
Abstract:
Parameter‑efficient fine‑tuning (PEFT) aims to adapt pretrained models with a small trainable parameter subset, however, most existing methods choose this subset from fixed architectural heuristics rather than using dynamic, task‑aware criteria. We introduce FisherAdapTune, a Fisher‑guided Adaptive Fine‑Tuning framework that progressively selects parameter groups by tracking the temporal drift of their Fisher geometry. Starting from a PAC‑Bayesian view of fine‑tuning, we decompose the generalization error bound into Fisher‑weighted update costs and show that parameter groups whose curvature contribution has stabilized can be frozen to reduce the error bound without interrupting the remaining adaptation dynamics. FisherAdapTune formulates this criterion with a scale‑invariant Jensen‑Shannon distance between consecutive Fisher distributions, yielding an adaptive active parameter set. We evaluate our approach on a downstream segmentation task, and results show FisherAdapTune improves the in‑distribution performance and zero‑shot transfer in multiple settings, validating that Fisher structural drift is a useful signal for efficient, task‑aware adaptation. We release our \hrefhttps://github.com/AtlasAnalyticsLab/FisherAdapTunecode publicly to enable further application of our proposed approach.
Authors:Taehyoung Kim, Tim Schoenbrod, David Eckel, Henri Meeß
Abstract:
Recent learning‑based path planners use neural networks to process visual map representations and approximate heuristics for classical search algorithms, yielding near‑optimal paths with reduced search effort. However, these methods are tied to the shortest‑path objective implicit in their supervision, which limits their flexibility to accommodate alternative criteria. We introduce FlexPath, a two‑stage framework that decouples feasibility from preference. In Stage 1, we use imitation learning to acquire a task‑independent spatial prior over feasible paths from visual map inputs. In Stage 2, differentiable Path Shape Objectives (PSOs) adapt this prior toward task‑specific criteria without relearning path structure, requiring only efficient objective‑level adaptation. A single pretrained model can be adapted to multiple objectives. For shortest‑path planning, FlexPath reduces search effort on TMP by 14.3% compared to the state‑of‑the‑art TransPath, while also finding lower‑cost paths on average and demonstrating strong zero‑shot generalization across three unseen domains. For obstacle clearance with minimum clearance distance 2, it achieves 96.8% full obstacle avoidance while maintaining low search cost. The framework further extends to semantic‑aware avoidance and waypoint guidance via objective‑level adaptation, and remains compatible with classical planners at inference time. Data and code are available at https://github.com/FraunhoferIVI/FlexPath.
Authors:Bharath Sivaram Narasimhan, Karthik R Narasimhan
Abstract:
As recommender systems transition toward agentic, multi‑turn conversational interfaces, evaluation paradigms have struggled to keep pace. Current benchmarks often rely on "LLM‑as‑a‑judge" evaluations, which introduce subjectivity, high costs and inconsistency. We present τ‑Rec, a benchmark for agentic recommender systems that replaces subjective evaluation with verifiable rewards and a reveal‑tagged elicitation (RTE) mechanism that controls how task constraints surface during dialogue. By testing agents against structured catalog predicates and employing a pass^k reliability metric, τ‑Rec provides a systematic test for consistent reasoning. Our evaluation of nine configurations across five model families ‑‑ GPT‑5.4, Claude Sonnet 4.6, Gemini 2.5 Flash, DeepSeek V4 Flash, Qwen3‑32B and GPT‑5 mini ‑‑ reveals a steep reliability cliff, where even the best model achieves only ~57% at pass^1 and ~38% at pass^4, highlighting a critical gap in current conversational agent deployment. All code and data are publicly available at https://github.com/nbharaths/tau‑rec.
Authors:Qijun Wang, Chunqi Qian, Huacheng Zeng
Abstract:
In today's digitally connected world, keyboards remain the primary interface for inputting sensitive information, making them a persistent target for eavesdropping attacks. While prior keystroke inference techniques have exploited side‑channel signals such as acoustics and vibrations, they typically rely on conspicuous, short‑range sensors and require victim‑specific data for model training, limiting their practicality, scalability, and stealth. In this paper, we present RadKey, an RF backscatter system for covert, long‑range, through‑wall keystroke eavesdropping. RadKey comprises two components: a compact batteryless backscatter tag and an RF reader. The tag captures keystroke‑induced vibrations and acoustic signals, modulating them onto the frequency shift of its backscattered RF signal using two magnetically‑coupled LC resonators. This design also enables spectral separation between the excitation and backscatter signals, mitigating self‑interference for the RF reader and thus extending eavesdropping range. The RF reader demodulates the backscattered RF signal to infer typed content. It employs a dedicated signal processing pipeline that extracts user‑ and keyboard‑independent keystroke features across time and frequency domains, enabling strong generalizability. To further enhance adaptability, RadKey integrates an LLM for online adaptation, leveraging LLM outputs as pseudo ground‑truth labels to refine the classifier during runtime. We have built a prototype of the full RadKey system and evaluated it through extensive over‑the‑air experiments. Results show that RadKey achieves accurate and robust keystroke inference across diverse users in real‑world settings. A demo video is available at: https://radkey‑submission.github.io/RadKey/
Authors:Yaochen Zhu, Harald Steck, James McInerney, Aditya Sinha, Yinhan He, Nathan Kallus, Jundong Li
Abstract:
Direct preference optimization (DPO) is a simple and effective alignment strategy for large language models (LLMs) based on pairwise preferences. In recommender systems, however, user feedback is rarely pairwise. For a given context, e.g., a user, a session, or a conversation, we typically observe set‑wise preferences with multiple positive items, where every positive item should outrank every unobserved or explicitly negative item, with no prescribed order among the positives or the negatives themselves. A natural generalization is to use the Plackett‑Luce (PL) reward model, which extends the Bradley‑Terry reward model underlying vanilla DPO from pairwise preferences to full rankings of candidates. However, we show that adapting the PL model to set‑wise preferences requires marginalizing over all positive orderings, where the resulting expression is combinatorial in complexity. To address this fundamental challenge, we propose Mult‑DPO, a novel DPO objective with a tractable multinomial surrogate likelihood over set‑wise preference events for the user‑preference alignment of LLM‑based recommender systems. The multinomial construction is not itself a ranking distribution, but it is defined on the same reward‑induced weight space and admits a closed‑form DPO‑style objective, enabling direct alignment of LLMs with multiple candidates through a classification‑style objective. In addition, we prove that the multinomial DPO loss is a tractable upper bound on the marginalized PL DPO loss when optimizing against the set‑wise preference data. We further characterize the tightness of this bound in terms of the relative total weight of positives versus negatives, which provides insights into tightening the bound with richer or harder negatives. Finally, we extend Mult‑DPO to the alignment of LLMs with multiple preference levels. Code is available at https://github.com/yaochenzhu/Mult_DPO
Authors:Nathan Molinier, Adrian A. Marth, Reto Sutter, Christoph Germann, Jacob A. Connolly, Mathieu Guay-Paquet, Nathan D. Schilaty, Kenneth A. Weber, Julien Cohen-Adad
Abstract:
Lumbar spine conditions are a leading cause of disability worldwide, yet reliable quantification of degeneration from MRI remains challenging. In clinical practice, analysis is predominantly performed in two dimensions (2D), as manual three‑dimensional (3D) assessment is time‑consuming. However, 2D measurements suffer from limited reproducibility, particularly when anatomical structures are not aligned with the imaging plane. Existing automated approaches are often restricted to 2D, rely on discrete grading, or lack robustness and interpretability. We introduce SpineReport, an open‑source, fully automated framework for comprehensive 3D morphometric analysis of lumbar spine MRI. Leveraging robust anatomical segmentations, the method extracts quantitative metrics from key structures, including the spinal canal, spinal cord, vertebrae, intervertebral discs, and foramina. These include both morphological and signal‑based features, enabling cross‑subject and longitudinal assessment. SpineReport further generates subject‑specific reports that allow comparison with cohort distributions, improving interpretability and objective characterization of spinal morphology. Clinical relevance was evaluated against radiologist‑reported severity grades for central canal, lateral recess, and foraminal stenosis. Metrics showed strong associations with central canal stenosis severity, with T2‑weighted CSF signal providing the highest performance (AUC = 0.95). Canal AP diameter and area ratios also demonstrated strong correlations and high discriminative ability (AUC > 0.80). For lateral recess stenosis, associations were moderate, with lateral CSF signal being the most informative (AUC = 0.73). No significant associations were observed for foraminal stenosis despite robust region‑of‑interest extraction. SpineReport is released as an open‑access tool: https://ivadomed.github.io/SpineReport/
Authors:Yu Han, Kailing Li, Yang Jiao, Yulin Dai, Yuqian Fu, Linhai Zhuo, Tianwen Qian
Abstract:
Training large language models (LLMs) as autonomous agents via reinforcement learning (RL) has enabled frontier models to achieve superhuman performance in long‑horizon tasks. However, existing RL algorithms operate at the trajectory level, performing policy optimization only after collecting complete episode rollouts. This coarse‑grained approach faces fundamental challenges in multi‑turn agent settings where rewards are sparse, delayed, and credit assignment across individual steps is critical. In this work, we propose State‑Score‑Supervised Policy Optimization (3SPO), a novel RL algorithm that performs post‑step policy optimization with dynamic state score supervision. At each step, 3SPO computes the state score based on historical success rates, supervising step‑wise credit assignment, adaptive rollout and post‑step policy optimization without requiring value function estimation or additional auxiliary models. Theoretically, under a per‑state bandit abstraction, we show that the proposed score‑supervised allocation mechanism achieves logarithmic allocation regret and provide sample‑complexity guarantees for action identification, score distinguishability, and filtering stability. Experiments on ALFWorld and WebShop with Qwen2.5‑1.5B/7B‑Instruct show that 3SPO consistently outperforms GRPO by +22.6% on ALFWorld and +15.6 points on WebShop, while using comparable resources to achieve 2.4× more state exploration and 1.8× faster convergence. Code is available at https://github.com/genalyu/3SPO.
Authors:Gijs van Nieuwkoop, Siamak Mehrkanoon
Abstract:
Precipitation nowcasting is increasingly being approached with deep learning models that learn directly from recent radar observations. Although such models can efficiently capture short‑term precipitation motion, they often lack broader contextual information about the meteorological conditions under which rainfall develops. This paper investigates whether lightweight temporal context can improve radar‑based nowcasting, particularly for high‑intensity rainfall. We propose the Time‑Aware Small‑Attention U‑Net (TA‑SmaAt‑UNet), which extends the core SmaAt‑UNet model with temporal conditioning layers that use cyclical encodings of time‑of‑day and time‑of‑year to modulate intermediate feature representations. Experiments on KNMI radar precipitation data show that temporal conditioning is most beneficial for rare, high‑intensity precipitation events, while also improving the representation of seasonal variability and predicted rainfall‑intensity distributions. A layer conductance analysis further indicates that the added temporal conditioning layers are actively used by the model despite their small parameter cost. These findings suggest that simple, physically motivated temporal context can improve the realism and reliability of deep learning‑based precipitation nowcasts. The implementation of our models and training setup is available on \hrefhttps://github.com/gijsvn/TA‑SmaAt‑UNetGitHub.
Authors:Dmitry Manning-Coe, Thomas Read, Anna Soligo, Oliver Clive-Griffin, Chun-Hei Yip, Rajashree Agrawal, Jason Gross
Abstract:
Dictionary learning methods like Sparse Autoencoders (SAEs) and crosscoders attempt to explain a model by decomposing its activations into independent features. Interactions between features hence induce errors in the reconstruction. We formalize this intuition via compact proofs and make five contributions. First, we show how, in principle, a compact proof of model performance can be constructed using a crosscoder. Second, we show that an error term arising in this proof can naturally be interpreted as a measure of interaction between crosscoder features and provide an explicit expression for the interaction term in the Multi‑Layer Perceptron (MLP) layers. We then provide three applications of this new interaction measure. In our third contribution we show that the interaction term itself can be used as a differentiable loss penalty. Applying this penalty, we can achieve ``computationally sparse'' crosscoders that retain 60% of MLP performance when only keeping a single feature at each datapoint and neuron, compared to 10% in standard crosscoders. We then show that clustering according to our interaction measure provides semantically meaningful feature clusters, and finally that sleeper agents have significant interactions. Code is available at https://github.com/chainik1125/crosscoders‑feature‑interactions/tree/arxiv.
Authors:Anirudh Sekar
Abstract:
We introduce RKSC (Reasoning‑Aware KV Cache Sharing), a training‑free inference framework that eliminates two structural redundancies in multi‑branch LLM reasoning pipelines. ASKS (Attention‑Similarity KV Sharing) computes the prefix KV cache once and broadcasts it to all semantically similar branches via hidden‑state cosine similarity, strictly generalising the token‑exact prefix caching used by vLLM and SGLang. CGEE (Confidence‑Gated Early Exit) applies two complementary exit mechanisms: (1) it skips the verification forward pass entirely when generation confidence is decisive across branches, and (2) it terminates the verification pass at an intermediate layer when per‑layer entropy stabilises, using lightweight hooks on the transformer backbone. RSBCM (Reasoning‑Selective Block Cache Manager) prevents unbounded cache growth via attention‑weighted depth‑priority eviction. Across five model families (7B‑10B), four benchmarks, and 1,000 evaluated problems, RKSC achieves a mean speedup of 3.008x over the No‑KV baseline (peak 3.990x), a 1.66x mean improvement over vLLM‑equivalent prefix caching, with a CGEE‑induced error rate of only 0.37% (6 errors out of 1,616 verify calls). No fine‑tuning or architecture changes are required. Code is available at https://github.com/AnirudhSekar/RKSC.
Authors:Ayana Hussain, Soumya Sharma, Golnoosh Farnadi, Nicholas Vincent, Héber Hwang Arcolezi, Ulrich Aïvodji
Abstract:
Large language models (LLMs) are becoming widely deployed as personal AI assistants with access to sensitive user data, making privacy a major challenge for their design and evaluation. Prior work focuses mainly on individual‑level risks, overlooking interdependent privacy (IDP)‑‑where one person's data may be revealed by others without their knowledge or consent. We address this gap by introducing IDP‑Bench: the first LLM benchmark for IDP scenarios, grounded in the Contextual Integrity (CI) framework. We evaluate eight open‑source LLMs on their understanding of IDP scenarios across three levels of IDP reasoning using two LLM judges. Results show strong co‑ownership recognition (6/8 models exceed 90%) but persistent weaknesses in identifying CI parameters (information attribute, primary subject) and IDP‑specific parameters such as secondary subjects, where 7/8 models score below 74%. Models also struggle to judge sharing appropriateness (5/8 scoring below 77%). While the ability to judge the appropriateness of sharing improves with scale, performance tends to decline in smaller models, and prompt sensitivity remains high on IDP‑specific questions‑‑highlighting the need for more targeted study of IDP in LLM privacy research. Data \& code available \hrefhttps://github.com/tisl‑lab/Interdependent_Privacy_Benchhere.
Authors:Liuyin Wang
Abstract:
Long‑term memory is the missing layer for LLM agents: across sessions they forget, and the common workaround ‑‑ replaying the whole history into the prompt ‑‑ is expensive, slow, and, as distractors accumulate, less accurate. Most memory systems win on cost or latency but still lose to the full‑context baseline on accuracy, and benchmark numbers are reported on inconsistent, non‑reproducible harnesses, so one system appears at wildly different scores across sources. We present Engram, an open‑source, dual‑process memory engine on a bi‑temporal data model. A fast write path appends lossless episodes with no LLM on the critical path; an asynchronous path extracts atomic (subject, predicate, object) facts, builds a bi‑temporal knowledge graph, and resolves contradictions without an LLM call per fact ‑‑ invalidating, never deleting, so every fact keeps provenance and a supersession chain. A hybrid read path fuses dense, lexical, graph, and recency/salience signals, applies a point‑in‑time ("as‑of") filter, and assembles a compact, provenance‑tagged context. On the full 500‑question LongMemEval_S, graded by the official category‑specific judge, Engram's lean configuration ‑‑ answering from a ~9.6k‑token retrieved slice, never the full history ‑‑ scores 83.6% vs. 73.2% for full‑context (+10.4 points, McNemar p < 10^‑6) at ~8x fewer tokens (9.6k vs. 79k), with 0/500 errored. The gain needs a hybrid read path: facts alone lose recall, while facts plus retrieved chunks recover detail. We also contribute a neutral, in‑repo evaluation harness with the official judge baked in and the full‑context baseline in every table, publish the raw per‑question logs, and document the measurement‑integrity pitfalls (truncation, home‑grown judges, full‑history leaks) that silently distort memory benchmarks. Every number ships with a command to reproduce it.
Authors:Yuhao Zhang
Abstract:
Sparse Mixture‑of‑Experts (MoE) large language models achieve strong quality with low per‑token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token‑dependent routing. Expert pruning is a direct remedy, but prior criteria typically score experts independently and overlook that MoE inference is inherently \emphcoalitional, where outputs arise from routed top‑k expert combinations. We propose SHAPE, a task‑driven pruning framework that explicitly models \emphintra‑layer expert cooperation. SHAPE formulates routing traces on a small calibration set as an empirical cooperative game and assigns interaction‑aware expert values via a Shapley‑style attribution over observed top‑k coalitions, enabling the identification of experts that are essential for high‑utility collaborations rather than merely frequent. To preserve MoE topology under a global pruning budget, SHAPE further introduces a \emphquality‑coverage selection rule that retains, in each layer, the minimal expert subset covering an α fraction of non‑negative Shapley mass, while using bisection to match a target keep rate. Experiments on three modern MoE backbones (Qwen3‑30B‑A3B, GPT‑OSS‑20B, and DeepSeek‑V2‑Lite) across diverse benchmarks show that SHAPE consistently improves robustness over global and layer‑wise pruning variants, maintaining competitive accuracy under 20% and 40% expert pruning without additional training and delivering clear reductions in peak GPU memory footprint. The open‑source code is available at https://github.com/Alizen‑1009/Shapley‑Moe.
Authors:Chong Liu, Luxuan Fu, Xuyu Feng, Zhen Dong, Bisheng Yang
Abstract:
The paradigm of digital twin cities is shifting from coarse visual mapping toward more precise and actionable digitization of urban assets. However, existing datasets predominantly focus on coarse visual perception, lacking the strict multi‑modal alignment and attribute and status diagnosis required for automated infrastructure maintenance. To bridge this gap, we introduce WHU‑Infra3D, a large‑scale, multi‑modal benchmark dataset dedicated to roadside infrastructure inventory. Covering 53.8 km across three cities, WHU‑Infra3D uniquely integrates panoramic imagery and LiDAR point clouds with rigorous 2D‑3D instance association and cross‑frame tracking. Comprising over 175k multi‑view 2D bounding boxes alongside thousands of 3D infrastructure instances, the dataset provides over 181k detailed attribute and status annotations (e.g., rust, occlusion) to empower operational health assessment. We establish comprehensive baselines across five core tasks: 2D detection, 2D cross‑view matching, 3D geo‑identification, 3D point cloud segmentation, and attribute recognition. Extensive evaluations expose significant cross‑city domain gaps and inherent vulnerabilities of current models on long‑tailed defective statuses, establishing WHU‑Infra3D as an essential testbed for advancing scalable, AI‑driven urban infrastructure inventory and lifecycle management. The WHU‑Infra3D dataset is available at https://github.com/WHU‑USI3DV/WHU‑Infra3D.
Authors:Nazmus Shakib Shadin, Xinyue Zhang, Jingyi Wang, Miao Pan
Abstract:
Federated Learning (FL) combined with Split Learning (SL) is a privacy preserving paradigm that enables training deep neural networks (DNNs) on resource constrained devices while reducing overall training cost. However, determining the optimal split point, meaning the layer where the model is divided still remains a critical challenge, especially when clients have heterogeneous hardware capabilities. Fixed split points can overload weak devices and increase the communication and server load, which slows convergence and reduces stability. This paper introduces QSplitFL, a novel capability‑aware Deep Q‑Network (DQN) framework for optimal split point selection in Split learning based Federated Learning (SFL) environments. Unlike existing approaches that rely on high‑dimensional model weight representations, QSplitFL employs a lightweight state representation derived directly from client hardware metrics, including CPU utilization, memory, battery level, and network latency. The proposed framework incorporates a decayed loss‑drop reward function that prioritizes early convergence, and a committee‑based DQN architecture with majority voting to mitigate reward hacking. Extensive experiments on MNIST, Fashion‑MNIST, CIFAR‑10, and CIFAR‑100 datasets using CNN, ResNet50, MobileNetV4, and ConvNeXt architectures demonstrate that our approach achieves better convergence and higher accuracy compared to existing methods, while effectively adapting to heterogeneous device resources. The source code is publicly available at https://github.com/AIPO‑Lab/QSplitFL.
Authors:Manel Slokom, Malek Slokom, Thierno Kante
Abstract:
Privacy and data sharing are often in tension. Many organizations use synthetic data to reduce privacy risk and still share useful data. For tabular data, auditing privacy remains hard. In many cases, even humans cannot easily tell if a table is real or synthetic. In this paper, we propose a method based on LLM discrimination. We ask an LLM to classify each table sample as REAL or SYNTHETIC. We test two settings: C1 with table only, and C2 with table plus distributional metadata. We use LLaMA as an open model and Gemini as a reference model. In our experiments, we run three synthesis models, CTGAN, TVAE, and Gaussian Copula, on two public datasets, UCI Adult and ACS Census. We collect 451 valid trials. Our results show clear differences between models. On Adult, LLaMA reaches DRS=0% in reported cells, while Gemini reaches DRS=100% for CTGAN and TVAE. On Census, LLaMA predicts SYNTHETIC for most samples, while Gemini stays high in C1 but drops for CTGAN and TVAE in C2. We also compare with a classifier two‑sample test (C2ST) and record linkage as distributional baselines, and with a human pilot of 2 annotators and 240 trials. Our results show that LLM discrimination is a practical privacy audit signal when model choice, per provider reporting, and data encoding are handled with care. For reproducibility, code and experiment scripts are available at https://github.com/SlokomManel/LLM‑as‑a‑Discriminator.
Authors:Yue Zhao, Hongyan Li, Yong Chen, Luo Ji
Abstract:
Emotional interaction is increasingly crucial for conversational AI, yet current systems lack a self‑emotion determination mechanism to drive the streaming text‑to‑speech (TTS) synthesis. We propose an emotion‑planning framework that determines the emotion prior to the textual generation, grounding the downstream emotional TTS in a streaming manner. The framework is implemented by a plug‑and‑play LLM module, initialized from pretrained LLMs, and trained by reinforcement learning (RL) with emotions as the actions. A hybrid reward is employed which combines imitation signals with theory‑driven scoring, in which the theory of Plutchik's wheel of emotions is adopted. By experiments on DailyDialog, EmoryNLP, IMEOCAP, and MELD, our method outperforms prompting and finetuning baselines on both emotion determination and response quality. We finally implement an entire streaming pipeline for real‑time deployment, with the speech quality confirming the framework's emotional alignment, contextual coherence, and expressive fluency. Codes, cases, and demos are available in https://sixingdeguo.github.io/EmoQ‑page/.
Authors:Arsalan Shahid, Gordon Suttie, Philip Black
Abstract:
Foundation models are moving from response generation into operational roles. They plan across steps, call tools, request human input, coordinate with other agents, and increasingly carry responsibility for work that affects customers, claims, code, contracts, and clinical decisions. Production deployments are no longer one human supervising one model. They are multi‑human, multi‑agent collaborations that cross teams, time zones, and trust boundaries. The technical surface for this collaboration remains weakly specified. When an agent drafts a response and a human edits it before it ships, the moment of human judgement is the most valuable signal in the system. In current practice it is recorded, if at all, in application code, chat threads, ticket comments, and tribal memory. Two protocol standards address adjacent concerns: MCP standardises agent access to tools and data, and A2A standardises agent‑to‑agent interoperability. Neither defines the shared workspace in which humans and agents perform accountable work together. This paper presents CHAP, the Collaborative Human‑Agent Protocol. Under CHAP, the override that used to vanish into a chat thread becomes a structured event carrying a diff, a rationale, and a content hash. The handoff between shifts becomes a portable envelope rather than a pinned message. The human approval of an agent's draft becomes a non‑repudiable signed decision that can be replayed years later. The protocol achieves this through a small Core (workspaces, participants, tasks, artefacts, and an append‑only evidence log) together with composable profiles that add review, modes, routing, deliberation, handoff, identity, signatures, and transparency‑backed audit as deployments require them. Specification, reference implementation, conformance suite, and worked examples are available at: https://github.com/BrightbeamAI/chap
Authors:Yuheng Chen, Teng Hu, Yuji Wang, Qingdong He, Zhucun Xue, Qianyu Zhou, Jason Li, Lizhuang Ma, Jiangning Zhang, Dacheng Tao
Abstract:
The fidelity and structural diversity of training datasets fundamentally determine the capabilities of video generation models. While commercial systems showremarkableabilitytogeneratecinematicnarratives, the progress of open‑source models remains limited by the scarcity of high‑quality training data. To bridge this gap, we introduce CineDance‑1M, a large‑scale, open research Text‑to‑Audio‑Video (T2AV) dataset designed specifically for multi‑shot, long‑form joint audio‑video generation. Averaging 92.8 seconds and 24.2 continuous shots per video, it provides configurable, structured annotations for both audio and video modalities. This exceptional quality is achieved through a rigorous three‑stage curation pipeline: i) diverse sourcing and comprehensive cleansing, ii) film‑theory‑inspired narrative parsing, and iii) hierarchical dual‑modal captioning. For a comprehensive assessment, we propose CineBench, featuring a diverse prompt suite and a six‑dimensional, human‑aligned metric system tailored for complex narrative audio‑video evaluation. Furthermore, we adapt LTX‑2.3 into CineDance, which demonstrates exceptional single‑modality quality alongside precise audio‑video alignment and robust subject and environment consistency, effectively validating our curation strategy and the high quality of CineDance‑1M. We anticipate that this work will serve as a solid foundation for accelerating future research in multi‑shot, long‑form joint audio‑video generation. Our project page is available at https://aliothchen.github.io/projects/CineDance/.
Authors:Hankun Lin, Ruqi Zhang
Abstract:
Ensuring the reliability of Large Language Models (LLMs) under distribution drift requires inference‑time adaptation. While inference‑time alignment methods such as Best‑of‑N and rejection sampling are widely used, they frame the task as a sampling‑intensive, reward‑guided search, leading to two key limitations: their performance is bounded by the base model's generation quality, and their reliance on imperfect reward models makes them vulnerable to reward hacking. To address these challenges, we introduce Gradient‑Guided Reward Optimization (GGRO), a lightweight inference‑time method that performs targeted, minimal intervention during decoding via gradient guidance. Specifically, GGRO monitors token‑level entropy to identify high‑uncertainty regions indicative of drift or misalignment. Upon detection, it responds by injecting nudging tokens, generated using gradient signals from an off‑the‑shelf reward model, to steer the generation trajectory rather than merely re‑ranking samples. Experiments show that GGRO consistently improves inference‑time alignment across safety, helpfulness, and reasoning benchmarks. It also increases coverage of high‑quality responses and robustness to reward hacking, with minimal computational overhead. Code is available at https://github.com/lhk2004/GGRO.
Authors:Junke Zhu, Hao Zhang, Yutian Zhu, Ang Li, Chenxiao Hu, Meng Gai, Fei Zhu, Zhangjin Huang, Sheng Li
Abstract:
Ray tracing enables 3D Gaussian fields to serve as a representation for physically based light transport. Faithful inverse rendering requires forward rendering and backward optimization to be defined within a consistent light‑transport pipeline. Existing inverse rendering methods estimate G‑buffers via splatting and optimize materials in screen space, tying the recovered properties to a rasterization‑based pipeline. This pipeline mismatch, together with simplified rendering equations that neglect indirect illumination, often leads to inconsistent shading, visible artifacts, and inaccurate material‑lighting estimation under path‑traced rendering. Therefore, we propose a splatting‑free path‑traced inverse rendering framework for 3D Gaussian fields, where forward light transport and backward gradient propagation are defined within a unified ray‑tracing pipeline. Our key idea is to define a path‑space equivalent interaction model for overlapping Gaussian primitives, under which Monte‑Carlo‑based path tracing is unbiased for the induced light‑transport integral, while pathwise gradients are replayed over the same ray‑traced interactions rather than splatting‑derived screen‑space buffers. The framework optimizes materials and a compact Spherical‑Gaussian environment under the full rendering equation with ray‑traced visibility and multi‑bounce light transport. Extensive experiments demonstrate competitive material inversion and improved path‑traced rendering quality, producing more plausible shadows, reflections, and relighting results under global illumination.
Authors:Berker Demirel, Valentino Maiorca, Marco Fumero, Theofanis Karaletsos, Francesco Locatello
Abstract:
Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimental factors. In many applications, especially in scientific domains, such models are attractive to explore conditions for which real samples are rare, expensive, or not yet observed. However, this creates a circularity for evaluation: standard conditional quality metrics require a reference target distribution, but in the extrapolative regime that distribution is unavailable by definition. We address this problem with a post‑hoc, per‑sample trust score for assessing conditional samples using only the training distribution. The score combines two estimable quantities: global realism, measuring compatibility with the real data manifold, and attribute‑wise faithfulness, measuring whether a sample is closer to the requested attributes than to plausible alternatives. We show that the score can recover meaningful comparisons across extrapolated generations, under a mild coverage condition on the observed attributes. These comparisons enable effective filtering, ranking, and abstention of generations and can be used directly on off‑the‑shelf pretrained models. In biological imaging, selected samples preserve real morphological structure better and improve downstream predictive performance, while similar gains are observed on controlled vision benchmarks. Finally, we show how the score can be applied during generation, enabling abstention before full decoding. Code is available at https://github.com/berkerdemirel/faithful‑cond‑gen.
Authors:Apratim Bhattacharyya, Shweta Mahajan, Sanjay Haresh, Rajeev Yasarla, Reza Pourreza, Litian Liu, Risheek Garrepalli, Roland Memisevic
Abstract:
Learning everyday skills, like cooking a dish, relies increasingly on instructional media such as online videos. This opens the door to the use of video (and multimodal) large language models (LLMs) as task guidance assistants. A crucial capability for the real‑world success of a prospective task guidance assistant is it's ability to intervene proactively as soon as a mistake is apparent in order to guide the user. To evaluate this crucial capability, we introduce Ego‑MC‑Bench (Mistake Corrections), a benchmark for evaluating reactive, step‑by‑step task guidance in realistic cooking scenarios. Extensive experiments show that Ego‑MC‑Bench is highly challenging for state‑of‑the‑art video LLMs. We argue that a key reason is the limited availability of training data for fine‑tuning models on this task. Although there exists a wide range of cooking video datasets, existing datasets lack examples of mistakes along with appropriately timed interventions. To help address this data limitation, we also introduce Ego‑CoMist, a counterfactual synthetic dataset created by transforming non interactive cooking videos into supervised training examples showing proactive interventions. We show that fine‑tuning on Ego‑CoMist yields performance gains especially for smaller and more efficient video LLMs that are well suited for delivering assistance on edge devices.
Authors:Yoojin Nam, Jinhoon Jeong, Namkug Kim
Abstract:
As autonomous research agents and AI co‑scientist systems push large language models (LLMs) from drafting toward end‑to‑end manuscript production, the bottleneck shifts from generation to verification. Fluent LLM output can hide fabricated citations, numbers that drift from source tables, and unmet reporting‑guideline items; existing tools generate without verifying, and self‑critique inherits the blind spots that produce confident fabrication. We describe an architecture pairing generation with verification, resting on three principles: decompose the workflow into self‑contained skills, gate every stage transition with halt‑on‑failure, and resolve each integrity question with the cheapest sufficient mechanism, a deterministic, re‑executable check where one suffices and a prose‑level probe only where interpretation is unavoidable. This determinism‑where‑possible split, organized as an integrity‑gate taxonomy, is the core contribution. It is realized as MedSci Skills, an open‑source toolkit of 43 skills with a 21‑detector deterministic tier, evaluated on three public‑dataset pipelines (STARD, PRISMA, STROBE) and a seeded‑defect ablation. Across the three pipelines every content‑hash manifest verified clean and the gates surfaced real defects; on 27 identical injected defects the deterministic gates detected all 27 with no false positives on the matched clean fixtures, whereas a single‑prompt LLM reviewer detected 11, its misses in code, bibliography, and style defects the prose hides. Determinism‑where‑possible verification yields an auditable, re‑executable trail that exposes the evidence a human needs to check an LLM‑assisted manuscript: feasibility and reproducibility evidence, not a claim of human‑competitive quality, which a separate blinded study addresses. MedSci Skills is MIT‑licensed and archived (v3.8.0).
Authors:Qinghua Xing, Yinda Chen, Yaping Jin, Zhenhe Wu, Bohan Lin, Hang Zhou, Xinghao Chen, Hanting Chen, Zhiwei Xiong
Abstract:
Large language model agents increasingly rely on skills: reusable procedural documents encoding workflows, tool use, implementation patterns, validation checks, and domain rules. Skill rewriting is often treated as prompt compression, but shorter skills can make agents more expensive by removing sparse operational anchors that prevent exploration, debugging, and recovery. We study skill rewriting through this economic lens. Our controlled framework profiles skill structure, rewrites skills using information‑preservation strategies, and evaluates the rewrites under fixed task instructions, environments, and verifiers. Experiments on SkillsBench reveal distinct quality‑‑cost trade‑offs across strategies: API/code anchoring, workflow guarding, and rule/formula anchoring benefit different task families, with no universally dominant template. In the main held‑out evaluation, the learned policy reduces total cost by 7.0% and downstream agent‑token cost by 6.0%; in frozen cross‑model transfer, the corresponding reductions average 14.7% and 13.7%, while verifier quality is preserved. These results position skill design as cost‑aware operational knowledge engineering rather than prompt compression. Resources: https://github.com/1Reminding/Skill_EE.
Authors:Juan S. Santillana
Abstract:
Reference‑free faithfulness metrics verify each atomic claim a model makes against ground truth, and are increasingly used to evaluate grounded generation. We show they share a blind spot: they measure only precision ‑‑ are the stated claims supported? ‑‑ and therefore reward abstention, since a model can score near‑perfect faithfulness by saying almost nothing. We make this measurable using Formula 1 telemetry, a domain where strategic ground truth is derived deterministically and, crucially, completely: for each decision we know the full set of facts that mattered. This completeness ‑‑ absent in open‑domain faithfulness benchmarks ‑‑ lets us measure recall (coverage of the relevant facts) exactly, alongside precision. On a multilingual (EN/ES/PT) benchmark of 7,253 decision instances spanning 157 races, the most precise frontier model covers under half of the relevant facts and ranks last by F1, so requiring coverage reorders the systems; the same effect reappears in a second complete‑oracle domain (NOAA weather forecasts). Fine‑tuning small models (1B‑7B) on the complete oracle closes the precision‑recall gap entirely (F1 ~0.98), beating every zero‑shot frontier system regardless of scale. We pair faithfulness with coverage into a single score, validate the metric (controlled perturbation; agreement across a model‑free regex extractor and a cross‑family LLM extractor, system‑level Spearman 1.0), and give a verifier‑guided generation method that improves precision and recall without references. We release the benchmark, structured annotations, metric, baselines, and an interactive demo.
Authors:Yufei Chen, Zihan Wang, Yubao Tang, Yukun Zhao, Maarten de Rijke, Zhaochun Ren
Abstract:
Multimodal generative retrieval formulates multimodal retrieval as discrete identifier generation, eliminating the need for explicit similarity search over external embeddings. Existing approaches construct identifiers via residual quantization and decode them with trie‑constrained beam search. This combination introduces an indexing‑decoding gap: identifier learning objectives, including reconstruction and contrastive losses, do not explicitly enforce prefix discriminability during decoding. As a result, even well‑optimized identifiers can be irreversibly pruned early in beam search due to low‑rank prefixes. We theoretically characterize this gap and derive a survival bound that relates prefix retention to three controllable factors in indexing and decoding. Building on this bound, we propose PRO, prefix retention optimization, a unified framework comprising three mechanisms: (i) prefix ranking distillation aligns quantized prefix rankings with those induced by pre‑quantization embeddings using a listwise loss; (ii) vocabulary scheduling increases codebook sizes from shallow to deep residual quantization levels to reduce early competition from non‑target prefixes; and (iii) geometric score fusion vectorizes each candidate prefix and incorporates its similarity to the query into beam search scoring, further reducing the indexing‑decoding mismatch. Experiments on nine multimodal retrieval tasks show that PRO improves retention of target identifier prefixes and outperforms existing multimodal generative retrieval baselines.
Authors:Zibin Liu, Shunkun Liang, Banglei Guan, Yang Shang, Qifeng Yu, Ji Zhao
Abstract:
Despite the rapid advancements in event‑based motion estimation, current geometric methods primarily focus on velocity estimation. However, absolute pose estimation, which is equally crucial for key applications such as robotic navigation and augmented reality, remains relatively underexplored. Consequently, the simultaneous recovery of absolute pose and velocity from event streams remains an open and challenging problem. To address this gap, we propose a geometric framework for absolute pose and velocity estimation by leveraging 3D lines in the scene and the events they trigger. At the core of the framework lie two key geometric constraints: the orthogonality between a 3D line and the normal vector of its corresponding event plane, and the collinearity of an event with the 2D projection of its associated line. Based on these constraints, we present both linear and polynomial solvers for absolute pose estimation. The former enables efficient computation, while the latter provides a globally optimal solution for rotation. For velocity estimation, we develop an efficient linear solver and a more accurate optimization‑based solver to recover both angular and linear velocities. Notably, our methods require a minimum of three event‑line correspondences to determine the 6‑DoF absolute pose or velocities independently. Extensive experiments in simulation and on real‑world datasets demonstrate that our methods achieve state‑of‑the‑art performance, with significant improvements in accuracy and computational efficiency compared to existing methods. The demo code is publicly available at https://github.com/Zibin6/EventPoseVelocity.
Authors:Yan Wang, Qifan Zhang, Jiachen Yu, Tian Liang, Dongyang Ma, Xiang Hu, Zibo Lin, Chunyang Li, Zhichao Wang, Miao Peng, Nuo Chen, Jia Li, Yujiu Yang, Haitao Mi, Dong Yu
Abstract:
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra‑long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek‑V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query‑critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone‑free decoupled training strategy. By formulating the indexer as a standard dual‑encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long‑term global memory. Across primary long‑context evaluation suites (e.g., LongBench‑v2, LongMemEval, and RULER), FM‑DS‑V4 compresses the average physical KV cache footprint down to merely 13.5% of the full‑context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.
Authors:Siyu Chen, Miao Lu, Beining Wu, Heejune Sheen, Fengzhuo Zhang, Shuangning Li, Zhiyuan Li, Jose Blanchet, Tianhao Wang, Zhuoran Yang
Abstract:
Self‑evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend on extensively curated or teacher‑generated training data, or, when the generator runs unsupervised, reward it by a difficulty heuristic that need not improve the solver. We introduce INFUSER, an iterative co‑training framework with two co‑evolving roles: a Generator that drafts questions and reference golden answers from a pool of unstructured, automatically collected documents, and a Solver that improves by training on them. The solver is trained with standard correctness rewards against the generator‑provided answers, while the generator is rewarded by an optimizer‑aware influence score that measures whether each proposed question would actually improve the solver on the target distribution. Because this continuous, noisy influence score is poorly served by standard GRPO, we propose DuGRPO, a dual‑normalized variant of GRPO, for generator training. Together, these turn the document pool into an adaptive curriculum that favors questions useful to the current solver, not just hard ones. On Qwen3‑8B‑Base, INFUSER outperforms strong self‑evolution baselines with over 20% relative improvement on Olympiad and SuperGPQA benchmarks, and an 8B INFUSER co‑evolving generator outperforms a frozen 32B thinking generator on math and coding. Ablations confirm each design choice is necessary, and two extensions, applying INFUSER to an instruction‑finetuned anchor and augmenting it with rule‑verifiable RLVR data, further demonstrate the flexibility and generalizability of the framework. Code is available at https://github.com/FFishy‑git/INFUSER.
Authors:Ankai Hao, Ke Chen, Huan Li, Lidan Shou
Abstract:
Feature engineering remains a cornerstone of tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for its automation, giving rise to LLM‑powered Automated Tabular Feature Engineering (LATTE). However, the field lacks standardized, cost‑aware evaluation platforms, and the combinatorial explosion of design choices obscures true algorithmic progress. To bridge these gaps, we systematically deconstruct 15 representative LATTE methods into a unified 6‑dimensional taxonomy. Based on this abstraction, we introduce LATTEArena, a standardized, modular, and extensible benchmarking framework that decouples monolithic pipelines into reusable execution blocks. By distilling the massive combinatorial space, we evaluate 24 core LATTE configurations across 7 research questions. Our head‑to‑head benchmarking goes beyond predictive accuracy to quantify token efficiency and execution robustness, yielding 17 empirical findings on cost‑effectiveness trade‑offs. Furthermore, we provide 3 concrete recommendations for optimal real‑world deployment. By enabling controlled component‑level comparisons, LATTEArena shifts the paradigm from ad‑hoc prompt engineering to systematic context management. All code, datasets, and over 4,000 execution logs are publicly available to foster a dynamic, community‑driven benchmark. Our framework, leaderboard, and all artifacts are hosted on the LATTEArena project website at https://goodenhak.github.io/LATTEArena.
Authors:Yanxiong Li, Guoqing Chen, Qianqian Li, Sen Huang
Abstract:
In the task of few‑shot class‑incremental audio classification, the number of classes is assumed to always increase without considering the possibility of decrease. However, the number of classes generally increases or decreases in practice. In this paper, we investigate a problem of Few‑shot Class‑variable Incremental Audio Classification (FCIAC), in which the number of classes increases or decreases. We propose a FCIAC method using prototype adaptation and pseudo class‑variable training. The model in our method consists of an encoder and a classifier. The classifier is initialized by a class‑variable prototype adaptation network, whose structure dynamically changes with the change of classes. In addition, we design a pseudo class‑variable training strategy to enhance the model's adaptability to changing classes. Experiments on three public datasets show that our method exceeds previous methods in average accuracy. The code is at: https://github.com/cgq2971‑afk/FCIAC.
Authors:Stéphane Eilles-Chan Way, Hugo Percot, Quentin Cappart, Tias Guns, Louis-Martin Rousseau
Abstract:
Decision‑focused learning has shown great promise for addressing predict‑then‑optimize problems, particularly in the presence of under‑specified models. However, its practical deployment is often hindered by high computational costs and limited scalability, as it requires solving a constrained optimization problem for each training instance at every iteration. To address these challenges, we propose a novel framework that incorporates Lagrangian decomposition into the decision‑focused learning paradigm. Specifically, we introduce a new surrogate objective along with two loss functions for evaluating and training the underlying prediction model. We further propose two variants of our approach, which offer different trade‑offs between computational efficiency and solution quality. Our framework can be seamlessly integrated with standard decision‑focused learning methods, including Smart Predict‑then‑Optimize (SPO+) and Implicit Maximum Likelihood Estimation (IMLE). Through experiments on two standard benchmarks, the multi‑dimensional knapsack problem and quadratic portfolio optimization, we demonstrate that our approach achieves competitive performance while remaining amenable to parallelization. In particular, it consistently outperforms traditional decision‑focused learning methods on large‑scale instances, involving up to eight times more variables than those typically considered in related work. The implementation is available at https://github.com/corail‑research/DFL‑LD.
Authors:Sheng-Wei Chan, Yung-Che Wang, Hsin-Jui Pan, Chia-Min Lin, Jen-Shiun Chiang
Abstract:
Document image binarization aims to separate foreground text from degraded backgrounds while preserving thin, broken, and low‑contrast strokes. Although deep learning methods have improved binarization performance, most existing approaches rely on convolutional, transformer‑based, or generative architectures, while Mamba‑based state space models remain largely unexplored for this task. In this work, we investigate Mamba‑based feature propagation and observe that direct state‑space propagation may dilute weak foreground cues during long‑range modeling, especially faint ink traces, fragmented characters, and boundary‑sensitive stroke details. To address this problem, we propose DeepMine‑Mamba, a Mamba‑based binarization framework equipped with a novel Anti‑Dilution Gate that estimates propagation‑induced feature changes and selectively restores stroke‑sensitive local responses while suppressing unnecessary background enhancement. Experiments on DIBCO/H‑DIBCO benchmarks under a strict leave‑one‑year‑out protocol show that DeepMine‑Mamba achieves competitive overall performance, with strong average FM and Fps across benchmark years. Ablation results further demonstrate that the Anti‑Dilution Gate improves stroke preservation and reduces perceptually significant binarization errors.
Authors:Md Mahfuzur Rahman Siddiquee, Fazle Rafsani, Jay Shah, Teresa Wu, Catherine D Chong, Todd J Schwedt, Baoxin Li
Abstract:
Abnormality detection is a crucial yet challenging task in medical image analysis. Distinguishing abnormalities from normal data by learning to reconstruct normal‑only data alleviates the reliance on labeled datasets. However, many studies, even if unsupervised, rely on a labeled validation set to select the best model for inference from multiple training iterations. For many diseases labeled data are unavailable and substantially time consuming to obtain. To address this, AUCp ‑ a novel metric that supports abnormality detection for unsupervised and self‑supervised methods is proposed. Instead of evaluating the realism of reconstructed images to select the best of model for inference, it focuses on actual detection performance and without requiring an annotated test set. Assuming the pseudo ground truth of all unannotated samples in the test set as abnormal/positive and using traditional AUC calculation, AUCp scores are derived. Given a large and representative training set of normal samples, we show mathematical and empirical evidence that model selection using AUCp scores improves disease detection in terms of unsupervised and self‑supervised methods over conventional metrics. Using two unsupervised methods for neurologic disease detection and self‑supervised methods on diverse datasets, our results demonstrate that the AUCp score effectively identifies the optimal model for inference, significantly enhancing abnormality and disease detection. The corresponding implementations are available in https://github.com/mahfuzmohammad/AUCp.
Authors:Yunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han, Zhihe Lu, Shanghang Zhang
Abstract:
World action models inherit the predictive capability of world models, enabling action generation to be guided by anticipated future observations. However, they rely primarily on vision and often fail in contact‑rich manipulation, where critical cues arise from physical interaction. In this paper, we propose Dream‑Tac, a unified Tactile‑World Action Model that jointly models actions, future visual observations, and tactile dynamics. Specifically, Dream‑Tac introduces (i) contact‑gated visuotactile fusion to selectively integrate tactile signals and (ii) a contact‑aware attention bias to better regulate cross‑modal interactions during manipulation. To support real‑time deployment, we further design a dual‑level acceleration strategy, reformulating the contact‑aware bias to preserve the fused attention path during training and introducing cache‑based diffusion acceleration at inference, achieving up to 2.9× faster training and 1.8× faster inference. Across six contact‑rich manipulation tasks, Dream‑Tac improves action accuracy by 31.7% on average, demonstrating the effectiveness of unified visuotactile world modeling.Code is available at https://github.com/LYFCLOUDFAN/Dream‑Tac.
Authors:Muhammed Rasin
Abstract:
We study a capability the dominant paradigm in synthetic tabular data does not provide: exact satisfaction of a declared analytical outcome with no source data. Imitation methods (copulas, GANs, diffusion) learn a real distribution and sample from it, and are judged on fidelity to real data. A large, practical class of needs is different: generating data with no source data ("cold start") that reproduces a declared outcome (a revenue curve, a churn rate, a group share) across a relational schema. Off‑the‑shelf imitation tools offer no interface for such targets, and no sampler can hit an exact aggregate, because sampling has variance. On a real public dataset, off‑the‑shelf learned synthesizers trained on that very data miss the declared monthly aggregate by 74 to 86 percent; a per‑period steelman cuts the miss to about 19 percent and still cannot reach 0; a closed‑form generator reaches exactly 0. We name this task outcome‑conformant synthesis, argue its evaluation axis is conformance rather than fidelity, and show the two axes are orthogonal. We contribute: (1) a formal account showing a widely‑used family of exact‑aggregate generators is exactly conditional‑sum sampling of a Gamma population (via Lukacs' characterization), with closed‑form exactness, a closed‑form marginal CV, and scale‑invariance; a controlled experiment maps the boundary, enforcing the exact aggregate costs at most 0.006 in 1‑Wasserstein distance to an arbitrary external marginal, the rest being shape‑family mismatch; (2) SpecBench, to our knowledge the first benchmark to measure conformance to analytical outcomes for cold‑start relational synthesis; and (3) a closed‑form, deterministic reference system. Exact aggregation alone is trivial; the contribution is conformance jointly with closed‑form marginals, integrity, determinism, and zero source data. We concede fidelity to imitation where real data exists.
Authors:Ruihua Han, Shuai Wang, Chengyang Li, Rui Gao, Xinyi Wang, Zhe Liu, Guoliang Li, Yupu Lu, Qi Hao, Jia Pan, Hengshuang Zhao
Abstract:
Simulation plays a key role in automated robotics research supported by large language models (LLMs). However, existing simulators often require custom code or complex interfaces, creating a barrier to rapid prototyping and automated algorithm development. To this end, we propose the Intelligent Robot Simulator (IR‑SIM), a lightweight skill‑native navigation simulator designed for rapid scenario construction, benchmarking, and robot learning. In IR‑SIM, scenarios are entirely defined by YAML configuration files that specify mobile robot kinematics, geometric collision checking, LiDAR sensing, visualization, and behavior modules. This design makes robotic simulation fully describable and reproducible, allowing scenarios to be generated and modified from text prompts through the proposed IR‑SIM agent skills. The resulting scenarios can be used for automated benchmarking of navigation algorithms and for automated generation of training data for learning methods. Furthermore, IR‑SIM provides bridges to high fidelity simulators and real world deployment, allowing users to validate their algorithms in more realistic settings after prototyping without extra coding. The experiments showcase the convenience and versatility of IR‑SIM in multiple tasks: constructing navigation scenarios from natural language, training a collision avoidance policy, benchmarking social navigation policies, and bridging to high fidelity simulators and real world deployment. The project website is available at https://github.com/hanruihua/ir‑sim.
Authors:Syed Rifat Raiyan, Mohsinul Kabir, Hasan Mahmud, Md Kamrul Hasan
Abstract:
Mathematical reasoning has long served as a stringent test of machine intelligence; over the past decade, it has moved from a niche problem within NLP to one of the most consequential AI frontiers. This survey provides a unified account of the field's evolution, from early rule‑based math word problem (MWP) solvers and template‑driven geometry systems, through neural expression generation and LLM prompting, to contemporary reasoning models, multi‑agent systems, neuro‑symbolic theorem provers, and verified discovery workflows. We organize the landscape along four axes: (i) informal reasoning over text and diagrams, spanning MWP solving, multimodal geometry, and VLMs; (ii) formal reasoning in proof assistants, including autoformalization, tactic prediction, compiler‑guided repair, and proof search; (iii) mathematical discovery, where systems propose constructions, improve bounds, or assist attacks on open problems; and (iv) the inference and training‑time techniques, including CoT prompting, tool use, process reward models, and RLVR, that increasingly connect generation with verification. We catalog major benchmarks across grade‑school arithmetic, competition mathematics, geometry, formal proving, multimodal and multilingual reasoning, and expert evaluation, and we examine benchmark saturation, contamination, reporting mismatches, and the distinction between pass@1, majority voting, and verifier‑assisted pass@k. We critically assess failure modes: brittleness under perturbation, reward hacking, multimodal grounding failures, fragile formalization, and the energy cost of reasoning‑scale inference. Drawing on recent perspectives from working mathematicians, we identify future directions centered on verified‑discovery workflows, reasoning efficiency, and infrastructure to make AI‑assisted formalization broadly usable. Companion materials: https://github.com/Starscream‑11813/awesome‑AI4Math.
Authors:Matteo Spanio, Mohammad Torabi, Andrea Poltronieri, Antonio Rodà
Abstract:
Symbolic music evaluation for large language models remains fragmented across representations, datasets, and metrics. We introduce LilyBench, a LilyPond‑based benchmark that jointly evaluates symbolic music generation and music understanding on the same family of open‑weight LLMs. The benchmark includes a 200‑prompt generation suite and ten understanding tasks adapted from ABC‑Eval, covering syntax, metadata prediction, structural sequencing, and music recognition. Generation quality is evaluated using compile rate, MusPy descriptor distributions via Jensen‑Shannon similarity, and LilyBERT‑based Fréchet Music Distance (FMD). Experiments on four open‑weight models show that executable LilyPond generation is achievable in zero‑shot settings, while structural understanding tasks remain challenging despite strong performance on composer and genre recognition. Our experiments also reveal systematic disagreements between descriptor‑based and embedding‑based metrics, suggesting that symbolic music evaluation benefits from metric triangulation rather than single‑score ranking. We release the benchmark, prompt bank, and evaluation code to support future research in symbolic music generation and understanding at https://github.com/CSCPadova/lilybench
Authors:George Ling, Lijin Yang, Hao Yang, Zhongzhan Huang
Abstract:
We present BLUE, a minimal method for better language use in vision‑language‑action (VLA) models for autonomous driving (AD). Through extensive analysis, we reveal that language matters on only a small fraction of routes, but on those routes it can greatly improve or degrade performance. Generating language at every frame is therefore inefficient, since most computation is spent on frames that do not benefit from language. We further show that pretrained VLA hidden states potentially already encode whether language will benefit a given frame, even though scene complexity and kinematic features alone struggle to predict this. Based on this finding, BLUE trains a lightweight gate on frozen VLA hidden states to decide per frame whether to activate language generation or predict actions directly, without modifying the backbone or requiring additional human annotation. With just a 0.11M‑parameter gate, BLUE sets a new state of the art on both benchmarks, achieving 76.2% success rate on Bench2Drive and 36 driving score on Longest6 v2, while delivering 2.54x inference speedup and 8.9% success rate improvement over the backbone. BLUE provides a practical path toward efficient language‑augmented AD, showing that VLA models can retain the benefits of language at a fraction of the cost. Our code, data, logs and checkpoints are fully available on https://github.com/George‑Ling3/BLUE.
Authors:Shi Ying Chang, Chiok Yew Ho, Yichen Li, Yintong Huo
Abstract:
AI coding assistants have significantly improved developer productivity by automatically suggesting code that aligns with user intent, and many of these tools are now integrated directly into Integrated Development Environments (IDEs). Developers interact with code in two distinct cognitive modes: Flow and Command. While developers require tools that directly complete or infill code in unfinished programs during Flow mode, they also need tools that can comprehend intentions expressed as natural‑language instructions and convert them into executable code in Command mode. Although instruction‑tuned Large Language Models (LLMs) dominate many application scenarios due to their abilities to infer and fulfill developers' intents, it remains unclear whether the same paradigm is equally suitable for different code‑related tasks. Therefore, it is necessary to understand how instruction tuning affects the feasibility of CodeLLMs as coding assistants. To fill this gap, we conduct the first empirical study that uncovers a key trade‑off caused by instruction tuning across programming modes, which we term the Instruction‑Tuning Tax. Our results show that instruction tuning is not a free lunch: although instruction‑tuned models are more capable of following instructions and leveraging structured guidance, these gains often come at the cost of weaker infilling performance. We further extend our study through both qualitative and quantitative analyses, including manual failure categorization, behavioral metrics that capture generation fidelity, and intermediate‑checkpoint evaluation throughout the tuning process. Summarizing our results into seven findings and four implications, our study offers a new perspective on the development of AI‑powered coding tools and highlights the need to carefully balance instruction‑following ability with effective code generation assistance.
Authors:Danilo Danese, Angela Lombardi, Giuseppe Fasano, Matteo Attimonelli, Tommaso Di Noia
Abstract:
Large and demographically balanced datasets are essential for reliable neuroimaging biomarkers. Full‑resolution 3D brain MRI synthesis can support data augmentation in this setting, but existing approaches either incur prohibitive computational cost at volumetric scale or rely on lossy latent compression that may compromise anatomical detail. As a result, practical 3D generative augmentation often requires specialized compute infrastructure. We propose WaveDiT, a conditional flow matching framework operating in the coefficient space of a 3D Haar Discrete Wavelet Transform. The model combines factorized spatio‑depth attention with band‑wise heteroscedastic uncertainty modeling derived from higher‑order wavelet statistics. Predicted log‑variance is integrated directly into both the flow objective and conditioning pathway, enabling adaptive precision consistent with the heavy‑tailed and input‑dependent variance structure of anatomical detail. This formulation supports full‑resolution 3D synthesis under practical memory and time constraints on a single modern GPU. Evaluation on a multi‑site cohort demonstrates improved alignment between generated and real MRI distributions, together with enhanced downstream brain age prediction and region‑level anatomical agreement relative to diffusion, latent, and wavelet‑based baselines. Code is available at https://github.com/sisinflab/WaveDiT
Authors:Joonyong Park, Jungwoo Kim, Junyoung Koh, Yuki Saito
Abstract:
AI‑generated music detectors can appear robust on standard benchmark splits, yet their deployments require transfer to generator sources absent during training. We study this problem with source‑restricted evaluation on \textscMoM‑open, an open reconstruction of MoM‑CLAM that replaces the non‑redistributable real corpus with FMA and MTG‑Jamendo while preserving the fake‑generator protocol. To isolate the role of representation, we introduce \textscCoMoE, a compact fixed classifier for comparing heterogeneous audio token spaces while keeping the downstream architecture and training recipe unchanged. Experiments show that standard and real‑source‑restricted splits are nearly saturated, whereas fake‑source restriction exposes large differences between token spaces: X‑Codec tokens are strongest when training on Udio alone, while MERT‑derived tokens are stronger when training on Suno‑v3.5 alone. These results suggest that codec‑style discrete token spaces should be treated as a primary experimental axis under generator shift in AI‑generated music detection. Our code and data are available at https://github.com/MAAP‑LAB/CoMoE.
Authors:Pratuat Amatya, Vinay Setty
Abstract:
We present a multilingual fact‑checking system deployed at Factiverse, designed for high‑throughput and low‑latency operation across diverse languages. The system follows a modular pipeline with three stages: claim detection, evidence retrieval and re‑ranking, and veracity prediction. We fine‑tune XLM‑RoBERTa‑Large for claim detection, mmBERT‑base for three‑label stance classification (Supports/Refutes/Mixed), and a SetFit‑based multilingual re‑ranker for claim‑‑evidence matching. We compare these components against strong LLM baselines, including GPT‑5.2, Claude Opus~4.6, and Qwen3‑8b. Experiments on production data spanning 114 languages for claim detection and 28 languages for veracity prediction show that task‑specific fine‑tuning provides strong and stable multilingual performance, while the fine‑tuned retrieval model remains competitive with modern proprietary embeddings. Same‑hardware latency measurements further show large efficiency gains for encoder‑based components, supporting their use in production deployments with tight cost and privacy constraints. Overall, compact fine‑tuned, self‑hosted models remain a practical and effective foundation for multilingual fact‑checking at scale. Code and data used for this study are available at https://github.com/factiverse/factcheck‑editor.
Authors:Boshu Lei, Kostas Daniilidis, Antonio Loquercio
Abstract:
We present an online reinforcement learning (RL) algorithm for fine‑tuning flow‑matching policies in continuous‑control problems. Our key insight is to view RL‑based policy improvement as a transport of action densities towards regions of high reward, which naturally aligns with the transport formulation of flow matching models. Prior methods either approximate the current or optimal policy distribution or resort to distillation, which introduces biased gradients or sacrifices multimodal modeling capacity. In contrast, our approach for RL with Density Transport, which we name \emphRLDT, constructs a transport field from a maximum‑entropy RL objective using Stein Variational Gradient Descent (SVGD). Then, it finetunes a pretrained flow matching policy to align with this field. Training with this alignment objective is nontrivial because flow‑matching policies generate actions via a multi‑step process, making direct gradient‑based optimization challenging. To overcome this challenge and stabilize training, we approximate policy actions from intermediate denoising steps via expected‑target estimation. This allows the transport‑field update to propagate into the network parameters without unstable backpropagation through time. Experimental results demonstrate that RLDT outperforms competitive baselines in reward quality and convergence speed. This performance holds across diverse continuous‑control tasks, encompassing both dense and sparse rewards, as well as state‑ and vision‑based long‑horizon robot manipulation. The project webpage is \hrefhttps://rpfey.github.io/rldt/https://rpfey.github.io/rldt/.
Authors:Beiwen Zhang, Yongheng Liang, Guowei Zou, Haitao Wang, Hejun Wu
Abstract:
Constructing efficient and reliable policies to assist humans is indispensable for human‑AI collaboration. Existing methods mainly follow two lines of work. Most prior work relies on multi‑agent reinforcement learning (MARL) to learn black‑box policies, which limits interpretability and raises safety concerns. Recent methods query large language models (LLMs) at each decision step, causing slow responses and high inference costs. We propose Collaboration Policy Tree (Co‑pi‑tree), a closed‑loop method that learns an executable policy tree consisting of a partner‑behavior prediction tree and an agent‑action selection tree. Co‑pi‑tree constructs a policy by distilling LLM reasoning into policy tree code. It then evaluates the policy through partner interaction, obtains feedback, and uses natural language to summarize the interaction feedback to improve problematic branches. Experiments in Overcooked‑AI show that Co‑pi‑tree improves average reward by 35.4% over the baseline average, while reducing the number of LLM queries by 77.7% and test‑time latency by 97.1%. Project page: https://beiwenzhang.github.io/Co‑pi‑tree/
Authors:Xu Zhang, Peang Wang, Wei Wang
Abstract:
Recently, large time series models (LTSMs) have gained increasing attention due to their similarities to large language models, including flexible context length, scalability, and task generality, outperforming advanced task‑specific models. However, prior studies indicate that pre‑trained LTSMs may exhibit a poorly conditioned non‑convex loss landscape, leading to limited trainability. As a result, direct fine‑tuning tends to cause overfitting and suboptimal performance, sometimes even worse than training from scratch, substantially diminishing the benefits of pre‑training. To overcome this limitation, we propose Smoothed Full Fine‑tuning (SFF), a novel fine‑tuning technology. Specifically, we construct an auxiliary LTSM via random initialization to obtain a smoother loss landscape, and then linearly interpolate its weights with those of the pre‑trained model to smooth the original landscape. This process improves trainability while preserving pre‑trained knowledge, thereby enabling more effective downstream fine‑tuning. From an optimization perspective, SFF perturbs sharp minima without significantly harming flat regions, facilitating escape from poor local basins toward smoother and more generalizable solutions. Extensive experiments on benchmark datasets demonstrate consistent improvements across eight representative LTSMs, including Timer, TimesFM, MOMENT, UniTS, MOIRAI, Chronos, TTMs, and Sundial, on diverse downstream tasks. The code is available at the link: https://github.com/Meteor‑Stars/SFF.
Authors:Bochao Yin, Xuan Lu, Zhengyu Qi, Xiaoyu Shen
Abstract:
Multi‑condition retrieval requires systems to identify documents that satisfy multiple distinct constraints, moving beyond mere topical relevance. While query decomposition is widely adopted as an intuitive remedy, its effectiveness across different retrieval pipeline stages remains underexplored. In this paper, we conduct a stage‑aware empirical study and uncover a stark, stage‑dependent effect: decomposition during initial retrieval frequently harms retrieval performance due to semantic dilution, yet substantially improves reranking by enabling more fine‑grained constraint verification. Motivated by these insights, we propose a principled Stage‑Aware Decomposition framework that retains the monolithic query during initial retrieval to preserve global semantic context, while employing sub‑queries exclusively during reranking for fine‑grained constraint matching. Extensive evaluations on the MultiConIR and SSRB benchmarks demonstrate that our framework consistently improves ranking performance for compositional queries across multiple retrieval and reranking models. We release our code at https://github.com/EIT‑NLP/Query‑Decompose.
Authors:Chenhan Jin, Shengze Xu, Qingsong Wang, Fan Jia, Dingshuo Chen, Tieyong Zeng
Abstract:
Data pruning (DP), as an oft‑stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well‑defined pruning method while striving for near‑lossless performance. However, existing approaches, which commonly select highly informative samples, can lead to biased gradient estimation compared to full‑dataset training. Furthermore, the analysis of this bias and its impact on final performance remains ambiguous. To address these challenges, we propose OrderDP, a plug‑and‑play framework that aims to obtain stable, unbiased, and near‑lossless training acceleration with theoretical guarantees. Specifically, OrderDP first randomly selects a subset and then chooses the top‑q samples, where unbiasedness is established with respect to a surrogate loss. This ensures that OrderDP conducts unbiased training in terms of the surrogate objective. We further establish convergence and generalization analyses, elucidating how OrderDP affects optimal performance and enables well‑controlled acceleration while ensuring guaranteed final performance. Empirically, we evaluate OrderDP against comprehensive baselines on CIFAR‑10, CIFAR‑100, and ImageNet‑1K, demonstrating competitive accuracy, stable convergence, and exact control ‑‑ all with a simpler design and faster runtime, while reducing training cost by over 40%. Delivering both strong performance and computational efficiency, our method serves as a robust and easily adaptable tool for data‑efficient learning. The code is publicly available at https://github.com/shengze‑xu/OrderDP.
Authors:Zehao Yu, Jiakun Zheng, Weiji Xie, Jiyuan Shi, Chenyun Zhang, Chenjia Bai, Xuelong Li
Abstract:
Recent progress in robot manipulation has been largely driven by learning from large‑scale demonstrations. For humanoid robot loco‑manipulation tasks, however, existing data sources force an unsatisfying tradeoff between trajectory quality and scalability. Real‑world teleoperation provides the highest‑quality trajectories but requires dedicated physical space and time‑consuming scene resets. Simulation offers an alternative way out of this dilemma: it can produce clean, embodiment‑aligned data at scale without any physical hardware. In this paper, we propose OASIS, a simulation‑data‑driven framework for humanoid loco‑manipulation. OASIS automatically reconstructs realistic object assets from real‑world images using a 3D generative model. Based on these assets, trajectories are first collected through teleoperation in simulation, and then augmented under diverse domain randomizations in a post‑processing stage. With the resulting simulation data, we further design a hierarchical visuomotor policy for humanoid loco‑manipulation. Extensive experiments on the real humanoid robot show that, under zero‑shot deployment, the policy trained on our simulation data achieves higher success rates on most tasks than that trained on real‑robot teleoperation data, owing largely to the broad lighting and environmental variations covered by our simulation rendering, which real‑robot data fails to capture. The project page is available at https://oasis‑humanoid.github.io/.
Authors:Yuan Zhang, Shiqi Zhang, Yedong Shen, Shuai Dong, Jiajun Deng, Xin Zhang, Yuxuan Gao, Jiajia Wu, Xin Nie, Zhiyuan Cheng, Jianmin Ji, Yanyong Zhang, Xingyi Zhang, Jia Pan
Abstract:
Vision‑Language‑Action (VLA) models achieve strong benchmark performance but still struggle in real‑world deployment with unseen objects, background shifts, and different robot embodiments. We argue that this stems from the lack of a unified geometry‑aware manipulation representation, leaving existing VLAs vulnerable to low‑level trajectory supervision, misaligned 3D features, and embodiment differences. To address this, we propose GEAR‑VLA, a VLA framework for learning unified geometry‑aware action representations for generalizable robotic manipulation. GEAR‑VLA adopts coarse‑to‑fine action learning, where multi‑source embodied pretraining equips the VLM with embodied reasoning and discrete action understanding before latent action tokens connect action semantics to a gradient‑decoupled DiT continuous action expert. It further performs semantic‑aligned 3D integration by aligning a trainable 3D spatial backbone with the VLA representation while freezing the original VLM‑aligned visual pathway. To share this representation across robots, GEAR‑VLA uses embodiment canonicalization, where embodiment‑aware states and embodiment‑invariant actions confine robot differences to the low‑level interface. Extensive simulation and real‑world experiments demonstrate strong generalization: GEAR‑VLA achieves state‑of‑the‑art performance on LIBERO, zero‑shot LIBERO‑Plus, and RoboTwin 2.0, reaches 85.9% success on AgileX and 81.0% on the pretraining‑unseen LDT‑01 embodiment, and obtains 90.1% success on a 6,360‑trial universal grasping benchmark with 212 unseen objects. Code and models will be released at https://github.com/babynabeauty/GEAR‑VLA.
Authors:Changliang Xia, Chengyou Jia, Minnan Luo, Zhuohang Dang, Xin Shen, Bowen Ping
Abstract:
Although video virtual try‑on (VVT) has achieved significant progress, existing methods still exhibit two fundamental limitations: first, they are restricted to single‑garment transfer, rendering simultaneous multi‑object try‑on highly impractical; second, their heavy reliance on explicit external priors (e.g., garment masks) inevitably destroys crucial physical dynamics and degrades visual quality. To bridge this gap, this paper proposes the novel Try‑On Anything task, which aims to simultaneously transfer diverse wearable objects onto a person in a video in a single inference pass. To support and standardize this paradigm, we introduce TryAny‑Bench, a comprehensive benchmark encompassing a paired video dataset alongside a tailored evaluation protocol. Furthermore, we present OmniTryOn, an external‑prior‑free generative framework designed to tackle this task. Specifically, OmniTryOn employs a First Frame Wearable Cache strategy, which directly provides diverse wearable objects for the generation process through the initial video frame. To maintain consistency, we propose the Spatiotemporally Consistent RoPE (STC‑RoPE), which inherently establishes robust spatiotemporal anchors to strictly preserve complex human motions and background dynamics. Optimized by the proposed Gradual Try‑On (GTO) training strategy, our model progressively masters robust multi‑object synthesis. Extensive experiments on TryAny‑Bench demonstrate that OmniTryOn significantly outperforms existing specialized video virtual try‑on models and general video editing baselines, establishing a powerful new standard for the Try‑On Anything task. Our dataset, code, and models are available at https://github.com/xcltql666/OminTryOn.
Authors:Bingjia Huang, Xiangyu Li, Xiang Wang, Liang Mi, Zixu Hao, Weijun Wang, Hao Wu, Kun Li, Yunxin Liu, Ting Cao
Abstract:
Generative robot policies fail unpredictably at deployment: they hesitate at critical moments, drift off‑task, or commit to unrecoverable actions. Existing online failure detectors either require white‑box access to policy internals or add runtime overhead through resampling and observation‑side signals. Our empirical analysis shows that emitted action chunks themselves already carry strong predictive signal for impending failures in generative robot policies. Motivated by this observation, we introduce ActProbe, a lightweight, pure action‑space detector that uses two compact signals available from a single forward pass: Temporal Consistency Error (TCE) between consecutive action chunks and Action Chunk Magnitude (ACM) of the current chunk. ActProbe maps these signals to per‑step failure probabilities with a task‑conditioned LSTM‑MLP architecture. Across a diverse suite of generative robot policies and benchmarks, ActProbe raises alerts before failures become visually recognizable, improving the accuracy (F1)‑timeliness Pareto frontier of failure detection by an average hypervolume gain of +12.7% over both internal‑ and external‑feature baselines, with a +9.0% early‑detection ROC‑AUC lead on unseen tasks. ActProbe further transfers to deployment, predicting failures on unseen real‑robot pick tasks and accelerating RL fine‑tuning (PPO) with 2.9x fewer environment interactions.
Authors:Cuong Dang, Jiahao Zhang, Hieu Ta Quang, Dung Le, Lu Cheng, Suhang Wang
Abstract:
While confidence calibration is essential for trustworthy decision‑making in safety‑critical applications, the robustness of calibrated GNNs to adversarial structural perturbations remains largely unexplored. However, studying calibration attacks on graphs presents unique technical challenges: (1) the discrete nature of graph structures complicates gradient‑based optimization, (2) existing underconfidence objectives fail to drive predictions toward uniform distributions, and (3) GNNs are highly sensitive to edge perturbations, often causing unintended label changes that violate attack constraints. To address these challenges, we propose a Unified Graph Calibration Attack (UGCA) framework designed for worst‑case (white‑box) analysis of GNN calibration robustness. UGCA introduces a KL‑divergence loss to encourage uniform predictive distributions, a reranking mechanism to reduce label flipping, a hybrid loss to recover labels when violations occur, and beam search to explore a broader adversarial search space. We further provide theoretical insights linking model generalization, dataset complexity, and calibration vulnerability, showing that models with higher accuracy or trained on datasets with more classes are more susceptible under this threat model. Extensive experiments demonstrate that UGCA substantially increases Expected Calibration Error while preserving classification accuracy. Our code is publicly available at https://github.com/CaptainCuong/Graph‑Calibration‑Attack.git.
Authors:Lianyu Hu, Xiaoyu Ma, Zeqin Liao, Yang Liu
Abstract:
Chain‑of‑thought (CoT) reasoning has proven effective for enhancing problem‑solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approaches suffer from a fundamental limitation: they perform reasoning entirely in text without accessing visual features during the reasoning process. After initial visual encoding, image information becomes inaccessible, forcing models to reason based solely on whatever was captured in the initial description, which forms a `vision‑blind reasoning' paradigm that limits fine‑grained visual extraction, error verification, and adaptive attention. We propose Text‑Visual Interleaved Chain‑of‑Thought (TVI‑CoT), a framework that enables explicit interleaving of textual reasoning and visual feature access through learnable control tokens <THINK>, <LOOK> and <ANSWER>. These tokens allow dynamic switching between reasoning and visual grounding, attending to relevant image regions conditioned on the evolving reasoning state. Experiments on eight benchmarks demonstrate state‑of‑the‑art results among MLLM‑based CoT methods and notable performance boost compared to the baseline: +6.1% on MMMU, +3.8% on MathVerse, +3.4% on MathVista, and +3.4% on ScienceQA. Code is available at https://github.com/hulianyuyy/TVI‑CoT.
Authors:Yanyan Wu, Boyi Zhang, Yanlin Liu, Xinyu Fang, Jining Luan, Meiqi Zhang, Jiacheng Liu, Hao Zeng, Dexu Yu, Chang Liu, Hanwen Du, Yongxin Ni, Youhua Li
Abstract:
Financial portfolio trading is naturally formulated as a reinforcement learning problem, where an agent sequentially rebalances assets under changing market conditions to balance return, risk, and transaction costs. Yet in non‑stationary markets, raw OHLCV states and short‑horizon return rewards often provide an under‑specified learning interface, motivating large language models as a way to inject financial knowledge into state and reward design while constraining open‑ended generation. To this end, we propose GIFT, an LLM‑guided framework for state‑reward interface design in PPO‑based financial reinforcement learning. Rather than using the LLM to make trading decisions, GIFT uses Factor‑guided State Enhancement to generate state features from financial‑factor primitives, Risk‑rule‑guided Reward Shaping to generate auxiliary rewards from portfolio‑risk rules, and Diagnostic‑guided Refinement to revise candidate interfaces using PPO rollout diagnostics. After refinement, GIFT fixes the selected state‑reward interface before evaluation, with no further LLM queries or interface updates at test time. Comprehensive rolling‑window experiments across diverse market regimes and portfolio scenarios demonstrate that GIFT improves learning‑signal quality and out‑of‑sample risk‑adjusted portfolio performance over baselines. Code and data are available at: https://github.com/KAG778/GIFT .
Authors:Jamal Seyedmohammadi, Pai Chet Ng, Angelo Genovese, Zhixiang Chi, Jeannie Lee, Konstantinos N. Plataniotis
Abstract:
Palmprint modality offers a privacy‑preserving biometric solution, yet its deployment is hindered by the domain gap between controlled enrollment and unconstrained authentication. Existing datasets are largely restricted to controlled setups and fail to capture the compound variability of real‑world environments. In this paper, we introduce X‑Palm, a cross‑domain dataset comprising 6,006 palm images from 103 individuals (206 hands). To the best of our knowledge, X‑Palm is the first palmprint dataset providing novel paired‑identity acquisition specifically designed to bridge the gap between reliably controlled multispectral enrollment and unconstrained mobile authentication while encompassing a broad spectrum of in‑the‑wild variability. Unlike existing datasets that focus on single to a few variations, X‑Palm addresses the massive modality and environmental shifts encountered in practical deployments by capturing paired data for identities across two distinct domains: (1) a controlled Multispectral Palmprint setting using our custom‑developed scanner, and (2) an unconstrained smartphone palmprint setting that is participant‑driven, incorporating simultaneous variations in hardware, hand pose, illumination, background, camera‑to‑hand distance, perspective, and palm surface conditions (e.g., moisture and occlusions). Our extensive benchmarks of 12 SOTA models reveal that while existing methods achieve high performance on controlled data, they experience severe performance collapse on X‑Palm. Conversely, models trained on X‑Palm demonstrate consistent robustness across domains, positioning X‑Palm as a valuable resource for training a model towards real‑world, cross‑domain generalization. Data access instructions and the related benchmarking codes are publicly available at: https://github.com/X‑Palm/X‑Palm‑2026
Authors:George Andronchik, Pavel Lokhmakov
Abstract:
This paper reads six engine‑level measurements together ‑‑ 1.1 host attack surface, 1.2 information leakage, 1.3 defense‑in‑depth stackability, 1.4 public CVE history, 1.5 patch cadence, and 1.6 upstream fuzzing posture ‑‑ to describe how five AI‑sandbox products isolate guest code from the host kernel. No single axis is a sufficient basis for a comparative judgement; the cross‑axis reading is the load‑bearing analysis. Three high‑level findings: (1) engine classes (microVM, userspace kernel, OCI container) separate cleanly on every architectural axis, but products within a class do not; (2) product pin policy is the dominant operator‑facing variable ‑‑ engine‑side patch latency aggregates to ~0 days for coordinated disclosures, while downstream lag spans 0 days to 471+ days to "opaque" to infinity; (3) fuzzing investment splits into three tiers, and the strongest combination ‑‑ microVM x continuous public fuzzer ‑‑ is unoccupied in this set, leaving the "0 published CVEs x no upstream fuzzer x no academic study" intersection structurally unmeasured. We report per‑axis orderings, per‑product portraits, and a threat‑model qualification matrix; no overall ranking is proposed. Companion repository (code, Apache‑2.0): https://github.com/orbitalab/RnD‑ai‑sandboxes‑sec‑study‑part‑1. License: CC BY 4.0.
Authors:Li Jiang, Haoran Xu, Yichuan Ding, Amy Zhang
Abstract:
On‑policy distillation (OPD) has become a central post‑training tool for large language models (LLMs), providing dense per‑token teacher supervision along the student's own rollouts. In this work, we identify a common structural cause underlying OPD, which we call prefix failure. Under prefix failure, dense per‑token supervision induces a bimodal teacher mixture and fragmented gradients that token‑level loss truncation or reweighting fail to address. This observation motivates us to move beyond token‑level loss interventions toward trajectory‑level output corrections. We thus propose Trajectory‑Refined Distillation (TRD), a trajectory‑level correction method that revises the student's rollout under the teacher guidance while within on‑policy support. By correcting problematic prefixes before distillation, TRD mitigates prefix failure at its source. Moreover, TRD improves the exploration by exposing the student to alternative valid derivations under teacher guidance, even when the original rolls are already correct. TRD can also be applied to on‑policy self‑distillation (OPSD), a parameter‑sharing variant that uses the student model conditioned on privileged informations as the teacher. Across a wide range of benchmarks and base models at multiple scales, TRD consistently outperforms prior baselines, improving single‑attempt accuracy and broadening reasoning coverage. Code is available at https://github.com/louieworth/trd
Authors:Yiqi Zhou, Junhao Lu, Jiale Yu, Zhuo Xu, Yang He, Yue Yuan, Shan Shen, Daying Sun
Abstract:
Digital Compute‑in‑Memory (DCiM) reduces data movement and has become a promising solution for energy‑efficient edge AI. However, most existing DCiM frameworks still primarily target integer or fixed‑point arithmetic, and provide limited support for compiler‑integrated and accuracy‑configurable floating‑point computation. Directly integrating conventional IEEE 754 floating‑point units into dense SRAM‑based DCiM arrays, however, incurs high area and power overhead. To address this challenge, this work presents an accuracy‑configurable floating‑point multiplier integrated into the OpenACM framework for SRAM‑based DCiM. An exact IEEE~754‑compliant multiplier is first implemented as a baseline, and a mantissa‑segmentation‑based approximate multiplier is then proposed to reduce hardware cost while preserving numerical fidelity. Post‑layout results show up to 69% logic area reduction and 72% power savings over exact floating‑point designs without delay overhead. Evaluations on image processing tasks and ResNet‑18 inference further demonstrate negligible accuracy degradation. These results indicate that compiler‑integrated approximate floating‑point multiplication is a practical approach for enabling efficient and configurable floating‑point support in SRAM‑based DCiM systems. The Floating‑Point Multiplier is available on https://github.com/ShenShan123/OpenACM
Authors:Wenwei Huang, Jia Wei, Jianlong Zhou
Abstract:
Multi‑contrast brain MRI provide complementary soft‑tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi‑contrast images. While current approaches excel in image synthesis, they often struggle to synthesize critical tumor regions and exploit contextual information in multi‑contrast brain MRI effectively. To address this issue, we propose a synthesis‑centric, segmentation‑assisted closed‑loop framework with retrieval augmentation synthesis. Our method overall takes a generative adversarial architecture, which aims to synthesize missing contrasts from any combination of available ones with a single model. To explicitly capture tumor semantics and focus synthesis on tumor regions, we add an auxiliary segmentation branch that predicts tumor masks and feeds them back as semantic conditioning in synthesis branch, thereby learning tumor‑aware representations in the model and improving synthesis fidelity. Furthermore, we propose a dual‑bank retrieval augmentation strategy. It dynamically queries two external knowledge bases, namely a tumor masks memory bank for crucial tumor context and cross‑image contrast feature memory bank for global style information, to augment synthesis. Verified on two public multi‑contrast magnetic resonance brain datasets: BraTs2020 and UCSF‑BMSR, the proposed method is effective in handling medical brain images synthesis tasks and shows superior performance compared to previous methods. Code is available at:https://github.com/iBizzard/SSCF.git
Authors:Jingyan Xu, Hong Shi, Yi Shan, Penghui Liu, Yunhao Bai, Ningyuan Li, Xueyang Liu
Abstract:
Large language models answer knowledge‑intensive questions using both parametric memory and retrieved evidence, but neither source is uniformly reliable. Retrieval can fill knowledge gaps, yet distracting passages may override correct closed‑book answers. We study this post‑generation conflict as answer‑level source arbitration: given Direct and RAG answers from the same frozen model, decide which source to trust. We propose TRUSTMARGIN, a training‑free, plug‑and‑play arbitration layer that scores the two existing candidates with the model's own likelihoods. It combines a parametric‑prior margin, which tests whether memory accepts the retrieved answer, with an evidence‑binding margin, which discounts passage‑only salience and measures question‑specific support. TRUSTMARGIN selects between Direct and RAG without fine‑tuning, external judges, or additional generation. Across 2WIKIMQA and CWQA with three LLaMA scales, TRUSTMARGIN consistently improves over Direct generation and BM25‑RAG, recovers part of the Direct/RAG oracle gap, and generalizes to multiple training‑free RAG pipelines.
Authors:Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang, SeungJae Lee, Jungwook Choi, Mingu Kang
Abstract:
Low‑rank projection has emerged as a promising approach for compressing the KV cache by exploiting hidden‑dimension redundancy. However, prior methods rely on fixed or heuristic rank selection and struggle to achieve aggressive compression with minimal accuracy degradation. We propose STAR‑KV, an adaptive low‑rank KV cache compression framework with fine‑grained rank control. STAR‑KV encompasses 1) a differentiable thresholding mechanism that enables optimal rank selection at both attention‑head and block levels, 2) a hybrid decomposition strategy that applies different low‑rank factorizations according to the sensitivity of key and value projections, and 3) a low‑rank‑aware mixed precision quantization that leverages data statistics for near lossless low‑bit quantization. Evaluated across multiple LLMs and benchmarks, STAR‑KV achieves up to 75% KV cache compression and up to 20x overall KV cache reduction when combined with quantization. Enabled by custom Triton‑based GPU kernels, STAR‑KV delivers up to 6.9x speedup for the attention module and 3.1x end‑to‑end generation throughput. Our code is publicly available at: https://github.com/PriyanshBhatnagar/STAR‑KV.
Authors:Gianluca Scarpellini, Ron Shprints, Peter Holderrieth, Juno Nam, Pranav Murugan, Rafael Gómez-Bombarelli, Tommi Jaakola, Maruan Al-Shedivat, Nicholas Matthew Boffi, Avishek Joey Bose
Abstract:
All‑atom generative modeling of 3D biomolecular complexes has emerged as the dominant paradigm for predicting the structure of proteins and protein‑ligand systems. Generating structures at the atomic level of fidelity, however, typically requires expensive iterative diffusion rollouts, making both conventional deployment and inference‑time search techniques computationally costly. In this paper, we introduce the Denoiser Cofolding All‑Atom Flowmap (DeCAF) framework for distilling state‑of‑the‑art all‑atom cofolding models into all‑atom flow maps that produce high‑quality samples in only a few inference steps. We build DeCAF on a denoiser‑based formulation of flow maps with endpoint losses that naturally support SE(3) rigid alignment, which we show is critical for training accurate models. We further derive a simple change of variables that lets DeCAF operate in the σ‑space noise schedule of EDM‑style architectures, enabling direct distillation from pretrained cofolding diffusion models. Equipped with DeCAF's flowmap lookahead, we introduce a purpose‑built inference‑time framework that improves sampling through reward‑guided search. Empirically, DeCAF‑Boltz statistically improves over Boltz‑1x in both accuracy (RMSD) and physical validity scores of protein‑ligand poses at strict NFE budgets on the challenging Runs N' Poses, while also showing a more optimal Pareto frontier across all inference compute budgets on PoseBusters. Distilling the state‑of‑the‑art Pearl cofolding model, DeCAF‑Pearl outperforms diffusion‑based cofolding models and matches its teacher on success rate while using 5x fewer NFEs. We release our code at https://github.com/genesistherapeutics/decaf.
Authors:Danqin Zhao, Yicun Liu, Xingwei Tan, Thomas T. Hills
Abstract:
Existing scientific relation extraction benchmarks mainly target domains such as computer science, where entities are tasks, methods, datasets, materials, or metrics. This leaves a gap in variable‑oriented empirical fields such as psychology, where findings are expressed as relations among constructs, measurements, interventions, and outcomes. We introduce variable‑centered empirical graph extraction, the task of mapping scientific abstracts to typed graphs whose nodes are normalized variables and whose edges represent empirical and hierarchical relations. To support this task, we construct EmpiriGraph‑Psy, a benchmark of 210 psychology abstracts annotated by domain‑trained annotators with normalized variables, concept hierarchies, empirical relation types, and validation states. We evaluate frontier and open‑weight LLMs using both direct extraction and a staged graph‑construction pipeline that separates variable extraction, normalization, hierarchy construction, evidence selection, relation extraction, and edge validation. The staged pipeline substantially outperforms direct extraction, with the best configuration achieving a macro‑F1 of 0.74. Error analysis shows that moderation relations and concept hierarchies remain the most challenging cases, highlighting the difficulty of extracting higher‑order empirical claims and implicit abstraction structure from scientific abstracts.
Authors:Xiaojun Wu, Cehao Yang, Honghao Liu, Xueyuan Lin, Wenjie Zhang, Zhichao Shi, Xuhui Jiang, Chengjin Xu, Jia Li, Jian Guo
Abstract:
LLM agents increasingly rely on external inference conditions: prompts, tools, memory, SOPs, skills, and harness feedback. These assets can improve task execution without changing model weights, but they are often revised by heuristic reflection or by reusing observed successes and failures as if counts alone were reliable belief. We introduce Bayesian‑Agent, a native and cross‑harness framework that treats reusable skills and SOPs as hypotheses about whether a frozen model will succeed under a particular prompt, context, and harness environment. Bayesian‑Agent records verified trajectory evidence, maintains a feature‑conditioned categorical posterior over each skill, and maps posterior state into inspectable actions such as patch, split, compress, retire, and explore. Model‑facing prompts receive executable guardrails and failure‑mode patches, while posterior summaries remain available for audit. With \textttdeepseek‑v4‑flash, incremental repair improves SOP‑Bench from 80% to 95%, Lifelong AgentBench from 90% to 100%, and RealFin‑Bench from 45% to 65%. We further evaluate Bayesian‑Agent's native backend and optional GenericAgent, mini‑swe‑agent, and Claude Code backends. The results include positive, negative, saturated, and case‑study settings, suggesting that agent skill evolution is best viewed as posterior‑guided harness optimization rather than uncalibrated prompt accumulation. The source code is available at https://github.com/DataArcTech/Bayesian‑Agent.
Authors:Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rocktäschel, Amos Storkey
Abstract:
As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open‑ended interactive tasks. Yet existing evaluations rarely test these demands together, instead emphasising single‑agent tasks, short interactions, or highly structured multi‑agent settings. We introduce alem, a JAX‑based benchmark for open‑ended multi‑agent coordination built on Craftax‑like dynamics. Alem embeds procedurally generated coordination tasks, soft specialisation, communication, and controllable coordination difficulty into a long‑horizon survival world with exploration, crafting, trading, and combat. We evaluate 13 modern LLMs zero‑shot within homogeneous teams, with trained MARL agents as reference points. Current LLM agents remain far from solving alem, averaging only ~6% normalised return, but their failures are not uniform. On the hardest coordination setting, zero‑shot Gemini‑3.1‑Pro‑High approaches MARL agents trained for one billion steps, while GPT‑5.4‑High achieves strong base‑task reward but much lower coordination reward. This contrast shows that individual task competence does not imply coordination competence. Ablations show that communication is the largest contributor to coordination, while memory and reasoning help when used to maintain multi‑step plans. Overall, our results identify coordination as a distinct bottleneck for frontier LLM agents, separate from single‑agent capabilities. Alem makes this bottleneck measurable and provides a controlled testbed for developing agents that communicate, allocate roles, and execute shared plans. Code is available at https://github.com/alem‑world/alem‑env.
Authors:Josiah D. Kunz, Kamal Choudhary
Abstract:
Finite element analysis (FEA) is essential for structural design but remains computationally expensive, particularly when evaluating multiple design iterations or load scenarios. Machine learning surrogate models offer a promising alternative, yet most approaches struggle with a critical limitation: generalizing across varying geometries. This work presents a mesh graph network (MGN) for predicting von Mises stress fields in 2D structural components with arbitrary hole geometries. Unlike traditional machine learning approaches that use absolute node coordinates as features, the proposed model builds on existing MGN frameworks that encode node types (e.g., fixed boundary, free surface, hole edge), relative edge features (distance between neighbors), and global features (applied load). This architecture is inherently translation‑ and rotation‑invariant, enabling generalization to unseen geometries without retraining. The MGN was trained on 11 plate geometries under 20 load conditions and evaluated on 7 unseen geometries and 3 unseen loads. In the most favorable case, the model achieves R^2 \geq 0.97 on an unseen geometry and unseen load, compared to R^2 \approx 0.01‑‑0.86 for conventional models (Random Forest, Gradient Boosting , K‑Nearest Neighbors) trained on identical data. However, even in less favorable cases, the MGN model still outperforms conventional models. This work extends the mesh‑based simulation framework of Pfaff et al. (arXiv:2010.03409) to structural mechanics, demonstrating that graph neural networks can serve as efficient surrogates for finite element analysis across varying geometries.
Authors:Annie Chu, Jason Brent Smith, Bryan Pardo
Abstract:
Audio effects (FX) shape sound in contemporary music practice. However, most interfaces present them as discrete modules and parameters that favor targeted adjustment over exploratory listening. This separation can make it difficult to build intuition about the broader space of possible transformations or to move fluidly between searching and refinement. We present FXplorer, an interface that organizes audio effects within a perceptually informed 2D space, allowing sound transformations to be browsed as a continuous landscape rather than as isolated presets. By combining established spatial interaction approaches and interpretable DAW‑style controls with recent embedding‑based machine learning methods for similarity and semantic search, the system brings exploration and parameter refinement into a single workspace. FXplorer supports composition, production, or performance by allowing users to edit and interpolate between effect presets interactively.
Authors:Yufei Wei, Shuhao Ye, Chenxiao Hu, Yiyuan Pan, Dongyu Feng, Rong Xiong, Yue Wang, Yanmei Jiao
Abstract:
Recovering the relative 6‑DoF pose between two image groups underlies cross‑sequence relocalization and multi‑camera rig odometry. Each group carries known intra‑group geometry from visual odometry or rig calibration, and pretrained multi‑view backbones already fuse such geometry into visual features. Yet current models treat all views as an unstructured set, leaving cross‑group reasoning as the missing piece. We introduce \ours, which keeps the foundation model entirely frozen and adds three lightweight trainable modules to bridge the two groups: a perceiver resampler, a cross‑group bridge with merged self‑attention, and a multi‑frame pose head. The trainable footprint totals about 32M parameters, under 6% of the full model, and is supervised only by relative poses. Across four datasets that span indoor and outdoor simulation, real‑world cross‑season capture, and zero‑shot sim‑to‑real transfer, \ours attains state‑of‑the‑art accuracy on both tasks, while every baseline is retrained with its full original supervision. Code is available at https://github.com/WeiYuFei0217/G2G.
Authors:Jaineet Shah
Abstract:
When an LLM agent fails ‑‑ issues a refund it should not have, calls the wrong tool, leaks data ‑‑ existing tooling answers what happened (observability) or whether it passed (evaluation), but not which step caused the failure. The obvious heuristics are wrong: the step that executes the harmful action is usually not the step that decided on it, and LLM‑judge attribution is correlational and unreliable (state‑of‑the‑art step‑level accuracy on the Who&When benchmark is about 14%). We present Causal Agent Replay (CAR), which answers the question by intervention: it models an agent run as a structural causal model, applies a do‑operation to a step, and re‑executes the trajectory forward under the same stochastic policy, measuring the shift in the outcome distribution. We define an intervention algebra over agent steps, a single‑step contrastive estimator whose point‑of‑commitment rule resolves a confound specific to stochastic run‑forward, and a budget‑bounded Monte‑Carlo Shapley estimator that splits credit across interacting steps. Every effect is reported with confidence intervals. We validate against synthetic structural causal models with planted ground truth: the contrastive estimator recovers the pivotal step, and Shapley recovers a two‑step interaction (0.44, 0.45, ~0; efficiency sum 0.909 versus the analytic 0.91). CAR is open source and runs on hosted or free local models.
Authors:Qi Liu, Gang Yue, Mingyu Yin, Lisai Zhang, Yidi Wu, Yaole Wang, Yaohui Wang, Chang Yao, Jingyuan Chen, Lin Ma
Abstract:
Recent advances in Diffusion Transformers have driven rapid progress in video generation and editing, yet these capabilities are still handled by separate, task‑specific models. Building a unified framework that supports diverse video tasks remains an open challenge: existing unified attempts either require dedicated auxiliary encoders or lack explicit mechanisms to distinguish heterogeneous conditioning tokens, struggling when the number and type of visual conditions vary across tasks. We propose TIDE, a unified framework that integrates instruction‑based editing, reference‑guided editing, and multi‑reference generation. At its core, we introduce per‑token task embeddings that assign each input token a task‑specific identifier, enabling the model to explicitly disambiguate target, source, and reference tokens. To simultaneously capture high‑level semantic understanding and fine‑grained structural fidelity, we design a dual‑path conditioning scheme that couples a vision‑language model with a VAE latent path for complementary signals. We further devise a multi‑task progressive training strategy that incrementally introduces tasks of increasing complexity, effectively harmonizing diverse objectives and enabling smooth generalization across heterogeneous task distributions. Extensive experiments on multiple video editing and generation benchmarks demonstrate that TIDE achieves state‑of‑the‑art performance across all evaluated tasks. Our project page is available at https://LittleWork123.github.io/tide.
Authors:Tanush Swaminathan, Runmin Jiang, Letian Zhang, Min Xu
Abstract:
LLM‑based scientific agents have shown strong capacity for autonomous research, yet their safety layers remain structurally divorced from core reasoning: they inspect pipeline outputs rather than shaping the deliberation that produces them. This separation opens two failure modes: safety signals accumulated at one stage are discarded before the next, and sequences of individually benign tool calls can compose into harmful outcomes that no single‑step filter detects. To address these challenges, we introduce SciTrace, a framework that weaves safety reasoning into every stage of the scientific agent pipeline. SciTrace couples two complementary mechanisms: a Safety‑Intrinsic Reasoning Loop (SIR) that maintains a cumulative risk state across the Thinker, Experimenter, Writer, and Reviewer stages through joint task‑and‑safety deliberation, and a Compositional Tool‑Chain Verifier (CTV) that performs trajectory‑aware safety checks before execution, catching risks that surface only across multi‑step tool sequences. Evaluated on 240 high‑risk research tasks and 120 tool‑related risk tasks spanning six scientific domains, SciTrace achieves state‑of‑the‑art (SOTA) safety among compared frameworks across four backbone models: it consistently improves tool call safety and adversarial robustness while preserving scientific output quality, and it uncovers 78.8% of the compositional tool‑chain escapes that single‑step monitors miss. The project website is available at https://opensciagent.github.io/SciTrace/.
Authors:Kewei Li, Rongying Zhang, Xueli Wang, Xiwen Gong, Zhongjian Wang, Lan Huang, Ruochi Zhang, Fengfeng Zhou
Abstract:
Token aggregation is a common bottleneck in models that map token representations to sample‑level predictions, yet most pooling methods operate only in the original token domain. We propose FLaG, a plug‑in aggregation module that transforms token representations with the real FFT, summarizes spectral components with learnable latent queries, applies a channel‑wise gate, and reconstructs enhanced time‑domain tokens for final pooling. We evaluate FLaG on antimicrobial peptide (AMP) activity prediction with ESM2, image classification with ResNet18 on CIFAR‑10 and CIFAR‑100, and text classification with RoBERTa on IMDB and GLUE. FLaG achieves its clearest gains on the ESM2‑8M antimicrobial peptide tasks and on CIFAR‑100, while remaining competitive with strong text baselines on IMDB and GLUE. Then we probe its behavior on the AMP setting with band knockouts, gate summaries, residue perturbations, latent‑query readouts, and structure‑proxy stratification. We find that low‑frequency bands contribute the most overall, and the remaining higher‑band pattern is more sample‑specific. The gate acts as a broadly shared spectral reweighting stage and the cross‑attention patterns are sample‑specific with mild query‑wise differentiation, and higher‑helix peptides exhibit stronger average spectral sensitivity in both bacteria. The supplementary materials, source code and data are released at https://www.healthinformaticslab.org/supp/ and https://github.com/Kewei2023/AMPCliff/tree/FLaG.
Authors:Rui Dai, Shuran Zheng
Abstract:
Recent research has established empirical scaling laws to predict model performance on multi‑domain data mixtures. However, a theoretical understanding of these model loss behaviors remains absent. In this work, we propose a unified framework to explain the underlying mechanics of data mixing. Our approach extends theoretical perspectives originally developed for standard neural scaling laws (e.g., Kaplan and Chinchilla) to the multi‑domain setting. Based on the distributional assumption that domains overlap on fundamental skills while diverging on specialized skills, we identify two key factors that govern the domain losses of models trained on different data mixtures: Capacity Competition, where the allocation of finite model capacity couples domain losses globally, and Noise Reduction, where optimal weights shift toward harder‑to‑learn domains to minimize overall noise. Empirical evaluations show that our framework outperforms existing baselines by fitting the loss landscape with a lower Mean Relative Error and identifying higher‑performing training mixtures. Most importantly, our model successfully extrapolates across scales, predicting highly effective mixtures for large, unseen scales using parameters fitted on smaller ones. In addition, our model achieves these results using significantly fewer parameters compared to previous empirical laws. Our code is available at https://github.com/meiqwq/Explaining‑Data‑Mixing‑Scaling‑Laws.
Authors:Jiangshuan Pang, Wangyang Tang, Jing Yan, Zhixuan Cheng, Youzhe He, Zhenkun Zhuang, Tao Zhou, Shiping Liu
Abstract:
MRI preprocessing defines the input distribution seen by brain MRI foundation models, yet it is usually treated as routine data cleaning rather than a modeling choice. We ask how much preprocessing is worth its computational cost for self‑supervised 3D MRI pretraining. Keeping the corpus, 3D ViT backbone, masking protocol, and downstream evaluations fixed, we compare a graded P0‑P7 preprocessing spectrum for masked autoencoding (MAE) and joint‑embedding predictive learning (JEPA) on 20,000 heterogeneous brain MRI volumes, then transfer the encoders to IDH prediction, MCI classification, brain age regression, and GLI/PED tumor segmentation. The results do not support a simple "more is better" rule. P0/P1 are numerically unstable, making P2 the lowest‑cost feasible level; beyond P2, choosing the best feasible preprocessing level improves aggregate utility by only 3.4 percentage points for MAE and 1.8 percentage points for JEPA, with most paired gains statistically unresolved. Stronger preprocessing is beneficial only in selected regimes: IDH improves modestly, AGE and GLI/PED are often near or best at P2, and MCI shows the clearest empirical P7 gain. Cross‑level MCI transfer further shows that much of the P7 advantage can be recovered by applying stronger preprocessing downstream, without requiring P7 throughout pretraining. These findings recast MRI preprocessing as a downstream‑aware cost‑utility decision rather than a default escalation pipeline. Code is available at https://github.com/PangJiangShuan/PreBrain.
Authors:Jiechen Huang, Hector R. Rodriguez, Dingcheng Yang, Zuochang Ye, Yibo Lin, Wenjian Yu
Abstract:
As capacitance extraction accuracy of rule‑based pattern matching becomes difficult to sustain at advanced nodes, a growing trend emerges to develop deep‑learning‑based 2D capacitance models. However, existing MLP‑ and CNN‑based methods constrain their input to fixed metal‑layer combinations in a specific process node, limiting their usability in practice. Recognizing the inherent similarity between capacitance matrix and the prevailing attention mechanism, we propose AttentionCap, a customized Transformer for capacitance matrix learning, with a Gram representation framework, a physics‑aligned symmetric‑attention output layer, and a novel normalized Laplacian loss. We also introduce a process‑node embedding to enable multi‑node learning. Trained on synthetic data, AttentionCap attains 0.67%/3.99% self/coupling‑capacitance error on unseen real designs under a multi‑layer and multi‑node setting, surpassing the CNN‑Cap baseline with 4.6×/5.7× lower self/coupling error and 192× faster inference speed. A pretrained AttentionCap accurately transfers to an unseen node with only 5K samples and 4K finetuning steps. With sufficient accuracy on unseen real designs and strong transferability to new process nodes, AttentionCap offers highly practical value for modern EDA workflows. Code and data are available at https://github.com/THU‑numbda/AttentionCap.
Authors:Xinyu Guan, Qianyang Zhao, Yuming Deng
Abstract:
Tool‑using LLM agents often fail not because relevant text is absent, but because decisive evidence is not selected, compressed, or surfaced at action time. We present CICL, a decision‑aware context layer that turns instance evidence into a context graph, routes deterministic, Opus‑assisted, Qwen, Codex/GPT‑5.5, and Qwen‑QLoRA judgments through a shared eight‑field schema, scores units by action shift, outcome uplift, necessity, and negative‑transfer risk, and packs high‑utility evidence as typed memory cards for a budgeted agent. The design separates the measured decision signal from the judge model, so frontier annotation, local surrogates, and lightweight rankers can be compared under one auditable protocol. Empirically, CICL yields a concrete open‑benchmark gain while exposing its limits. On 50 SWE‑bench Verified file‑retrieval instances, direct Qwen3.6‑plus reranking of BM25 top‑50 candidates raises hit@1 from 0.58 to 0.78 and MRR@10 from 0.634 to 0.790, with all 2,500 judgments parseable. Controlled diagnostics show action‑criticality: at budget 120, CICL reaches F1 0.620 on v1 and 0.425 on v3, and removing the top‑utility semantic v3 unit collapses F1 to 0.000. Supplementary checks add Qwen‑QLoRA agreement over 710 candidates, a small 200‑label real‑code Opus‑assisted signal, and a three‑instance patch smoke validating retrieval‑to‑patch plumbing without claiming official SWE‑bench success. RepoBench‑R summaries still beat cards, and compact rankers do not yet replace the heuristic. CICL contributes a reproducible measurement and selection layer for decision‑critical context, not an end‑to‑end coding‑agent repair claim.
Authors:Bingxuan Dai, Hongsong Wang, Jie Gui
Abstract:
Designing 3D metamaterial microstructures that meet the intended functions remains a major challenge, as it typically requires domain expertise, iterative simulations, and extensive manual tuning. Existing work on inverse design that automatically generates microstructures based on desired target properties often suffers from limited design diversity and faces challenges in ensuring the physical feasibility of the generated structures. To address this issue, a property‑informed diffusion‑based network is proposed that enables the generation of 3D microstructures directly from textual descriptions. Unlike traditional property conditioning methods, our approach leverages rich guidance in terms of semantics and physical properties in the text input to support diverse structure synthesis. To enforce consistency between the generated structures and the target textual prompts, a dual alignment strategy is adopted, including contrastive text‑structure alignment and test‑time reward‑guided alignment. Experimental results show that the model is capable of generating semantically meaningful and physically plausible structures across a wide range of material categories. Our approach has good potential for interactive microstructure design and opens up new directions for combining language‑based interfaces with inverse material discovery. Code is available at: https://github.com/hongsong‑wang/PropDiff‑TMG
Authors:Yi Duan, Zhao Yang, Jiwei Zhu, Ying Ba, Chuan Cao, Bing Su
Abstract:
DNA cis‑regulatory elements (CREs) such as enhancers control gene expression levels. Accurately predicting regulatory activity from DNA sequences is valuable but challenging, as it requires understanding complex biological regulatory processes. Existing methods typically regress activity scores from sequences in a black‑box manner, limiting both interpretability and regression performance. Meanwhile, large language models (LLMs) benefit from explicit reasoning processes, yet directly applying LLMs to raw DNA sequences performs poorly. In this paper, we bridge this gap by introducing R3LM, a framework that teaches LLMs reasoning‑informed regression on regulatory DNA through structured biological knowledge. Specifically, we design a biologically grounded data format that structures DNA's regulatory information for improved LLM understanding, and construct CRE‑ReasonBench, the first dataset that associates DNA sequences and activity scores with mechanistic reasoning traces. Through two‑stage training that first teaches LLMs reasoning over structured biological information then performs regression, R3LM achieves state‑of‑the‑art performance on enhancer prediction across three cell types, outperforming both LLMs with raw sequence input and specialized DNA models while providing interpretable mechanistic explanations. We expect R3LM as an interpretable reward model that can effectively assist biologists in CRE design. Code is available at https://github.com/DuanYi516/R3LM.
Authors:Yichen Chen, Siying Li, Yuhang Liang, Lijun Wang, Renyang Liu
Abstract:
Fraud detection in payment, e‑commerce, and telecommunications systems requires accuracy at the individual level, robustness under severe class imbalance, and ease of understanding for risk managers. Existing methods fall at least one of these requirements: automated machine learning systems search a fixed numerical space without semantic awareness of the dataset; graph neural network‑based methods require pre‑defined relational graphs and remain opaque at the individual‑decision level; and the design of general‑purpose large language model (LLM) agents does not consider the recall and precision constraints specific to real‑world fraud detection. In this paper, we propose SAGE, the first end‑to‑end LLM‑driven multi‑agent framework for fraud detection. SAGE coordinates three dedicated agents that make decisions based on a six‑layer Data Diagnostic Tree (DDT) and a Markov decision process guided by natural‑language gradients, automatically optimizing the model under a fraud‑specific reward. On five fraud datasets and five LLM backbones, SAGE wins 96.00% of method‑‑dataset comparisons and improves F1 by an average of 40.86% over baselines. The code is available at https://github.com/yichenC1c/SAGE.
Authors:Ruben Dario Florez-Zela
Abstract:
Vision‑based driver monitoring systems are increasingly deployed in safety‑critical intelligent transportation settings, yet they are almost always compared on classification accuracy alone. This paper argues that accuracy is insufficient to characterize a model's fitness for real‑world deployment, and proposes the Human‑Centered Benchmarking Framework (HCBF), which evaluates models across four dimensions: accuracy, explainability, efficiency, and robustness. The framework is applied to four representative lightweight architectures, MobileNetV3, ShuffleNetV2, EfficientNet‑B0, and DeiT‑Tiny, on the MRL Eye Dataset for eye‑state classification. While the models are nearly indistinguishable on clean‑set accuracy, each leads in exactly one dimension, and all four lie on the Pareto frontier. A Human‑Centered Score computed under three deployment‑oriented weighting scenarios ranks ShuffleNetV2 first throughout. However, this aggregate winner retains less than half of its performance under sensor noise and fails by classifying closed eyes as open, whereas the transformer remains robust. These findings show that aggregate ranking can mask dimension‑specific vulnerabilities that are operationally decisive, underscoring the value of multi‑dimensional, human‑centered evaluation.
Authors:Xiaoqian Wu, Yejie Guo, Xiaoyang Chen, Lixin Yang, Cewu Lu, Yong-Lu Li
Abstract:
We are surrounded by various objects with movable, articulated parts, e.g., box, handle, door. An accurate and generalizable perception of articulated parts is essential to enhance robotic manipulation capabilities. Building on this need, recent efforts in articulated parts perception have followed two main directions: One line of work uses pose‑based representation, which requires high manual cost; in parallel, affordance‑based methods extract future object motion from point tracking without additional manual efforts, but suffer from low‑quality data. In this paper, we propose a new representation of articulated parts, Geometric Primary Structure (GPS), an abstraction of the part geometry structure to balance scalability and quality. For efficient and scalable data collection, GPS is integrated with a portable Virtual Reality (VR) device and requires only one minute to annotate one object sequence. This direct human annotation provides higher quality than the estimated affordance. With this efficient VR‑GPS system, we collect 41K frames for 234 objects across six part classes, and train a generalizable GPS model with a single RGB‑D object image as input. For object manipulation, we deploy a heuristic policy based on GPS prediction. Without any in‑domain fine‑tuning, our method achieves an 73% success rate, covering 270 initial states for 9 objects. Our code, data and reusable tool are available at https://enlighten0707.github.io/gps.
Authors:Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen, Thanh Q. Duong, Linh D. Le, Duy M. H. Nguyen, Vien A. Ngo, An T. Le
Abstract:
Vision‑Language‑Action (VLA) policies are typically shipped as Python/PyTorch stacks that assume a workstation‑class GPU, a mismatch for the hardware on which robots actually run. We present vla.cpp, a portable C++ inference runtime built on llama.cpp. To our knowledge, it is the first ggml‑class engine to natively serve the flow‑matching and diffusion VLA inference pattern, in which a cached vision‑language prefix is consumed by a cross‑attending action expert integrated over several solver steps. A single runtime serves seven architectures spanning five backbone and four action‑head families behind one request/response protocol, with each model packaged as a self‑contained bundle. On LIBERO‑Object, the engine matches a state‑of‑the‑art checkpoint to within one episode out of 200, and runs BitVLA at 100% success in 1.3 GiB of memory. The same bundle runs unchanged across three hardware tiers, from a consumer GPU down to an 8 GB embedded module. A cross‑hardware roofline analysis shows that batch‑1 VLA inference is compute‑bound, so utilization rather than bandwidth is the deployment lever; an IMMA ladder GEMM derived from this analysis cuts BitVLA per‑step latency by 4.5x. We then frame an on‑robot stress test on an ALOHA arm that isolates the latency constraint under which a learned VLA must replan against a moving target on the hardware it was trained for. Code, demo videos, and the reproducible benchmark scaffold are available at https://fai‑modelopt‑tech.github.io/vla‑cpp.github.io/.
Authors:Jianhui Wei, Jie Tan, Hengchuan Zhu, Xiaotian Zhang, Yan Zhang, Ziyi Chen, Daoan Zhang, Wei Xu, Zuozhu Liu
Abstract:
Recent agent frameworks such as Claude Code, Codex, and OpenClaw are strong at tool use and orchestration, but whether they can handle long video generation, a long‑horizon multimodal task, remains underexplored. Unlike earlier video agents whose pipeline is handcrafted, these frameworks can build and refine their own workflows. We introduce VideoWeaver, an agent harness and benchmark that evaluates and evolves skills for long video generation, where an agent turns a single instruction into a long video by composing foundation skills into its own workflow rather than following a predefined pipeline. The benchmark has 16 task categories and 285 cases, with references spanning text, image, audio, video, and their combinations. Because errors can arise at any stage and not just in the final video, we propose an agent‑as‑judge that inspects both the execution trace and the final video, grounding its scores in evidence such as metadata and intermediate files. Using this feedback, we further design a skill evolution algorithm that refines and merges the agent's skills. Across multiple frameworks and models, we find that an explicit composition skill improves the generation process over using foundation skills alone, that skill evolution further improves output quality, and that performance varies notably across harness and model choices. The proposed agent‑as‑judge also aligns well with human judgments, especially on process metrics. Code and dataset is available at https://github.com/JianhuiWei7/VideoWeaver
Authors:Jiaqi Tang, Jianmin Chen, Youyang Zhai, Wei Wei, Runtao Liu, Mengjie Zhao, Xiangyu Wu, Qingfa Xiao, Qifeng Chen
Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real‑world visual corruptions. While existing robustness enhancement approaches exist, they are limited: black‑box feature alignment lacks interpretability, and white‑box text‑based reasoning cannot restore lost pixel‑level details. This work investigates a fundamental research question: Can MLLMs recover corrupted visual content by themselves? To address this, we propose Robust‑U1, a novel framework that equips MLLMs with explicit visual self‑recovery capability for robust understanding. The approach comprises three core stages: supervised fine‑tuning for initial reconstruction, reinforcement learning with dual rewards (pixel‑level SSIM and semantic‑level CLIP similarity) for aligning high visual quality, and multimodal reasoning that jointly considers both the corrupted input and the recovered image. Extensive experiments demonstrate that Robust‑U1 achieves state‑of‑the‑art robustness on the real‑world corruption benchmark and maintains superior performance under adversarial corruptions on general VQA benchmarks. Analysis confirms that high‑quality visual recovery directly enhances reasoning performance, establishing self‑recovery as a critical mechanism for robust visual understanding. The source code is available at https://github.com/jqtangust/Robust‑U1.
Authors:Juntong Shi, Brian L. Trippe, Jure Leskovec, Stefano Ermon, Minkai Xu
Abstract:
Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models. Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target. However, due to the huge gap between their distributions, the sampling requires a large number of particles and is thus expensive to compute. In this paper, we introduce PoE‑Bridge, a novel decoding framework that drastically improves generation speed and accuracy by introducing an intermediate distribution to bridge the gap. The distribution is constructed as a Product‑of‑Experts (PoE) of the DLM proposal and the AR target. With the intermediate distribution, we first use the DLM to draft multiple continuations in parallel, then apply rejection sampling to verify the drafted tokens and move the resulting candidates toward the PoE. We then use importance sampling to further correct the PoE‑aligned candidates toward the AR target. We further propose several improved techniques, including mixed‑temperature sampling for enhanced diversity and elastic rejection windows for reducing wasted verification. Empirically, PoE‑Bridge achieves significantly improved accuracy with 5× speedup over the standard DLM decoding approach, and recovers at least 95% of the target AR model's performance, efficiently advancing most of the quality gap on challenging mathematical reasoning and coding tasks. Our code is available at https://github.com/juntongshi48/poe‑bridge.
Authors:Hongkyu Koh, Ikbeom Jang
Abstract:
Electrocardiogram (ECG) classification models often suffer from severe label scarcity, making semi‑supervised learning (SSL) an attractive strategy for reducing annotation costs. In clinical settings, however, unlabeled pools frequently contain out‑of‑distribution (OOD) anomalies or diagnostic groups absent from the labeled set. Standard SSL forces incorrect pseudo‑labels onto these unseen classes, producing overconfident predictions. To address this, we propose SafeECGMatch, a calibration‑aware safe SSL framework for single‑label ECG classification under label distribution mismatch. Methodologically, SafeECGMatch employs a dual‑branch architecture extracting time‑frequency latent representations via ECG‑specific augmentations. Crucially, it dynamically aligns confidence with empirical accuracy through adaptive label smoothing and temperature scaling, calibrating both the multiclass classifier and the OOD detector across temporal and spectral domains. This joint optimization allows trustworthy OOD rejection and reliable pseudo‑labeling. Evaluated on the PTB‑XL and PhysioNet/CinC Challenge benchmarks, SafeECGMatch achieves state‑of‑the‑art accuracy and calibration, advancing reliable knowledge discovery in physiological time‑series. Code is available at https://github.com/labhai/SafeECGMatch.
Authors:Yuxin Cai, Zongtai Li, Maonan Wang, Muyi Bao, Haokun Zhu, Ruofei Bai, Ding Zhao, Zirui Li, Wenshan Wang, Wei-Yun Yau, Ji Zhang, Chen Lv
Abstract:
Object navigation requires a robot to search for an unobserved target in an unknown environment by deciding where to explore next under partial observability. Effective search resembles human‑like exploration: selectively probing visually promising frontiers while relying on spatial memory to avoid redundant revisits. We propose IntentNav, a spatial‑visual imitation framework that learns human‑like ObjectNav policies from human demonstrations. To infer high‑level search intent from low‑level human actions, we introduce Frontier‑based Human‑Intent Labeling, which looks ahead in human demonstrations and labels the frontier that best explains the demonstrator's future search direction. We construct a spatial‑visual candidate space, where BEV memory tracks explored regions, unexplored frontiers, and trajectory history, while egocentric visual memory provides semantic cues for each candidate. A VLM policy is trained to select among these grounded candidates, using Intent‑Aligned Objective to encourage consistent and human‑like exploration. IntentNav achieves state‑of‑the‑art performance on the MP3D, HM3D‑v1 and HM3D‑v2 ObjectNav benchmarks. The proposed candidate‑level navigation interface transfers zero‑shot to wheeled, quadruped, and humanoid robots without further VLM fine‑tuning. \hrefhttps://anonymous.4open.science/w/IntentNav/Project page.
Authors:Jianling Gao, Chongyang Tao, Jiayuan Bai, Liu Yang, Xuanguang Pan, Jinrui Liu, Shihao Xing, Xiaohan Xu, Jie Liang, Shuai Ma
Abstract:
Existing text‑to‑SQL benchmarks are largely centered on SQLite, making it difficult to evaluate whether models can generalize across heterogeneous SQL dialects. However, real‑world database systems differ substantially in syntax, functions, type systems, and execution semantics, so the same natural language intent often requires dialect‑specific SQL realizations. We introduce UniQL, a human‑verified benchmark for cross‑dialect text‑to‑SQL evaluation. UniQL aligns 1,534 natural language questions with executable SQL annotations across 16 SQL dialects, yielding 24,544 dialect‑specific queries. All dialects share the same intents, aligned schemas and database contents, enabling controlled evaluation of dialect generalization. UniQL is constructed through a hybrid pipeline combining database migration, SQL translation, execution‑guided verification, iterative rule summarization, and human validation. Experiments on both open‑source and closed‑source LLMs show that current models remain far from dialect‑universal, with substantial performance variation across database systems and limited transfer from SQLite success to other dialects. These findings highlight the need for aligned cross‑dialect benchmarks and more dialect‑aware text‑to‑SQL methods. Code and data are available at https://github.com/JerryGao818/UniQL
Authors:Zichen Zhu, Yuheng Sun, Mingxuan Zhu, Wenjie Ma, Situo Zhang, Zhexiang Wang, Ziyue Yang, Danyang Zhang, Kunyao Lan, Zihan Zhao, Dingye Liu, Siqi Xiang, Lu Chen, Kai Yu
Abstract:
Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes. Creations by generative models may contain artifacts, implausible details, or stylistic drift away from photorealism and offer little insight into why an edit was made. We propose IEA, a conversational Image Editing Agent that learns to operate parameterized tools in an explicit, interpretable action space. IEA is trained via a three‑stage multitask pipeline: (1) SFT on distilled expert edits, (2) GRPO with rewards for likeness improvement, tool usefulness, and intent summarization, and (3) large‑scale synthetic fine‑tuning to jointly master image editing, refinement, and user intent summarization. By manipulating 16 editing tools step by step, IEA produces transparent edit traces that can be inspected and debugged. In quantitative experiments, it attains a lower pixel distance on the edit task and a higher ROUGE‑L on the summary task than strong baselines. In user studies, it ranks best among tool‑calling methods for instruction following while surpassing generative methods in overall perceptual quality. Our results validate interpretable, tool‑centric VLMs as a reliable path to human instruction‑guided image retouching.
Authors:Atharv Thakur
Abstract:
This paper makes several improvements to existing level set based approaches to computing shape optimizers for the Dirichlet eigenvalues subject to a volume constraint. The most notable changes in formulation include an overhaul of the classical level set construction and root‑finding procedures as well the use of a regularized approximation to the standard objective function. Our resulting computational minimizers are either comparable to or improvements on the best known minimizers from the literature. We conclude with a survey of subproblems within the field that may benefit from numerical experiments; these include the existence of cusps on the boundary, the end‑behavior of eigenfunction weights in the p‑parameterized problem, and the nature of Weyl asymptotics as they relate to the Pólya conjecture.
Authors:Haoming Wen, Shi Chen, Qingyu Shi, Siyuan Liu, Minrui Luo, Jingzhao Zhang, Tianxing He
Abstract:
Current open‑weight large language models (LLMs) are prone to malicious finetuning attacks, which could compromise the safety alignment of LLMs with only a few steps of supervised finetuning (SFT) on poisoned datasets. Existing alignment‑stage defenses are primarily designed to defend against attacks that use parameter‑efficient finetuning methods. However, they fail to defend against stronger attacks that use full‑parameter finetuning. In this paper, we propose Patcher, a method inspired by adversarial training and bi‑level optimization, to combat such attacks. Patcher strengthens the simulated attack by scaling up the optimization steps in the adversarial loop, thus forcing the defender to find model parameters that are insensitive to stronger attacks. Furthermore, we propose an efficient parallel algorithm to implement Patcher, decreasing the wall‑clock time of training while preserving Patcher's performance. Extensive experiments show that Patcher substantially improves the model's robustness compared to vanilla SFT alignment, and transfers to diverse attack scenarios and model sizes. Code is available at https://github.com/haomingwen/patcher.
Authors:Alexey Kresin, Tchifou M. Dieffi, Tomer Caspi
Abstract:
Debiasing methods based on principal component analysis (PCA) are broadly used to reduce gender bias in word embeddings used in LLMs, yet it remains unclear what aspects of bias they actually remove and how destructive this process is. These methods are based on the understanding that bias resides in a low‑dimensional subspace, with the assumption that most of it can be captured by a few principal components. In this work, we conduct a systematic geometric analysis of PCA‑based gender debiasing and investigate what is actually removed from the embedding space. Our experiments across multiple embeddings show that direct gender bias is primarily concentrated in the first principal component, supporting the low‑rank bias hypothesis. However, associative bias measured by WEAT does not align with these principal directions and is instead spread across multiple embedding dimensions. Furthermore, as expected, we demonstrate that removing an increasing number of principal components leads to a consistent degradation of the embedding geometry, affecting semantic structure and vector relationships. These results reveal that PCA‑based debiasing operates as a trade‑off: while it effectively reduces certain forms of direct bias, it fails to eliminate distributed associations and introduces geometric distortion. Moreover, there is no universal optimal level of debiasing, as the balance between bias reduction and semantic preservation depends on the chosen metric and embedding. Overall, our findings suggest that bias in word embeddings is not purely low‑rank and that simple subspace removal methods may be insufficient for comprehensive debiasing.
Authors:Zekai Zhang, Jinglin Zhang, Qinghui Chen, Gang Li, Da Chen, Shuainan Jing, He Wang, Dagang Li, Cong Liu, Cong Bai, Shengyong Chen
Abstract:
Large‑scale Visual‑Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large‑scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text‑visual interaction for fine‑grained understanding. To address these challenges, we introduce a Large‑Scale Multi‑Modal Industrial Open‑Closed benchmark (MMIOC‑1M) containing over one million samples across 14 super‑categories, 29 industrial scenes, and 351 defect subcategories. To our knowledge, MMIOC‑1M is the first unified largest benchmark supporting both open‑vocabulary and closed‑set industrial detection, providing valuable pre‑training data for LVLMs in industrial scenarios. Furthermore, we propose a Refined Text‑Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert‑assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy‑based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text‑visual interaction module that enhances cross‑modal semantic alignment and understanding. Extensive experiments demonstrate that RTVPNet achieves state‑of‑the‑art performance on MMIOC‑1M, LVIS, and COCO benchmarks while maintaining computational efficiency. The dataset and code are available at https://github.com/hellozzk/MMIO.
Authors:Zhanke Zhou, Xiangyu Lu, Chentao Cao, Brando Miranda, Tongliang Liu, Bo Han, Sanmi Koyejo
Abstract:
RL with verifiable rewards can substantially improve LLM reasoning, yet standard GRPO‑style training often treats easy, hard, and learnable questions alike through uniform sampling and weighting, leading to inefficient compute allocation. We study GRPO by tracking token log‑probabilities, group‑normalized advantages, and the induced token‑level update weights. This reveals three recurring dynamics as training proceeds: (1) confidence inflation, (2) advantage contraction, and (3) hierarchical convergence. These findings suggest that the utility of each update depends strongly on both question difficulty and the model's current competence. Motivated by this, we propose Confidence and Difficulty‑adaptive Policy Optimization (CoDaPO), which assigns each question a bounded value from rollout confidence and empirical difficulty. CoDaPO then uses this value to reweight policy updates and resample high‑value learnable questions within mini‑batches, thereby increasing discovery within the learnable band under a fixed compute budget. Across twelve benchmarks, CoDaPO consistently improves accuracy over existing RL methods. Our code is publicly available at https://github.com/tmlr‑group/CoDaPO.
Authors:Shuyao Gao, Minghao Huang
Abstract:
Empirical research on the labor‑market impact of artificial intelligence has converged, since Frey and Osborne (2017), on a continuous‑gradient representation in which each occupation is assigned a real‑valued exposure score on [0,1] obtained by linear aggregation across capability dimensions. This continuity is rarely articulated as an assumption and has not been tested at the micro‑action level where substitution actually occurs. We decompose 1,961 ONET Detailed Work Activities into 15,817 micro‑actions using a multi‑agent LLM pipeline with 31‑expert HITL calibration, then project the DWA‑level Occupational Automation Index from our prior work onto a 7‑macro semantic typology. The result is a bipolar structure. Tool‑Mediated Physical (M2, mean OAI = 0.054) and Planning & Design (M7, mean OAI = 0.499) form two extremes separated by Cohen's d = 2.41 (H = 172.88, p = 6.21e‑34). The geometry is robust under three independent stress tests: resolution (K=7 to K=15, polar gap widens from 0.45 to 0.57), encoder swap to BGE (LLM‑class OAI lead replicates at 3.37x), and Eloundou's GPT‑4 task ratings (DWA‑level rho = 0.635). The six middle macros form a low‑contrast band between the poles (TOST at d=0.2 admits only 1/15 pairs as equivalent), not a flat plain. The geometry's stability does not, however, extend to its content. Across a decade, the polarity has inverted. Frey‑Osborne (2013) placed Tool‑Mediated Physical near the highest computerisation risk and Planning & Design near the lowest; our LLM‑era OAI reverses that order, with macro‑level FO‑Eloundou Spearman rho = ‑0.750, p = 0.020, against the original Oxford Martin appendix. Which pole is high is therefore contingent on the era's dominant capability frontier, while the stable geometry itself is the structurally robust object.
Authors:Swarna Chakraborty, Gabriel De Castro Araújo, Syeda Tasmi Faria, Marcelo M. Carvalho, Mylene C. Q. Farias
Abstract:
Point Cloud Quality Assessment (PCQA) methods typically predict scalar Mean Opinion Scores (MOS), which quantify overall perceptual degradation but do not reveal its causes. In contrast, human observers naturally reason in terms of specific distortions such as blur, color shifts, point density changes, missing regions, and geometric deformations. To close this gap, we introduce DAL‑PCQA, a distortion‑aware, language‑annotated dataset for PCQA. DAL‑PCQA augments benchmark point clouds with multi‑level distortion severity labels, discrete quality categories, and structured natural language descriptions aligned with human perception. We define a point‑cloud‑specific distortion taxonomy that covers both photometric and geometric artifacts. Statistical analysis reveals characteristic degradation patterns across distortion types and quality levels. To assess the utility of these annotations, we compare zero‑shot and fine‑tuned multimodal models for generating perceptual quality descriptions. Experiments show that distortion‑aware supervision substantially improves lexical and semantic alignment with ground‑truth descriptions. By enabling interpretable, distortion‑level reasoning, DAL‑PCQA facilitates language‑driven, explainable point cloud quality assessment. The dataset is publicly available at https://github.com/swarna96/DAL‑PCQA.
Authors:Katelyn Xiaoying Mei, Yi-Li Hsu, Minjoon Choi, Zongwan Cao, Chenjun Xu, Bingbing Wen, Su Lin Blodgett, Lucy Lu Wang
Abstract:
Human evaluation plays a critical role in assessing the quality of generated text. However, the reliability and reproducibility of these evaluations depend on transparent and well‑documented protocols ‑‑ details that are frequently missing in current practice. In this work, we conduct a large‑scale analysis of human evaluation protocols for evaluating long‑form generation tasks in CL conference publications from 2023‑‑2025, including a full manual review of 284 papers and LLM‑assisted analysis for another 1.8k+ papers. We define a set of 20 reportable criteria related to reproducibility of human evaluation studies, and apply these criteria to systematically examine reporting norms and practices within the community. We find widespread under‑reporting of important aspects of human evaluation study design, leading to ambiguity about what was measured and how, who contributed judgments, and how judgments should be interpreted. Based on these findings, we outline actionable recommendations to support more transparent and reproducible reporting in future research. Our analysis code and annotated dataset can be found at: https://github.com/larchlab/Illusions‑of‑the‑Gold‑Standard
Authors:Siyang Song, Micol Spitale, Zijian Wu, Xiangyu Kong, Cheng Luo, Cristina Palmero, German Barquero, Sergio Escalera, Michel Valstar, Mohamed Daoudi, Fabien Ringeval, Andrew Howes, Elisabeth Andre, Hatice Gunes
Abstract:
In dyadic interactions, various human facial reactions could be appropriate for responding to each human speaker behaviour. Following the successful organisation of the REACT 2023, 2024 and 2025 challenge series, a body of generative deep learning (DL) models have been developed for the problem of multiple appropriate facial reaction generation (MAFRG). This year, we propose the REACT 2026 challenge encouraging the development and benchmarking of Machine Learning (ML) models that can generate multiple personalised, appropriate, diverse, realistic and synchronised human‑style facial reactions expressed by a specific human listener for responding to each given speaker behaviour. As a key of the challenge, we continuously provide challenge participants with MARS dataset introduced by REACT 2025 but additionally provide individual‑level Big‑Five personality labels and EEG recordings. This introduces a new one‑to‑many personalised facial reaction generation setting combining human expressive behavioural, affective and neurophysiological signals, which remains largely unexplored in current dyadic interaction modelling. This paper also presents the challenge guidelines and new baselines on the four proposed sub‑challenges: Offline generic and personalised MAFRG as well as Online generic and personalised MAFRG, respectively, which are publicly available at https://github.com/reactmultimodalchallenge/baseline_react2026.
Authors:Didi Zhu, Changrui Chen, Stefanos Zafeiriou, Jiankang Deng
Abstract:
When a multimodal large language model answers a visual reasoning question correctly, is the prediction actually supported by the task‑critical visual evidence? Correct answers can coexist with flawed reasoning, making accuracy alone an incomplete test of grounding. We introduce VisualFLIP, a paired benchmark with 1,374 images arranged as same‑question perturbation pairs across cardinality, attribute, spatial, and logic tasks. Each pair keeps the question fixed but minimally changes the evidence so the gold answer deterministically flips. We evaluate 24 MLLMs with pair accuracy, which requires solving both sides of a pair, and Collapse Rate (CR), which measures how often a model that solves at least one side repeats the same non‑empty answer for both images. Together, these metrics show that paired correctness and evidence dependence are related but distinct: capable models can still fail to update after task‑critical visual changes, and collapse becomes more severe for some models when the edited image follows an earlier answer in a sequential setting. Further details are available on our project page: https://didizhu‑judy.github.io/VisualFLIP/
Authors:Chung-En Sun, Linbo Liu, Tsui-Wei Weng
Abstract:
Are tool‑calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially safer after a few regular agentic tasks ‑‑ a phenomenon we term the cold‑start safety gap. To study this systematically, we introduce Safety Over Depth for Agents (SODA), a benchmark that controls how many regular agentic tasks the agent completes before encountering a safety threat, supporting up to 20 preceding tasks. Evaluating 7 models from 4 families, safety improves by 9‑‑52% as the number of preceding regular agentic tasks increases from zero to twenty. Representation analysis confirms that model hidden states gradually shift toward a safety‑aligned region as more preceding tasks are present. By systematically studying which part of the preceding conversation matters most, we find that the regular agentic tasks themselves are the primary driver of safety, while the agent's own prior responses have less effect on safety but are essential for preserving later utility. This conclusion is further supported by evaluation on open‑source safety benchmarks (AgentHarm, Agent Safety Bench) and utility benchmarks (BFCL, API‑Bank), confirming that warming up the agent with regular agentic tasks before deployment makes it safer and preserves full capability. Based on these findings, we recommend a simple deployment strategy: having the agent complete a few regular agentic tasks before possible exposure to safety‑critical requests mitigates the cold‑start safety gap. Our code is available at https://github.com/Trustworthy‑ML‑Lab/Agent‑Cold‑Start‑Safety‑Gap
Authors:Anish Laddha, Nitesh Pradhan, Gaurav Srivastava
Abstract:
Large language models (LLMs) are widely used as judges for evaluating model outputs, but their high cost, latency, and opacity limit scalability. We introduce SLMJury, a framework for evaluating small language models (SLMs) as judges across two paradigms: closed‑ended binary correctness and open‑ended quality scoring. We benchmark 16 SLM judges (0.6B‑14B parameters) from four model families across ten benchmarks: eight closed‑ended tasks spanning mathematical, scientific, and general reasoning (N=64,824 judgments per configuration), plus SummEval and MT‑Bench for summarization and conversational scoring. We formalize judging as a budget‑conditioned function and study five dimensions. Four findings emerge. (1) The overthinking effect is domain‑dependent: for most judges quick 10‑token verdicts match or beat extended reasoning on mathematical judging (by 2‑7% where they help), while reasoning wins on general tasks by up to 23%. (2) Domain generalization separates model families, with math‑to‑general accuracy gaps ranging from under 10% to nearly 40%. (3) Closed‑ended and open‑ended judging draw on different capabilities: the best binary judge (Phi‑4) drops to rank 9 on MT‑Bench, while reasoning‑trained models invert this ordering. (4) Under the Reflect‑Critique‑Refine (RCR) debate protocol, multi‑agent debate degrades accuracy across all tested configurations, whereas the top judges resist six adversarial personas with <=0.55% variance. Reliable automated evaluation does not require large proprietary models, yet no single SLM dominates. The leaderboard is available at https://anishh15.github.io/SLMJury/, and our framework code and pip package are publicly available at https://github.com/anishh15/SLMJury and https://pypi.org/project/slmjury/.
Authors:Alma Andersson, Aya Abdelsalam Ismail, Edward De Brouwer, Doron Haviv, Tommaso Biancalani, Kyunghyun Cho, Gabriele Scalia, Aïcha BenTaieb, Hector Corrada Bravo
Abstract:
Understanding cellular phenotypes and how they respond to perturbations is critical for disease biology and therapeutic design. Single‑cell RNA sequencing enables characterization at cellular resolution, yet the combinatorial space of conditions makes exhaustive experimental mapping infeasible. We introduce single‑cell Concept Bottleneck Generative Models (scCBGM), a framework for interpretable and precise counterfactual editing of individual cells. scCBGM adapts concept bottleneck architectures for single‑cell data through decoder skip connections and a cross‑covariance penalty that promotes disentanglement without dimensional constraints. We extend the framework to flow matching models, enabling concept‑guided editing in both encoding‑decoding and generation regimes. To enable rigorous evaluation, we develop a synthetic benchmark with ground‑truth counterfactuals. Across multiple real datasets, scCBGM demonstrates superior performance in combinatorial generalization and counterfactual prediction, supported by cell‑level validation on synthetic data and population‑level benchmarks on real datasets.
Authors:Blake B. Gaines, Jinbo Bi
Abstract:
It is well established that ReLU networks define continuous piecewise‑linear functions, and that their linear regions are polyhedra in the input space. These regions form a complex that fully partitions the input space. The way these regions fit together is fundamental to the behavior of the network, as nonlinearities occur only at the boundaries where these regions connect. However, relatively little is known about the geometry of these complexes beyond bounds on the total number of regions, and calculating the complex exactly is intractable for most networks. In this work, we prove new theoretical results about these complexes that hold for all fully‑connected ReLU networks, specifically about their connectivity graphs in which nodes correspond to regions and edges exist between each pair of regions connected by a face. We find that the average degree of this graph is upper bounded by twice the input dimension regardless of the width and depth of the network, and that the diameter of this graph has an upper bound that does not depend on input dimension, despite the number of regions increasing exponentially with input dimension. We corroborate our findings through experiments with networks trained on both synthetic and real‑world data, which provide additional insight into the geometry of ReLU networks. Code to reproduce our results can be found at https://github.com/bl‑ake/ICLR‑2026.
Authors:Siyi Chen, Hugo Hadfield, Alex Zook, Mikaela Angelina Uy, Chan Hee Song, Erwin Coumans, Xuning Yang, Faisal Ladhak, Qing Qu, Stan Birchfield, Jonathan Tremblay, Valts Blukis
Abstract:
Open‑vocabulary long‑horizon manipulation requires robots to reason over flexible instructions and complex multi‑object scenes while adaptively planning, executing, monitoring, and recovering from failures. We address these demands with a closed agent loop in which a VLM orchestrates heterogeneous robot capabilities as interruptible tools. Unlike in virtual AI agents, the timing of decisions, actions and tool calls is important in a physical world that does not pause for reasoning. We refer to this setting as Physical Orchestration, and propose VoLoAgent, a VLM that plans, monitors, and recovers by treating a VLA/WAM as an interruptible tool it steers mid‑rollout alongside vision models and action primitives. To evaluate these long‑horizon capabilities, we introduce RoboVoLo, a high‑fidelity benchmark for open‑vocabulary long‑horizon manipulation across common sense, memory/state tracking, complex references, and world knowledge, with both task‑level success and failure‑mode diagnostics. Experiments show VoLoAgent substantially outperforms single VLA/VLM or tool‑based systems, with validation on real‑robot experiments. Project page: https://chicychen.github.io/VoLo/
Authors:Maryam Zaman, Muhammad Khuram Shahzad
Abstract:
Adversarial attacks pose a serious and growing threat to Machine Learning (ML)‑based Intrusion Detection Systems (IDS), where imperceptible perturbations to network flow features can systematically mislead classifiers into accepting malicious traffic as benign. The IDS‑Anta framework partially addresses this through Z‑score normalization, Singular Value Decomposition (SVD), and Multi‑Armed Bandit (MAB) classifier selection with Thompson Sampling, yet its classifier pool lacks sufficient structural diversity for robust adversarial resistance. This work introduces IDS‑Anta++, which incorporates XGBoost and LightGBM gradient boosting models into the ensemble and wraps the extended pool in a three‑layer black‑box defense: Isolation Forest anomaly screening, median feature smoothing, and six‑way majority voting. Experiments conducted on CIC‑IDS‑2017, CEC‑CIC‑IDS‑2018, and CIC‑DDoS‑2019 under both Fast Gradient Sign Method (FGSM) and Zeroth Order Optimization (ZOO) attacks confirm detection accuracy above 99% on clean data, with measurable robustness gains under adversarial conditions relative to the baseline IDS‑Anta configuration.
Authors:Rishabh Makwana, Mamta, Deeksha Varshney, Oana Cocarascu
Abstract:
Vision‑Language Models (VLMs) have demonstrated strong performance across multimodal tasks, yet their safety robustness remains an open challenge. While prior work has shown that structured visual prompts such as flowcharts can effectively jailbreak VLMs, existing studies are largely limited to English‑centric settings. In this paper, we introduce MLingualFC, a multilingual multimodal benchmark designed to evaluate jailbreak vulnerabilities of VLMs across diverse languages using structured flowchart representations. MLingualFC encodes harmful instructions into flowchart images across five languages (Hindi, Punjabi, Spanish, Romanian, and German). We evaluate state‑of‑the‑art multilingual VLMs, including Qwen2.5‑VL, Gemma‑4, and Pangea, under a black‑box threat model. Our results reveal significant multilingual safety gaps. Flowchart‑based attacks achieve high attack success rates (ASR) in case of Latin script languages, demonstrating that visual encoding of harmful content effectively bypasses safety alignment across languages. In contrast, non‑Latin script languages such as Punjabi exhibit substantially lower ASR, suggesting potential limitations in visual text recognition rather than stronger safety alignment. These findings highlight that current VLM safety mechanisms fail to generalize across languages and modalities. Resources are available at https://github.com/Rishabhpm23/MLingualFC
Authors:Yuchen He, Baolong Bi, Shenghua Liu, Huaming Liao, Yuyao Ge, Bolin Wan, Siqian Tong, Juan Chen, Jiafeng Guo, Xueqi Cheng
Abstract:
Although multi‑objective reinforcement learning (MORL) is central to aligning large language models with complex human preferences, the prevailing practice of static weighted summation overlooks a more fundamental phenomenon: reward learning is markedly asynchronous across objectives. Well‑learned dimensions quickly produce homogeneous, low‑variance signals whose residual noise contaminates the aggregated reward (in GRPO) or occupies a fixed share of the advantage budget (in GDPO), interfering with the scarce yet high‑value signals carried by under‑learned dimensions. To address this asynchrony, we propose Stage‑Aware Dynamic Weighting (SAW), a lightweight, algorithm‑agnostic dynamic weighting mechanism. SAW utilizes the coefficient of variation (CV) as a scale‑invariant proxy for real‑time informativeness, reweighting each dimension's reward or advantage contribution by its relative informativeness within the batch. Unlike gradient‑based methods that require multiple forward and backward passes, SAW relies solely on batch‑level statistics, introducing nearly negligible computational overhead. Experiments on tool‑calling and text summarization tasks demonstrate that SAW consistently improves both training efficiency and final performance under both GRPO and GDPO frameworks, confirming it as a general‑purpose plug‑in for multi‑reward LLM alignment. Our code is available at https://github.com/Zhaolutuan/SAW
Authors:Sumit Chongder
Abstract:
Noisy intermediate‑scale quantum (NISQ) processors are entering an early fault‑tolerance regime where full quantum error correction carries prohibitive resource costs, yet lightweight error detection can meaningfully improve algorithmic success rates. Existing compilation and error‑detection toolchains treat these concerns in isolation, with no principled way to balance detection overhead against success probability under latency constraints. We present an integrated hardware‑aware compilation and data‑driven quantum error‑detection (QED) framework that jointly optimises qubit mapping, SWAP insertion, and syndrome‑schedule placement via a noise‑weighted cost function and a learned multi‑objective scheduler. Simulation experiments on an HPC cluster using GPU‑accelerated density‑matrix simulation (NVIDIA cuQuantum SDK) across VQE, phase‑estimation, and Grover benchmarks, three noise profiles, and circuit sizes of 6‑20 qubits (depths 10‑160), show that joint co‑design raises algorithmic success probability by up to 68 percent (95 percent CI: 60 percent to 76 percent) over SABRE on an 8‑qubit VQE instance with post‑selection.
Authors:Maksim Shandybo, Ivan Bespalov, Daniil Yefimov, Marina Kosheleva, Alexander Loukianov
Abstract:
Parsing historical documents with complex, non‑standard layouts remains a fundamental bottleneck in large‑scale archival digitization. Unlike modern typography, historical newspapers exhibit severe physical degradation and highly irregular page structures that confound even state‑of‑the‑art vision‑language models, presenting severe out‑of‑distribution challenges. We address this gap with an automated pipeline specifically designed for parsing historical newspapers, documents characterized by particularly intricate multi‑column layouts. Our approach combines a fine‑tuned YOLO architecture for layout analysis and block detection, trained on 1,426 fully human‑annotated scanned pages, with a novel semantic assembly module that reconstructs articles by jointly modeling lexical‑semantic similarity via TF‑IDF, visual embeddings from our fine‑tuned YOLO, and geometric layout constraints. This multi‑modal integration yields state‑of‑the‑art performance, achieving an F1 score of 0.904 on block‑to‑article mapping. Notably, end‑to‑end evaluation against vision‑language models (Qwen3.6‑35B‑A3B and Qwen3.6‑Plus) demonstrates that PereStruct achieves substantially higher fidelity (BLEU approximately 0.96 vs 0.34), validating that modular architectures excel where generic VLMs fail on complex historical layouts. To support reproducibility and advance research in this domain, we release both the training corpus of 599 annotated pages and a curated PereStruct benchmark of 93 pages with expert‑verified ground‑truth block‑to‑article mappings. This framework establishes a robust foundation for high‑fidelity digitization and semantic reconstruction of complex archival materials.
Authors:Yaoting Wang, Ziyi Zhang, Wenming Tu, Shaoxuan Xu, Wenjie Du, Cheng Liang, Weijun Wang, Yuanchao Li, Guangyao Li, Hao Fei, Yuanchun Li, Henghui Ding, Yunxin Liu
Abstract:
Recent advances in Omni‑Multimodal Large Language Models (Omni‑MLLMs) have enabled strong integration of vision, audio, and language. However, their audio‑visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI‑Bench, a cognitively inspired benchmark that evaluates Omni‑MLLMs across three stages, perception, understanding, and reasoning, through cross‑modal tasks requiring joint audio‑visual interpretation. This design enables fine‑grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI‑Bench‑PriSe, an extension that probes models' primitive audio‑visual sensation using unfamiliar, low‑semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open‑source and closed‑source models reveal substantial limitations in current Omni‑MLLMs. Based on these findings, we present a four‑level AVI taxonomy. Overall, AVI‑Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI. Project website: https://fudancvl.github.io/AVI‑Bench/
Authors:Lecheng Yan, Yichong Zhang, Ben Pan, Xiaoyu Zheng, Jiawei Qian, Anqi Wu, Wenxi Li, Chenyang Lyu
Abstract:
Editing a long‑form video from heterogeneous footage requires more than selecting clips: an agent must preserve narrative intent across material preparation, timeline construction, post‑production, and revision while leaving enough evidence to diagnose failures. We present Crayotter, an open‑source multimodal multi‑agent system for prompt‑driven video editing. Crayotter organizes production into three phases: coverage‑aware material preparation, artifact‑based editing research, and tool‑grounded timeline execution. Each phase externalizes inspectable artifacts, including coverage reports, multimodal analyses, editing blueprints, tool calls, and intermediate renders. These artifacts make an editing run traceable and allow failed segments to be diagnosed and selectively revised instead of requiring a full restart. We evaluate Crayotter on 23 editing themes against CapCut‑Mate and CutClaw. Under human evaluation, Crayotter achieves an average score of 3.40/5, compared with 2.44 and 1.70 for the two baselines, with consistent gains in theme alignment, narrative coherence, and editing smoothness. We additionally describe a replayable trajectory schema and verifiable reward design that prepare these workflows for future policy optimization. Code, traces, and examples are publicly available at https://github.com/idwts/Crayotter.
Authors:Bo Zhang, Borui Zhang, Chenghao Jiang, Minglei Shi, Xiaofeng Wang, Zheng Zhu, Jie Zhou, Jiwen Lu
Abstract:
Personal AI agents must increasingly operate across APIs, shells, web surfaces, and desktop GUIs, yet many systems remain tuned to a single interface and offer limited support for user teaching and auditability. We present Syll, an open‑source, self‑hosted multimodal agent harness that unifies MCP/API tools, CLI execution, and visual GUI control in a modular runtime, enabling agents to coordinate computer use across heterogeneous interfaces while streamlining how users and agents exchange information. At the core of Syll is a bidirectional user‑agent interaction layer: users teach procedures through direct demonstration, which Syll compiles into reusable skills; agent execution is translated back into multimodal evidence ‑‑ logs, keyframes, and approval checkpoints ‑‑ for inspection and control. Syll further externalizes memory, skills, routines, and governance as editable local artifacts, supporting straightforward inspection, extension, and downstream development. Our implementation has been validated on production desktop applications including Adobe Photoshop, Adobe Audition, Stardew Valley, macOS Finder and others. We report mechanism‑oriented studies that validate multimodal routing, teachable GUI replay, and persistent local artifacts. We hope Syll can serve as a practical open‑source foundation for personal automation that users can teach, inspect, and continuously extend.
Authors:Guangzhi Sun, Yixuan Li, Yudong Yang, Chao Zhang
Abstract:
Audio‑visual large language models (LLMs) hold strong promise for long‑form video understanding, yet their long‑video inference is fundamentally limited by the linear growth of video tokens and key‑value (KV) caches. We present OmniMem, a memory‑efficient streaming framework designed specifically for audio‑visual LLMs. Unlike existing compression methods that treat all tokens uniformly, OmniMem introduces a modality‑aware memory allocation strategy that separately manages visual and audio contexts, addressing the severe token imbalance between the two modalities. OmniMem further preserves informative and non‑redundant KV states through perturbation‑aware memory selection, enabling compact memory without sacrificing long‑range understanding. To strengthen compression under realistic deployment constraints, we also explore budget‑aware fine‑tuning, which encourages the model to consolidate useful information into retained memory. Experiments on VideoMME Long, LVBench, and LVOmniBench with video‑SALMONN 2+ and Qwen‑2.5‑Omni show that OmniMem consistently improves over strong training‑free compression baselines by 2‑4% absolute accuracy under the same memory budgets, with an additional 1‑2% gain after fine‑tuning.
Authors:Neel Tushar Shah, Manglam Kartik
Abstract:
We present CARTOGRAPH, a verification layer for AI scientists that couples unresolved‑subspace experiment steering (select), explicit ambiguity closure (resolve), and residual‑based library inadequacy detection (refuse). Under a local linear‑Gaussian bridge, raw unresolved projection is the isotropic unresolved Fisher‑information trace, while CARTOGRAPH‑A is the exact unresolved A‑optimal rule; closed‑form EIG and Box‑Hill arise as local comparators rather than global equivalents. Across five testbeds, CARTOGRAPH‑A beats raw projection 129W/0T/15L at d = 8 (p < 10^‑21) in a replicated structured cascade. More distinctively, the framework tentatively identifies three out‑of‑library pharmacokinetic mechanisms and then revokes those identifications as residuals expose structural misfit, while one perturbed in‑library control stays identified throughout. In low‑dimensional pharmacokinetic and filtered EPA settings, near‑ties against disagreement are predicted by theory and observed. Finally, in a retrospective audit of 40 positive claims from the published A‑Lab autonomous materials system, the refuse guard flags all 4 claims later marked inconclusive under manual reanalysis while passing 32/36 confirmed claims. Code is available at https://github.com/ai4science‑boed/cartograph.git
Authors:Fang Wu, Shuting Jin, Xiangru Tang, Mark Gerstein, Xiangxiang Zeng, Yejin Choi, Jure Leskovec, Jinbo Xu
Abstract:
Protein function is largely determined by molecular surface geometry and physicochemical complementarity, yet most protein design methods condition only on backbone structure. We introduce SurfDesign, a surface‑conditioned protein design framework that models molecular surfaces as continuous geometric manifolds and integrates them with pretrained protein language models. SurfDesign employs surface‑based equivariant message passing to capture surface normals, curvature, and directional geometry, together with a parameter‑efficient fine‑tuning strategy. Focusing on functional protein design, we show that SurfDesign consistently outperforms prior surface‑conditioned and backbone‑only methods on de novo binder and enzyme design benchmarks. We also report strong performance on inverse‑folding benchmarks as a diagnostic of structural compatibility. Our results highlight manifold‑aware surface representations as a principled foundation for functional protein and enzyme design. Code is available at https://github.com/smiles724/SurfDesign.
Authors:Roghayeh Taghavi, Aysa Hasanazde Bashkandi, Amir Ali Bengari, Mohammad Amin Raji, Mohammad Salahi Ardekani, Parisa Mardukhian, Parvaneh Rezaei, Ramin Mousa
Abstract:
The emergence of machine learning and deep learning has revolutionized the efficiency of diagnostic, therapeutic, and administrative systems in healthcare. However, this rapid adoption has come at the cost of requiring significant computing power and energy consumption, as well as e‑waste disposal and carbon emissions. One of the challenges of these models is choosing the right model for classification tasks. To this end, researchers attempt to identify the optimal model using their data through trial and error, which involves energy consumption and waste. The goal of this study is to develop a model‑based recommender system for medical image classification. For this purpose, a data set was collected from 3,000 articles in the field of medical image classification. This dataset, publicly available under the name MedicalRec‑Bench, contains over 5,000 records of models tested in various tasks, including Skin Cancer Classification, Tumour Classification, Wound Classification, Breast Cancer, and MRI classification. The dataset was evaluated in four different modes, depending on the number of features: MedicalRec I (5 features), MedicalRec II (9 features), MedicalRec III (11 features), and MedicalRec IV (18 features). Collecting all values for the features is challenging due to non‑reporting by the authors; hence, the dataset contains significant amounts of missing values. The Medical Recommender System (MedicalRec) is a transformer‑based model used for item recommendations in this study. This model achieved remarkable results in the evaluation on the dataset and in the evaluation with 12 base models. This model achieved a maximum HitRate@100 of 75.5%. The dataset and implementations are available through the GitHub link: https://github.com/Ramin1Mousa/MedicalRec
Authors:Prabal Shrestha, Bohan Jiang, Haoning Xue, Huan Liu, Xinyi Zhou
Abstract:
Multimodal large language models (MLLMs) have shown strong performance on objective tasks such as video understanding and reasoning. However, it remains unclear whether they can approximate subjective human responses, which depend not only on content comprehension but also on individuals' social contexts. To address this gap, we evaluate MLLMs as synthetic participants in an emerging task: assessing perceived sensory engagement with short videos. Grounded in the Perceived Message Sensation Value (PMSV) framework, we compare ratings from recruited human participants and profile‑conditioned MLLM simulations (n=673) using a 17‑item scale measuring emotional arousal, dramatic impact, and novelty. We find that even leading MLLMs (Gemini 3 Flash and Qwen 3 Omni) show limited agreement with human participants. The models exhibit distinct downward mean‑shift and central‑tendency biases in their rating distributions. They both introduce and flatten subgroup differences, while showing inconsistent sensitivity to participant profiles. Prompting strategies affect these metrics differently, modestly improving some aspects while worsening others. These results highlight both the challenges and opportunities of developing MLLMs as synthetic participants in video‑based research. Data and code: https://github.com/MINDLab25/mllm‑human‑simulation‑eval
Authors:Zeyuan Wang, Dongyang Hou, Cheng Yang, Xuezhi Cui, Linrui Xu, Bo Yu, Gaozhi Zhou, Ziyu Li, Liangtian Liu, Kai Ouyang, Wang Guo, Lili Zhu, Chao Tao
Abstract:
Large language model (LLM)‑based agents provide a novel paradigm for the automated processing of remote sensing(RS) data. Their success in complex RS tasks rely on extensive specialized tool libraries. However, tool documentation often exceeds the context window limits of LLMs, making precise tool retrieval essential for agentic workflows. Existing tool retrieval methods face "semantic asymmetry" bottleneck: natural language queries typically express macro‑level intentions lacking tool‑specific semantics, while tool documentation provides fine‑grained technical descriptions lacking operational context for workflows. To bridge this semantic gap, this paper proposes a bidirectional semantic complementary tool retrieval method. First, on the query side, we introduce a planning‑based query enhancement mechanism that leverages the reasoning capabilities of agents to decompose abstract intentions into logical subtasks, thereby actively supplementing the query with missing functional semantics. Second, on the tool side, addressing the strong coupling characteristics of RS tool chains, we construct a dynamic tool dependency graph with continual learning capabilities. By employing a neighborhood information aggregation mechanism, contextual information from precursor tools is explicitly injected into the current node representation, enriching tool descriptions with contextual semantics. Experimental results on the RS dataset GeoPlan‑bench and the general‑purpose dataset API‑ Bank demonstrate that the proposed method not only significantly improves tool retrieval accuracy for complex RS tasks but also exhibits robust extensibility for transfer to general‑domain tasks. The source code and dataset are available at https://github.com/geox‑lab/BSCTR.
Authors:Ritvik Pandey, Sid Manchkanti, Mohammed Wazir Adain, Mohammed Hadi, Dushyanth Sekhar
Abstract:
We introduce PulseBench‑Tab, an open multilingual benchmark for evaluating table extraction from document images. The benchmark comprises 1,820 human‑annotated tables spanning 9 languages and 4 scripts (Latin, CJK, Arabic, Cyrillic), drawn from 380 real‑world source documents including financial filings, government reports, and regulatory disclosures. Tables range from 2 to 1,183 cells, with 48.1% containing merged or spanning cells. Alongside the dataset, we propose T‑LAG (Table Logical Adjacency Graph), a novel evaluation metric that models tables as directed graphs over cell adjacencies and computes structural and content fidelity in a single score via optimal bipartite matching. We evaluate 9 commercial and open‑source table extraction systems across the benchmark and report per‑language breakdowns. The full dataset, scoring code, and all provider outputs are publicly available.
Authors:Shengli Zhou, Xiangchen Wang, Guanhua Chen, Feng Zheng
Abstract:
Large language models (LLMs) have recently been applied to 3D vision‑language (3D‑VL) tasks, which require spatial reasoning to identify target objects relative to anchors. Scene graphs are commonly employed to represent such relations, but reasoning over complete graphs incurs high token costs and computational inefficiencies, motivating the need for pruning. Existing pruning methods primarily rely on spatial proximity and often remove task‑relevant relations, thereby undermining reliable spatial reasoning. To address these limitations, we derive a key requirement for scene graph pruning: preserving spatial relations that are most pertinent to the specific 3D‑VL task. Guided by this insight, we propose the Conceptual‑Adjacent Scene Graph Pruner (CAPruner). CAPruner integrates fuzzy semantic relevance with spatial proximity to estimate the importance of relations, enabling the selection of critical relations in a task‑specific context. Moreover, to avoid costly relation‑level annotations, CAPruner is trained by supervising the aggregated scores of each node's incident edges. Extensive experiments demonstrate that CAPruner effectively preserves relations essential for spatial reasoning, leading to substantial performance improvements of LLMs on 3D‑VL tasks. Code is available at https://github.com/fz‑zsl/CAPruner.
Authors:Lin Mu, Guoji Wang, Li Ni, Lei Sang, Zhize Wu, Peiquan Jin, Yiwen Zhang
Abstract:
Large Language Models (LLMs) have shown strong potential for recommendation (LLMRec) due to their powerful reasoning and generalization abilities. However, effectively aligning the textual semantics modeled by LLMs with the collaborative signals remains a key challenge. Existing methods either translate collaborative information into textual prompts or inject pre‑trained embeddings into the LLM, both of which treat structural information as static input and fail to capture high‑order relational dependencies. To bridge this gap, we propose GraphLoRA, a novel framework that generalizes low‑rank adaptation from independent to structure‑aware propagation. GraphLoRA embeds a trainable graph message‑passing network within the low‑rank adaptation pathway, enabling structural signals to propagate through the parameter space. This design allows collaborative topology to explicitly guide parameter updates, fostering deep integration between graph‑structured and textual semantic information. Extensive experiments on multiple benchmarks demonstrate that GraphLoRA not only outperforms state‑of‑the‑art LLM‑based recommendation methods but also achieves superior generalization, effectively balancing structural reasoning capability with computational efficiency. Code is available at \hrefhttps://github.com/wgj15965/GraphLoRAhttps://github.com/wgj15965/GraphLoRA.
Authors:Meixi Song, Dizhe Zhang, Hao Ren, Ruiyang Zhang, Bo Du, Ming-Hsuan Yang, Lu Qi
Abstract:
In this work, we focus on extending SHARP, the popular photorealistic view synthesis method, for universal monocular rendering across a continuum of camera systems, from conventional perspective cameras to wide‑field‑of‑view, fisheye and omnidirectional panoramic settings. To overcome the pinhole‑specific assumptions of SHARP, our key idea is to align various images in a unified omnidirectional latent space. Thus, we propose UniSHARP, which performs implicit alignment in both feature and Gaussian spaces. Specifically, Gaussian primitives are arranged along rays and radial distances in a ray‑based universal representation, while 2D semantic and 3D spatial features extracted from UniK3D‑inspired encoders are jointly decoded to generate the complete Gaussian cloud. To comprehensively evaluate our method, we construct a benchmark covering diverse imaging systems across various scenes. The benchmark is further stratified by field of view (FoV) to enable fine‑grained assessment of the universal monocular rendering task. Extensive experiments on the proposed benchmark demonstrate the effectiveness of UniSHARP, outperforming alternative methods by a large margin. The project page can be found at: https://insta360‑research‑team.github.io/Unisharp‑website/
Authors:Hanhui Wang, Yiming Xie, Haiwen Feng, Zhaoyang Lv, Shenlong Wang, Huaizu Jiang
Abstract:
We introduce StreamForce, a streaming video generation framework that enables physically grounded control through continuous force inputs. Unlike prior video models that train separate models for different force types, assume fixed forces, or rely on non‑causal processing, StreamForce is a causal and unified model that responds instantly and coherently to both local and global, time‑varying forces. To achieve this, we design a unified force representation as a control signal and develop a distillation pipeline for force‑controllable video generation. Our model combines autoregressive efficiency with force responsiveness, sustaining stable photometric and dynamic realism. StreamForce runs at up to 16.6 FPS on a single GPU, achieving state‑of‑the‑art performance in both force adherence and motion realism. Project website: https://neu‑vi.github.io/StreamForce/
Authors:Johannes Theodoridis, Johannes Maucher, Andreas Schilling
Abstract:
We propose Differences in Detection (DnD), an intuitive method to compare two object detection models. Based on the same matching algorithm, it complements the standard metrics of mean Average Precision (mAP) and TIDE error analysis with the ability to compare two models directly. More specifically, we calculate the intersection of ground truth labels that are recognized by both models, followed by the corresponding difference sets and the complement set of ground truth labels that are missed by both models. The resulting comparison is more direct and intuitive than a comparison of independent summary statistics. It reveals individual and shared mistakes and becomes particularly interesting when combined with error types. In this case, the differences in detection errors can be analyzed naturally in a standard confusion matrix. While valuable in itself, we believe that one of the best applications of DnD is to guide explainability methods such as ODAM towards metric‑relevant examples, grounded in structured subsets. The code for our method is available here: https://github.com/JohannesTheo/differences‑in‑detection
Authors:Songhao Wu, Zhongxin Chen, Yuxuan Liu, Heng Cui, Cong Li, Rui Yan
Abstract:
Large language models exhibit impressive zero‑shot capabilities across a wide range of downstream tasks. However, they struggle to function as off‑the‑shelf embedding models, leading to suboptimal performance on massive text embedding benchmarks. In this paper, we identify a potential cause underlying this deficiency. Our motivation stems from an unexpected observation: text embeddings tend to align with frequent but uninformative tokens when projected onto the vocabulary space. We argue that this excessive expression of high‑frequency tokens suppresses the model's ability to capture nuanced semantics. To address this, we introduce EmbedFilter, a simple linear transformation designed to refine text embeddings derived from LLMs directly. Specifically, we uncover that the unembedding matrix within LLMs encodes a latent space that is actively writing these frequent tokens into embedding space. By filtering out this subspace, EmbedFilter suppress the influence of high‑frequency tokens, thereby enhancing semantic representations. As a compelling byproduct, this enables an inherent dimensionality reduction, lowering index storage and speedup retrieval while fully preserving the refined embedding quality. Our experiments across multiple LLM backbones demonstrate that LLMs equipped with EmbedFilter achieve superior zero‑shot downstream performance even with significantly reduced embedding dimensions. We hope our findings provide deeper insights into the mechanisms of LLM‑based representations and inspire more principled designs to improve text embeddings training. Our code is available at https://github.com/CentreChen/EmbFilter.
Authors:Jiayu Wang, Weijiang Lv, Bowen Fu, Jing Fu, Jiayi Song, Lingyu Zhang, Lanxuan Xue, Luodi Chen, Zepeng Xin, Kaiyu Li, Xiangyong Cao
Abstract:
As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long‑horizon coding tasks and even autonomous experiment execution. Despite their evolution from research assistants into autonomous research agents, these systems still exhibit significant limitations in field sensitivity, research ethics, and nuanced scientific judgment. Consequently, frontier agents remain unable to fully replace human researchers. To bridge this gap, we conceptualize the AARR (Act As a Real Researcher) benchmark series. Unlike existing benchmarks that primarily assess macro‑level execution capabilities, AARR focuses on whether agents can emulate the professionalism, thoroughness, and nuanced reasoning that characterize human researchers in granular research scenarios. In this work, we propose AARRI‑Bench (Act As a Real Research Intern), the first benchmark in this series. We conduct extensive experiments across frontier models and agentic systems, revealing that even the best‑performing configuration (Mini‑SWE‑Agent with Claude Opus 4.7) achieves only 68.3% success rate, frequently overlooking subtle yet critical details that are obvious to real human researchers. Our results indicate that developing researcher‑like AI requires further exploration of research behavior, rather than merely complex scaffolding. Our data is released at https://github.com/AARR‑bench/AARRI‑bench.
Authors:Jiahao Meng, Yue Tan, Qi Xu, Kuan Gao, Weisong Liu, Yanwei Li, Jason Li, Lingdong Kong, Haochen Wang, Qianyu Zhou, Jiangning Zhang, Guangliang Cheng, Yunhai Tong, Lu Qi, Minghsuan Yang
Abstract:
Video understanding is being rapidly transformed by multimodal large language models (MLLMs), as research moves from short clips to long, multimodal, and knowledge‑intensive video scenarios. These scenarios require models to handle sparse evidence, long‑range dependencies, multimodal alignment, and reliable inference under limited computational budgets. This work presents a human‑view perspective on LLM‑based video understanding, organized around three functional abilities: watching, remembering, and reasoning. Rather than treating video tasks as isolated benchmarks, this view provides a unified structure for analyzing how video MLLMs acquire evidence, preserve context, and produce grounded outputs. We introduce a formulation that characterizes video understanding systems by their perceptual representations, memory states, reasoning traces, and final predictions. Based on this formulation, we identify challenges in spatio‑temporal perception, efficient long‑video processing, memory modeling, streaming understanding, and faithful reasoning. Representative methods are organized by their roles in video MLLM systems. Watching covers fine‑grained, comprehensive, audio‑visual, and efficient perception. Remembering includes offline and streaming memory, while reasoning covers text‑only reasoning and thinking with videos. We further examine application domains such as egocentric, sports, instructional, medical, and narrative videos, and cover training datasets and evaluation benchmarks across task types, supervision formats, modalities, and capability dimensions. Finally, we outline open problems and future directions for scalable, memory‑aware, and evidence‑grounded video intelligence. Related works will be continuously traced at https://github.com/marinero4972/Awesome‑HumanView‑VideoUnderstanding.
Authors:Simon Schug
Abstract:
Sparsity allows scaling model parameters without proportionally increasing computational cost. While mixture of experts (MoE) models are made increasingly sparse, individual experts typically remain large and dense. Here, we demonstrate that further increasing sparsity by shrinking each expert to consist of a single neuron and selecting a tiny fraction of many available neurons can improve compute efficiency and interpretability. Counterintuitively, the key to achieving both is removing the nonlinearity typically applied to the experts, resulting in a network of sparsely gated linear neurons (sgatlin). In an isoflop comparison, we find that replacing all transformer feedforward layers with sgatlin improves perplexity in language models across different compute budgets. At the same time, the sparsity and linearity of the resulting feedforward circuits present new opportunities for model interpretability. In a small‑scale case study, we demonstrate that feedforward circuits in sgatlin can be interpreted without having to train additional replacement models. We find that they form semantically structured clusters and are causally implicated in factual recall. Our findings paint a possible path towards compute‑efficient and interpretable transformer feedforward layers.
Authors:Rohan Shravan
Abstract:
This paper reports on training a hundred‑billion‑parameter sparse mixture of experts on a single eight‑GPU node, end to end. LightningLM 0.1V is a recurrence‑backbone language model family grown in four stages from a small dense seed, through a 5B and a 9B mixture of experts, to a 120B model with 460 routed experts under top‑12 routing. Each larger model is grown from the trained weights of the smaller one; active parameters rise monotonically from 1.78B at the dense seed to 5.93B at 120B (about 5% of the 118.67B stored). The full lineage runs on single nodes, the larger stages at 8K context, reaching a released training loss of 1.78 at 120B scale. This is a systems and experience report. It is organized around three disciplines. Reversibility: a reversible recurrence stack reconstructs activations in the backward pass instead of storing them, holding activation memory flat as the model grows. State‑preserving growth: each expansion (dense to MoE, shallow to deep, few experts to many) is given as a reproducible principle paired with the failure that results from getting it wrong; several failures are silent. Single‑node economics: the 120B trains through TQP, a strategy of quantized base expert weights and trained low‑rank adapters that carries optimizer state on 2.26B adapter parameters rather than 100B+ resident in routed experts, cutting expert‑path optimizer state by a factor of ~45. What is new is the integration of known primitives, not any primitive in isolation: one grown lineage running end to end on a single node, documented at practitioner level, with per‑domain held‑out loss as evidence that targeted capabilities (multilingual Indic competence, code) were learned by construction. Model family, tokenizer, and training code are released.
Authors:Yifan Duan, Qixiang Xu, Hengtao Wu, Zhanxun Liu, Wenhao Guan, Junxi Liu, Ziyang Ma, Kelu Xu, Xie Chen
Abstract:
In recent years, audio generation has made significant progress in tasks such as text‑to‑speech (TTS), text‑to‑audio (TTA) and text‑to‑music (TTM). However, generating long‑form and controllable audio from complex audio scene descriptions remains a significant challenge, as such scenes often require coordinated speech, sound effects, music, songs, temporal structure, and post‑production. In this work, we introduce Audio‑Oscar, a multi‑agent framework for generating audio from complex descriptions. Audio‑Oscar coordinates a set of specialist agents, each responsible for a different aspect of the audio scene, including character modeling and voice design, speech generation, fine‑grained timeline planning, model selection, non‑speech generation, and audio post‑production. Audio‑Oscar further incorporates feedback‑driven refinement. In addition, to address the lack of suitable benchmarks for evaluating audio generation from complex audio scene descriptions, we construct ASG‑Bench, an Audio Scene Generation Benchmark containing both scene descriptions paired with reference audio and text‑only scene descriptions. Each scene is annotated with target audio events and temporal statements to evaluate whether the generated audio faithfully realizes the required scene content and temporal structure. Experimental results show that Audio‑Oscar can effectively generate audio that matches complex scene descriptions. Project samples are available at https://audiooscar.github.io/. Our code is available at https://github.com/ziye26/Audio‑Oscar.
Authors:Kaijie Shi, Wanglong Lu, Huiling Chen, Vinicius Prado da Fonseca, Ting Zou, Hanli Zhao, Xianta Jiang
Abstract:
Biosignals‑free shared‑autonomy control of upper‑limb prosthetic hands aims to enable natural and low‑effort manipulation without relying on EMG or other physiological signals. Recent imitation‑learning‑based approaches have shown promising results, but their scalability is limited by the cost and variability of collecting large amounts of real‑world human demonstration data. In this work, we present a scalable simulation framework that automatically generates diverse reach‑to‑grasp demonstrations from a wrist‑mounted virtual camera. The framework combines physically feasible grasp synthesis, natural reaching trajectories retargeting, and reach‑‑grasp‑‑lift execution in procedurally generated indoor environments. It records wrist‑view observations, proprioception, and actions to build a large‑scale demonstration dataset for imitation learning. Through extensive simulation benchmarks, we evaluate object and scene generalization and compare several representative state‑of‑the‑art imitation learning methods. Results show that the simulated demonstrations are sufficiently rich and consistent for effective policy learning. In three realistic settings, the learned sim‑to‑real policy achieves over 90% grasp success, surpasses baseline methods, and exhibits stronger generalization, highlighting the promise of simulation‑driven training for biosignals‑free shared‑autonomy prosthetic grasping. The demonstrations are available at \hrefhttps://sites.google.com/view/sim‑prosthetic‑grasp/homehttps://sites.google.com/view/sim‑prosthetic‑grasp/home.
Authors:Huixi Intelligence, :, Chen Zhang, Chenyang Zhou, Guanglei Ding, Guanghui He, Haibin Gao, Jiajia Chen, Jianyong Zhang, Lianyi Yu, Ningyi Xu, Ping Xu, Qingchen Li, Yingjun Hu, Yijia Zhang, Yuxi Liu
Abstract:
Vision‑Language‑Action (VLA) models have shown strong potential for robotic manipulation, but real‑time deployment on edge hardware remains challenging. In this work, we identify VLM visual and context tokens as a major source of deployment latency: for GEMM‑dominated projection operators, computation grows linearly with the number of input tokens when model dimensions are fixed. Motivated by this observation, we propose RhinoVLA, a deployment‑oriented VLA model co‑designed with the Huixi R1 edge SoC. RhinoVLA adopts a token‑efficient Qwen3‑VL backbone and a continuous Action Expert, reducing the VLM‑side token and computation burden while preserving pretrained multimodal capability. To support cross‑robot learning, RhinoVLA further introduces a unified interface that combines View Registry, 72D physical state‑action slot space, and robotinstance LoRA, allowing heterogeneous robot observations and action schemas to be aligned under a shared policy. On the deployment side, RhinoVLA is optimized through hardware‑aware compilation, mixed‑precision execution, and parallel visual encoding. Experiments show that RhinoVLA achieves downstream performance comparable to π0.5 at a similar parameter scale, while reaching 11.69 Hz end‑to‑end inference on Huixi R1, meeting the 10 Hz real‑time closedloop control target. The project will be open‑sourced at https://github.com/HuixiAI/RhinoVLA.
Authors:Huzaifa Shaaban Kabakibo, Animesh Trivedi, Lin Wang
Abstract:
As scalable inference services become popular, the cold start latency of an inference engine becomes important. Today, vLLM has evolved into the de facto inference engine of choice for many inference workloads. Although popular, due to its complexity and rapid evolution, there has not been a systematic study of its startup latency. With major architectural innovations such as the V1 API and the introduction of torch.compile, this paper presents the first detailed performance characterization of vLLM startup latency. We break down the startup process into six foundational steps and demonstrate that it is predominantly CPU bound. Each step exhibits consistent and interpretable scaling trends with respect to model‑level and system‑level parameters, enabling fine‑grained attribution of latency sources. Building on these insights, we develop a lightweight analytical model that accurately predicts vLLM startup latency for a given hardware configuration, providing actionable guidance for resource planning in large‑scale inference environments. All benchmarking datasets, analysis tools, and prediction scripts are open sourced at https://github.com/upb‑cn/vllm‑startup‑profiler.
Authors:Ivan Sviridov, Artem Oskin, Ivan Panin, Iaroslav Bespalov, Dmitry Dylov, Ivan Oseledets, Aleksandr Nesterov
Abstract:
Adapting large language models (LLMs) to clinical workflows often requires costly fine‑tuning or manual prompt and pipeline engineering. We study LLM‑guided MAP‑Elites evolution as an inference‑time alternative for discovering medical decision strategies and provide an implementation repository at https://github.com/univanxx/llm_guided_evo_medical. We formulate urgency triage, interactive consultation, and medical image classification as evolutionary searches over executable artifacts optimized by task‑specific fitness functions. Across all three settings, evolution improves over manually designed baselines under practical constraints. In triage, evolved programs increase Semigran accuracy from 77.3% to 87.1% and emergency recall from 0.60 to 0.97, while improving safety‑weighted held‑out MIMIC‑ESI performance. In interactive consultation, evolved policies improve the accuracy‑‑cost frontier across Llama‑3, Qwen‑3.5, and Gemma‑4 and transfer to held‑out iCRAFTMD. In PneumoniaMNIST, prompt‑only evolution improves frozen MedGemma VLMs while preserving strict JSON outputs. Qualitative analysis shows that the gains come from interpretable program‑level mechanisms, calibrated triage boundaries, targeted evidence acquisition, selective commitment, and finding‑oriented visual decision rules, rather than superficial prompt rewording alone.
Authors:Hangtao Zhang, Yucheng Zhao, Sishun Liu, Ziqi Zhou, Zeyu Ye, Wei Wan, Minghui Li, Shengshan Hu, Yanjun Zhang, Yi Liu, Leo Yu Zhang
Abstract:
Jailbreak prompts can bypass alignment guardrails in large language models (LLMs) and elicit unsafe outputs, making reliable deployment‑time detection critical. Prior detection approaches largely rely on a fixed metric space, e.g., raw inputs, gradients, or hidden features, in which benign and jailbreak prompts are linearly separable. We show this assumption breaks under (i) pseudo‑malicious prompts that are benign by intent but contain safety‑related keywords, and (ii) adaptive attacks that explicitly optimize against the deployed detector. To overcome this limitation, we shift our focus from identifying a universal metric space to analyzing the more robust neighborhood structure of the underlying data manifold. We present Manifold Trajectory Kinetics (MTK), which treats an LLM as a kinetic system transforming inputs into outputs and detects jailbreaks by tracking how a prompt's neighborhood structure evolves across layers. Benign prompts remain close to benign neighborhoods throughout inference, whereas jailbreak prompts exhibit a characteristic trajectory that begins near malicious seeds and later strategically shifts toward benign neighborhoods to evade refusal.Across four LLMs and ten jailbreak attacks, MTK achieves strong robustness to both failure modes: on pseudo‑malicious prompts, it attains a jailbreak true positive rate of 95% at a false positive rate of 5% on benign prompts and 2% on pseudo‑malicious prompts, and under adaptive attacks, it maintains a true positive rate of 85%. We further demonstrate the superior performance of MTK for jailbreak detection in vision‑language models. Our code is available at https://github.com/Rookie143/mtk.
Authors:Aravind Sundaresan
Abstract:
Serial LLM inference backends ‑‑ such as Ollama ‑‑ process requests one at a time under FCFS admission, causing Head‑of‑Line Blocking (HOLB) under mixed workloads at high utilisation: short factual queries can be delayed by minutes behind long generation jobs. While cloud‑scale deployments mitigate HOLB via continuous batching (vLLM, Orca), these solutions require tens of GB of VRAM for concurrent KV‑caches ‑‑ infeasible for memory‑constrained edge and local deployments that rely on serial request dispatch. We present \clairvoyant, a drop‑in sidecar proxy for any serial OpenAI‑compatible backend (e.g., Ollama, llama.cpp). \clairvoyant predicts response length from 19 lightweight lexical features via an ONNX‑exported XGBoost classifier, achieving 0.029\,ms per‑request latency (four orders of magnitude below typical generation time). Because admission scheduling depends on relative ordering rather than exact prediction, the system optimises ranking fidelity, achieving 62‑‑96% in‑distribution and 52‑‑66% cross‑distribution accuracy across natural conversation datasets. We find that curated instruction datasets are degenerate training sources for length prediction: GPT‑imposed brevity constraints reduce Long‑class representation to under 0.02% of examples, making natural conversation logs the only viable training source. End‑to‑end GPU benchmarks on an RTX~4090 show 70‑‑76% P50 latency reduction for short requests under maximum queue pressure (100 concurrent requests) and 17% under steady‑state Poisson arrivals (ρ=0.74). \clairvoyant is open‑source and requires no modifications to the inference backend.
Authors:Ziyang Ma, Ruiqi Yan, Ruiyang Xu, Jie Fang, Zhikang Niu, Yi-Wen Chao, Wenming Tu, Tianrui Wang, Auden, Qi Chen, Wenxi Chen, Jiaying Chi, Yanru Huo, Zixuan Jiang, Xiquan Li, Yalin Li, Junxi Liu, Minghao Liu, Binghao Qiang, Yijia Shan, Zheshu Song, Tian Tan, Zixiang Wang, Zeyu Xie, Zhifei Xie, Xiaoyu Xing, Qixiang Xu, Chen Yang, Guanrou Yang, Shan Yang, Yifan Yang, Steve Yves, Haotian Zhang, Haina Zhu, Kai Yu, Liefeng Bo, Eng-Siong Chng, Xie Chen
Abstract:
We introduce MMAE, a Massive Multitask Audio Editing benchmark, serving as the first comprehensive evaluation testbed designed for general‑purpose instruction‑based audio editing. Spurred by the shift toward intelligent creation, interactive editing has rapidly expanded from visual domains, pioneered by models like Nano‑banana 2 for images and Gemini‑Omni for video, into audio. However, the current evaluation infrastructure lags severely, remaining highly fragmented and restricted to specific subdomains or basic operations. Unlike existing benchmarks that are limited in scope, MMAE extends to a broad spectrum of real‑world scenarios, encompassing 7 distinct audio modalities, including sound, speech, music, and their mixtures. Furthermore, we establish a comprehensive taxonomy spanning 6 levels of task complexity, from basic modifications to multi‑hop reasoning and multi‑round editing, 2 levels of granularity, and 8 distinct operation types. Meticulously curated through human‑agent collaboration, MMAE comprises 2,000 high‑fidelity samples paired with a pioneering rubric‑based evaluation framework. By decomposing free‑form tasks into 17,741 verifiable criteria, this robust rubric‑based paradigm enables a precise, multi‑dimensional assessment of both instruction following and context consistency. Our extensive evaluation of leading models reveals that current systems remain far from achieving reliable edits. Strikingly, the Exact Match Rate (EMR) consistently falls below 5% and plummets to an absolute 0% in complex, mixed‑modality tasks, exposing critical bottlenecks in precise execution and structural robustness. We hope MMAE will serve as a catalyst for future advances in the intelligent creation community, providing a clear diagnostic roadmap and establishing a standardized, long‑lasting evaluation paradigm for next‑generation audio editing systems.
Authors:Yuhang Zhou, Yixin Cao, Guangnan Ye
Abstract:
Reasoning prefixes shape the future trajectory of LLM problem solving, yet existing process reward models usually evaluate them through local step correctness. We argue that correctness is a useful but indirect proxy for the effect we ultimately care about: whether a prefix increases the probability of successful completion. We define this effect as prefix gain, the solve‑rate improvement induced by conditioning lightweight student model group on a prefix, and use it to train a Prefix Utility Model (PUM) with a simple pairwise ranking objective. PUM learns outcome‑grounded prefix utility and can score both complete trajectories and partial reasoning prefixes. Across Best‑of‑N selection, beam search, and reinforcement learning on mathematical reasoning, PUM provides a strong prefix‑level supervision signal, especially when candidate pools are large, search budgets increase, or rule‑based rewards are sparse. We release all data, models, and code at https://zhiqix.github.io/pum‑project‑page.
Authors:Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi
Abstract:
Streaming 3D Gaussian Splatting requires highly scalable, progressive representations. Existing progressive methods rely on discrete layering, accumulating separate splat sets for each level of detail. This structural independence between layers inherently leads to error accumulation, severe splat redundancy, and uncontrolled quality transitions. We propose EvoGS, the first continuous‑layering representation. Organized as an Evolution Tree, EvoGS generates finer details via an explicit, wavelet‑inspired parent‑child refinement. This empowers child nodes to structurally correct ancestral errors, yield inherently sparse and highly compressible inter‑layer signals. Extensive experiments show EvoGS eliminates splat redundancy from over 65% to under 25%. Compared to state‑of‑the‑art baselines, it reduces transmission payload and GPU VRAM footprint by up to 2.4× and 5.5×, respectively, and achieves smooth quality transitions optimal for real‑time adaptive streaming. Project page: https://yuang‑ian.github.io/evogs/
Authors:Yihong Xu, Eloi Zablocki, Yuan Yin, Elias Ramzi, Ellington Kirby, Alexandre Boulch, Matthieu Cord
Abstract:
End‑to‑end planners for autonomous driving typically generate a set of candidate trajectories, score each one, and return the highest‑scoring candidate. However, the scorer is applied only after the proposals are generated and cannot influence the set of trajectories: a weak set of candidates limits planning performance regardless of the scorer's quality. We instead treat the scorer as a learned trajectory‑level reward function and search for trajectories that maximize it. Our method, TOAD, runs the Cross‑Entropy Method at test time, warm‑started from the planner's proposals. It requires no retraining and is plug‑and‑play for existing planners. Across six base planners, TOAD improves results on NAVSIM‑v1 (94.7 PDMS), NAVSIM‑v2 (56.3 EPDMS), and the closed‑loop HUGSIM benchmark. The code will be made publicly available via the project page: https://valeoai.github.io/TOAD/.
Authors:Duc Tri Tran, Trung Thanh Nguyen, Vijay John, Phi Le Nguyen, Yasutomo Kawanishi
Abstract:
Video Text Spotting (VTS) is essential for urban surveillance and intelligent transportation systems, enabling automated reading of street signs, vehicle markings, and scene text in video streams. However, reliable recognition remains challenging due to dynamic video factors common in surveillance scenarios, including motion blur, occlusion, and scale variation, which degrade frame‑level recognition. Existing VTS methods typically perform recognition independently on each frame, leading to inconsistent and inaccurate results across sequences. To address these limitations, we propose TraRA (Trajectory‑level Recognition Aggregation for VTS), a plug‑and‑play method that performs trajectory‑level text recognition by leveraging temporal and multimodal consistency. TraRA integrates two key modules: (1) the Temporal Clustering and (2) the Vision‑Language Aggregation. The former refines noisy trajectories by grouping temporally and visually coherent text instances, while the latter employs a Low‑Rank Adaptation‑enhanced Vision‑Language model to fuse visual cues with linguistic context across frames. By aggregating information over entire text trajectories, TraRA achieves robust text recognition even under challenging surveillance conditions. Extensive experiments on four public benchmarks, including road and urban scene datasets (RoadText, BOVText, ArTVideo, and ICDAR15), demonstrate that TraRA consistently improves tracking and recognition performance over state‑of‑the‑art VTS methods. The source code is available at https://github.com/trid2912/TraRA.
Authors:Daniel Cieślak, Andrzej Czyżewski
Abstract:
Physics‑informed machine learning is often assessed by curve error, although engineering use depends on downstream decisions: ranking candidates, avoiding infeasible designs and limiting regret. We introduce pinn‑gym, an open benchmark for material‑conditioned lattice design that couples a transparent reduced‑order crush‑and‑impact oracle with five printable polymer cards, dimensionless force‑response targets and a protocol spanning curve fidelity, physical admissibility, top‑k retrieval and mass regret. Across per‑material, pooled and cross‑material settings, low nRMSE is frequently insufficient to identify useful design selections. Physics‑informed losses alter trade‑offs rather than monotonically improving all metrics, and dimensionless conditioning improves comparability without making transfer symmetric. The benchmark is not a certified material model; within the released oracle, candidate generator and material cards, pinn‑gym provides a reproducible testbed for evaluating PIML surrogates as decision systems rather than curve predictors alone.
Authors:Herilalaina Rakotoarison, Steven Adriaensen, Tom Viering, Carl Hvarfner, Samuel Müller, Frank Hutter, Eytan Bakshy
Abstract:
Information‑theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration‑exploitation framework for Bayesian optimization (BO). However, their practical implementation relies on complicated and slow approximations, i.e., a Monte Carlo estimation of the information gain. This complexity can introduce numerical errors and requires specialized, hand‑crafted implementations. We propose a two‑stage amortization strategy that learns to approximate entropy search‑based acquisition functions using Prior‑data Fitted Networks (PFNs) in a single forward pass. A first PFN is trained to be conditioned on information about the optima; second, the α‑PFN is trained to predict the expected information gain by training on information gains measured with the first PFN. The α‑PFN offers a flexible learned approximation, which replaces the complex heuristic approximations with a single forward pass per candidate, enabling rapid and extensible acquisition evaluation. Empirically, our approach is competitive with state‑of‑the‑art entropy search implementations on synthetic and real‑world benchmarks, while accelerating the different entropy search variants across all our experiments, with speed ups over 50x. Source code: https://github.com/automl/AlphaPFN.
Authors:Okan Umur, Ali Emre Güşlü, Ibrahim Delibasoglu
Abstract:
The rapid advancement of video editing and generative artificial intelligence technologies has made realistic video manipulation increasingly accessible. Although existing datasets have significantly advanced research in deepfake detection, object removal, and video inpainting, they do not adequately model scenarios in which a short manipulated segment is inserted into an otherwise authentic video and the original video continues afterward. In this study, we review representative datasets from the literature, analyze their characteristics, and discuss their limitations with respect to temporally localized realistic manipulation detection. Based on this analysis, we motivate the need for a new dataset specifically designed for authentic videos containing short and highly realistic manipulated intervals. Finally, we evaluate two complementary approaches on our custom‑curated test set to establish an initial benchmark for this challenging scenario. The first employs a linear probe on DINOv3 features, assessed under three thresholding strategies. The second leverages DINOv3 features with a consecutive frame similarity‑based method to detect temporal manipulation boundaries. Together, these experiments provide an initial benchmark for partially manipulated video detection and highlight the need for content‑adaptive thresholding mechanisms. The dataset, code, and supplementary materials are publicly available at https://github.com/OkanUmur/temporally‑localized‑video‑manipulation‑detection.
Authors:Yinzhou Tang, Jingbo Xu, Yu Shang, Zihao Song, Chen Gao, Wei Wu, Yong Li
Abstract:
World Action Models (WAMs) offer a promising approach to embodied intelligence, yet existing methods rely heavily on video prediction as action priors and lack adaptive multimodal reasoning, limiting their effectiveness on long‑horizon, complex tasks. We observe that WAMs require different multimodal reasoning modes under different execution contexts: textual reasoning is essential during task transitions to guide high‑level action prediction, while visual reasoning is critical during fine‑grained manipulation for precise control. Motivated by this observation, we propose AdaWAM, a world action model with adaptive multimodal reasoning abilities. AdaWAM integrates a lightweight dynamic router that autonomously triggers textual or visual reasoning as needed during task execution. Experiments on both simulated and real‑world embodied tasks show that AdaWAM substantially improves inference efficiency while outperforming state‑of‑the‑art embodied policies. Codes and demos are available at: https://adawam.github.io/.
Authors:Zheng Gai, Li Xincheng, Jiang Wangyingjie, Zhao Panwei
Abstract:
The kappa statistic is the most widely used measure of inter‑rater agreement for categorical data. Despite its popularity, applied researchers often encounter two major hurdles: (i) determining the sample size required to achieve a desired level of agreement with given power, and (ii) computing appropriate kappa coefficients with proper interpretation. Existing R packages such as irr and kappaSize provide these functionalities but require programming skills and lack an integrated, user‑friendly interface. We present CATEKAPPA, an R package that bridges this gap by combining sample size planning (via kappaSize) and agreement analysis (via irr) into a single Shiny‑based web application. The package supports Cohen's kappa for two raters, Fleiss' kappa for three or more raters, and Light's kappa, and provides automatic interpretation using the Landis & Koch scale. Users can either launch an interactive graphical interface or use command‑line functions for scripting. The package is freely available on CRAN.
Authors:Jilles S. van Hulst, Jakub M. Tomczak, W. P. M. H. Heemels, Duarte J. Antunes
Abstract:
Variational autoencoders (VAEs) learn low‑dimensional latent representations of high‑dimensional data. When the data lies on a manifold with non‑Euclidean topology, the standard Gaussian prior introduces a topological mismatch that degrades reconstruction quality and prevents faithful representation. We present a constructive mathematical framework that resolves this mismatch for all manifolds that admit a product covering space. These are manifolds expressible as products of elementary factors (circles, intervals, or lines) or as quotients of such products by a finite symmetry group. The class includes cylinders, tori, Möbius strips, Klein bottles, and real projective spaces. Factorized distributions over the elementary factors yield product topologies with closed‑form, decoupled KL divergences, so that each latent factor can be shaped independently while keeping training tractable. We catalogue reparametrizable encoder‑prior pairs for periodic, bounded, and unbounded supports, and provide coordinate transformations that allow standard neural networks to output non‑Euclidean parameters with smooth gradients. For quotient manifolds, the decoder receives group‑invariant features of the covering‑space coordinates, so that identified points produce identical outputs. Anchor constraints fix the coordinate system relative to the data or create soft topological holes. Experiments on synthetic manifolds and real‑image datasets (rotated and cyclically shifted MNIST) confirm that a topology‑matched prior aligns KL regularization with the data manifold. The resulting topology‑aware models outperform the Gaussian baseline at all practically relevant regularization strengths. The code is available at https://github.com/JvHulst/VAE‑Topology.
Authors:Xing Yue, Linjuan Wu, Daoxin Zhang, Yongliang Shen, Weiming Lu
Abstract:
Open‑ended reward modeling requires judges that can follow subtle, domain‑specific preferences when verifiable answers are unavailable. Existing rubric‑based methods often address this by generating criteria online for each query, but the extra generation step can add inference overhead and produce rigid or misaligned guidance. We introduce Eval‑Skill, an exploration‑guided method that synthesizes reusable evaluation skills for reward modeling and reframes reward guidance as context evolution rather than parameter training or per‑query rubric generation. Using only 100 cases per domain for skill evolution, Eval‑Skill synthesizes reusable domain‑level evaluation skills through two progressive stages, workflow generation followed by principle generation, with exploration and selection interleaved across both stages. Once generated, a skill is directly injected into the judge context. Across multiple RM benchmarks, Eval‑Skill consistently improves diverse judge backbones; on RewardBench 2, it yields significant gains over vanilla judging for each main backbone (+13.44% for Qwen3‑8B, and 18.51% for DeepSeek‑V4‑Flash). Further analyses of evolution‑time scaling, generalizability, and transferability show that compact evaluation skills offer an efficient new paradigm for LLM‑based evaluation. Code is available at https://github.com/xing‑stellus‑yue/Eval‑Skill.
Authors:Zhenyu Yang, Zemin Du, Shengsheng Qian, Changsheng Xu
Abstract:
Zero‑Shot Composed Image Retrieval (ZS‑CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption without training samples. Existing ZS‑CIR datasets often suffer from complete irrelevance between reference and target images due to noisy image sources, and do not achieve a true zero‑shot scenario as they use public image datasets that models like CLIP have been trained on. To tackle these challenges, we introduce ZeroSight, a novel benchmark for ZS‑CIR. It includes a dataset with consistent reference‑target pairs sourced from videos, a data construction pipeline, and evaluation methods that consider the ranking of multiple positive and negative target images. We ensure visually and semantically consistent reference‑target pairs by extracting frames from a single video and generating relative captions using LLM‑assisted methods. To ensure a true zero‑shot scenario, we use video data published after March 31, 2022, ensuring it was not included in CLIP's pre‑training data. Additionally, we propose a training‑free MLLM‑driven method, SC4CIR (Symmetric Consistency for CIR), which can effectively identify hard negative targets through 3 symmetric consistency checks. This method is plug‑and‑play, seamlessly integrating with various CIR methods and significantly improving performance. Our experimental results from 27 methods reveal that current ZS‑CIR datasets and evaluation metrics result in inflated retrieval performance, exaggerating the capabilities of CIR methods. Our benchmark and models can be accessed at https://github.com/sotayang/ZeroSight.
Authors:Minseong Kim, Jinyeong Park, Sungho Park, Jibum Kim
Abstract:
Multi‑modal approaches used for 3D CAD generation require substantial computational resources, necessitating efficient training. To address this, we propose GuideCAD, which leverages semantically rich visual‑textual representations having only a small number of trainable parameters to generate 3D CAD models. Specifically, GuideCAD uses a mapping network that converts image embeddings into prefix embeddings, enabling a pretrained large language model to integrate visual and textual information. As a result, a transformer‑based decoder predicts the construction sequence using the visual‑textual embeddings in order to generate the 3D CAD model. For experimental evaluation, we construct a new dataset, referred to as GuideCAD, which consists of text‑image pairs. Each pair includes a text prompt that represents a 3D CAD construction sequence and its corresponding 3D CAD image. Our experimental results show that GuideCAD generates comparably high‑quality 3D CAD models while using approximately four times fewer parameters and achieving twice the training efficiency compared to fine‑tuning approaches. We have released the source code and dataset for our method at: https://github.com/mskimS2/GuideCAD
Authors:Yongliang Miao, Fengyuan Liu, Wei Shi, Yanguang Liu, Fei Sun, Na Zou, Mengnan Du
Abstract:
Supervised fine‑tuning (SFT) is a prevailing method for adapting large language models to reasoning tasks by imitating offline expert demonstrations, often treating a single expert trajectory as the target behavior. However, reasoning is not simple path imitation: rigidly following one demonstrated solution may overfit to surface forms and suppress the model's own reasoning distribution. We propose Rollout‑Adaptive Supervised Fine‑Tuning (RASFT), a policy‑aware SFT framework that calibrates expert supervision according to problem‑level solvability estimated from verified on‑policy rollouts. For each problem, RASFT strengthens expert guidance when the current policy struggles, while relaxing rigid imitation and incorporating correct self‑generated trajectories when the model already exhibits reliable reasoning behavior. To preserve useful reasoning priors, RASFT further introduces a clipped inverse ratio between the frozen reference model and the current policy to constrain excessive policy drift. Experiments across multiple models on six mathematical reasoning benchmarks and two code reasoning benchmarks show that RASFT achieves better overall performance than SFT, SFT variants, and representative RL methods. The code is available at https://github.com/zjd1sq/RASFT.
Authors:Bokai Zhao, Yiyang Zhang, Long Bai, Tai Ma, Hanqing Chao, Minfeng Xu
Abstract:
Computational pathology requires visual representations that transfer across diverse clinical endpoints and remain robust to variation in magnification, staining, scanner type, slide preparation, and input resolution. We present DaX, a pathology vision foundation model that adapts DINOv3‑style self‑supervised learning to whole‑slide histopathology. DaX is initialized from natural‑image DINOv3 weights and incorporates continuous magnification training, cross‑scale tissue views, orientation‑agnostic and acquisition‑robust augmentation, multi‑input‑size training, and Gram‑anchored dense consistency. These designs aim to connect local cellular morphology with global tissue architecture while stabilizing dense token‑level representations across input scales. We further construct a WSI‑level benchmark comprising 161 clinically meaningful tasks from 44 public datasets, covering 28,182 patients and 34,394 slides across four clinical domains and nine task categories. All models are evaluated under a fixed patient‑level cross‑validation protocol with fold‑level statistical ranking, enabling reproducible comparisons that are less sensitive to split‑dependent variation. Across this benchmark, DaX achieves the highest mean performance across tasks and consistently strong task‑level ranking scores, with gains spanning diagnostic pathology, biomarker and molecular profiling, tissue/specimen context, and risk, response, and prognosis. These results support DaX as a transferable visual encoder for computational pathology and provide a standardized evaluation framework for future pathology foundation models. Project page: https://alibaba‑damo‑academy.github.io/DaX/benchboard/.
Authors:Xinyi Li, Zhen Fang, Yongxin Deng, Jinyuan Luo, Hongnan Ma, Changdae Oh, Zijing Shi, Shanshan Ye, Hanchen Wang, Shu-Lin Chen, Yadan Luo, Mengyue Yang, Sean Du, Sharon Li, Ling Chen
Abstract:
Hallucination detection is essential for the reliable deployment of large language models (LLMs). However, existing evaluations face two core challenges: inconsistent inference configuration and evaluation, and limited coverage of downstream domains and tasks. Consequently, reported detector performance is often difficult to compare, reproduce, and generalize beyond specific experimental settings. We introduce OpenHalDet, a unified benchmark for hallucination detection across diverse generation scenarios. OpenHalDet standardizes the evaluation pipeline, from prompt construction and response generation to truthfulness annotation, detector scoring, and metric computation. It supports heterogeneous detector families under different access settings, including black‑box methods that use only generated outputs, gray‑box methods that rely on probability‑based signals, and white‑box methods that exploit internal model signals. By bringing diverse tasks, models, and detectors into a shared framework, OpenHalDet enables controlled comparison and provides a systematic view of how different detection paradigms behave in LLM applications. We release OpenHalDet as an open and extensible codebase to facilitate reproducible evaluation and future development of hallucination detection methods. The code and datasets are available at https://github.com/Nellie179/Hallucination‑Detection.
Authors:Sunoh Kim, Daeho Um
Abstract:
Vision‑language models (VLMs) such as CLIP achieve strong zero‑shot recognition but remain highly fragile under adversarial perturbations. Recent test‑time adaptation defenses improve robustness by leveraging many augmented views, but this leads to impractical slowdown and a clear robustness‑throughput trade‑off. To address this challenge, we present Stability and Suitability‑guided Test‑time Prompt Tuning (SS‑TPT), evaluating the quality of each augmented view via two complementary scores: (1) stability, measuring prediction invariance to weak augmentations, and (2) suitability, measuring feature‑space density among views. These stability and suitability (SS) scores guide both adaptation and inference through an SS‑guided consistency loss and an SS‑weighted prediction, amplifying trustworthy views while suppressing corrupted ones. Extensive experiments demonstrate that SS‑TPT significantly outperforms prior state‑of‑the‑art methods, achieving superior robustness‑throughput trade‑offs across diverse datasets and varying numbers of views, thereby demonstrating both strong practicality and generality. Our code is available at https://github.com/sunoh‑kim/SS‑TPT.
Authors:Zhixian Zhao, Shuiyuan Wang, Wenjie Tian, Jingbin Hu, Ziyu Zhang, Lei Xie
Abstract:
While Audio Language Models (ALMs) demonstrate strong semantic understanding, they struggle with complex affective interactions. Specifically, textual semantic dominance often overshadows acoustic nuances, and a lack of cognitive depth leads to generic, emotion‑agnostic responses. We propose CogAudio‑LLM\footnote \urlstylesame https://github.com/zxzhao0/CogAudio‑LLM, a novel cognitive affective reasoning framework. To mitigate semantic dominance, we build LIME‑440K, a ``lexically‑identical, multi‑emotion'' dataset designed to facilitate acoustic‑semantic decoupling. We introduce EIPS, a 4‑step Chain‑of‑Thought (CoT) mechanism incorporating psychological reasoning. For inference efficiency, multi‑stage training explicitly establishes EIPS via supervised fine‑tuning, then distills this logic into an implicit generation process. Finally, we design DR‑SAPO (Dual‑Route Soft Adaptive Policy Optimization) to dynamically balance the logical rigor of the CoT with the empathetic quality of the direct response.
Authors:Sunoh Kim, Daeho Um
Abstract:
Vision‑language models such as CLIP have achieved remarkable zero‑shot recognition capabilities, yet their robustness against adversarial perturbations remains limited. Test‑time counterattack (TTC) was recently proposed to improve CLIP's robustness by perturbing an input image to steer it away from a corrupted state during inference. However, TTC remains fragile under strong attacks because its counterattack relies on a directly corrupted original view and employs a noise‑driven hard‑gating scheme that cannot adapt to varying corruption severity. To address these limitations, we introduce Multi‑view guided Adaptive Counterattack (MAC), which performs counterattacks for multi‑view with corruption‑aware soft weighting. Specifically, MAC first constructs augmented views of an input image to obtain diverse embeddings. It then performs counterattacks to refine corrupted embeddings of views. Next, MAC adaptively scales the counterattack intensity for each view based on its estimated corruption degree. Finally, the adaptively counterattacked views are aggregated to yield a robust final prediction. Extensive experiments across 20 datasets and diverse attack scenarios demonstrate that MAC substantially improves robustness while preserving high inference speed and memory efficiency with its tuning‑free design. Our code is available at https://github.com/sunoh‑kim/MAC.
Authors:Yixuan Zhou, Guoyang Zeng, Xin Liu, Xiang Li, Renjie Yu, Jiancheng Gui, Jiaheng Wu, Ziyang Wang, Xudong Shen, Runchuan Ye, Zhisheng Zhang, Jiuyang Zhou, Bingsong Bai, Weiyue Sun, Mengyuan Deng, Qundong Shi, Zhiyong Wu, Zhiyuan Liu
Abstract:
We present VoxCPM2, a https://info.arxiv.org/help/prep#abstractsfully open‑source multilingual and controllable speech generation foundation model that extends the hierarchical diffusion‑autoregressive modeling paradigm of VoxCPM. VoxCPM2 advances the framework in three key dimensions: (i) capability, by unifying 30 languages, 9 Chinese dialects, natural‑language voice design, style‑controllable voice cloning, and high‑fidelity continuation cloning within a single backbone; (ii) quality, through an asymmetric AudioVAE that encodes at 16 kHz and reconstructs at 48 kHz, enabling implicit super‑resolution with high encoding efficiency; and (iii) scale, by jointly scaling the model to 2B parameters and the training data to over 2 million hours of multilingual speech. To support these diverse capabilities within one model, we introduce a unified sequence organization that expresses all generation modes through different arrangements of the same input building blocks, allowing joint training under a single set of parameters and objective. VoxCPM2 achieves state‑of‑the‑art or competitive performance on public zero‑shot and instruction‑following TTS benchmarks. On our internal 30‑language evaluation set, it attains an average WER of 1.68%. These results demonstrate that hierarchical continuous‑latent modeling, without relying on any external discrete speech tokenizer, offers a viable and powerful foundation for large‑scale multilingual and controllable speech generation. The model weights, fine‑tuning code, and inference tools are publicly released under the Apache 2.0 license to foster community research and development.
Authors:Donggyu Lee, Youngbin Ki, Jeonghun Kang, Taehwan Kim
Abstract:
While highlight detection for long‑form videos is of great practical importance, most existing methods remain limited to short‑form content, largely due to the absence of a suitable benchmark. To bridge this gap, we introduce SVHighlights, to the best of our knowledge, the first benchmark for highlight detection in extremely long sports videos, each exceeding one hour in duration, across multiple sports categories. SVHighlights is constructed from pairs of full‑length sports videos and their corresponding official highlight videos using a dataset generation pipeline, enabling scalable label generation without conventional per‑clip saliency annotation. The benchmark comprises 320 videos with an average duration of 2.00 hours and a total of 640.18 hours, substantially exceeding previous datasets. Existing methods also face fundamental challenges on long videos: models trained on short clips fail to generalize to hour‑long content, and their clip‑level scoring lacks the broader context needed to identify highlights. To address this and provide a strong baseline, we present TF‑SELECTOR, a training‑free segment‑based approach that divides each video into context‑aware segments by merging adjacent shots sharing the same semantic content, and predicts segment‑level saliency scores using a large language model with multimodal inputs including visual captions, transcripts, and audio volume. Experiments demonstrate that TF‑SELECTOR achieves superior performance across most metrics compared to Video Temporal Grounding (VTG)‑tuned baselines, with improvements of +3.12 in HIT@1, +4.06 in HIT@K, and +2.95 in IoU. These results establish SVHighlights as a challenging testbed for long‑form highlight detection and demonstrate that a simple segment‑based strategy can effectively scale to hour‑long videos.
Authors:Rahul Nair, Chun Tao
Abstract:
Deploying Small Language Models (SLMs) on edge devices requires efficient fine‑tuning strategies that adapt models to new tasks without degrading their general capabilities. In this study, we benchmark five sub‑1B models (135M‑1B) on mathematical reasoning tasks and uncover a critical vulnerability: Full Fine‑Tuning (Full FT) actively harms performance in models under 300M parameters, often dropping accuracy below zero‑shot baselines. This "negative transfer" makes Parameter‑Efficient Fine‑Tuning (PEFT) not just an efficiency preference, but a stability requirement. We find that while Low‑Rank Adaptation (LoRA) and Weight‑Decomposed LoRA (DoRA) perform comparably, their strengths vary by task; DoRA excels in complex reasoning (GSM8K), while LoRA dominates pattern matching (OrcaMath). In particular, Full FT is outperformed by LoRA on aligned models (Qwen2.5‑0.5B) and even by simple 5‑shot In‑Context Learning on the smallest architectures (SmolLM2‑135M). Based on these findings, we recommend defaulting to PEFT for all aligned sub‑1B models and caution against Full FT for any architecture smaller than 500M parameters to prevent catastrophic forgetting. Reproduction of this work can be found at https://github.com/gulguluu/tiny‑slm‑finetune‑compare.
Authors:Wenhao Zhang, Ramin Ramezani, Tao Han, Kai Hwang, Minyi Guo
Abstract:
Modern image‑analysis pipelines often convert images into structured semantic variables, such as facial attributes, object concepts, and scene descriptors. Learning directed dependencies among these variables can produce interpretable visual semantic graphs, but continuous directed acyclic graph learning is limited by the cost of enforcing acyclicity. We present polyDAG, a polynomial acyclicity framework for efficient continuous causal discovery in visual semantic graphs. polyDAG replaces the matrix‑exponential acyclicity constraint with a finite polynomial trace constraint and proves that the new constraint is zero exactly for acyclic graphs. We further derive a geometric‑series implementation that avoids the explicit summation loop while preserving the same acyclicity condition. Experiments on synthetic Erdos‑Renyi graphs and CelebA facial visual attributes show that polyDAG improves efficiency and structure recovery. Averaged over the revised synthetic protocol with d in 100, 200, 500, polyDAG reduces mean structural Hamming distance from 318.4 to 285.4 and improves mean F1 score from 0.725 to 0.756. At 100 nodes, the geometric variant runs in 3.44 seconds compared with 5.16 seconds for the exponential baseline, corresponding to a 33.4 percent speedup. Code and data are publicly available at https://github.com/wenhaoz‑fengcai/polyDAG.
Authors:Pei Yang, Hai Ci, Yanzhe Chen, Qi Lv, Han Cai, Mike Zheng Shou
Abstract:
Vision‑language‑action (VLA) models have advanced rapidly across backbones, training recipes, and data scale, yet the action decoder, which converts the backbone's hidden state into a continuous control signal, has barely changed and remains a single‑point predictor across the majority of current VLAs. Whether implemented via autoregressive token bins, L1 regression, or flow‑matching denoising, the resulting decoder treats the action space as unstructured, leaving the geometric proximity of neighboring actions unexploited during training. To advance this, we introduce ActionMap, a voxel heatmap action head that drops into an existing VLA in place of its native action decoder. For each new action, the head predicts a voxel heatmap over the action space, where each voxel directly stores the probability of the corresponding action. Across LIBERO simulation and real‑world Franka manipulation, our heatmap head surpasses two architecturally distinct backbones at matched training steps (e.g., +8.2% over OpenVLA‑OFT's L1 regression head on the LIBERO four‑suite average), converges at comparable or faster rates on both backbones, and remains markedly more data‑efficient at low training data. The cross‑backbone consistency indicates that action representation is a real lever for VLA performance, distinct from further backbone or recipe scaling. Project Page: https://showlab.github.io/ActionMap/.
Authors:Yuan Zeng, Yujia Shi, Yuhao Yang, Dongxia Liu, Zongqing Lu, Wenming Yang, Qingmin Liao
Abstract:
Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross‑ID features with those learned from same‑ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state‑of‑the‑art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.
Authors:Zhiwei Xu, Shihao Wu, Hanseul Cho, Wei Hu, Yixin Wang
Abstract:
Classical scaling laws for language model pretraining balance model size against training dataset size under a fixed compute budget, assuming abundant data and a single pass over the corpus. As training compute grows faster than the supply of natural language data, pretraining is likely to enter a data‑constrained, compute‑rich regime where models train for multiple epochs over a finite dataset. We study data‑constrained pretraining along two axes, regularization and scaling. For regularization, we study masked‑input regularization (MIR), an auxiliary next‑token prediction loss on randomly masked inputs. MIR tests whether the random masking central to diffusion language models can benefit autoregressive pretraining without architectural changes or inference overhead. Across 72M to 1.4B parameter models, we find that MIR added on top of strong weight decay improves validation loss over autoregressive strong‑weight‑decay‑only models, with downstream gains at 1.4B. For scaling, we propose SoftQ, a scaling law that couples model size and data size to capture their interaction under repeated data. Classical alternatives such as the Chinchilla law use an additive form that decouples these terms, making them misspecified in the data‑constrained regime. We find that SoftQ fits data‑constrained experiments substantially better than these alternatives, and estimates MIR's gains as equivalent to roughly 1.3 times as much unique training data. We release our code at https://github.com/yixinw‑lab/dc_pretrain.
Authors:Yuan Zeng, Yujia Shi, Zongqing Lu, QingMin Liao
Abstract:
Human Image Animation has seen significant advancements, primarily driven by diffusion models. However, existing methods typically demand substantial training data and resources to achieve high‑quality results, limiting generalization and accessibility. In this work, we introduce \emphFreeAnimate, a training‑free framework that leverages the inherent capabilities of image diffusion models to enable temporal consistency, identity preservation, and background stability. Our approach incorporates a novel preview generation strategy that provides temporal and structural priors from generated preview frames, effectively guiding pose alignment and background consistency without training. Additionally, FreeAnimate introduces Inversion‑Boosted Attention and Reference‑Anchored Self‑Attention modules to guarantee temporal consistency and identity preservation. Experimental results demonstrate that FreeAnimate outperforms existing training‑free competitors and training‑based baseline methods, achieving generation quality comparable to state‑of‑the‑art methods and offering robust generalization across diverse datasets. Our project page is at https://freeani.github.io/.
Authors:Kangjian Zhu, Haobo Jiang, Jianjun Qian, Jin Xie
Abstract:
In this paper, we propose a cross‑view fusion framework that enhances the robustness of 6‑DoF grasp pose estimation in corner views. Our framework alleviates occlusion by incorporating an auxiliary view and avoids the time‑consuming, task‑agnostic multi‑view reconstruction through a post‑fusion strategy. To enhance cross‑view fusion, we propose a self‑supervised contrastive learning strategy that leverages cross‑view associations to regularize point cloud features. In brief, a cross‑view point pair is considered a match if the two points correspond to the same 3D location, and a non‑match if they represent distinct grasp directions. The learning strategy significantly enhances the spatial consistency and direction distinctiveness of point features, thereby facilitating cross‑view fusion and improving estimation robustness. Furthermore, we propose a cross‑view‑aligned cylinder integration module to fuse grasp‑relevant geometry into a comprehensive representation. Specifically, the module first aligns the cross‑view points and features according to their similarity to enhance the robustness against noise. Subsequently, these points are registered into the cylindrical coordinate frame, emphasizing the rotation‑symmetric geometry which is important for grasping. Finally, local self‑attention and seed cross‑attention layers are alternately employed, respectively enabling interactions within single views and across views, which supports fine‑grained representation of grasp‑relevant geometry. Our framework achieves strong performance on the GraspNet‑1Billion benchmark and in real‑world applications. Code is available at https://github.com/KJZhuAutomatic/Cross‑view‑Grasp.
Authors:Xiang Yang, Feifei Li, Mi Zhang, Geng Hong, Xiaoyu You, Mi Wen, Min Yang
Abstract:
Diffusion transformers (DiTs) equipped with multimodal attention (MM‑Attn) have become a dominant paradigm for image generation. However, preventing the generation of harmful content remains a critical challenge, particularly in image‑to‑image (I2I) editing tasks. Existing safety mechanisms are primarily designed for text‑to‑image (T2I) synthesis or U‑Net‑based architectures, which limits their effectiveness for unified safety mitigation in DiT‑based frameworks. To bridge this gap, we propose Unified Visual Safety Regulator (UVR), a training‑free safe generation framework that regulates unsafe semantics in generated images. UVR is grounded in an analysis of attention dynamics from the perspective of information flow in MM‑Attn. We identify a task‑independent start‑up stage, during which unsafe semantics in output patches rapidly emerge and can be accurately localized, followed by task‑specific semantic amplification and interference stages, where harmful signals are further propagated and entangled with benign content. Based on these observations, UVR mitigates unsafe generation through unified, targeted attention modulation and explicit restriction of harmful information flow over the identified unsafe output patches. Experiments across various concepts show that UVR achieves state‑of‑the‑art safety performance by achieving 91% and 77% erase rate in image synthesis and editing tasks, while preserving visual quality and fidelity with minimal degradation. Code is available at https://github.com/deng12yx/UVR.
Authors:Yuan Zeng, Zilue Gao, Yujia Shi, Zongqing Lu, Wenming Yang, QingMin Liao
Abstract:
Estimating hand‑surface contact pressure from an egocentric view is crucial for AR/VR devices, robotic imitation, and ergonomic analysis. Existing methods often discretize pressure signal and process frames independently, leading to quantization errors and temporal inconsistencies. We present \emphEgoPressDiff, a conditional video diffusion framework that generates UV‑pressure maps from visual input. The core of our approach is a multi‑modal conditioning strategy, introducing a PoseNet and a Vertex Encoder to efficiently extract features from hand pose and 3D mesh vertices. These signals, along with depth information, guide the generative process to ensure the pressure fields are physically grounded. To effectively fuse these heterogeneous features, we further propose a Distribution‑Calibrated Spatial Layer, which aligns their statistical properties before combination. Evaluated on the EgoPressure ego‑view setting, EgoPressDiff achieves state‑of‑the‑art results, improving Volumetric IoU by over 34% relative to prior baseline, while reducing MAE and maintaining high temporal accuracy. Our project page is at https://egopressdiff.github.io/.
Authors:Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang, Yuhang Zang, Tianyi Wei, Xiaohang Zhan, Jiaqi Wang, Tong Wu, Xingang Pan, Dahua Lin
Abstract:
Group Relative Policy Optimization (GRPO) has demonstrated remarkable success in aligning text‑to‑image (T2I) flow models with human preferences. However, we have identified that the learning loop of current flow‑based GRPO is fundamentally decoupled from the learner's current capability, suffering from critical blind spots at both prompt selection and advantage estimation: (i) Existing methods sample prompts randomly, overlooking the substantial impact of data selection on reinforcement learning (RL) efficacy‑‑a factor proven crucial in GRPO for large language models; (ii) They evaluate sample quality solely relying on intra‑group statistics, lacking a global perspective to accurately measure true policy improvement. To address these issues, we propose Adaptive GRPO (AdaGRPO), a novel capability‑aware RL algorithm tailored for flow models. Specifically, AdaGRPO consists of two principal components: (i) Online Curriculum Filtering Strategy: Dynamically tracks the model's proficiency and adaptively selects prompts that best match its current learning boundary; (ii) Cross‑Level Advantage Fusion: Synergistically integrates fine‑grained intra‑group advantages with macro‑level global advantages, providing a comprehensive and unbiased policy evaluation. As a lightweight, plug‑and‑play module, AdaGRPO can be seamlessly integrated with existing frameworks such as Flow‑GRPO, DanceGRPO, and Flow‑CPS. Extensive experiments demonstrate that AdaGRPO consistently drives performance gains while significantly stabilizes GRPO training for flow models.
Authors:Yu Yu, Chen Lyu
Abstract:
With the remarkable progress of Code Large Language Models (Code LLMs) in achieving semantic correctness, execution efficiency has become an increasingly important dimension for evaluating their practical utility. However, existing approaches typically treat full programs as a single optimization target during training, without explicitly modeling the structural factors that influence efficiency. As a result, although these models can generate semantically correct code, they fail to learn, at a fine‑grained level, the underlying skeleton features that lead to efficient implementations. To address this limitation, we propose SkelDPO (Skeleton‑Guided Direct Preference Optimization), a skeleton‑guided preference optimization framework that systematically enhances the efficiency of code generation. SkelDPO first identifies efficient and inefficient implementations from the code dataset and, through comparative analysis, locates their efficiency‑prone and inefficiency‑prone points, forming alignment signals between efficiency and inefficiency skeletons. During training, a joint code and skeleton preference loss is introduced, enabling the model to learn semantic correctness while reinforcing its understanding of efficiency‑critical components in code. Results show that SkelDPO consistently surpasses existing methods: compared with SOTA method that relies solely on efficient and inefficient code preference optimization, it improves Pass@1, Beyond@1, and Effi@1 by 3‑6%, 3‑7%, and 2‑5%, with greater improvements observed on complex tasks. Overall, SkelDPO provides a new perspective on skeleton‑level efficiency alignment, breaking the limitation of conventional preference optimization that relies solely on correctness or efficiency pairs. All datasets and source code are publicly available at: https://github.com/icpcSkelDPO/SkelDPO.
Authors:Ming Dai, Sen Yang, Boqiang Duan, Boyuan Tong, Jiedong Zhuang, Wankou Yang, Jingdong Wang
Abstract:
Reasoning Video Object Segmentation (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning to achieve precise pixel‑level localization. Existing methods are limited to reasoning over fixed initial inputs and lack the capacity to actively acquire further visual evidence, which is often essential for resolving complex references in long or intricate videos. To address this, we propose VideoSEG‑O3, the first multi‑turn reinforcement learning framework for RVOS that emulates the human ``coarse‑to‑fine'' cognitive process. It employs a multi‑turn temporal‑spatial chain‑of‑thought to capture fine‑grained details by iteratively pinpointing critical intervals and keyframes. Additionally, to enable the policy to perceive segmentation quality beyond mere text probability of \texttt[SEG] during the RL stage, we introduce SEG‑aware logit calibration, which integrates pixel‑wise segmentation feedback directly into the token‑level logits. Furthermore, we design a decoupled thinking trace to hierarchically decompose the reasoning process into temporal, spatial, and linguistic dimensions, and construct VTS‑CoT, a specialized cold‑start dataset featuring comprehensive reasoning trajectories. The code and models will be released at https://github.com/Dmmm1997/VideoSEG‑O3.
Authors:Dahee Kwon, Haeun Lee, Jaesik Choi
Abstract:
Recent text‑to‑image models built on large‑scale Transformer backbones and flow‑based objectives deliver strong text‑image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity‑enhancement methods alleviate this issue, but typically require expensive sampling or auxiliary optimization, incurring non‑trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero‑frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock‑in that limits downstream variation. Building on this observation, we propose DC Attenuation for diVersity Enhancement (DAVE), a training‑free representation‑level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt‑consistent diversity while maintaining competitive image quality.
Authors:Kasra Borazjani, Fardis Nadimi, Payam Abdisarabshali, Owen Palinski, Allan Salihovic, Dinh Nguyen, Minghui Liwang, Seyyedali Hosseinalipour
Abstract:
This paper presents a forward‑looking vision for integrating the emerging multi‑modal multi‑task federated foundation models (M3T FedFMs) into vehicular networks, with the goal of unifying the expressive power of multi‑modal multi‑task foundation models (M3T FMs) with the privacy‑preserving and distributed learning capabilities of federated learning (FL). Given the largely underexplored nature of this research direction, we first introduce the fundamental training/fine‑tuning principles of M3T FedFMs. We then discuss a range of their representative use cases in vehicular networks, illustrating the significant potential of M3T FedFMs to enable next‑generation vehicular intelligence. Afterwards, we identify key constraints inherent to vehicular environments that challenge the practical deployment of M3T FedFMs, and articulate a set of forward‑looking research directions to address these challenges. Furthermore, through a case study conducted on a real‑world vehicular dataset (i.e., Waymo Open Dataset), we demonstrate the promise of M3T FedFMs for vehicular networks and release our implementation to facilitate reproducibility and stimulate research in this emerging area (repository: https://github.com/KasraBorazjani/vehicular‑fedfm)
Authors:Yikai Zhang, Gaoxiang Jia, Jie Ding, Boxiang Wang
Abstract:
TorchKM is an open‑source library for kernel machines, including support vector machines, kernel logistic regression, and kernel quantile regression, with GPU acceleration. The library features a scikit‑learn‑style API and is designed to exploit GPU‑friendly linear algebra, accelerating the full training and model‑selection pipeline through intelligent reuse of matrix operations. Benchmarks show competitive predictive performance with substantial speedups over standard baselines. The efficiency and programmable design also make TorchKM a kernel‑learning component for AI‑driven workflows. Code and documentation are available at https://github.com/YikaiZhang95/torchkm, and the package can be easily installed via PyPI.
Authors:Zhiling Yan, Dingjie Song, Hanrong Zhang, Wei Liang, Yuxuan Zhang, Yutong Dai, Lifang He, Philip S. Yu, Ran Xu, Xiang Li, Lichao Sun
Abstract:
Self‑evolving agents requires adaptation after deployment, but existing approaches assume a usable learning loop, such as curated skills, successful trajectories, or verifier signals. Real open‑world deployments may provide none of these, offering only a task prompt. In this work, we study open‑world self‑evolution, where an agent must build both its skills and its own verification signals from scratch, using open‑world resources but no target‑task supervision. We propose OpenSkill, a framework that bootstraps this loop: it acquires grounded knowledge and verification anchors from documentation, repositories, and the web, synthesizes them into transferable skills, and refines those skills against self‑built virtual tasks grounded in the anchors rather than in target answers. The open world thus supplies both the knowledge to be learned and a supervision‑independent practice environment, with target‑task supervision reserved for final evaluation. Across three benchmarks and two target agents, OpenSkill attains the best automated pass rate while satisfying the no‑supervision constraint. Analysis shows its skills transfer across models without model‑specific adaptation, and its self‑built verifier aligns with ground‑truth outcomes despite never accessing them.
Authors:J. S. van Hulst, W. P. M. H. Heemels, D. J. Antunes
Abstract:
This paper develops the Dynamic Gaussian Process (DGP), a framework for estimating functions governed by integro‑difference equations (IDEs). IDEs model continuous functions that evolve with discrete‑time dynamics and arise naturally from time‑discretization of linear partial differential equations (PDEs). The DGP extends Gaussian process regression to time‑varying functions and extends Kalman filtering to infinite‑dimensional states. The DGP posterior remains a Gaussian process with closed‑form mean and covariance updates, and separable kernel structure reduces the problem to a finite‑dimensional Kalman filter on basis function coefficients. This paper extends the DGP to vector‑valued states, enabling the treatment of higher‑order PDEs, and provides a stability and approximation error analysis for the basis function approximation. The functional L2 estimation error decomposes exactly into in‑subspace and out‑of‑subspace contributions, and all approximation errors vanish as the number of basis functions grows. The framework is demonstrated on the heat equation and on the wave equation, the latter with a vector‑valued state. Code is available at https://github.com/JvHulst/Dynamic_Gaussian_Processes.
Authors:Zepeng Zhang, Aref Einizade, Jhony H. Giraldo, Olga Fink
Abstract:
Missing data is a common challenge in spatiotemporal systems, arising in applications such as air quality monitoring and urban traffic management. Traditional machine learning approaches, like recurrent and graph neural networks, rely on iterative propagation, which tends to accumulate errors over time and space. Recent diffusion‑based methods mitigate error propagation but require iterative sampling and often depend on problem‑agnostic Gaussian priors, limiting both efficiency and effectiveness. To address these limitations, we propose GiFlow, a Graph‑Informed Flow Matching framework for spatiotemporal imputation. GiFlow replaces the typical Gaussian prior with a graph‑informed prior constructed via spatiotemporal filtering of observable signals, which better aligns the source distribution to the target and thereby simplifies the generation trajectory. The flow field is parameterized by a hybrid vector field model that integrates spatial attention, temporal attention, and spatiotemporal propagation, enabling joint modeling of spatial and temporal dependencies. Extensive experiments on both synthetic and real‑world datasets demonstrate that the proposed GiFlow outperforms the state‑of‑the‑art approaches in spatiotemporal imputation. The code is available at https://github.com/zepengzhang/GiFlow.
Authors:Xi Xuan, Wenxin Zhang, Yufei Zhou, King-kui Sin, Chunyu Kit
Abstract:
Court judgments are central to legal practice and jurisprudence, yet discourse analysis of Hong Kong judgments has received limited attention, owing largely to the absence of expert‑annotated corpora. We introduce the Hong Kong Judgment Discourse Dataset (HKJudge), the first sentence‑level expert‑annotated legal discourse corpus. HKJudge includes criminal judgments across all five levels of HK's court hierarchy, comprising ～290k sentences and ～6.5 million tokens, fully annotated by legal linguistics experts. We design a two‑tier discourse schema that captures what facts a court finds, how it reasons, and what it rules. At the sentence level, each sentence is assigned one of 26 rhetorical roles. At the span level, sentences are further annotated with three sentencing elements (charge, imprisonment term, fine). Ten legal linguistics annotators produced the annotations with an inter‑annotator agreement of κ= 0.8. We formulate two tasks on HKJudge, termed rhetorical role classification and legal element extraction, and provide the first benchmark evaluation of four BERT‑based models, two open‑source LLMs under zero‑shot and fine‑tuning settings, and four commercial LLMs on both tasks. Our work demonstrates the value of sentence‑level discourse annotation for modeling the structure of HK judgments and provides a rich data foundation for future work on legal judgment prediction. The HKJudge dataset and code are available at https://github.com/xuanxixi/HKJudge.
Authors:Ujjwal Bhatta, Utsabi Dangol, Sumaly Bajracharya, Rodrigue Rizk, KC Santosh
Abstract:
Sparse rewards and heterogeneous task sequences remain persistent challenges in Reinforcement Learning (RL), often resulting in slow convergence, weak generalization, and inefficient exploration. We propose Uncertainty‑Aware LLM‑Guided Policy Shaping (ULPS), a novel framework that integrates a calibrated Large Language Model (LLM) into the RL training loop to provide structured, uncertainty‑modulated behavioral guidance. ULPS employs an A‑based oracle to synthesize optimal symbolic trajectories, which are used to fine‑tune a BERT‑based language model. During training, this model supplies action suggestions whose influence is conditioned on epistemic uncertainty estimated via Monte Carlo (MC) dropout. An entropy‑based blending mechanism adaptively balances LLM guidance and the learned policy (via Proximal Policy Optimization, PPO), allowing the agent to prioritize reliable priors while preserving adaptability. We evaluate ULPS on the MiniGridUnlockPickup benchmark and observe consistent improvements in success rate, reward efficiency, and sample complexity over unguided, uncalibrated, and standard RL baselines. ULPS achieves more than 9% improvement in execution accuracy after fine‑tuning, requires fewer environment interactions, and yields higher reward AUC. Our results demonstrate that integrating symbolic A trajectories, pretrained language priors, and uncertainty‑aware control offers a principled and effective approach to multi‑task reinforcement learning in sparse‑reward domains, with potential extensibility to partially observable and multi‑agent settings.
Authors:Tang Li, Yanlin Chen, Mengmeng Ma, Xi Peng
Abstract:
Despite high accuracy, Vision Transformer (ViT) predictions can be driven by spurious cues, raising the need to understand their inner workings before safe deployment. Sparse autoencoders (SAEs) provide a promising lens for decomposing model representations into human‑interpretable concepts, yet adapting SAE‑based interpretation to ViTs remains challenging due to limited control over concept coverage and subjective, non‑scalable feature interpretation. To fill the gaps, motivated by neuroscience‑inspired principles, we propose ViSAE, a mechanistic interpretability toolbox for understanding ViT inner workings through concept circuits. ViSAE consists of three components: (1) A probing suite with 64K images and a 16K visually grounded concept vocabulary, improving concept coverage efficiency by 20x over ImageNet and interpretation accuracy by 28.7% over existing concept sets. (2) Top‑down concept reading and Bottom‑up circuit tracing algorithms that automatically recover ViT inner workings via concept circuits. (3) Applications for auditing and steering ViT behavior. Through concept editing, ViSAE improves the worst‑group accuracy on WaterBirds by 48.2%, outperforming existing methods by 23.8%. Our data and code: https://github.com/deep‑real/ViSAE.
Authors:Jun-You Lin, Ying Choon Wu, Tzyy-Ping Jung
Abstract:
Objective. EEG foundation models (FMs) report strong accuracy on clinical resting‑state EEG. However, high accuracy under subject‑disjoint cross‑validation remains ambiguous: it can reflect a genuine clinical biomarker, or subject‑identity features that correlate with the label. We name this the Identity Trap and ask whether it can be diagnosed at the representation level before fine‑tuning. Approach. We propose FMScope, a frozen‑representation protocol packaging five diagnostics: variance decomposition, subject‑axis erasure, aperiodic 1/f ablation, layer‑wise label probing, and within‑subject direction consistency. We apply it to three pretrained FMs (LaBraM, CBraMod, REVE) across four datasets in a 2x2 layout: subject relation of label x presence of a consensus cross‑subject EEG marker. Main results. (i) The Identity Trap is universal: frozen subject‑variance is 13‑89x a random null in 12/12 pairs, rising in all 12 under fine‑tuning (+10 to +63 pp). This dominance is a removable linear axis: erasing it improves label decoding where the label varies within subject (+6 to +12 pp in primary cells; +4 to +27 pp across external cohorts). (ii) Aperiodic 1/f is one subject carrier: removing it drops the subject probe by 9‑19 pp on LaBraM and CBraMod. REVE saturates subject identity without measurable aperiodic dependence. (iii) Fine‑tuning amplifies label‑variance only in cells with a literature‑established cross‑subject marker. Significance. The Identity Trap is a physically‑grounded instance of shortcut learning: the preferred cue has a measurable physiological component, and subject‑disjoint splitting alone cannot rule it out. FMScope separates gains reflecting a biological marker from those reflecting subject identity.
Authors:Richard Li, Aditya Prakash, Andrew Wen, Saurabh Gupta, Yilun Du, Pulkit Agrawal
Abstract:
Human video datasets used for cotraining robot manipulation policies largely consist of curated demonstrations where motions are orchestrated to resemble robot behavior and 3D hand poses are captured with specialized hardware. A more plentiful source of data is everyday Internet video, but it is an open question what factors enable transfer from such videos to robots. We investigate this using a new dataset of 532 human videos with 28 hours of high‑quality triangulated hand labels and natural motions. We find that hand pose quality affects transfer, but even with accurate hands, the inherent motion gap hinders transfer unless the vision and policy networks specialize to each embodiment. Our cotraining recipe yields consistent improvements, with an absolute success rate gain of 29.7% in the low‑robot‑data regime across six manipulation tasks.
Authors:Sam Buchanan, Druv Pai, Peng Wang, Yi Ma
Abstract:
In the current era of deep learning and especially generative models, there is significant investment in training very large deep neural networks. Thus far, such models have been "black boxes" that are difficult to understand in the sense that they have opaque internal mechanisms, leading to difficulties in interpretability, reliability, and control. Naturally, this lack of understanding has led to both hype and fear. This book is an attempt to "open the black box" and understand the mechanisms of large deep networks, through the perspective of representation learning, which is a major factor ‑ arguably the single most important one ‑ in the empirical power of deep learning models. A brief outline of this book is as follows. Chapter 1 will summarize the threads that underlie the whole text. Chapters 2, 3, 4, 5, and 6 will explain the design principles of modern neural network architectures through optimization and information theory, reducing the process of architecture development (long having been described as a sort of "alchemy") to undergraduate‑level linear algebra and calculus exercises once the underlying principles are introduced. Chapters 7 and 8 will discuss applications of these principles to solve problems in more paradigmatic ways, obtaining new methods and models which are efficient, interpretable, and controllable by design, and yet no less ‑ sometimes even more ‑ powerful than the black‑box models they resemble. Chapter 9 will discuss potential future directions for deep learning, the role of representation learning, as well as some open problems.
Authors:Nishit Anand, Ashish Seth, Sreyan Ghosh, Dinesh Manocha, Ramani Duraiswami
Abstract:
Retrieving music using natural language descriptions has improved with contrastive audio‑text models such as CLAP, but current systems remain limited to coarse semantic queries. When descriptions specify fine‑grained musical attributes such as tempo, key, chord progression, or rhythmic structure, existing models often fail to retrieve the correct audio. We show that this limitation stems from the contrastive learning objective itself: despite being trained on long captions, CLAP‑based models effectively utilize only the first few tokens, discarding much of the information encoded in detailed prompts. Then, we propose FIGMA (FIne‑Grained Music RetrievAl), a multi‑view contrastive architecture that addresses this limitation by jointly optimizing global audio‑text alignment and frame‑level, token‑wise alignment. This design enables FIGMA to capture both high‑level semantic context and fine‑grained musical attributes within a unified representation space. Moreover, we formalize the task of Fine‑Grained Music Retrieval and construct Fine‑Grained Music Caption dataset (FGMCaps), a large‑scale dataset of 380K music‑caption pairs for training along with a 10K test set, both annotated with tempo, key, chord progression, beat count, as well as genre and mood. Extensive experiments demonstrate that FIGMA consistently outperforms existing CLAP‑based music retrieval models across multiple music retrieval benchmarks, including out‑of‑domain evaluations, with relative improvements of up to 73.3%.
Authors:Jingbo Gong, Yikai Wang, Yushi Lan, Yuhao Wan, Ziheng Ouyang, Rui Zhao, Ming-Ming Cheng, Qibin Hou, Chen Change Loy
Abstract:
Object insertion aims to seamlessly composite a reference object into a specified region of a background image. Recent diffusion‑based methods achieve high visual quality but formulate insertion as a simple 2D inpainting task, providing no explicit control over the object's 3D pose and limiting their practical applicability. We propose DIRECT (Decomposed Injection for Reference Composition and Target‑integration), a novel framework that integrates interactive pose manipulation with high‑fidelity 2D image synthesis to enable pose‑controllable object insertion. Our method decomposes the insertion conditions into three complementary components: appearance guidance capturing visual details from the reference object, geometry guidance derived from the user‑adjusted 3D proxy, and context guidance from the target background. By injecting them through separate pathways, DIRECT avoids feature entanglement and simultaneously preserves reference appearance, follows the user‑specified pose, and adapts the object to the target scene. We also introduce an automated data construction pipeline to improve the diversity and quality of training data. Experiments show that DIRECT outperforms previous methods in both geometric controllability and visual quality.
Authors:Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak, N. J. Mayne, Miles Cranmer
Abstract:
In the sciences, regression tasks often require predicting high‑dimensional outputs from few training examples. Multi‑output Gaussian processes excel in low‑data regimes but typically struggle with high‑dimensional outputs. Compress‑then‑predict pipelines such as PCA‑GP (principal component analysis plus Gaussian process regression) handle high dimensionality, but rely on bases optimized for reconstruction rather than prediction. To address this gap, we propose a model that represents each output as a linear‑Gaussian decoding of a low‑dimensional latent state drawn from a Gaussian process prior. By analytically marginalizing the decoder weights, we couple compression and prediction in a single objective that scales to high‑dimensional outputs. We refer to this model as Gaussian process latent factor regression (GPLFR). We demonstrate GPLFR by building the first spatially resolved emulator of global climate models for rocky exoplanets.
Authors:Ziyue Li, Yang Li, Tianyi Zhou
Abstract:
Large language models (LLMs) perform inference by following a fixed depth and order, non‑recurrent execution of all layers. We reveal the wide existence of training‑free, flexible, dynamic program‑of‑layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic‑depth methods, often while executing fewer layers, and that these gains persist under out‑of‑distribution evaluation. Our results suggest that fixed‑depth execution captures only a narrow subset of an LLM's latent reasoning capacity.
Authors:Bo Liu, Yuan Xie, Yuan Gao, Xiaolong Luo, Peng Ye, Tao Chen, Fujun Han
Abstract:
Sudoku is a representative constraint satisfaction problem that requires global structural reasoning under strict discrete constraints. The existing works of solving Sudoku mainly focus on two dominant approaches, i.e., traditional heuristic and deep learning solver. However, they suffer from two complementary limitations: learning‑based solvers lack hard correctness guarantees, while complete symbolic solvers are still prone to long‑tail search. To address these shortcomings, we propose a novel diffusion model‑guided approach, termed as DiBS, for the branch selection search process. Specifically, DiBS keeps the symbolic solver complete and uses the diffusion model as a branch‑ordering guide. The core method is ranking candidate values under the current partial assignment and lightweight consistency signal. Furthermore, we provide an in‑depth theoretical proof to reveal how it works and why it works. Experiments on the challenging Royle 17‑clue Sudoku benchmark show that our DiBS substantially reduces search cost relative to strong heuristic baselines, especially in nodes, backtracks, and long‑tail percentiles. Besides, these results confirm that learned global guidance is effective on hard instances where branch‑order mistakes are most expensive. All codes are available at https://github.com/shanxierdan/DiBS.
Authors:Zhida Sun, Yulin Zhang, Zheng Gu, Min Lu, Bongshin Lee, Daniel Cohen-Or, Hui Huang
Abstract:
Traditional statistical graphics are precise but often lack the visual appeal, memorability, and engagement of pictorial charts. We present a generative framework for the automated synthesis of pictorial charts that bridges the gap between semantic expression and structural faithfulness. Rather than treating charts merely as images to be stylized, we frame the problem as a dual‑conditioned generation task guided by two parallel external control signals: a text prompt capturing the semantic context of the editing intent, and a context image providing the abstract statistical chart's global structure. To reinforce these controls within a Multi‑Modal Diffusion Transformer, we introduce two complementary feature‑level mechanisms: structural alignment to anchor spatial layouts to the input chart, and semantic alignment to transfer expressive textures from reference images. Generalizing across major visual channels (i.e., length, area, angle, and position) and diverse semantic domains, our method produces pictorial charts that are both artistically compelling and structurally consistent. Extensive quantitative evaluations and perceptual user studies demonstrate that our framework outperforms traditional controllable generation and image editing baselines, providing a foundation for high‑fidelity, data‑driven generative modeling in expressive visual storytelling. Project page: https://ssalign.github.io/.
Authors:Huayi Zhou, Wei Gao, Dekun Lu, Ruiji Liu, Zhanqi Zhang, Ziyang Zhang, Jian Chen, Wenlve Zhou, Sheng Xu, Shumin Li, Kangyi Guo, Shichen Xu, Zixin Huang, Yongyi Su, Kui Jia
Abstract:
End‑to‑end manipulation policies, combined with web‑scale pretrained Vision‑Language Models (VLMs), show the promise for generalizable and dexterous robotic manipulation. However, they inherit two key limitations from 2D foundation models: 1) the reliance on 2D RGB inputs that ignores the intrinsically 3D nature of manipulation; and 2) the lack of spatial 3D alignment between input‑output spaces as well as across diverse robot embodiments, camera setups, and trajectory datasets. In this paper, we present a series of contributions to address these issues. First, we introduce aligned vertex map and vertex spectrum ‑‑ a pixel‑wise 3D representation that elevates 2D visual inputs to 3D, using camera calibration and optional depth. This novel input representation marries 3D awareness with the generalization of 2D large VLMs. Then, we propose to align the inputs and outputs of manipulation policies by expressing per‑pixel 3D information of each camera view and robot actions to a shared coordinate. Based on this, we designate a canonical Bird's‑Eye‑View (BEV) alignment frame and innovatively propose to construct BEV images, producing a view‑invariant representation robust to camera pose variations. To enable training and evaluation at scale, we develop a comprehensive data processing pipeline to perform such alignments; we also introduce a novel temporal alignment scheme for trajectories across diverse robots, human operators, and datasets. These contributions collectively mitigate input and output spatial‑temporal misalignments, improving the consistency and generalization for real‑world manipulation. Pretrained checkpoint, source code and data processing pipeline are available in https://hnuzhy.github.io/projects/Dex‑BEV.
Authors:Ümit Mert Çağlar, Alptekin Temizel
Abstract:
Semantic segmentation of remote sensing imagery requires models that capture both global context and local detail under tight computational budgets. Prior work typically optimizes for one of these axes: attention for global context, convolution for local detail, or compactness for efficiency. While hybrid approaches aim to capture both, they require architectural changes and encoder backbones with computational overhead, limiting efficiency and performance. We present LALE (Lightweight‑transformer Architecture for Land‑cover Estimation), an end‑to‑end remote sensing image segmentation architecture, that bifurcates its encoder by resolution: lightweight ConvMixer stages handle high‑resolution local features, while transformer stages handle low‑resolution global context, confining the quadratic cost of self‑attention to deep, downsampled feature maps. An all‑MLP multi‑scale decoder, together with RMSNorm and StarReLU throughout, further reduces compute and parameter count. On the large‑scale ARAS400k remote‑sensing segmentation benchmark, LALE establishes a strong efficiency‑performance trade‑off against CNN, transformer, and hybrid baselines. Our smallest variant, (just 1.6M parameters), reaches within 2.6 F1 points of the best baseline (UPerNet) while using 4.5x fewer parameters, 7x less storage, 17x fewer GMACs, and delivering 1.8x higher throughput. The codebase for LALE is publicly available at https://github.com/caglarmert/LALE.
Authors:Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra, Xuelin Chen, Erkut Erdem, Aykut Erdem, Duygu Ceylan
Abstract:
Generative video models have achieved remarkable visual fidelity and temporal coherence, yet intentional camera control remains elusive. Existing frameworks treat camera motion as a byproduct of pixel synthesis, producing trajectories that are stochastic, spatially inconsistent, and indifferent to the human subject driving the scene. In this work, we present Auteur, a method for language‑driven, human‑centric camera framing in generative video. Our core insight is that professional filmmakers conceive shots not as world‑space trajectories but as framings defined relative to the actor, encoding shot size, angle, and composition as functions of human pose and motion. We formalize this intuition as a human‑centric camera parameterization and introduce a Domain‑Specific Language (DSL) that is convertible to standard 6‑DoF camera parameters. A fine‑tuned multimodal large language model then acts as a virtual director, mapping natural language descriptions and coarse human motion to sparse DSL keyframes that are deterministically interpolated into continuous camera trajectories, which are then provided as input to video generators. We train and evaluate Auteur on a new dataset of 34K aligned text, human motion, and DSL‑annotated camera trajectories drawn from procedural synthesis and real‑world movie footage from the CondensedMovies dataset. Auteur enables cinematographic framing of human‑centered scenes, a capability largely absent in prior generative models. To assess this behavior, we propose new framing‑focused metrics, and our experiments show that Auteur consistently outperforms existing methods. Project page is https://cyberiada.github.io/Auteur/
Authors:Muyi Bao, Yuxin Cai, Hang Xu, Zongtai Li, Jinxi He, Jingfan Tang, Chen Lv, Ji Zhang, Yaqi Xie, Wenshan Wang
Abstract:
Vision‑language models (VLMs) have become a common foundation for vision‑and‑language navigation in continuous environments (VLN‑CE). Yet most VLM‑based methods cast navigation as low‑level action prediction, an interface that is ambiguous, tied to short‑horizon motion primitives, and inefficient due to repeated VLM querying. We propose Goal2Pixel, a pure pixel‑based paradigm that reformulates VLN‑CE as navigable pixel grounding. Rather than predicting actions, Goal2Pixel uses the image plane as a unified spatial interface between VLM reasoning and robot motion: the model predicts a visible navigable pixel to the agent, which is back‑projected into a 3D waypoint for forward navigation. For non‑forward actions, we append auxiliary directive regions to the image plane, where the left/right/bottom regions are interpreted as turning left, turning right, and stopping, respectively. To enable long‑horizon navigation, we propose a visibility‑aware keyframe memory for compact and informative history representation. To adapt pretrained VLMs to navigable pixel grounding, we introduce semantic embeddings and coordinate‑aware auxiliary losses. Goal2Pixel achieves competitive state‑of‑the‑art performance while requiring fewer VLM inference calls than prior methods. On R2R‑CE Val‑Unseen it achieves 54.1% SR and 52.5% SPL with just 7.75 VLM calls per episode, 6x fewer than the 46.62 required by direct action prediction at 32.9% SR. The same trend holds on RxR‑CE.Project Page: https://baobao0926.github.io/Goal2Pixel/.
Authors:Haoji Hu, Huaqing Mao, Yijun Lin, Xiaowei Jia, Jinwei Zhou, Minoh Jeong, Yao-Yi Chiang
Abstract:
Pairwise dependence measures such as correlation and causality are fundamental to temporal data mining, yet there is still no principled and robust way to quantify dependence between heterogeneous data types, especially between continuous time series and discrete temporal event sequences. Existing approaches rely on ad hoc transformations or mutual‑information estimators that are highly sensitive to quantization, repeated values, and event redundancy, leading to biased or unstable results in practice. We propose a nonparametric mutual information estimator that directly measures the dependence between time series and event sequences without data transformation, learning, or ad hoc discretization. Our method models the continuous‑discrete duality of real‑world time series to handle quantization and repeated‑value artifacts and introduces a latent event clustering strategy to mitigate bias from event co‑occurrence and redundancy. Together, these yield a robust and unified framework that bridges discrete and continuous mutual information. We evaluate the proposed estimator on four representative tasks: discrete‑continuous time‑delayed mutual information for causality analysis, global and local temporal repetition discovery, discrete covariate selection for time series forecasting, and continuous feature selection for classification. Experiments on synthetic and real‑world datasets show consistent improvements over existing methods in accuracy, robustness, and interpretability, positioning our approach as a general‑purpose dependence operator for heterogeneous temporal data, similar to Pearson correlation for homogeneous time series. Code available at: https://github.com/HaojiHu/Multimodal‑Temporal‑Data‑Quantification
Authors:Xiwen Chen, Wenhui Zhu, Jingjing Wang, Peijie Qiu, Zhipeng Wang, Huayu Li, ZhengXiao He, Xuanzhao Dong, Prayag Tiwari, Mingkun Xu, Yujian Xiong, Feng Luo, Abolfazl Razi, Brendan Hogan Rappazzo, Anderson Schneider, Yuriy Nevmyvaka
Abstract:
Aligning Large Language Models (LLMs) with human preferences is often formulated via Direct Preference Optimization (DPO). However, the standard Bradley‑Terry instantiation of DPO is limited in modeling common departures from transitivity in human preferences. To address this, recent work has introduced Self‑Play Preference Optimization (SPPO), which iteratively refines the policy by training on self‑generated win‑lose pairs. Our investigation, however, reveals a critical instability in SPPO: the optimization is prone to policy degeneration when the preference oracle assigns overly confident wins to semantically indistinguishable responses. To mitigate this, we propose S‑SPPO, a dual‑space semantic calibration framework comprising: i) Supervision Calibration via semantic gating, which anneals win rate targets toward the maximum‑entropy baseline as semantic overlap increases; and ii) Representation Calibration via latent repulsion to enforce geometric diversity to prevent manifold collapse and maintain latent diversity between chosen and rejected samples. Theoretically, we show that the calibration preserves the constant‑sum game structure, facilitating convergence to a Nash Equilibrium. Empirically, S‑SPPO avoids the performance degradation seen in prior methods, achieving 52.19% win rate and 47.46% length‑controlled win rate on AlpacaEval 2.0 with Llama‑3‑8B, without using additional human‑annotated preferences during training. The code will be available at https://github.com/xiwenc1/s‑sppo.
Authors:Žiga Kovačič, Kevin Ellis
Abstract:
To study the ability to infer physical dynamics from videos and extrapolate them forward in time, we assemble a dataset of 2D Material Point Method (MPM) physical simulations covering rich physical phenomena such as deformable objects, fluids, kinetic objects, and emitters. We study code generation and video diffusion approaches on this dataset, identifying their strengths and weaknesses by varying the amount of physically relevant side information. The code generation model, beyond giving a working demonstration of automatic synthesis of MPM simulations, reveals that such an approach struggles with inferring physical parameters from visual input, but relative to video diffusion, produces physically and temporally stable extrapolations forward in time, while the video diffusion model more strongly identifies geometric properties from visual input but produces physically implausible extrapolations.
Authors:Anushka Tiwari, Kaiyi Ji
Abstract:
While prompt‑based parameter‑efficient continual learning mitigates catastrophic forgetting by isolating task‑specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored. We address this limitation by proposing Selective bAckward refinement for positive Backward knowledge transfER (SABER), a replay‑free framework that enables controlled backward transfer in prompt‑based continual learning. SABER determines when backward refinement is beneficial using complementary task‑correlation criteria based on prompt‑gradient geometry and loss‑distribution similarity, and how to perform refinement safely by restricting updates to non‑interfering directions in the prompt parameter space. Extensive experiments across multiple continual learning benchmarks and diverse pretrained backbones, including T5‑Large, LLaMA, and Qwen, demonstrate that SABER consistently achieves positive backward transfer while maintaining strong overall average performance. Code is available at https://github.com/OptMN‑Lab/SABER‑ICML‑2026/.
Authors:Nicolas Hrubec, Jürgen Cito
Abstract:
This paper presents a replication and extension of the recently introduced state‑in‑context agent framework. We independently re‑implement the DirectSolve variant and evaluate it on the SWE‑bench Verified benchmark. We report end‑to‑end full‑benchmark results using GPT‑5‑mini and run selected ablations with GPT‑4.1. In addition, we investigate a complementary research question: What is the impact of token‑reducing input transformation strategies on the performance of software engineering agents? Based on a preliminary prompt analysis, we identify source code as the dominant contributor to token consumption. We therefore apply a series of code minification techniques that remove or shorten non‑essential lexical elements while preserving program semantics. The proposed transformations are integrated into the agent and systematically evaluated. Experiments show that minification reduces average input token usage by 42% with a 12 percentage‑point drop in resolution rate. These findings demonstrate that lightweight source code transformations can yield substantial efficiency gains while retaining a substantial fraction of the baseline performance, indicating a promising path toward more cost‑effective agents. The full implementation is publicly available on GitHub: https://github.com/ipa‑lab/minified‑state‑in‑context‑agent